Dear Fellow Scholars, this is Two Minute Papers

with Károly Zsolnai-Fehér. A neural network is a very loose model of

the human brain that we can program in a computer, or it’s perhaps more appropriate to say

that it is inspired by our knowledge of the inner workings of a human brain. Now, let’s note that artificial neural networks

have been studied for decades by experts, and the goal here is not to show all aspects,

but one intuitive, graphical aspect that is really cool and easy to understand. Take a look at these curves on a plane. These

curves are a collection of points, and these points you can imagine as images, sounds,

or any kind of input data that we try to learn. The red and the blue curves represent two

different classes – the red can mean images of trains, and the blue, for instance, images

of bunnies. Now, after we have trained the network from

this limited data, which is basically a bunch of images of of trains and bunnies, we will

get new points on this plane, new images, and we would like to know whether this new

image looks like a train or a bunny. This is what the algorithm has to find out. And this we call a classification problem,

to which a simple and bad solution would be simply cutting the plane in half with a line.

Images belonging to the red regions will be classified as the red class, and the blue

regions as the blue class. Now, as you see, the red region cuts into the blue curve, which

means that some trains would be misclassified as bunnies. It seems that if we look at the problem from

this angle, we cannot separate the two classes perfectly with a straight line. However, if we use a simple neural network,

it will give us this result. Hey! But that’s cheating, we were talking about straight lines.

This is anything but a straight line. A key concept of neural networks is that they

create an inner representation of the data model and try to solve the problem in that

space. What this intuitively means, is that the algorithm will start transforming and

warping these curves, where their shapes start changing, and it finds, that if we do well

with this warping step, we can actually draw a line to separate these two classes. After

we undo this warping and transform the line here back to the original problem, it will

look like a curve. Really cool, isn’t it? So these are lines, only in a different representation

of the problem. Who said that the original representation is the best way to solve a

problem? Take a look at this example with these entangled

spirals. Can we separate these with a line? Not a chance. But the answer is – not a chance

with this representation. But if one starts warping them correctly, there will be states

where they can easily be separated. However, there are rules in this game – for

instance, one cannot just rip out one of the spirals here and put it somewhere else. These

transformations have to be homeomorphisms, which is a term that mathematicians like to

use – it intuitively means that that the warpings are not too crazy – meaning that we don’t

tear apart important structures, and as they remain intact, the warped solution is still

meaningful with respect to the original problem. Now comes the deep learning part. Deep learning

means that the neural network has multiple of these hidden layers and can therefore create

much more effective inner representations of the data. From an earlier episode, we’ve

seen in an image recognition task that as we go further and further into the layers,

first we’ll see an edge detector, and as a combination of edges, object parts emerge,

and in the later layers, a combination of object parts create object models. Let’s take a look at this example. We have

a bullseye here if you will, and you can see that the network is trying to warp this to

separate it with a line, but in vain. However, if we have a deep neural network,

we have more degrees of freedom, more directions and possibilities to warp this data. And if

you think intuitively, if this were a piece of paper, you could put your finger behind

the red zone and push it in, making it possible to separate the two regions with a line. Let’s

take a look at a 1 dimensional example to see better what’s going on. This line is the

1D equivalent of the original problem, and you see that the problem becomes quite trivial

if we have the freedom to do this transformation. We can easily encounter cases where the data

is very severely tangled and we don’t now how good our best solution can be. There is

a very heavily academic subfield of mathematics, called knot theory, which is the study of

tangling and untangling objects. It is subject to a lot of snarky comments for not being

well, too exciting or useful. What is really mind blowing is that knot theory can actually

help us study these kinds of problems and it may ultimately end up being useful for

recognizing traffic signs and designing self-driving cars. Now, it’s time to get our hands dirty! Let’s

run a neural network on this dataset. If we use a low number of neurons and one

layer, you can see that it is trying ferociously, but we know that it is going to be a fruitless

endeavor. Upon increasing the number of neurons, magic happens. And we now know exactly why! Yeah! Thanks so much for watching and for your generous

support. I feel really privileged to have supporters like you Fellow Scholars. Thank you, and I’ll see you next time!

This episode has some pretty cool figures and animations (thanks for Christopher Olah!), but since this wasn't about pure computer graphics, the visuals are not as spectacular as in the fluid simulation papers. Did you still find it as interesting? Please let me know here in the comments section. 🙂

This is absolutely fascinating. Everyone just normally shows the diagram of the neural networks and I never really understood what was going on, but the way you showed how the graph gets manipulated with subsequent layers made things much clearer. Thanks!

I thought it was very interesting and the animations really helped eith understanding! Also, loved the sound of your mind being blown. Pretty much the same sound I make.

Great to see this 🙂

By the way, on Olah's blog even the comments occasionally hold a gem worthy to be checked out. Some people link interesting papers down there.

That being said, those papers probably aren't typically well suited for this channel. They often are abstract and not very visual. Still, if you are interested in digging deeper, this is highly recommended. And maybe there even are one or two papers that actually are visual enough for this channel hidden in there.

Really like your explanation about these stuff. Keep up with the good work 🙂

OMG This was so beautiful! Thank you!

I'm not a "scholar" but I find it fascinating, thanks!

Epic stuff. Keep it up 🙂

the best one I've seen till now, really cool! thanks sir!

Simply the best explanation of NNs and DNNs I've ever seen. Keep this awesomeness up!

Great video, thanks. I would be really thankful if you could share your opinion on something:

Does this at all aimed to represent the inner workings of the human brain?

I have an example: I have a 2-year old and it is enough to show her an image of a crocodile once or any new object, lets say a crane ONCE.

She then basically recognizes ALL crocodiles and cranes if she sees.

So it is not about comparing lots of images and having a more sharp line between objects but feels like understanding the basic concept of a crocodile.

It may sound a bit platonic, but hopefully get what I mean. She doesn’t need a lot of data to draw a line between the two objects. In extreme cases she needs only one example to recognize that type of object.

I am thinking of this a lot, but i may totally miss the point as a laic. Köszönöm!

Wow, your videos are really great and professional! I thought this was gonna be all stuff I already knew since it was so short and just an introduction, but you managed to stick some parts in there which I had not previously seen, like those visualizations and that thing about knot theory. Great work!

great indeed thanks! I subscribe!

just one passage I do not have clear: why it must draw a straight line? can't it draw a curved line in the first place or a circle in the bull eye example?

Thanks, keep up the good work 🙂

Very very well done! That is the way students can get difficult matter in minutes. Thanks for this …

One minor thing: may be a (better) motivation should be given, why a straight "line" is necessary in the first place.

Very much appreciate such a clear and direct explanation on deep learning.

I have been looking for easy ways to understand Deep learning for a class project, since we only working on Multi Layers Neural networks. And i find your video easy to understand. but now i am thirsty for more.

Please advise with resources or links i can use to gain deep knowledge on this topic.

Thanks in advance

Great video

Hey, really interesting video. Could you recommend any resources for learning more about the mathematics surrounding neural networks?

I have to say, this was one of the best, clear and easy to understand videos I've ever watched about this subject. Exactly what I searched for, very well done, thanks a ton!

Great work!

but how does it decide from all of the edges that a combination of edges make a certain face, even u let it learn for 10000000 years it wont know that thats me unless i told it at least once am i right ??

what i mean is the out put already known we just want to filter to the output !

correct me if am wrong please

I've just discovered this channel and and I have to say big thank you for your work. Man you are doing amazing stuff, I could not imagine how much effort have you put inside each video but the result is 10000000x better.

Concise yet very meaningful. Keep up the great work and useful work. Subscribing to your channel.

Brilliant presentation! I am (K)not kidding 😀

It's pretty good in that it avoids getting lost in too much depth (pun intended) but it severely lacks in that the only explanation for deep learning is that it involves neural nets with a number of hidden layers. Which doesn't differ from basic feed forward supervised learning neural nets.

This was amazing! I am a new subscriber and I found this video very interesting!

Awesome visualisations.Really helped a lot in understanding the NNs.Can u do some more videos explaining what kind of models can a NN take care of and what it depends on?

These videos are brilliant!!

Honestly, I still don't understand diddly squat. But I am glad that there are people out there who do. Just make sure that SkyNet has a sense of humor before flipping the switch, please.

In decades of teaching and research I found that this intuitive explanation of this episode to be the clearest demonstration of NN as a classifier.

BRAVO!

Great video. Can you please clarify what you mean at 5:00 with 'increasing the number of neurons'? I understand that if you increase the dimensions, you will be able to warp and manipulate your graph to find a suitable place for your straight line, and then scale back down to original dimension, but what do you mean by increasing the neurons? I'm also assuming by layers you mean dimensions.

How could I have missed this awesome video for so long?

As I mathematician, I really liked the explanation: "… homeomorphisms, which is a term that mathematicians like to use.". 😉

In terms of the classification problem, is there an advantage of using this over the K nearest neighbors algorithm?

How exactly the data be if images when represented as points in higher dimensional space looks like a knot.Awesome.

So many new informations 😮😮😮

never seen these visualisation with deep networks. very cool video!

Absolutely heretical

Nice explanation

you know your stuff when you can explain a complex subject it in the simplest form, for anyone, anywhere to understand AND appreciate. Kudos.

hats off, you have done a great job. My question is do deep neural networks transform space or transforms data. From what I gather they transform space.

Thank you for explaining it beautifully.

This is so concerning , how Artificial Intelligence – Deep learning etc. can start mimic human :/

The Danger to human species from A.I. depicted in Hollywood Movies is not too far .

Amazing explanation and visualization, thank you!

Really crisp and to the point explanation.

You wouldn't find such a description of Neural Nets normally. Great video !

?

Thanks a lot for this video, you are the only one who actually provides interesting presentations with clear explanations

That is the best explanation I have ever heard for NN. Thank you

Simply the best simple explanation for neural networks and deep learning. Thank you very much for this very nice and outstanding piece of work!

One of the most difficult tasks is to take a complex subject and simplify it to where anyone can understand. And to do it 2 min?

Your work is absolutely amazing, thank you! I can only imagine the effort put into each video. And to think you present papers from so many different fields, you are truly a inspiration.

Best explanation ever.just Amazing

Really the best and simple explanation sir..

I am wondering can we apply deep learning( concept of warp of dimensions )to the problem of string theory which mainly deals with the multi dimensional objects like torus..knots…

If we can solve a "2d entanglement" using 3 dimensions, could we solve a 3D entanglement using 4 dimensions?

this is truly amazing

wow… fascinating channel. Thanks for the clearest explanation ever..

Came here from 3 blue 1 brown. This amazing visualization helped me understand the basic concept of ML really well!

You managed to capture one of the most breathtakingly beautiful ideas behind neural networks in a five minute video, and conveyed it with simplicity and elegance.

Very well done.

puffff, everyone mind blowing explosion

Love this stuff! Reminds me of 3blue1brown.

Wow, excellent video! I learned a bit more of Deep Learning, I saw a linear transformation, visually, in data classification

This is by far the best video of illustration of ANN I have seen on the internet.. keep making this stuff

* Insert mind being blown sound here. *

Awesome Explanation!!

Thanks for the video

Maybe the best video on youtube. I watch 4:30 hours a day.

As a non-scientific person, I found this video verry clear and interesting. Do you know where I can find other 3D representation of data, used for data sorting? Like the ones we see at the end (the intricated TOR and the spiral-like donut)

thanks !

Great

We gonna fail CVMA UofG

#glasgow

Good stuff, liked and subbed. I was heavily into NNs, GAs, classifiers etc many years ago, i think it's time to jump back in 🙂

what a great channel! keep up the good work man.

SOMEONE HELP!!!! 😭 … I'm the type of person who (litterally) struggles 4 attempts per math course!

I'm trying to make sense of NN's & Deep Learning Algorithms so I can survive the A.I era … HELP!!!!!!😭😭

Awesome explanation – all the vector and linear algebra makes sense now – thank you 😉

that. was. awesome!

Mind…..blown. Wait, and this was 4 years ago??? I'm not sure, but I either just [email protected] a brick or a 3rd testicle just descended…

It's amazing. Thank you.