How can I learn about neural networks

[Anonymous No.11597138]

I don't just mean coding them, I want to know how they work.

How is a network of dot products able to find patterns and make predictions?

[Anonymous No.11597166]

>>11597138
It's essentially just a fancier form of regression. You have a 'network' of linear transformations to your data, but you don't know what the parameters (also called 'weights') of those transformations are supposed to be.

You can iteratively improve those parameters by starting with a random guess and gradually changing the parameters until your network predicts training data better. This is called 'gradient descent' because you're descending the 'gradient' of error within your parameter space, and your trained neural network will hopefully settle at a minimum that makes it a useful for model.

There's much more to making a good neural network than that, but that's the basic idea behind the theory.

[Anonymous No.11597236]

>>11597138
http://neuralnetworksanddeeplearning.com/

[Anonymous No.11597246]

>>11597138
>>11597166
>>11597236
Why is there so much buzz around this? Everyone and their mother is selling machine learning courses. Who cares?

[Anonymous No.11597252]

>>11597138
simple example
https://www.youtube.com/watch?v=aircAruvnKk

[Anonymous No.11597262]

>>11597246
its interesting until you actually study it

[Anonymous No.11597265]

>>11597246
Because it is useful. Deep learning does image analysis much better than humans can do. It'll replace a lot of pathologists in the next few decades.

Or the more popular example: self driving cars. It is much more simple (and effective) to use machine learning for feature detection in the car's camera feeds than trying to program something like that from scratch.

[Anonymous No.11597312]

>>11597252
>6.7 million views
>2 years old
I'm never gonna make it, there's no use learning this.

[Anonymous No.11597410]

>>11597312
>there's no use learning this.
True but sometimes it's better to learn concepts now so you can pick it up quickly when you actually need to use it. If you're not busy, might as well as learn something new.

[Anonymous No.11597426]

>>11597410
I watched it, apparently that kind of neural network is from the 80s and 90s so I'm way more late to the party than I thought.

[Anonymous No.11597433]

>>11597246

it's very very easy to do and its not difficult to understand
its 100% marketing
it's basically just matrix multiplication and inversion and you don't even have to know how to do it by hand or know linear algebra , since you can just call packages that do it for you

[Anonymous No.11597461]

>>11597138
Read the deep learning book by Goodfellow.

[Anonymous No.11597483]

>>11597426
Neural networks are from the 50s, though the field has been making alot of advancements lately.

[Anonymous No.11597625]

>>11597166
>>11597433
Anons explained it pretty well. However, there explanations were pretty superficials.

Neural networks are interesting because of the many different architectures you can imagine and deploy. CNN, RNN, siamese networks, GAN, etc. Each architecture solve differently a problem. You can get very creative.

Saying that NN is just "matrix multiplication" is like saying "astronomy is just looking at stars" or "painting is just putting oil on canvas".

[Anonymous No.11597759]

>>11597625
>Saying that NN is just "matrix multiplication" is like saying "astronomy is just looking at stars" or "painting is just putting oil on canvas".
So its completely accurate (assuming you are talking about oil paining)?

[Anonymous No.11597830]

>>11597138
There is really not much to it, it's just chained logistic regression. Read the wiki page and try to implement it yourself would be the best way to understand how they work

[Anonymous No.11597833]

>>11597262
kek

[Anonymous No.11597975]

pic related is probably the best intuition booster.

A neural network is a series of nonlinear projections. The idea is that at the end of the network, the input has been projected to be linearly separable.

It turns out that we can learn the parameters to these networks by gradient descent. This is a very non-trivial discovery. To put this in context, Neural Networks had been around for about 30 years before people realized you can use gradient descent on them.

I'm actually published in the field and I'll stick around in this thread if you guys don't act like complete faggots.

[Anonymous No.11597983]

>>11597138
Nobody knows or cares how they work

[Anonymous No.11597986]

>>11597975
>gradient descent
Explain this. And what are some neat things that NNs can do outside of classifying things (e.g. spam vs not spam)?

[Anonymous No.11597988]

>>11597138
>How is a network of dot products able to find patterns and make predictions?
Derivatives. It's moving towards the negative trajectory of the error function, where the error function is a function based on all the weights of the neural network (so that it can find the weights with the highest impact to the error, and change it).

[Anonymous No.11597997]

>>11597986
See >>11597988

[Anonymous No.11598069]

>>11597759
Yes, that's what the computer is essentially doing but it doesn't describe what a person working in the field actually does and what the hard part is.
It would also be completely accurate to say "a mathematician is someone who writes symbols" but it's a rather bad description of the task.

[Anonymous No.11598571]

>>11597975
why are ML students so bad at getting their ML projects to work? Why is building a novel convergent model so difficult?

[Anonymous No.11598580]

>>11598571
The reason they struggle is because there is a huge range of things that could be a problem. Could be the data, could be hyperparameters. You have to develop a kind of intuition for building and tuning these models.

There are also a plethora of tricks that you pick up after being in the game for a while, that new comers will have a hard time recognizing.

[Anonymous No.11598588]

>>11597246
Because of you have several gigabytes of data, a computer with several gpus and are lucky enough to find the correct architecture and training scheme for your problem, you can make most things "good enough" to sell to people

[Anonymous No.11598589]

>>11598580
any common missteps you've seen?

[Anonymous No.11598596]

>>11598589
Data normalization is a very common problem. It's one of the things that you don't really learn from books or courses. It also varies across data domains and even across models.

After reading hundreds of papers, you pick up a bag of standard normalization techniques to throw at most problems you come across.

[Anonymous No.11598602]

>>11597138
Andrew Ng course
Stanford has http://deeplearning.stanford.edu/tutorial/
From there it's mostly papers. There isn't actually a lot of theory (some theorems about approximation, some theorems about what kinds of functions can be approximated, but this kind of stuff has nothing to do with what most deep learning researchers work with).
There are a few books about data augmentation and etc, and working knowledge of statistics is always useful but it's mostly practical/intuitive knowledge.
Be aware that the reproduction crisis on deep learning makes the reproduction crisis on psychology look tame by comparison. Most papers are trash and you will be frustrated. "Modern day alchemy" is a meme for a good reason.

[Anonymous No.11598682]

>>11598602
sounds like a field that's moving too fast to understand what it's doing. I bet there's room in the near future for some lucky schizo to stumble on some beautiful theory/framework for analysing these models in a mathematically rigorous way. unfortunately, I'm a controls guy, and theory in my field is completely cemented, with new ideas being mostly ridiculed for not falling within existing frameworks.

[Anonymous No.11598708]

>>11597138
Sounds like most people ITT have just taken/familiar with Andrew Ng's introductory course and didn't dive any deeper than that.

ML is much, much deeper than gradient descent, simple classifiers and regression.

You're basically asking competent people to explain an extremely deep and nuanced topic in laymans terms because you're too lazy to do the work. Fuck off.

[Anonymous No.11598712]

>>11598708
If you think it's anything deeper you're in the wrong, or should explain why it's deeper

[Anonymous No.11598715]

>>11598708
>ML is much, much deeper than gradient descent, simple classifiers and regression.

Yes, but OP has no background and just wants a cursory overview. Drowning him in jargon and cutting-edge topics doesn't do him any educational favors.

[Anonymous No.11598722]

>>11598708
You don't know shit idiot.

[Anonymous No.11598788]

>>11598682
Unlikely at best, much simpler neural network models than cnns don't have a cohesive theory and they have been around since the 80s.
Plus theory has very little to do with the daily work (and interests) of practitioners.
Most of the field's growth is due to money coming from industries where it can be used, but there isn't a lot of money for theory. There are a few concepts with are important, but they are just that.
You get grants and found startups by recognizing cars on satellite photos and segmenting cells on microscope images, not by creating theory that is unlikely to bear monetary fruit in less than 40 years

[Anonymous No.11600048]

>>11598788
I could say the same for all of mathematics. it doesn't stop the schizos.

[Anonymous No.11600082]

[Anonymous No.11600146]

>>11600082
gif-bro is back! How you're doing these days?

[Anonymous No.11600210]

>>11600146
im gud fwend, i never left though dun worry. im always lurking