How did I learn Deep Learning myself?
On the occasion of TensorFlow has just celebrated 1 year of open source , I wrote this article to recount how I learned Deep Learning in the past year, and to synthesize knowledge myself, and to let everyone who is interested in Machine Learning In general, as well as Deep Learning in particular, reference.
I am an engineer on server-side, work mainly uses Rails, SQL and has absolutely nothing to do with Deep Learning. I study completely because of my hobby and because I can predict which skills in this area will be useful later. In college, I also had 1 year of studying sound recognition, so if I said that I didn't have any knowledge, it would not be true, but the confession at that time just made it through the speakers for graduation. Last year when big companies launched a series of Deep Learning frameworks, I found it was time to go back to learning carefully.
Starting from Basic Learning Machine
Machine Learning class through Mr. Andrew Ng's lock on Coursera.
Link https://www.coursera.org/learn/machine-learning
Mr. Andrew Ng is one of the leading people in both Machine Learning and Deep Learning. Mr. Ng had time to teach at Stanford, one of the founders of Google Brain, and now Chief Scientist at Baidu.
This key is really good for beginners. Mr. Ng lectures in great detail all necessary concepts, starting with a simple example like Linear Regression and more complicated models like Neural Network, SVM …
Through this course, I have grasped the core of Machine Learning, as well as knowing what it can do. A bit of regret is that the exercise is written in Octave / Matlab – the language that is mainly used for research and is rarely included in practical products. You may find it a little difficult to learn a whole new language, but rest assured that Octave / Matlab is quite easy to use and the designed exercise is not too difficult.
I recommend studying this course with a few friends, both for motivation and for helping each other in the learning process. The fact that I studied with a few people in the company and took turns explaining the answers of the exercises: very fun and effective.
If you find it difficult to solve the exercises, you can refer to your answers that have been publicized on GitHub .
Ready for Deep Learning
Completing the Machine Learning course will probably take about 3 months. After that, I took a short break and went on to take the second course on Deep Learning. This time I chose the Deep Learning course on Udacity taught by Google engineers.
Link https://www.udacity.com/course/deep-learning–ud730
Yes, because the engineers teach, I feel that this key is not as detailed as Mr. Ng's key. All videos are very short, and just introduce the Neural Network models in Deep Learning. I can understand everything in the video, but when I do my homework, I don't understand anything ?
The beauty of this key is that the exercises are designed to use TensorFlow, which is very useful when applied to practice. But if I just watch the video and do the homework right now, I think it's quite difficult to complete. To do all the exercises, I have to do the TensorFlow tutorials, read more reference books and related essays. In general, video is only 2 weeks to see, but it can take 3 months to solve all the exercises.
My reference answer is also publicly available on GitHub .
Dig deeper into Deep Learning
In general, after the above course, I also "understand well" what is CNN, RNN. But to apply in practice, it is still quite difficult. I started reading books that are considered Deep Learning textbooks. Books can be read online at http://www.deeplearningbook.org/
This book was written by Goodfellow, Bengio et al. This book is quite thick, so I printed it and put it into a paper book to read for convenience. Chapter 1 is the basic knowledge of math and Machine Learning. This chapter adds a lot of knowledge to the Coursera course, helping me to better understand the math as well as the relevance to the statistical probability of the algorithms studied. Chapter 2 is my favorite chapter. This chapter explains in detail the commonly used Neural Networks such as CNN, RNN, as well as its applications in practice. This chapter really helps me fill the knowledge gap after the course on Udacity. The final chapter is for researchers. It is very difficult to understand, so now I only read with the purpose of remembering the concept of the concept is the main, hope later can understand ?
In parallel with reading Deep Learning textbooks, I also watched videos of Stanford University's Deep Learning lectures. There are 2 public keys:
- Image processing key: http://cs231n.stanford.edu/
- Natural language processing course: http://cs224d.stanford.edu/
These two courses are very good, but to be honest, the video is too long and many of the things I think have been fully understood through Goodfellow's books, so I don't watch them all, but just focus on the things I care about.
Start writing the application yourself
After about 10 months to fully digest this place, I started using TensorFlow to make simple play applications. The most recent application is to write 1 chatbot on LINE that can return 1 sticker that matches the message content. The application is still in development because there is still a little data but I have spoken about it at a meetup for engineer in Japan. If you understand Japanese, you can refer to http://www.slideshare.net/vanhuyz/tensorflowline-botaws-lambda
Currently my company started to have a few new projects on AI, and the knowledge I gained over the past year will definitely help a lot.
Summary
Deep Learning is a difficult part, so you can't finish it in a few days. If you want to work in this field, you need time and a little passion. Hopefully, through my article, you can find the best learning method for yourself!