For Beginners in Machine Learning I
Change and response
If you are interested in machine learning, you have probably felt a sense of unease at least once when hearing names and phrases such as Google, Amazon, or the Fourth Industrial Revolution. When the news says AI will transform the labor market, someone says an undergraduate education is not enough for data analysis, or a friend shares a failed machine learning competition experience, you may worry that you alone are falling behind.
I had a similar experience. In the summer of 2012, when I received my PhD, I was a machine learning specialist full of pride and enthusiasm. I was confident that I could study machine learning theory and apply it, and public interest in statistics was beginning to rise rapidly. But that confidence did not last long. Within only a few years, neural networks became a worldwide academic trend and the direction of research changed very quickly. I became anxious not to fall behind: even before achieving a small result in my own work, I felt I had to keep up with another field. Senior academics advised me to be wary of that impatience.
Around that time, I joined a deep learning community on Facebook and browsed the news. A torrent of research results and opinions was being shared. Seeing all the posts and links, I thought, ‘I need to know this, learn Python, write my own paper, and what about my classes…’ Those thoughts weighed on me. It was impatience, and it kept growing.
From a little distance, it is easy to see that such impatience is not helpful. I believe the answer is not to watch what others are doing, but to do what I want to do. Perhaps I have shared my experience to express a simple fact everyone knows, out of sympathy for someone facing a similar concern. Since then I have gradually begun doing what I want to do and continue to follow related research. My impatience has diminished accordingly.
What follows is a modest invitation to stop searching for what everyone else is doing and begin learning data science and machine learning. It is for those who have already made that decision, or would like to.
The structure of knowledge
I come from what is often called a humanities background. When I first began studying statistics as a business undergraduate, I was not familiar with natural logarithms, had never encountered vector spaces, and knew no calculus of trigonometric functions. Mathematical Statistics I and II, which I took out of a desire to learn, were hell: the classes, assignments, and exams. As expected, the results were poor, and I had to start again from the beginning.
As in every discipline, knowledge has a structure. Even if a university machine learning course does not list prerequisites, the instructor assumes students have some prior knowledge. If every course had to be completely self-contained, it would be almost impossible to convey the necessary material within the available time.
Information, hierarchy, and restA structured sequence of courses greatly affects learning efficiency for learners as well as teachers. A question I often ask in class is, ‘Have you learned this?’ If there is little response, or if I have previously taught the material myself, I usually continue. That works if most students have the assumed knowledge, but not for those who do not. Gaps in learning arise intermittently and accumulate slowly, like clothes becoming wet in a drizzle. You may have experienced putting something aside at first, only to find that, after enough gaps accumulate, you cannot take another step. Over time, these unfilled gaps become devastating to learning; satisfaction with the course can fall, and self-reproach can follow. What prior knowledge can reduce these gaps?
The skills needed in machine learning fall into three broad areas: programming, probability and statistics, and optimization. Courses for learning machine learning are built around these areas, and at higher levels knowledge of all three is needed at once. Doctoral students in our lab who want to write papers in AI and machine learning must be proficient in all three. The same applies to advanced undergraduate courses.
To do well in advanced courses in the Department of Statistics, students need sufficient prior knowledge in all three areas.If ambition leads you to take these courses without the prerequisites, gaps in understanding will arise frequently and learning will become less efficient. I think it is important to overcome impatience and study steadily. If you ease the desire to master everything within two years, a path may become visible.
One good solution is to take prerequisite courses systematically. Most relevant departments design their course portfolios with this structure in mind, and students should follow it as closely as possible. Of course, enrollment limits, scheduling conflicts, a late start in a second major, and other circumstances can make the exact sequence difficult. Even so, systematic acquisition of machine learning prerequisites is clearly beneficial for efficient learning. At the same time, instructors should make sufficient efforts within class-time constraints to prevent learning gaps, and universities should schedule courses in a way that respects their sequence.