Big Data Education at the University of Seoul
Planning a career as a big-data specialist
It seems that more people are entering the University of Seoul hoping to become data professionals. A few years ago, so many projects appeared under the banner of big data that I sometimes thought it was excessive. Yet we still hear of a shortage of people who can work with data, and most US universities are expanding data-related majors and courses. It really does feel as though the age of data has arrived. I cautiously expect this trend to continue for some time. Let me begin with data scientists.
Among data-related occupations in workforce-demand forecasts, data scientists represent the smallest labor market. Nevertheless, many areas still need them, and their compensation tends to be high. Why can we not train large numbers of people to work with data all at once, like making products in a factory? Acquiring knowledge and technical skills may sound sufficient, but data analysis is not so simple: it requires a deep understanding of the domain. Technical skills may be comparatively easy to acquire, while building domain expertise can be a major barrier. Data scientists must therefore be versatile: able to communicate with domain experts, establish the main constraints and direction of analysis, examine technical problems as a whole, and find ways to solve them. Because they often act as the captain of a project, setting its overall direction and drawing conclusions, employers may need relatively few people but seek the very best. This is why a PhD is often preferred and at least a master’s degree is commonly sought.
Must everyone become such a superhero? I would strongly support any student who wants to study with that ambition, but not everyone can or needs to be one. In practice, the important thing is to solve the problem at hand, often as a team. Data-related work can broadly be divided into hardware and software roles. The former includes database administration for storing and processing data, computer networks, and security; the latter includes processing and preparing data, ongoing analysis, and problem-solving analysis, including programming. Even a superhero data scientist cannot know all of these equally well. More importantly, the problem to solve is not confined to analysis. MOLOCO, which provides personalized online advertising, reportedly competes above all on processing data and delivering recommendations very quickly. Excluding transmission, a machine learning algorithm may have only about 0.1 seconds to act: it is a race for speed. If a customer has a 0.5-second waiting-time limit for a query, some time goes to traffic and some to the machine learning algorithm. From the service manager’s standpoint, an improvement anywhere helps. A recommendation algorithm is unusable if it cannot return results within the service deadline, however much it improves recommendation quality. Reducing that time requires a comprehensive decision that considers not only the learning algorithm but also hardware, networks, databases, and many other technologies. Machine learning and AI are still only part of a service.
The conclusion is that machine learning and AI are not everything. Do them if you find them interesting. (In fact, we do the work by helping one another.)
Entering a master’s program
This is for people wondering whether to pursue a master’s degree. Some things on their minds when entering a program:
- Parents: ‘You must have a lot to study.’ (The tuition is breaking our backs, kid.)
- Relatives: ‘It must be hard to find work. I hear the job market is tough.’ (Oh dear, I should watch what I say.)
- Friends: ‘…’ (I start work next week.)
Graduate life certainly brings pressure. People around you are working toward something and constantly stimulate you; it is stressful. Yet I suspect you also know that this pressure can help you grow faster. One thing I can say is that graduate school lets you learn much faster and more efficiently than studying alone. A culture in which people remain interested in a field, continue researching it over a long period, and share what they learn was not created by a single advisor and cannot be monopolized by one person. The atmosphere, lab culture, and values built by the many people who have passed through the lab are neither easy to create nor easy to imitate. The power of a culture formed within such a small group can be seen in many ways.
Graduate school is more than adding skills and experience to a career: it is also about finding people with whom to share what you love for life. I therefore see self-deprecating accounts as the unfortunate stories of people who did not happen to meet good people with whom to share their interests, and as an opportunity for those of us who build a lab together to reflect on ourselves.
If you are considering a master’s degree as a big-data specialist, I recommend it: you can find lifelong colleagues with whom to share work you enjoy and, although the time is short, gain an opportunity to immerse yourself in something with an urgency that makes it feel as though the world depends on it.
A view of Mirae Hall, University of Seoul