Big Data Scientists

High school seniors are getting ready to graduate next month here in the United States. Many of them are considering a high-tech career. Some parents have asked me: “Where will the high-tech jobs be in four years?” and “What major(s) should I focus on?”.

I share with them with a new job title that most have never heard of before: Big Data Scientist.

The term “Big Data” has become big brother to the term “Cloud”. Wikipedia has an excellent introduction to big data. I also recommend that you read industry blogger Chuck Hollis’ opinion about how both terms (Big Data and Cloud) are coming together. If you want some real-world examples of how difficult it has become to analyze enormous amounts of data, read this article by The Economist.

So what does it take to become a big data scientist? Do you need a strong background in IT systems? Is it important to understand how to build information storage infrastructures so that massive amounts of data can be quickly analyzed? Should you become an expert in database technology?

My answer is no. While I believe that the study of IT infrastructure is interesting, and the experience of building these infrastructures is valuable, my opinion is that there will be fewer and fewer of these types of jobs available. Over the next four years, access to big data repositories will become more and more common. Cloud providers will figure out the plumbing for these systems. Big datasets will be ready for analysis.

The more relevant questions will be:

• • How do I effectively search big data?
• • How do I effectively analyze big data?
• • How do I effectively share (e.g. visualize) the results?

College students would do well to prepare themselves to answer these types of questions. Here are several majors that would serve them well:

1. Computer science. The algorithms that one uses to search big data repositories are going to be critical. Computer science can train and prepare students to use the right tool at the right time in the right situation.
2. Mathematics. Know your statistics. Numerical and statistical analysis will rule the day.
3. Social sciences. If you know your algorithms, and you know your math, you will really win if you understand people emotions and how to present data effectively. Professor http://www3.babson.edu/academics/faculty/tdavenport.cfm Tom Davenport, one of the big names in Big Data, has a B.A.in Sociology ('76), an M.A. in Sociology ('79), and a PhD in Sociology ('80). http://en.wikipedia.org/wiki/Anthropology Anthropology is an interesting angle as well.

Quora has an excellent set of answers to the question: “How do I become a data scientist”? Algorithms and math are at the top of the list.

My company (EMC) is hosting a Big Data Heroes contest this coming May. This contest will honor up and coming data scientists.

In the 1980s and 1990s, Computer Science Majors with Software Engineering degrees created nice careers for themselves. In 2011 they still can! Here are my closing tips:

• • Focus on algorithms for churning through massive amounts of data.
• • Consider adding a Mathematics major (or minor) with an emphasis on statistics.
• • Study data visualization (methods of presenting the results of your analysis).
• • Become an expert on collaboration software (blogging, wikis, Twitter, etc).
• • Learn what makes people tick, and learn how to most effectively present to them.

Steve
http://stevetodd.typepad.com
Twitter: @SteveTodd
EMC Intrapreneur