The IT industry thrives on the ‘latest buzz’, the new in thing, the flavour of the month.
The mega vendors like IBM, SAP, Oracle etc love marketing the hell out of whatever the latest buzz word is.
3-4 years ago this was Big Data. Everyone was told they had to have a Big Data strategy and you were a dead company if you didn't. I was asked by so many customers about whether they needed a data lake and what on earth a data lake was. Most of these companies were SMB and didn’t have the budgets or IT departments to undertake such a massive IT project. They also didn’t have petabytes of documents/video data, unstructured data, or the social media need. Basically, they didn’t need a big data strategy. In fact, they just needed a small data strategy!
The latest trend now is of course Cognitive. Mega vendors are telling you that you have to have a Cognitive strategy to survive or prevent yourself from being disrupted. So far, from what I have seen, most SMB companies are going to be fine without needing to think (or define) a Cognitive strategy. The fact is that most Cloud based services are doing the hard work for you. They are actively thinking about how to put the Cognitive capability into their offerings so that you can realise it in your day to day jobs. Take the example of Salesforce: they considered a Cognitive offering but really just decided that it was too hard and easier to just use IBM Watson - since all the hard work (and investment) had already been done.
The same buzz is happening in the job market for Cognitive skills, and right now the biggest buzz is that you need to ‘employ a data scientist to survive’, to disrupt the market or to stop yourself from getting disrupted.
So, let's just take a moment to review what a Data Scientist is.
According to the dictionary, a Data Scientist is “a person employed to analyse and interpret complex digital data, such as the usage statistics of a website, especially in order to assist a business in its decision-making.”
Great! Now, let's look at what skills you need to be a Data Scientist. A quick google search will find you pages such as http://blog.udacity.com/2014/11/data-science-job-skills.html.
So, it would seem to be a Data Scientist you need to be able to cover:
- R and Python programming
- Big Data skills (Hive/Spark/Hadoop etc)
- Machine Learning
- Data wrangling
- Data visualisation
Wow, that’s a big list! I can guarantee that if I did a count of people who would really qualify for this and actually be considered a Data Scientist expert - I could probably count them on my fingers. Also, these unicorns that are Data Scientists aren't cheap. A quick scan on seek.com.au will reveal it will cost you north of $150k for one (and probably more likely $200k).
Now, I am not arguing that a "Data Scientist" isn’t useful when you have a defined business problem, i.e. chronic pain that requires that set of skills to solve. Unfortunately, that’s not always the case though. You don’t employ people and then try to think of a business problem to solve - the business problem comes first.
So, having thought through all that - it is obvious that I am not a Data Scientist. I don’t want to go back to University for 4 years to learn those skills either.
There is a much smarter generation coming up that will have these skills but what they will lack is business experience.
Most businesses today mainly suffer from bad data: i.e. broken business processes, legacy source systems, wrong calculations or just a lack of access to existing data (and of course spreadsheet data, but don’t get me started there!).
So, what I believe most companies actually need is a ‘Data Doctor’ i.e acute pain, someone who can come in and treat the data issue to take away the business pain. A Data Doctor can add a lot more value, a lot quicker and a lot cheaper.
Right, now I’m off to change my Linkedin title…