What Kinds of People Live on Planet Hadoop?

Labor market analytics firm Wanted Analytics recently assessed the market for technology professionals and found that demand for people with proficient levels of Hadoop expertise had skyrocketed by around 33% since last year - it is true, Hadoop is hard technology to master and the labor market is not exactly flooded with an over-abundance of skilled practitioners.

Hadoop has been called a foundational technology, rather than ‘just' a database by some commentators - this almost pushes it towards being an ‘environment' rather than it being a single software product... and this all goes towards making Hadoop even harder to master, many will agree.

Interestingly, we can essentially define Hadoop as a free Java-based ‘programming framework' that supports the processing of large data sets in a distributed computing environment with robust data processing and storage power.

Hadoop confusion
Is Hadoop a foundational technology, a data environment or programming framework? It is all of these things of course, but this may be where some of the challenge (and, specifically, the skills challenge) lies.

What kinds of people live on Planet Hadoop? Well, quite apart from being able to breath a noxious mixture of data, storage and Java-flavored oxygen, Hadoop native beings (we'll call them Hadoopers) are deeply innovative.

Hadoopers are the kind of people that are driven by an inherent desire to push the ‘analytics envelope' so far forward that it becomes an operational business process that in itself drives innovation - and therefore greater profits.

These are the sort of people who are prepared to help ‘industrialize the analytical process' and make it part of the way a firm operates from first principles. As Adrian Jones of SAS has put it, the analytics factory provides an approach that offers flexibility for analysts and a structured framework for governance.

"Your data preparation process should make it easy to quickly move between the two states. This step requires the creativity of the business combined with the process and operational efficiency of IT. Taking a factory approach to data preparation will remove this gray area from many organizations, reducing the conflict that comes through duplications and inefficiencies," writes Jones.

Hard-wired into Hadoop
As operational processes throughout the business start to get hard-wired into Hadoop, we see analytics deployed in a production state where it becomes part of the core workflow of all operations across the business - the truly analytical company starts to flourish from this point onward.

This Hadoop planet (or company, or environment, or biosphere) is populated by people with ability and aptitude of course; Hadoop is no place for dummies unfortunately... but what other traits to Hadoopers exhibit?

"Resourcefully creative with an ability to think on their feet," - would be a good way of describing these folk, i.e., they are capable of surveying the analytics landscape and knowing instinctively which datasets to combine, splice, analyze and focus on - but more than anything, they can perceive the kind of outcome that ‘might' be useful without actually knowing how it should initially be quantified and qualified.

A steely stoicism
Being able to convert the analytic process into an operational process requires an aptitude for practicality and a steely stoicism with the patience to wait and look for the right outcomes.

Really successful Hadoopers are corporate beasts with an appreciation for their company's regulatory processes and systems, i.e., there's not much point in producing great Hadoop analytics if it remains untamed analytics. Good corporate Hadoop citizens are capable of making Hadoop insights ‘universally accessible' so that every stakeholder in the organization gets the appropriate level of access.

As Fiona McNeill, Product Marketing, SAS, puts it, "The difference with analytically mature, innovative organizations, is that insights are universally accessible - whether you work in HR, finance, sales, logistics, marketing or services. The data is recognized as a corporate asset and analytical methods become intellectual property."

Nimble feet, fingers and foreheads
This discussion could go on and on. Good Hadoop pros exhibit many, many attributes, but one that is appropriate to finish with is nimbleness. Although this term ‘nimble' has become hackneyed, overused and overtired across the IT industry, we will bid to use it this one last time.

Hadoop is all about trial and error in terms of finding out what works best in relation to any particular analytics job. Nimble Hadoopers are unyielding in their ability to keep trying out analytics scenarios until they get what they need - or, perhaps more important, until they get what they want for the business.

These are some of the many elements currently populating the prosperous parts of Hadoop, whether we survive in this brave new world or not is dependent on our ability to be this kind of Hadooper - welcome to the new world.

This post is brought to you by SAS.

SAS is a leader in business analytics software and services and the largest independent vendor in the business intelligence market.