On behalf of data scientists, I have a confession to make. Despite what you’ve been told, we are not living in the midst of a “Big Data Revolution,” at least not mathematically. In fact, over the past two decades there has been very little change in the algorithms and statistics behind the field of data science and its related predecessors: pattern recognition, machine learning, data mining and predictive analytics. Fundamentally, I’ve been teaching and doing the same things for more than 25 years.
But there is a real difference within this revolution and that is the data itself. Long ago, it was only large and high-end retailers that had the resources to consume high volumes of data across multiple shopper touchpoints that were required to create a single view of the shopper.
Not so long ago, retailers – small, medium and large – collected data separately for each sales channel their shoppers could engage with their brand. The data was stored in silos and analyzed in silos. The data complexity increased as more channels became available: mobile, mobile web, and 3rd party channels like Amazon and eBay.
Fast forward to today. More data is collected than ever before in ever-increasing numbers of online and offline channels, yet most retailers still keep data in silos. Even for the retailers who have successfully integrated these channels into a unified view of their customers, they face an even more daunting challenge: how to find meaningful and useful insights. Collecting and integrating data in of itself is not enough. Derived data must be created from the raw data to reveal these insights.
In the simplest terms, my view of customer intelligence is the practice of leveraging big data and applying machine learning techniques to provide measures retailers can use to better understand customer intent. Doing so effectively requires the alignment of three key elements: 1) Retail Experts, 2) Data Geeks (Analysts or Architects), & 3) Predictive Modelers or Data Scientists. Retail Experts are needed to frame a problem properly in a way that will provide value to the brand. Data Geeks know where the data is, what’s in the data and how to access the data. Predictive Modelers and Data Scientists then build models that achieve business objectives.