In a previous post, I discussed how Target used GuestID numbers and advanced algorithms to track and predict customer buying habits. Target is just one example of a retailer using big data to enhance business. The real juggernaut of retailers using big data should not be a surprise—Wal-Mart. The retailer captures point of sale data from 5,000 stores in six countries every second and throws all this data into a 30 petabyte Teradata database (1 petabyte=1 million gigabytes). Now that’s BIG data!
Wal-Mart’s data center is considered one of the most sophisticated in the world. Much like Target, Wal-Mart carefully analyzes each piece of data and uses big data algorithms which churn behavioral patterns and basically a roadmap for each store. Wal-Mart then uses this roadmap to help influence how each store is operated.
Although Wal-Mart has seen incredible success, they are starting to slip. They reported a 1.4% revenue decline and a 16.4% drop in operating profit in the 4th quarter. This situation now creates a question … What is Wal-Mart missing?
They often have the lowest prices available, military-like infrastructure, a tried and true business model and one of the world’s most sophisticated databases. However, Wal-Mart may be neglecting something really simple—their customers.
Big Data’s Fatal Flaw
Wal-Mart has lost contact with their customers who are their most important asset. They, as with many other businesses, have relied on the assumption that big data can tell them everything they need to know about their customers. Big data allows you to discover incredible and insightful information about customers—but it is only “cold” data. It does not tell the whole story. It is just numbers.
For example, do you think you could quantify the love of your life? Do you love your girlfriend because she is 5’6”, has Pantone 12-577 hair, drives a mid-sized sedan, and the digits of her cellphone number create that affection—probably not.
Big data tells a lot of the story but it misses out on the most important part: causation.
Correlation vs. Causation
Big data provides businesses with correlation. “Small data”, like interacting directly with customers, provides causation. Correlation is “a mutual relationship or connection between two or more things” but it is does not describe the entire picture. Causation is the reason for that correlation.
Both big data and small data, working together, are powerful tools for retailers. Once a specific causation for a customer’s behavior is identified through small data, big data can be used to correlate and verify the behavior in overall picture.
Consider this example …
- Can you positively correlate soft drink sales with drowning related deaths? Of course this makes no sense but it is statistically correct. Why would this be? In the summer, both drowning related deaths and soft drink sales increase almost equally.
- The reason? Warm weather means more people are swimming which increases the chance for drownings. Also, warm weather promotes soft drink sales sales as people are looking for something to quench their thirst in the summer heat.
- Therefore, it would be really silly if soft drink bottlers targeted areas with the most drowning cases to promote sales because their big data analysis correlated that result.
- The small data cause must also be discovered and considered. In this case, warm weather is the cause behind sales increases for soft drink sales.
Now consider the Lysol example …
- An example where causation was the deciding factor for the success of the product.
- Lysol was originally developed for NASA because in an airtight capsule the smallest odor can create havoc on astronauts. The Lysol prototype eliminated odors incredibly well. This example caused its developers to consider that it could have great success in the civilian realm.
- Lysol could be the cure for poorly smelling bathrooms, pet odors, etc. that people needed—so they introduced it to the public hoping to make millions in sales.
The millions of dollars were no where to be seen and the sales data made product success seem hopeless. Now if you were to base the success of Lysol on the sales numbers at the time you most likely would have scrubbed the whole thing. It just was not selling. However, certain they had a solid product, marketers started to interview customers and found the cause behind their misfortunes.
The pieces of small data they discovered helped them change a major flaw in the marketing of their product and it became the Lysol we all know today. This is one example where small data was an invaluable asset.
I guess I should tell you what Lysol marketers found. The original Lysol had no smell to it. Yes, it eliminated the worst odors, but customers really did not know it was working because the odors just vanished and that was it. Lysol added some perfume to their spray and presto! They had an enormously successful product.
So whats the takeaway?
Big data can tell you massive amounts of information and correlate many things; however, it is missing causation and cannot provide an accurate, complete story. The increase in drownings does not mean soft-drink sales personnel will be getting a bonus for a jump in sales. Or focusing on sales data is not an accurate indication of a product’s usefulness as in the Lysol example.
Big data and small data need to dance together to provide an accurate measure of trends and patterns for effective decision making.