How Customer Shopping Data Is Used to Promote Groceries — Using AI
One of the best ways to inspire customers is a well-designed thematic promotion. That’s why retailers’ circulars are full of recipes from famous chefs and stores are changing into showrooms of world cuisines. But with so many products and brands available, it’s never easy to design weekly promotions that will engage shoppers, attract them to the stores and increase their spend. Fortunately, analysing historical shopping data can help.
The key is to understand market structures — which products are complements (work well together) and which are substitutes (compete against each other). Complements are best grouped together— whether on the shelves or as a thematic promotion in leaflets or newsletter campaigns (Italian, Organic, BBQ…), while close substitutes can be reduced as they don’t represent a contribution to the overall category or store revenues.
Revealing market structures from shopping data
To illustrate how market structures work, we conducted research to analyze 12 months of shopping data collected in a large German city from 147 stores of the city’s largest grocery retailer. Details about this research were published in the Journal of Marketing Research.
We used advanced machine learning algorithms derived from the SO1’s personalization engine to generate a detailed product map where every product in a retailer’s assortment is depicted as a single dot. These dots naturally form various clusters according to how likely they are to co-occur in customers’ baskets across one or multiple shopping trips.
These product groupings do not consider a retailer’s own category definitions, nor how the products are positioned on the market. They only consider consumers’ views of market structures derived from shopping data from millions of shopping baskets .
Some clusters on the map are similar to retailers’ own product categories, such as juices, soft drinks, chocolate bars, wine or bottled water. But the majority of these clusters include products from several different categories, such as Italian, Barbecue, Baking, Breakfast, Vegetarian, Organic and even Children’s Products (e.g., small juice boxes, snacks for school, bologna in the shape of teddy bears, or Disney yogurt). Products from these clusters are mostly located on different shelves throughout the store, but customers tend to buy them together.
Below is an example of how such product cluster can look like. We zoom into an "American" BBQ sub-cluster, which is part of the bigger BBQ cluster.
The top part contains ingredients for “American” barbecue dishes such as hamburgers and hot dogs. Hot dog sausages and hot dog buns are located on the left, burger buns and beef patties on the right. In between, we find complementary ingredients that can be used with both hot dogs and hamburgers (e.g., sliced pickles, fried onions, and sauces).
Although this might seem obvious for humans, remember that the AI works completely autonomously based on data without any previous assumptions.
Discovering regional differences and seasonability
Discovering product groups as seen by customers is already a valuable insight to run promotions and design stores’ shelves, but the true value of this approach lies in its ability to observe differences across various store locations and even seasons.
To show what this can look like, we compare 16 product clusters across three sample stores. Although the stores are in the same city under the same brand and have almost identical assortments, we observe interesting differences between the sales across product clusters:
- Store A is located in a residential area with higher average income and education has higher sales of vegetables (+35.4% compared to average) and organic products (+149.9%) while lower consumption of recipe mixes (-65.5%) and lunch snacks (-38%).
- In comparison, store B, located in lower-income residential area, has considerably higher sales of BBQ products (+71.2% vs. -23.8% for higher income area) as well as German food (+31% vs. -45.4%) and dog food (+80.7% vs. -41.7%). Compared to store A, shoppers at store B prefer beer over wine (+31.2%) and consume more recipe mixes (+38.6%).
- In store C, located in the city center, we observe a big increase in consumption of lunch snacks (+200.1%), fresh juices (+184.2%) and ice cream (+69.3%), and lower consumption of baby products (-44%) and dog food (-70.8%).
Of course, these insights can be learned from sales data itself. But compared to just observing sales, this approach reveals, in an automated way, which exact products people buy for certain purposes (e.g. for BBQ, German cuisine, vegetarian meals…) in different locations. And not by the category as defined by the retailer, but as seen by customers.
Conclusion
Applying machine learning to analyse market structures proved to be a highly effective, automated and consumer-centric method. It allows retailers to group products in a way customers truly shop for them considering regional and seasonal differences, and it does so fully autonomously from the shopping data — taking away the guess work. Retailers can use these insights to optimize assortments, promotions and shelf design for each store individually.
To increase a loyalty program's performance, however, we recommend to personalize promotions for each customer individually. Once the AI understands the market structures, it can also understand customer preferences and recommend to them the right product at the right price—on demand and fully autonomously. You can read the full article at our blog.