September 30th, 2020

Sam's Club: 2020 Genius Awards Finalist (Marketing Analytics Adoption)

This is the second in a series of 12 blog posts showcasing the achievements of the finalists of the 2020 ANA Genius Awards, presented by Neustar. The 2020 Genius Awards winners will be announced at the 2020 ANA Masters of Marketing Conference, held October 21-23.



For membership-based brands like Sam's Club, true marketing ROI isn't calculated by simply measuring sales. Instead, the real value of marketing lies in being able to acquire the right members, keep them engaged with the most relevant personalized offers, and ensure they perceive enough value to renew their membership the first time and each subsequent year. For Sam's Club, it's not a question of how marketing can drive sales, but of how it can add value for members.

Here's how the leading membership discount retailer adopted a new suite of marketing technology powered by machine learning to improve the ROI of its traditional marketing outreach, centered on direct mail campaigns, to deliver greater value for its members.

The Challenge

Calculating the true ROI of marketing down to the individual member level

Up until about three years ago, Sam’s Club had a one-size-fits-all approach to marketing, with each member receiving the same communications and identical cookie-cutter offers. 

That all changed when the company formed a "Decision Sciences" team to facilitate data-driven decision-making across the organization. A new targeting, measurement, and optimization suite named "Omega" was born, and the marketing team updated its approach to include the use of randomized control trials, uplift models, and machine learning to deeply personalize and improve the relevance of their marketing communications for members. 

 "We went from chain-level marketing to member-level marketing using a member’s omnichannel behavior and experience to drive the messaging," explained Agastya Kumar Komarraju, Associate Director, Head of CRM Analytics, Sam's Club. 

The Approach

Leveraging uplift modeling to improve customer segmentation and personalization 

For the last two years, Sam's Club has relied on uplift modeling—a machine-learning technique that focuses on maximizing incremental sales—to improve its targeting and outreach.

Over time, by testing all direct mail campaigns through randomized control trials, Sam's Club developed a rich repository of experiment data, which became the foundation for the company's Omega targeting, measurement, and optimization suite. 

Armed with this data, the company realized they would be more successful if they shifted their approach to targeting. Instead of casting a wide net to find potential shoppers and develop campaigns to convert sales, they saw they'd have better success by narrowing in on a specific segment of customers, those with the potential to be influenced to make purchases as a result of relevant advertising, which they named the "Persuadables."

That way, they could minimize advertising spend and conserve resources by avoiding customer targeting on either end of the spectrum: those already loyal and likely to purchase, regardless of marketing exposure (the "Sure Things"), as well as individuals unlikely to be moved to shop at Sam's Club even after marketing exposure (the "Lost Causes").

By using uplift modeling, Sam's Club was able to create targeted outreach for this key priority segment, the Persuadables group described below, while avoiding the deprioritized segments—including the Sure Things, "Lost Causes and Do-Not-Disturbs detailed below—to drive up to 100% return on ad spend (ROAS) for the company, compared with random targeting. 

Priority Segment

Deprioritized Segments

  • Persuadables: Members who will only buy and renew if they receive marketing (Sam's Club's target segment)

  • Sure Things: Highly engaged members who will purchase regardless of whether they receive marketing ads or not

  • Lost Causes: Members who will not buy or renew whether they receive marketing ads or not

  • Do-Not-Disturbs: Members who might cancel their membership if they are reminded through a renewal notice that their membership is up for renewal 

While traditional response models -- such as decision trees and propensity models -- might suggest marketers target cohorts like Sam's Club's Sure Things segment, unable to distinguish them from individuals in groups like Sam's Club's Persuadables segment, uplift models overcome this limitation through predictive modeling. Often referred to in the industry as "incremental models," "true lift models," or "persuasion models," uplift modeling has enabled Sam's Club to contact fewer members while achieving better results.

Now, each campaign launched through Sam's Club Omega is set up as a truly randomized test, so that the marketing team can understand the incremental impact of the outreach, and, with measurement automated at various levels of campaign granularity, the company's uplift models optimize future marketing budgets based on the experimental data. Members are constantly graded, with the model ensuring only those who score with a high need for an additional push to convert are followed up with. Lastly, the company optimizes its merchandising mix by identifying those items that aren't driving incremental sales and replacing these ad space with items more likely to deliver an uplift in renewals. 

The Impact

By adopting the Omega platform, Sam's Club is maximizing ROAS, boosting membership renewal efforts, and optimizing its merchandising mix

Since Sam's Club overhauled its entire direct mail program using Omega, the brand has achieved a more than 100% lift in ROAS compared with randomized targeting, as well as up to a 51% lift in ROAS over decision tree targeted segments, and up to a 12% lift in ROAS over previously used propensity models. Now the marketing team can clearly pinpoint both ROAS and member renewal rates and which products drive the most incremental sales and tie that back to marketing initiatives.

Randomized controls have helped the team get a granular view of which products are instrumental in driving incremental sales and, more importantly, which are not, enabling Sam's Club to optimize its merchandising mix as well. For some of its poorly performing campaigns, they realized that some of the items had no incremental sales and were being purchased by members, regardless of whether they were advertised.

As a result, Sam's Club shifted from highlighting these to focusing on brand building advertisements to drive a longer-term impact on the company's brand equity. This, in turn, will provide further insights for the Omega measurement suite so the marketing team will be able to measure their impact not just on sales, but on other KPIs, such as awareness and brand perception. 

Final Thoughts

Beyond overhauling its direct mailing efforts, Sam's Club has been working on rolling out its uplift models and automated measurement to other channels, including email, the company's mobile app push notifications, and digital campaigns on its website.


More about the 2020 Genius Awards Finalists

Previous: Twitter I Next: Microsoft




Let's Connect

Learn How Your Company Can Benefit from the Power of Trusted Connections.

Contact Us   Give us a call 1-855-898-0036