How Can You Unlock New Marketing Opportunities With Data Deduplication?
Businesses have always collected data about their customers. General stores in the 19th century didn’t use phone numbers, emails, or loyalty cards, but they sure knew how to recognize their customers from one day to the next. And customers loved the personal attention.
Today, data collection is ubiquitous, but it’s also incredibly complex: from point-of-sale data to e-commerce transactions, from loyalty cards to customer service calls, data is flowing in at a furious rate, and from dozens of channels. And every single one of those channels comes with its own way of identifying customers—when was the last time you gave your email address at checkout to get a copy of your sales receipt?
It’s no surprise then that 20%-40% of customer records end up being duplicates, with each record containing only a fragment of the whole story. It’s costly, inefficient, and infuriating for customers who end up being bombarded with the same messages over and over again—or worse: conflicting messages based on misconstrued datapoints.
Thankfully, we now have robust data deduplication solutions in the industry to help marketers manage duplicate records. Case in point: we recently helped a major retailer unify its customer profiles and eliminate 100 million duplicate records from its CRM database.
Before we consider all the benefits this top retailer derived from its data deduplication efforts, let’s review how it got to that point.
Customer data is great - until it isn’t
The company started out decades ago as a brick-and-mortar specialty retailer, before UPC codes and loyalty programs were a thing. Its customer service was top notch, but it had no way to anticipate customer demand. As the organization grew in size and launched more stores, it got really good at managing its supply chain but had no clear idea of who its best customers were. Marketing was a scattershot affair.
When it hopped on the e-commerce bandwagon 20 years ago, the company immediately looked at it as a BOPIS (buy online, pick up in-store) opportunity rather than a standalone channel. This helped it get ahead of the curve on the identity front (physical stores had to have a way to cross-reference online buyers), but its primary focus at the time was order fulfillment. It didn’t matter if a customer ordered from a different account as long as they could present a valid order confirmation, or if they still lived at the same address since they were coming to the store to pick up their order.
Over time, data gaps and duplicates started to creep into the organization's systems. Some records contained names and phone numbers, others emails only, and yet others relied on customer IDs that had more to do with order IDs than anything else. With more than two million CRM updates coming in every day from a wide variety of data sources, the company couldn't easily tell if those updates belonged to an existing record or required the creation of a new record. In the heat of the moment, more often than not, a new record was created.
Duplicate records added up quickly. The company couldn't afford to stop everything on a dime and rewrite its whole data collection operation from scratch, so it ended up with millions of duplicate records and no good way to stem the tide. While it invested heavily in targeting and personalization capabilities, poor data prevented many of its targeted campaigns from reaching their full potential.
Does this all sound familiar?
Data deduplication is a process
The organization's first reaction was to export all of that data, clean it up once and for all, and re-import it into its CRM. That’s a useful step of course, but there are two important caveats.
One, you’ve got to make sure that you’re using a reliable truth-set to validate that data. It doesn’t help to combine two records if you have no idea if either of them corresponds to a real person. Which one has the correct name spelling, or street address? Our client didn’t use a top-of-the-line truth-set at first and let 80% of its duplicates slip through.
Two, you’ve got to impose strict restrictions on new data updates from that point on, or that hard work will quickly be diluted in a new onslaught of bad data. But we all know that’s not possible. You might be able to impose global address verification on all input forms, but there’s only so much that you can do to control the data that people enter on website registration forms, partner portals, or other applications that eventually feed into your CRM database. You can't foresee either whether someone is feeding you the wrong data, or telling you they moved or changed phone numbers recently. Do you want to turn away a lead because their email looks funky, or their street address doesn’t include an apartment number?
That’s why marketers need to look at data deduplication as a process, not a one-shot project. You want to streamline systemic sources of data duplication, of course, but also acknowledge that duplicate data will find its way into your CRM, one way or another, and that the only way to address the problem properly is to have data deduplication baked into your systems.
What benefits can you expect from data deduplication?
When done right, data deduplication can yield substantial savings and unlock many doors for your company.
If you’re doing direct mail, your stamp budget will thank you. Our client saved $2.5M in postage alone in its very next campaign and used the extra money to reach out to more prospects. It was able to reduce friction with existing customers and improve response rates—not just in its direct mail campaigns, but across all omnichannel campaigns too.
It’s important to note that most duplicate records have bits and pieces that can be salvaged. Data deduplication is not all about deleting records, but rather identifying what bits and pieces are trustworthy, and unifying those data fragments into a single view of the customer. For our client, deleting 100M previously undetected duplicate records was an achievement in and of itself, but it also resulted in the validation and enrichment of 200M other records.
Ultimately, data deduplication is all about unlocking new opportunities to build lasting relationships with your customers. It's not just for retail: marketers in other industries (e.g., insurance, banking, QSR, auto, tech, travel) want to send the right message on the right channel to the right person and at the right time too. They simply can’t do any of that successfully if they don’t have full confidence in their customer data.
Data deduplication is also one of the key pillars of what we call unified identity—a new approach to managing customer data that’s already allowed top brands to cut waste, improve reach, boost sales, and prune inefficient partners at a time of unprecedented change in the industry.
Want to reap the same benefits? Let’s get you started with a thorough data health assessment.