Provocateur:

Marketers who are intent on taking a data-driven approach can learn a great deal from pollsters’ strategies and missteps.

​

2016 was witness to some tumultuous election cycles, from the Brexit vote in the UK to the presidential election in the U.S. In the two aforementioned cases, the pollsters had it wrong from the start—and the results of both were shocking to many. With the millions of dollars spent on polls at various points of both elections, how did the pollsters get it so wrong?

​

While political analysts are poring over every detail, one thing is abundantly clear: The problem was in the data. The polling data that was used wasn’t up to snuff, and because of this, the end results were inaccurate.  

​

There are several parallels between political polling and marketing measurement, and marketers face many of the same challenges that pollsters do. While using the wrong data caused pollsters to miss the mark on predicting election results, a problem like this in marketing can also have severe implications, including allocating millions of wasted dollars towards channels or audiences that won’t ultimately drive desired business outcomes.

​

It’s all in the sample

Election polls are designed to take a random sample of the population in the hopes of being able to predict the outcome of the election. For polls to be accurate, the sample must share four key characteristics. First, it must be representative of the population that will vote in the election. However, it’s incredibly difficult to select a sample that has the exact consistency as the universe of people who will actually vote (versus the entire universe of registered voters).

​

Second, the sample size must be large. It’s important to include all segments of the voter population. If certain segments are missed, then the sample isn’t reflective of the entire population. But even with large sample sizes, it’s possible that the people you’ve selected don’t accurately reflect the true desires of the voting population. For instance, even if all major segments are included in the sample, there may be small pockets or sub-segments that aren’t represented.

​

Third, the polling data must be available at the finest possible level of granularity to avoid skewing the results.

​

Finally, the poll questions must be well written to avoid swaying the response. Slight variations in the questions can alter a respondent’s perspective on an issue and seriously impact the accuracy of the results. Moreover, people sometimes give politically correct answers to pollsters but vote differently at the polling booth.

​

With polling results so heavily depended on sampling methods that pose so many challenges, is it any wonder that the pollsters got it wrong? Anything short of the full data set and the sample will be imperfect. Getting polling right means dealing with that imperfection with the lowest margin of errors.

​

The impact on marketing

Many marketers rely on their own form of polls, otherwise known as surveys, to measure marketing effectiveness, predict performance, and guide decision-making. Like election polls, marketing surveys are never perfect. Although survey-based measurement can certainly be executed within a margin of error, surveys are prone to similar flaws, even when conducted by a reputed and experienced agency.

​

Accuracy issues arise if marketers don’t leverage the best possible samples that are robust, granular, and representative of the target universe of consumers, or if the questions aren’t asked in the right way. Additionally, surveys can take weeks or months to execute and only provide a distinct “snapshot in time,” making it difficult, if not impossible, to act quickly on the information and make decisions on a campaign-by-campaign basis. For companies on a growth trajectory or looking to scale, this just isn’t a practical option.

​

The solution: advanced measurement

To increase accuracy of measurement, marketers need an advanced measurement approach that eliminates the assumptions and subjectivity of survey-based methods. They need a mathematical approach that incorporates all available data across both converters and non-converters (as opposed to a sampling of just the survey responders), and provides insights on a recurring basis and on a much faster cadence to measure effectiveness throughout the life of a campaign.

​

Why all converters and non-converters? To illustrate the importance, let’s go to back to our polling example. Consider a scenario in which a pollster surveys 10,000 people and finds that 90% of responders are women voting for Candidate A. Candidate A’s team might then surmise that approximately 90% of all women will vote for their candidate. But what if only 10% of the 10,000 people surveyed responded to the poll? What about the people outside the 10,000 who were surveyed? By not accounting for the large percentage who didn’t take the poll, the pollsters will miss the opportunity to assess true sentiment and make assumptions based on data that didn’t tell the whole story.  

​

The same is true in marketing. If marketers look only at those who converted, while excluding those who did not, then they are making decisions based on skewed insights of overall marketing performance. A marketer only has the complete picture when he or she looks at the population that has been exposed to her marketing efforts as a whole.

​

Yes, 2016 was the year of polling missteps, but marketers can learn from pollsters’ mistakes. By tapping into all available data, marketers can draw more informed and accurate conclusions. As for the pollsters, there’s always the next vote, right?

About the Author

Anto Chittilappilly is cofounder, president, and CTO of Visual IQ. He’s responsible for the company’s overall engineering and technology innovation strategy.​

A seasoned entrepreneur and  marketing measurement and optimization specialist, Anto has a passion for helping advertisers measure and optimize their advertising and marketing spend. He is a recognized expert, author, and speaker on marketing attribution, marketing mix modeling, and marketing audience analysis.​

Find Anto at @achittil and on LinkedIn.