Red Light, Green Light: Calculating Confidence in Your Data

Accessing a whole world of destination data for the first time affects everyone differently. For some, it’s like being a kid let loose in a candy store. For others, it feels more like being handed all of the controls to a NASA spaceship. In other words, it can feel like a lot.

Whether you approach these insights with excitement or apprehension, we ensure you don’t have to do so alone. The destination intelligence you get through the Zartico Destination Operating System (ZDOS®) is designed to be user-friendly. And — just like in space-bound rockets — we’ve included several “data safety” features to keep you on track, so you can confidently use your insights to plan, measure, and report your destination efforts.

The Sample Size Calculator is a helpful feature included in each ZDOS® module. The sample size number at the top or bottom of the module updates each time you adjust the filters. It also changes color based on the confidence level with which you should evaluate this group of insights.

The Sample Size Calculator uses the familiar colors of a traffic light:

🟢 Green: Proceed and share with confidence! These insights are accurate and highly defendable based on data science best practices.

⚠️ Yellow: You may be going down a bit of a rabbit hole — proceed with caution. Think of this like the indicator on some cars that beeps when you start to drift out of your lane.

🛑 Red: Stop! Your sample size is too small to accurately reflect real-world trends. It’s time to zoom out by expanding either the date range or the geographic area you’re focusing on. 

Why does sample size matter?

When evaluating information about large populations, no data set will represent every single individual in the group. Instead, the available data serves as a representative sample of the whole. 

We are accustomed to applying this logic in everyday life — in surveys, political polling, clinical medical trials, and crash test ratings.

Generally speaking, the larger the sample size, the smaller the margin of error and the higher the confidence level. This means it is less likely that the result you are seeing is due to statistical noise, and more likely that it reflects actual trends.

The data sets we work with are plenty big enough — our geolocation data alone is approximately 5,000 times larger than the sample sizes routinely used in medicine and car crash testing. In addition to size, Zartico’s Data Science team applies techniques like normalization and hotspot filtering to ensure our insights are representative enough for strategic decision-making.

Of course, with great power (or data granularity), comes great responsibility — and some temptation. 

Filters like the ones in ZDOS® let you zoom in on specific date ranges, geographic areas, or segments of visitors. This is incredibly useful for isolating the impact of a marketing campaign or a major community event, or for digging deeper into the preferences and habits of a certain visitor origin market.

But if you go too far — if you try to analyze people visiting your downtown farmers market on a single Saturday in August, for example — then it’s likely that your sample will no longer be sufficiently representative of the whole. Take a glance at your Sample Size Calculator — the yellow or red is a gentle reminder to take a step back and look at the bigger picture.

Another temptation comes with the large sample sizes captured by big data — thinking that the numbers you see in the Sample Size Calculator reflect actual attendance or visitation totals. Even with very large numbers — often many times larger than what you might collect in a traditional intercept survey — it's important to remember that it is only a sample of the whole picture. The proportion of total visitors captured in that sample may fluctuate based on several factors. But keeping an eye on the Sample Size Calculator will ensure the sample you're looking at is statistically representative enough to use as the foundation of your strategic plans.

The Sample Size Calculator is one tool in a robust toolbox full of ways that Zartico empowers destinations to use data insights for strategic decision-making. Beyond the behind-the-scenes work of the Data Science team, ZDOS® users also benefit from a variety of training opportunities and from an active user community that’s eager to share their experiences.

What can your destination learn from Zartico’s place-based insights? Book a demo to find out.