How can you determine whether a sample accurately represents a population?

A survey can only be truly valuable when it’s reliable and representative for your business. However, determining the ideal survey sample size and population can prove tricky. In other words, who will you be surveying and how many people? No idea? No worries. We’re here to help!

Say you’re a market research manager at a furniture company and you are planning to launch a new furniture line by the end of 2016. However, before you launch the new line you wish to conduct an online survey on whether your line ‘Fall – 2016’ is more or less likely to be a hit or miss on the European Union (EU) market. So far, so good. Yet, the following question will almost instantly arise: “What is the population that I would like to survey?”. Or, who do you need to survey to gain valuable insights in the success of your new furniture line? In this case the answer is rather straightforward. Assuming that you are launching the new line on the European market, that minors do not buy furniture and that your furniture is reasonably priced, your population consists of all adults in the EU.

Open Sample Size Calculator

What is the survey sample size?

For obvious reasons it is impossible to survey those (roughly) 400 million adults in the EU. A sample of adults living in the EU offers the solution for this issue. A sample is a selection of respondents chosen in such a way that they represent the total population as good as possible. However, instantly a new question comes to the forefront: “How many people should my sample consist of?”. Using a correct survey sample size is crucial for your research. After all, a sample that is too big will lead to the waste of precious resources such as time and money, while a sample that is too small will not allow you to gain reliable insights.

So, how large should your sample be? Should you survey 1%, 5%, 10%, … of the adult citizens in the EU? Well, this depends largely on how accurate you want your survey data to be. In other words, how closely you want your results to match those of the entire population. There are two measures that affect the accurateness of the data.

• First of all there is the margin of error (or confidence intervals). In short, this is the positive and negative deviation you allow on your survey results for the sample. Or, in other words, the deviation between the opinions of your respondents and the opinion of the entire population. An example will shed some light on this statistical explanation. Suppose you set your margin of error on 5%. If – let’s hope so! – 90% of your survey respondents like the ‘Fall 2016’ line, a 5% margin of error means that you can be ‘sure’ that between 85% (90%-5) and 95% (90%+5) of the entire population actually likes the ‘Fall 2016’ line.
• Second there is the confidence level. This tells you how often the percentage of the population that likes the ‘Fall 2016’ line actually lies within the boundaries of the margin of error. Or, following on our previous example, it tells you how sure you can be that between 85% and 95% of the population likes the ‘Fall 2016’ campaign. Suppose you chose the 95% confidence level – which is pretty much the standard in quantitative research1 – then in 95% of the time between 85% and 95% of the population likes the ‘Fall 2016’ line2.

How many respondents does your survey require?

Once you have decided how accurate you want your sample data to be, you can start calculating how many respondents (people who have completely filled in the survey or completes as we call them at CheckMarket) you actually need.

Below you find an indicative table on how to calculate your number of completes. Remember that your population consist of approximately 400 million adults in the EU. As a consequence, the appropriate number of completes will be found on the last row of the table below. Depending on the confidence level and the margin of error, the number of completes will vary. As we chose a margin of error of 5% and a confidence level of 95% for our ‘Fall 2016’ campaign, you need approximately 400 completes (it is advisable to round to the nearest hundred) for your sample.

Alternatively, on the CheckMarket website, you find an easy sample size calculator to calculate the number of completes…

What about response rate?

Before you start sending out your survey to 400 respondents, remember there is such a thing as response rate. Response rate is the ratio of respondents that fill in the questionnaire they received compared to the total number of surveys you send out. For instance, if you send out your survey to 400 people and you receive 200 filled in surveys, your response rate is 50%.

For an online survey, conventionally, a response rate of 20% is considered as a good response rate, while a 30% response rate is considered to be really really good. As we calculated that we need 400 completes, this means that you will definitely have to send the survey to more than 400 people in order to reach those 400 completes. Obviously, you cannot predict beforehand what response rate you will achieve. However, assuming that your survey will achieve a response rate of 20%, we divide the objective of 400 completes by a response rate of 20%. As a consequence, you will have to send your survey to approximately 2.000 adults in the EU.

Which of the following determine how represent a sample is of a population?

How can a sample be representative of a population?

How can we ensure our samples accurately reflect the population?

What determines the accuracy of a sample?