Conducting surveys: part 1

Group Of Happy Doctors4 out of 5 doctors recommend…what? How do we know what 4 out of 5 doctor’s recommend? Probably because someone conducted a survey. Sounds simple, doesn’t it? But obtaining accurate results can actually be difficult.

Conducting surveys can provide valuable information to help you guide your business. Whether you are performing a brand survey, or a best-in-class research project, finding out what people think and what they do is an important activity. Sometimes you find out new information that you didn’t know, which is obviously valuable, but validating information that you already believed or suspected is just as important.

Where to start? The first thing to consider is who to survey. This may sound simple: you have a mailing list and you can simply send out your survey to your existing mailing list. While simple and convenient, this presents a number of problems as well.

To start with, we need to look at what our population consists of. The population is the entire community that we are interested in. This might be all consumers if you sell iPhones, or all B2B marketers if you sell business marketing solutions. For this population you need to come up with a sampling frame which includes all of the members of this population. Some sampling frames are easy to come by – a list of all students at a particular school, a list of all members of an organization. Some are more difficult to come by like coming up with a list of all business buyers in a whole country.

Whether or not you are able to obtain a list for your entire sampling frame, you often are not able to survey an entire population (saturation sampling)  simply because of the number of members. In most cases, you will select a sample from the entire sampling frame and use this smaller sample for your survey. While this makes the survey much easier to conduct, it also presents a number of challenges. Primarily, we need to know what the relationship is between the make-up of the sample to the make-up of the whole population. If 75% of the population thinks A, but in our sample we pick mostly the people that think B, our survey results will not be representative of the whole. There are a number of ways to address this.

The first way to select a subset is to simply pick members at random. This is the easiest way and can often result in a good representative sample. Another way to create a sample is to use systematic sampling and select every nth member of a list. Both of these methods, however, also have drawbacks.

If the sampling frame that we have is not actually the set of the whole population, and if it is skewed in some way, than a random or systematic sample pulled from the frame will probably be skewed as well. Of course, there is always the chance that a random selection itself can be skewed even if the sampling frame is complete. For example: if we have a population of all potential voters, we can easily use a voter registration list. But if we find out that anarchists are half as likely to register as people with all other types of political affiliation, we can be confident that the voter registration list is not an accurate representation of our entire target population. Or if we have a population that is 50% male and 50% female, but a random sample happens to include 60% female and 40% male, then our sample is not going to be representative.

One way to address this is to create a stratified sampling, often as a first step in a two pass sampling selection process. In a stratified sample, we split the whole list into groups and then proceed to apply a sampling technique like random or systematic to each group. After that, we combine the results. To look at our previous example where half of the population was male and half female, we can split the whole sampling frame into male and female groups, and then select x number of random samples from each, and combine the results. This will give us a sample that is guaranteed to have the same 50/50 ratio of male to female members as the larger sampling frame. Or in the case of voters, we can select twice as many anarchists as other political groups (assuming we do know for certain that they register exactly half as often). Stratified sampling can be used with any type of groups that are important to your research. If you are conducting a survey on favorite flavors of ice cream, it might not matter whether your ratio of male to female members is accurate but it might well matter that your age grouping is correct. If you are surveying voter affiliation with Hillary Clinton, it probably is important to get the male/female ratio correct in your sample in order to get accurate results.

To be continued in part 2…

This entry was posted in Business, Data and Analytics, Marketing. Bookmark the permalink.