This module discusses how to sample the target population, types of sampling methods, causes of bias, survey sizes and other issues to consider.

Learning Objectives

• Define population and sample and describe why sampling the population is important
• List and explain types of sampling
• Discuss potential causes of sampling bias
• Describe how to determine the appropriate sample size to survey

The target population for a research study is the entire group of individuals or entities that a researcher is interested in examining. This may be all employees of a company, all the residents of city, fathers of teenage daughters, college seniors, individuals with a particular disease or condition, or any other defined group that shares at least one common characteristic. It is typically not possible to survey the entire population of interest. Therefore, surveys are administered to a sample, or subset, of the target population. The governing principle is that sample will provide a reliable representation of the target population so that the conclusions drawn from the results can generalized to the larger group. To ensure that this is the case, there are a number of factors to consider in the research design and those will be addressed in this module.

Types of Sampling

The two broad types of sampling techniques are non-probability sampling and probability sampling. Non-probability sampling includes sampling techniques where members of the sample are specifically selected and therefore, all members of the population do not have an equal chance of being selected for the survey. This type is commonly used for hypothesis development, case studies and pilot studies but it may not be safe to assume that members of this sample represent the entire population. Therefore, probability sampling methods are more commonly used in survey research because all members of the population have an equal opportunity to be selected for the survey sample. With probability methods, the selection process is randomized and the potential for bias in the responses is reduced. In probability sampling methods, researchers may randomly select names or phone numbers from a list, or they may take a systematic approach and select every Nth name from a list or some similar method. Regardless of the exact way in which the respondents are chosen, it is important for the researcher to keep in mind that the survey process must be systematic, repeatable, and nonbiased in order for the results to be valid and able to be generalized.

Potential Causes of Bias

There are many ways bias may be introduced into survey research. It may be the wording of the questions themselves, as discussed in the previous module, or it may be the result of the sampling process. Following is a list of potential ways sampling bias may occur and these should be considered when designing the survey process:

• Selection bias – Subgroups of the population may be over or underrepresented in the sample depending on the method of selection. For example, if you used landline telephones to do a telephone survey, would this leave out a large segment of perhaps the younger population that only have cell phones?
• Self-selection bias – Membership in the sample is determined by the respondents themselves. This may occur in cases where convenience sampling is used, and respondents are asked to call in if they would like to participate.
• Non-response bias – Occurs when the subjects that respond are perhaps different in some way from those that choose not to respond. In this case, it may be that the correct group is being sampled, but some are refusing to respond. For example, an email survey would be more likely to draw responses from those with desk jobs or those that spend more time at a computer than perhaps blue-collar workers or those who use computers much less.
• Response bias – It may be that the responses do not truly reflect the beliefs, opinions or attitudes of the respondents accurately because of outside pressure, difficult terminology, ambiguous questions, or questions that are poorly worded or are sensitive in nature.

 Survey Sample Size

Once the appropriate target population has been determined and a method of sampling that population has been selected, the researcher will have to determine the appropriate sample size to survey. There are really four issues to consider finding the appropriate sample size:

• Margin of Error (also called Confidence Interval): A 5% margin of error is most used in surveys. This means 5% is added to the results in both directions resulting in a range that should be reliable. For example, if 75% of respondents like a particular restaurant, then actually 70-80% of the target population likes the restaurant.
• Confidence Level: A confidence level is likelihood that your sample represents the population. Confidence levels are commonly set at 95%. This means that if the researcher replicated the study 25 more times with similar respondents from the target population, the results would be the same 95% of the time.
• These two things help determine how many respondents are needed by plugging those numbers into a Sample Size Calculator.
• The Sample Size Calculator provides the target number of respondents. Researchers must take into consideration that the response rate will not be 100%. Therefore, the researcher should try to estimate the response rate for the survey. A response rate of 10-15% is a common conservative figure used when doing survey research.
• To calculate the actual number of surveys that must be administered to reach the target number of respondents, divide the sample size number by the estimated response rate.

As a researcher creates the design for a survey research project and attempts to make the decisions described above, there are other factors should be considered that will impact those decisions. Following is a list of examples:

• Is the population fully literate? Are there language issues? These issues may create biases or effect response rates.
• Will the population cooperate? Some target populations may be more responsive than others and therefore, the estimated response rate may need to be adjusted.
• Will geographic restrictions limit the ability of the researcher to reach the desired number of respondents?
• Can the respondents be found and are they easily accessible? For example, do they work night shifts or travel? This may affect the mode of delivery of the survey and the estimated response rate?
• The sensitivity and complexity of the questions may impact the estimated response rate.
• Will respondents be knowledgeable about the issues/questions?
• Can false respondents be avoided?

Suggested Readings

Aday, L. A., & Cornelius, L. J. (2011). Designing and conducting health surveys: a comprehensive guide. John Wiley & Sons.
Blair, J., Czaja, R. F., & Blair, E. A. (2013). Designing surveys: A guide to decisions and procedures. Sage Publications.
Dillman, D. A. (2011). Mail and Internet surveys: The tailored design method--2007 Update with new Internet, visual, and mixed-mode guide. John Wiley & Sons.
Fowler Jr, F. J. (2013). Survey research methods. Sage publications.
Kalton, G. (1983). Introduction to survey sampling (Vol. 35). Sage.
Kish, L. (1965). Survey sampling.
Kotrlik, J. W. K. J. W., & Higgins, C. C. H. C. C. (2001). Organizational research: Determining appropriate sample size in survey research appropriate sample size in survey research. Information technology, learning, and performance journal, 19(1), 43.
Thomas, S. J. (1999). Designing Surveys That Work! A Step-by-Step Guide. Corwin Press, Inc.