The purpose of this module is to examine the key issues related to quantitative research that must be addressed to ensure a quality research study that is valid, reliable, generalizable, and reproducible.

Learning Objectives

• Define validity, reliability, falsifiability, generalizability, and reproducibility as they relate to quantitative research
• Explain the importance of each in a quantitative study

If the results of quantitative research are to be considered useful and trustworthy, there are several key issues that must be considered and addressed as part of the experimental design and analysis. Following is a description of these issues:

Validity

The term validity refers to the strength of the conclusions that are drawn from the results. In other words, how accurate are the results? Do the results measure what was intended to be measured? There are several types of validity that are commonly examined, and they are as follows:

• Conclusion validity looks at whether there is a relationship between the variable and the observed outcome.
• Internal validity considers whether that relationship may be causal in nature.
• Construct validity refers to whether the operational definition of a variable reflects the meaning of the concept. In other words, it is an attempt to generalize the treatment and outcomes to a broader concept.
• External validity is the ability to generalize the results to another setting.

There are multiple factors that can threaten the validity in a study. They can be divided into single group threats, multiple group threats, and social interaction threats. For more information, on the threats to validity click here.

Reliability

Reliability is defined as the consistency of the measurements. To what level will the instrument produce the same results under the same conditions every time it is used? Reliability adds to the trustworthiness of the results because it is a testament to the methodology if the results are reproducible. The reliability is often examined by using a test and retest method where the measurement is taken twice at two different times. The reliability is critical for being able to reproduce the results, however, the validity must be confirmed first to ensure that the measurements are accurate. Consistent measurements will only be useful if they are accurate and valid.

Falsifiability

The term falsifiability means that for any hypothesis to have credence, it must be possible to test whether that hypothesis may be incorrect. A researcher should test his/her own hypothesis to prove or disprove it before releasing results to prevent another researcher from proving it wrong. If a theory or hypothesis cannot be tested in such a way that may disprove it, it will likely not be considered scientific or valuable to those in the field.

Generalizability

Generalizability refers to whether the research findings and conclusions that result from the study are generalizable to the larger population or other similar situations. The ability to generalize results allows researchers to interpret and apply findings in a broader context, making the finding relevant and meaningful.

Replication

Replication is the reproducibility of the study. Will the methodology produce the same results when used by different researchers studying similar subjects? Replication is important because it ensures the validity and reliability of the results and allows the results to be generalized.

Consideration of all of these issues is important to the results of a research study. For further details and specific examples, see the Resources Links on the right side of this page. Validity is seen by many as being the primary issue that should be examined. The following SlideShare presentation, General Issues in Research Design, discusses validity in further depth, along with other issues that should be addressed in research studies.

Ch04 General Issues in Research Design from yxl007

Suggested Readings

Adcock, R. (2001, September). Measurement validity: A shared standard for qualitative and quantitative research. In American Political Science Association (Vol. 95, No. 03, pp. 529-546). Cambridge University Press.

Becker, B. J. (1996). The generalizability of empirical research results.

Neuman, W. L., & Neuman, W. L. (2006). Social research methods: Qualitative and quantitative approaches.

Neuman, W. L., & Robson, K. (2004). Basics of social research. Pearson.

Onwuegbuzie, A. J. (2000). Expanding the Framework of Internal and External Validity in Quantitative Research.

Winter, G. (2000). A comparative discussion of the notion of validity in qualitative and quantitative research. The qualitative report, 4(3), 4.