The following module provides tips and best practices to consider when conducting comparative research.

Learning Objectives

• Describe the basic steps involved in any comparative research study
• List tips and best practices for conducting comparative research, including those related to case selection, data analysis and overall research design

The information below includes numerous tips and best practices commonly used to effectively conduct comparative research. The tips provide general guidelines that can be used to increase the quality of the project, as well as the reliability and validity of the results.

Guidelines for the Process of Conducting Causal Comparative Research

• Select a topic – Researchers commonly look for experiences or situations that have occurred in the real world.
• Review the literature – Reviewing the literature regarding the topic may help researcher identify the independent and dependent variables for the study. It may also help identify extraneous variables that may contribute to a cause-effect relationship.
• Develop a hypothesis – It should describe the impact of the independent variable on the dependent variable.
• Select the comparison groups – Researchers should use caution when selecting groups and attempt to choose groups that only differ in regard to the independent variable if possible. This will help control for extraneous variables and reduce their effect. Some researchers will use techniques, such as matching which is commonly used in experimental research, in an attempt to find corresponding groups that differ primarily by the presence of the independent variable.
• Select tool for measuring variables and collecting data –In comparative research, the researcher does not have to implement a treatment protocol, therefore, it is a matter of collecting data that allows for comparisons to be made between the groups. This could come from existing data sources, surveys, interviews, etc...
  • 3 Main Types of Data:
    • Nominal – Indicates the presence or absence of a particular attribute or characteristic
    • Ordinal – Makes comparisons between two attributes, characteristics, or groups in general terms such as more/less, faster/slower, earlier/later
    • Quantitative – numerical values that are continues and quantifiable such as the number of individuals, number of events, frequency of events, etc.
  • Analyze and Interpret the Results – Both descriptive and inferential statistics are commonly used. For descriptive statistical analysis, researchers typically use measures such the mean, frequency, and the standard deviation. Commonly used inferential statistics include the t-test, the analysis of variance, and the chi square. These tests help the researcher determine if there is a statistically significant difference between the groups. 

General Tips and Best Practices

• Limit the number of cases, groups, or countries – Comparative studies are best when cases are carefully selected and limited in number. The study usually involves a significant level of knowledge about each one which is difficult if there are too many comparisons to make. If there are too many cases, groups, or countries to compare it increases the chances of error and increases the likelihood of an irrelevant case impacting results. The cases, groups, or countries should be similar enough to fit into some type of category or classification that defines them and sets up the comparative study.
• Case Selection – Comparative studies do not use random selection to place cases, countries, or individuals into groups. In fact, the cases or groups are usually pre-determined or existing. Therefore, the selection of cases, groups, or countries that are similar enough to allow for comparison is key to the overall success of the study. Some researchers will use matching procedures to select the cases. One way to select cases is to identify extraneous variables that could play a role in the differences found among the groups and control for those variables by choosing cases that are similar with respect those extraneous variables. This will help to reduce the potential impact of extraneous variables and increase the validity of the results.
• Consider accessibility of similar data sets when selecting topics – Especially in cross-national comparative studies, it is important to consider in advance whether obtaining comparable data from the groups or countries is going to be possible. For example, if comparing the economies of two countries, the researcher should investigate what economic indicators are going to be available and whether they will allow for comparisons to be made.
• Develop an analytic frame once the cases or countries have been selected – The analytic frame is the aspect or characteristic of the comparison cases that the researcher decides to study. For example, if the researcher chooses to compare countries that have had governments overthrown in the last 50 years, he or she would need to select an analytic frame to narrow down the data to a particular aspect of interest. The researcher may, for example, decide to focus on the style of government that is put in place. Identifying the analytic frame keeps the researcher focused so there is no distraction from other data that may be “interesting” but not relevant, such as the nature of the battles that led to overthrowing of the original government.
• Interpreting Results - When interpreting the results, researchers should be cautious about stating that the independent variable caused the dependent variable. Due to the lack of randomization in participant selection and the presences of extraneous variables, it is probably best to state that the results show a possible effect or possible cause. Experimental research designs are really the only methods that can truly establish definitive cause-effect relationships.

Suggested Readings

Collier, D. (1993). The comparative method. Political Science: The State of Discipline II, Ada W. Finifter, ed., American Political Science Association.
Lijphart, A. (1971). Comparative politics and the comparative method. American political science review, 65(03), 682-693.
Harvey, P. H., & Pagel, M. D. (1991). The comparative method in evolutionary biology (Vol. 239). Oxford: Oxford university press.
Pagel, M. D. (1992). A method for the analysis of comparative data. Journal of theoretical Biology, 156(4), 431-442.
Ragin, C., & Zaret, D. (1983). Theory and method in comparative research: Two strategies. Social forces, 61(3), 731-754.
Ragin, C. C. (2014). The comparative method: Moving beyond qualitative and quantitative strategies. Univ of California Press.
Thomas, R. M. (1998). Conducting educational research: A comparative view. Greenwood Publishing Group.
Tuckman, B. W., & Harper, B. E. (2012). Conducting educational research. Rowman & Littlefield Publishers.
Van de Vijver, F., & Leung, K. (1997). Methods and data analysis of comparative research. Allyn & Bacon.