This module discusses research designs and methodologies commonly used for SoTL projects and provides insight into factors that should be considered when designing the project.

Learning Objectives

• Determine the type of data to be collected and use that to identify appropriate methodology and timeframe for the study
• Explain the basic types of research methodology: quantitative methods, qualitative methods, and mixed methods
• Describe commonly used methods for SoTL data collection

There are two main issues to consider when selecting the appropriate research design for a SoTL project. What type of data will be collected –numerical, descriptive, or a combination of the two? What is the timeframe for the project? The research question will ultimately guide the choice of design and methods of data collection based on the answers to these two questions.

Begin by examining the research question. What type of data will be needed to answer the question?   If the question requires collection and analysis of numerical data, the project will mostly likely require a quantitative design. SoTL questions of the “what works” nature often fall into this category. For example, a researcher may want to investigate the impact of discovery learning versus the traditional lecture format in a biology classroom. To do this, the researcher may want to conduct pre- and post-tests for the two different classrooms and compare the learning outcomes using statistical analyses. If the question is a “what is” type of SoTL question, the data may be more descriptive in nature and therefore, qualitative methods may be more appropriate. For example, a researcher wants to examine student perspectives on whether or not journaling has a positive impact on learning. Finally, the answer to question may require both types of data and therefore, a mixed methods approach would be applied. For example, perhaps a researcher wants to examine the percentage of students that participate in optional study groups and how that that participation impacts their perception of learning in the course.

The second consideration that will impact the design of the project is timeframe. Again, the research question will determine the timeframe for data collection. Some research questions may only require that data is collected as a snapshot during one moment in time. While this may sufficiently answer some questions, this type of data collection would not be useful for studying processes or changes over time. Therefore, when data needs to be collected at multiple points, the study will need a longitudinal design.

The resources on the page provide additional insight into basic research design considerations. Once an overall approach and design has been selected, the specific method of data collection will need to be determined. This can vary by discipline, but following is a brief list of commonly used data collection methods for SoTL projects as described by Kathleen McKinney in Scholarship of Teaching and Learning (2007):

• Portfolios and other reflective tools –These are self-reflective, qualitative data collection methods and commonly used in areas such as Fine Arts. These may include course portfolios, teaching portfolios, essays, and journals.
• Interviews and Focus Groups –These result in rich, qualitative data commonly obtained through the use of open-ended questions. Such methods allow for the participants to truly have voice, but anonymity is difficult.
• Observational Research – Includes classroom observations of a particular activity or of faculty and students’ behaviors. The data is commonly collected through the use of notetaking, audio recording or videotaping and the data may be quantitative or qualitative depending on the nature of what is being observed.
• Questionnaires – Self-reported data from a large number of respondents can be quickly and easily collected using online or paper-pencil questionnaires. The data collected is most often quantitative. Questionnaire data can be collected anonymously, which can make it more valid and reliable.
• Content Analysis –Involves the analysis or examination of written text to identify patterns/themes or to count categories. For example, student papers may be analyzed in such a way to determine learning outcomes.
• Secondary Analysis –Data that was gathered for another purpose can be used to answer a SoTL question if the researcher has access to the data. Institutional data and standardized test data are two examples.
• Experiments and Quasi-experiments –These methods create a more traditional research environment where participants are assigned to groups and one or more independent variable is manipulated. The data collected is usually quantitative. The advantage is that the ability to control variables allows the researcher to draw causal conclusions.
• Case Studies – Focuses on one class, course, or assignment and the data collection can come from analyzing records, making observations, and conducting interviews. The data can be rich and detailed and come from multiple sources.
• Multimethod Studies –Commonly used for broader studies and include a mix of methods for data collection. For example, focus groups and questionnaires may be both used to gather the data to answer a SoTL question and the results reflect an analysis of all the data combined.

The information provided above is an overview of basic research designs and common data collection methods for SoTL projects. Overall, the design will largely be determined by examining the question and approaches may vary by discipline. The resources on the right will provide additional information.

Suggested Readings

Atkinson, M. P. (2001). The scholarship of teaching and learning: Reconceptualizing scholarship and transforming the academy. Social Forces, 79(4), 1217-1229.
Creswell, J. W. (2012). Qualitative inquiry and research design: Choosing among five approaches. Sage.
Creswell, J. W. (2013). Research design: Qualitative, quantitative, and mixed methods approaches. Sage Publications, Incorporated.
Hatch, T. (2005). Into the Classroom: Developing the Scholarship of Teaching and Learning. Jossey-Bass, An Imprint of Wiley. 10475 Crosspoint Blvd, Indianapolis, IN 46256.
Hubball, H., & Clarke, A. (2010). Diverse methodological approaches and considerations for SoTL in higher education. The Canadian Journal for the Scholarship of Teaching and Learning, 1(1), 2.
Hubball, H., Clarke, A., & Poole, G. (2010). Ten‐year reflections on mentoring SoTL research in a research‐intensive university. International Journal for Academic Development, 15(2), 117-129.
Huber, M. T., & Morreale, S. P. (2002). Disciplinary Styles in the Scholarship of Teaching and Learning: Exploring Common Ground. AAHE Publications Orders, PO Box 1932, Merrifield, VA 22116-1932.
Hutchings, P. (2002). Ethics of Inquiry: Issues in the Scholarship of Teaching and Learning. Carnegie Publications, The Carnegie Foundation for the Advancement of Teaching, 555 Middlefield Road, Menlo Park, CA 94025.
McKinney, K. (2010). Enhancing learning through the scholarship of teaching and learning: The challenges and joys of juggling (Vol. 139). John Wiley & Sons.
O'Brien, M. (2008). Navigating the SoTL Landscape: A Compass, Map and Some Tools for Getting Starting. International Journal for the Scholarship of Teaching and Learning, 2(2), 1-20.