Data analysis involves a series of steps to ensure that the data collected are properly prepared and analyzed to answer the research questions effectively. The key tasks associated with preparing data for analysis include data preparation (data cleaning, data transformation, and data integration), exploratory data analysis (descriptive statistics, data visualization, correlation analysis), tests of assumptions for the selected statistical analysis (ensuring each requirement for a statistical test is met), hypothesis testing (formulating the hypothesis, selecting the statistical test, conducting the test), interpreting the results using null hypothesis significance testing, and reporting the results in accordance with APA reporting and formatting standards. This section will guide SoTL researchers through these steps, providing practical advice and examples from the scholarship of teaching and learning (SoTL) research.

Background

Data preparation is the first critical step in the data analysis process. Data integration involves aggregating all the data into a single source, such as a spreadsheet, to ensure that each data point matches the corresponding study participant. Data cleaning addresses missing data, outliers, and errors to ensure the dataset's accuracy and reliability (Kuhn & Johnson, 2013). Data transformation converts data into suitable formats for analysis, such as normalizing values or creating categorical variables from continuous data (Osborne, 2010).

Exploratory data analysis (EDA) provides a preliminary understanding of the data. Descriptive statistics summarize the main features of the dataset, providing measures of central tendency and dispersion based on the level of measurement (nominal, ordinal, interval, ratio) (Tukey, 1977). Through plots and graphs, data visualization helps identify patterns, trends, and relationships between variables (Cleveland & McGill, 1984). Correlation analysis assesses the strength and direction of relationships between variables, using correlation coefficients to quantify these associations (Dancey & Reidy, 2007).

Testing assumptions is essential before conducting any statistical analysis. Each statistical analysis has certain data requirements. These tests determine whether the data meet the assumptions required for the chosen statistical procedure, such as normality, homogeneity of variances, and independence (Field, 2013).

Hypothesis testing involves several steps. Formulating the hypothesis starts with developing a research question and supporting hypotheses, as discussed in the "Developing the Research Questions" section. Selecting the appropriate statistical test depends on the research design and the nature of the data. For example, independent-samples t-tests and ANOVA are used to compare groups, while regression analysis is used to examine relationships within a single sample (Cohen, 1988).

            Conducting the statistical tests requires the use of appropriate software tools. Common tools include Excel for basic statistical analysis, SPSS for more advanced analysis, and programming languages like Python and R for comprehensive data analysis (Pallant, 2020). You should select the tool you are most comfortable with, and that will accomplish the analysis task. Once the tool is selected, you should identify the variables by their role (i.e., independent/dependent, predictor/criterion), and determine how to use the software to assist you in completing the statistical analysis.

Interpreting the results using null hypothesis significance testing involves determining whether the observed effects are statistically significant. In null hypothesis significance testing (NHST), p-values play a crucial role in determining the statistical significance of the results. The p-value represents the probability of obtaining an effect at least as extreme or more discrepant as the one observed in the data, assuming that the null hypothesis is true. A low p-value indicates that the observed data is unlikely under the null hypothesis, suggesting there is evidence to reject the null hypothesis in favor of the alternative hypothesis. A threshold value, known as the alpha level (commonly set at .05), is used to determine statistical significance. If the p-value is less than the alpha level, the result is considered statistically significant, implying that the effect observed is unlikely to be due to chance alone. For example, in an independent-samples t-test comparing student engagement levels between two teaching methods, a p-value less than .05 would indicate a significant difference between the methods. It is important to note that while p-values indicate the likelihood of the data given the null hypothesis, they do not measure the effect's size or practical significance nor provide a probability that the null hypothesis is true. Hence, p-values should be interpreted alongside other statistics, such as confidence intervals and effect sizes, to draw more comprehensive conclusions about the research findings (Nickerson, 2000).

            Reporting the results involves presenting the findings in a structured format according to APA standards. This includes detailing the statistical methods used, presenting the results in tables or figures, and providing a narrative interpretation of the findings (American Psychological Association, 2020).