Identifying appropriate instruments and data sources for SoTL research requires careful consideration of the research problem and the specific educational context. In a university setting, whether face-to-face or online, researchers should begin by clearly defining the variables they intend to measure and selecting instruments that are both reliable and valid for those measurements. 

For measuring student engagement, researchers might choose standardized instruments like the NSSE for broader assessments or develop custom surveys tailored to specific course contexts, recognizing that the validity and reliability of custom surveys are not demonstrated, affecting the study's internal validity. For example, in an online course, a researcher might use a survey tool to measure the frequency and quality of student interactions within discussion forums. This approach allows for collecting detailed data that reflect the unique dynamics of the online learning environment. 

When selecting data sources, researchers should consider the availability and accessibility of existing records. Grades on individual assignments, attendance records, and end-of-course evaluations are commonly used data sources in face-to-face classrooms. These records provide quantifiable measures of student performance and can be analyzed to identify trends and correlations with instructional practices. In online classrooms, data sources may include logs of student activity within the learning management system, digital submissions of assignments, and automated analytics that track student progress and engagement, keeping in mind that researchers will likely only have access to those data that are available within the instructor’s view and that does not require IT or a database manager’s assistance.

Researchers must ensure that their chosen instruments and data sources align with the theoretical framework guiding their study. For instance, if a study is grounded in constructivist theory, the instruments should capture data on active learning strategies and student interactions. This alignment ensures that the collected data will be relevant and supportive of the theoretical underpinnings of the research.

Obtaining IRB approval is a crucial step in the data collection process. Researchers must submit detailed proposals outlining their data collection methods, including descriptions of the instruments and data sources, procedures for obtaining informed consent, and plans for ensuring data confidentiality. The IRB review process helps identify and mitigate potential ethical issues, ensuring the research complies with all relevant regulations and standards. 

For example, a SoTL researcher conducting a study on the impact of formative assessment on student learning might collect data through periodic quizzes administered via an online platform. These quizzes would serve as both an instructional tool and a data source, providing continuous feedback to students while generating data on their learning progress. To complement this primary data, the researcher might analyze grades from the course's final exam, offering a summative measure of student achievement.