AI Assist

Artificial intelligence (AI) chatbots can be valuable tools for exploring research ideas and refining research questions. AI can assist in brainstorming sessions, providing relevant literature, and offering feedback on the clarity and feasibility of your research question. By interacting with an AI chatbot, you can quickly gather information and generate new ideas that might not have surfaced otherwise. 

If you are beginning your search, you could ask an AI chatbot, "What are some recent studies on active learning in higher education?" The chatbot can provide summaries and references to recent research, helping you to identify trends and gaps. Additionally, AI can help refine your research question by suggesting ways to narrow or broaden its scope based on existing literature. 

For example, one might prompt the chatbot with:

I am a faculty researcher at a large university examining the scholarship of teaching and learning. I am interested in examining if using asynchronous video feedback using tools such as Loom are more or less effective than traditional written feedback for engaging students and helping them to improve their retention of the learning material. Please develop for me 5 to 10 research-oriented questions that would use quantitative research to explore this problem.

Using the results, such as those presented in the figure below, you can begin to see different perspectives on exploring the research question. These ideas can help you see which questions are significant, clear, and researchable for your situation. As a reminder, chatbots make errors, so not everything they share is accurate, so critically judging the responses is important. For example, I asked ChatGPT to provide me with quantitative questions, but Question 6 is qualitative, so it is important to verify all the outputs you receive. Also, keep in mind the chatbot does not know your resources; however, by sharing insights, the AI tool can help you narrow your focus to a researchable project.

Incorporating AI tools into the research question development process can enhance your efficiency and effectiveness, allowing you to focus more on designing and conducting your study. Embracing technology in this way aligns with contemporary research practices and leverages the wealth of information available at our fingertips.

AI Assist

Artificial intelligence (AI) tools may be used throughout the research process to assist in procuring information, understanding concepts, and leveraging technology to parse through large volumes of written works.

Review of the Literature

AI tools like Elicit (elicit.com) can assist in searching for related articles in peer-reviewed literature. Elicit is designed to automate literature searches, helping researchers find relevant studies quickly and efficiently. Elicit allows one to identify a topic or research question, and it searches the literature for the most relevant articles, offering summaries beyond the abstract to provide greater insights. Using AI to scan vast databases and identify pertinent articles, researchers can ensure a comprehensive literature review and uncover gaps that need further investigation.

Theoretical Framework

AI tools can help identify suitable theoretical frameworks by analyzing the research problem and suggesting relevant theories. AI can also identify variables associated with specific theories and highlight discrepancies if the selected variables do not align with the theoretical framework. This ensures that the research is grounded in a solid theoretical foundation and that the variables and instruments used are appropriate.

Quantitative Methodology

AI tools can ensure all aspects of quantitative research are considered during the research design. For instance, AI can suggest appropriate statistical tests, check for common methodological pitfalls, and provide templates for data collection instruments. These tools can help researchers design robust studies, select the right analytical techniques, and interpret results accurately.

Artificial intelligence (AI) chatbots can be powerful tools for exploring research ideas and ensuring alignment between variables, theoretical frameworks, and measurement instruments. Here are some ways AI can assist:

1. Exploring Research Ideas: Use AI chatbots to brainstorm research questions and variables. Prompt example: "What are some potential variables to study a research question about the effect of online teaching methods on graduate student performance?"

2. Ensuring Theoretical Alignment: AI can help ensure that variables align with the theoretical framework. Prompt example: "How can I align my variables with the social constructivist theory in my study on collaborative learning?"

3. Checking Variable Consistency: AI can verify that all variables pertain to the same unit of analysis. Prompt example: "Are my variables student GPA, participation frequency, and self-reported motivation consistent in a study on undergraduate learning outcomes?"

4. Offering Insights on Descriptive Statistics: AI can suggest appropriate descriptive statistics based on the level of measurement. Prompt example: "What descriptive statistics should I use for ordinal data collected from a faculty member end-of-course survey?"

By incorporating AI tools into the research process, novice researchers can enhance their efficiency and ensure rigorous alignment of their study components. AI can provide instant feedback and access to a wealth of information, facilitating a more streamlined and informed research process.

Artificial intelligence (AI) can be a valuable resource for SoTL researchers when selecting appropriate instruments and data sources. AI chatbots, for instance, can help researchers brainstorm ideas, refine their research questions, and identify potential data collection methods. Researchers can receive tailored recommendations that align with their specific needs by describing their educational environment and research objectives to the chatbot.

For example, a researcher might use an AI chatbot to explore potential instruments for measuring student engagement in an online course. The chatbot could suggest various survey tools, digital analytics platforms, and observational protocols based on the description of the course and the research objectives. Sample prompt: "What are some effective instruments for measuring student engagement in a graduate-level online course?"

AI can also assist in aligning instruments and data sources with the study variables and theoretical framework. Researchers can input their research questions and theoretical foundations into the chatbot, which can provide suggestions on relevant instruments and data sources. Sample prompt: "How can I align my data sources with constructivist theory in a study on collaborative learning?"

Ensuring the feasibility of using instruments and accessing secondary data sources is another area where AI can be helpful. Researchers can describe their classroom setting, whether face-to-face or online, and receive advice on practical data collection methods. Sample prompt: "What are feasible data collection methods for assessing student participation in a face-to-face undergraduate course?"

Figure 1

Example ChatGPT Response to Prompt

AI tools can also offer insights into the ethical considerations and IRB requirements for using different instruments and data sources. Sample prompt: "What ethical considerations should I be aware of when using online discussion logs as a data source in my research?"

By leveraging AI, SoTL researchers can streamline the process of selecting and validating their instruments and data sources, ensuring that their research is both rigorous and contextually appropriate.

AI Assist

Artificial intelligence (AI) can greatly assist SoTL researchers in various aspects of data collection. For obtaining necessary permissions, AI can help draft research proposals by providing templates and suggesting best practices based on previous successful submissions. AI chatbots can guide researchers through the IRB application process, offering tips on addressing common ethical concerns and ensuring all required information is included.

When planning data collection, AI tools can analyze the research prospectus and provide feedback on its comprehensiveness and clarity. AI can also help identify the most appropriate data collection methods and instruments by comparing the research objectives with existing SoTL studies. For example, an AI tool could suggest using a specific student engagement survey validated in similar studies.

In recruiting participants, AI can aid in developing recruitment materials, such as emails or LMS announcements. AI can offer different written options based on the intended audience to ensure higher participation rates.

When retrieving data from secondary sources, AI can streamline the process by quickly extracting relevant data from large datasets, ensuring accuracy and completeness. This can be helpful with sentiment analysis or other text-based machine-learning tasks.

Publicly available AI tools can be valuable in assisting SoTL researchers with data analysis. AI can help streamline various aspects of the process, from data preparation to hypothesis testing and interpretation of results.

For data preparation, AI tools can assist in automating data cleaning by identifying and correcting errors, handling missing data, and detecting outliers. When requesting assistance, you must be clear about which software tool you use to receive the necessary guidance. For example, if you are working in Microsoft Excel, be specific about the tool you are using and the data available. With ChatGPT, you might use a prompt such as:

An example from part of the response this prompt generated is as follows:

You can see that the AI tool will guide you through each step of the process until you become more familiar with each step.

During exploratory data analysis, AI can generate descriptive statistics and visualizations quickly, providing researchers with an immediate understanding of their data. AI-powered platforms like Google Data Studio or Tableau can create interactive dashboards, allowing researchers to explore data relationships dynamically. If you are not comfortable with these tools, the AI tool may be used similarly as described in the data preparation step to guide you through the process using your selected tool.

When testing assumptions and conducting hypothesis testing, AI can guide researchers in selecting the appropriate statistical tests based on their data and research questions. AI chatbots like ChatGPT can provide step-by-step instructions on performing statistical analyses in software such as Microsoft Excel, SPSS, Python, or R.

For interpreting results, AI tools can help researchers understand the implications of their statistical findings. For instance, AI can explain the meaning of p-values, confidence intervals, and effect sizes, ensuring researchers accurately interpret their results. This is particularly helpful if there are multiple variables or the effect size is reported in an unfamiliar format, such as f² instead of R² in regression.

AI can assist in formatting tables and figures according to APA standards to facilitate the reporting of results. AI tools can also generate initial drafts of the results section, which researchers can refine and expand upon.