Might their be validity issues with altering techniques CATs vs. no CATs in differing modules/Weeks in the same course.  Meaning is there the posibility that changes in average participation posts in a week might be influenced by how much the specific module topic appealing to the student rather than the use of CATs.

This is a great start quantifiying CATs and I appreciate you sharing.

Jason

Hi Jason,

Thank you for the feedback. The CATs were used the entire course for every module. There would be validitiy issues if we did what you suggested. We looked at two classes in which no CATs were used the entire course and two classes in which CATs were used the entire course in the same content. Does this help answer your question Jason?

Tom

Yes, I inferred the CAT was some modules and not others in the same course.  If you have time, I'd be interested in your experience and process to engage the IRB to use class info.

Thanks,

jason

Hi Thomas and John,  Thank you for your thorough efforts in this analysis of CAT's in the classroom.  I can tell that the KWL process that you have used has increased the productivity of the participation.  I would like to see with the CAT process how much of a difference the participation would be if the CAT's or the KWL were more aligned to specific objectives that the student needed to learn for that module or that week?  Did your team do any type of analysis that showed more or less participation when the CAT's asked the student's to explain or discuss a particular objective?  I would love to see any data on that process.  Thanks for your hard work and keep up the great efforts.  Matt.

Hi Matt,

 Thank you for your feedback. Every CAT/KWL is aligned with the course objectives for each Module. For example, in Module 2 students are asked to complete a KWL on the Senses which aligns with their Module assignment that week. We did not analyze specific responses in regards to CATs that ask students to explain or discuss, however within the KWL's students are asked what they know about the specific objective for that week which leads them to explain. We also did see that participation greatly increased, especially in the more difficult weeks which leads to a greater degree of discussion; however we did not investigate the quality of that discussion. This is something we could investigate in the future, however. Thank you for your thoughts. Please let us know if you have further questions/suggestions.

Tom

Hi guys,

There is a type on page 5 "leaned" instead of learned. I am using CATs but had not thought of using the KWL method. I used this much in my practice of the classroom for 18 years.  I am going to explore this more in my class and see what kind of results I get.

Gina

Thank you for the feedback Gina. We will make that change. I look forward to hearing about your experience with KWL's in the online classroom. We both know how effective they are in a ground/traditional course. Looking forward to your future success!

Tom

I am just curious, would answering the KWL be mandatory for students? Would you make it part of the DQ grade or part of participation?  I like the idea of evaluating the student's perception of what they learned that week and then sharing this with the class in a wrap up. The instructor could then use this data to determine if the week's objectives had bee met!

Hi John & Tom:

Very well written indeed!!! I loved your use of the diagrams to enhance the paper and to provided the reader with a visual cue to help them follow & even understand what your were/are presenting to your audience. So, with the information you have provided, how does Grand Canyon Universities online format work for the learner to understand  and even become involved in the weekly discussion when theere may only be 5 weeks in a course? Whay suggesitons do you have for the instructor or the student who does not want to become involved in the subject matter?

Thank you & very good work!

Mr Gregory E Ludvik, MS; RRT-NPS

Adjuct Online Faculty

College of Arts & Sciences/ College of Nursing & Health Sciences

Hello, Gentlemen! Excellent work on your essay! I think that if participating in Classroom Assessment Techniques earned a distinct grade, that students would be more likely to reply. I think students get confused that replying to the CATs can count toward participation credit, and some students will only post if they know they will get a grade for it. In my experiences, I find that some classes have several students who reply to the CAT posts, and other that do not. I think it depends on the class itself. You bring up some great ideas regarding the KWL strategy. I like how you provided data that showed class participation with CATs is higher than class participation with no CATs. I think that the more we post as instructors, the more engaged the students are. However, I do think that the same students repeatedly reply to the CATs each week and the same students do not reply at all. This is just from random observation of using my own CATs my classes. We always have the overachievers and underachievers in every course! Regardless, I think that your study would be very interesting using a larger sample size! I look forward to seeing more of your research on this topic because it directly applies to what I am doing in my classes. :-)

John & Tom, thanks for your submission. Your hypothesis that CATs lead to more participation which in turn leads to more student achievement is certainly worthy of study and debate.

I have a few suggestions/issues with the study that I would like to bring to your attention. I have numbered them below in what I feel is priority order.

1. Table #3 needs to be revised. The table is misleading because you forgot to account for class size. For example, the module three total numbers are 380 for "none CATs" courses and 675 "for CATs" courses which you claim leads to a 77% difference. However, when class size is taken into account the average student posts are 9.74 vs. 14.67 which is 66%.  I would recommend using avg. student posts here instead of the actual number of posts. 
2. Are instructor posts eliminated? If not, you may have a significant confounding variable here. If they were then you should mention this in your study. If they were not then I would recommend recalculating and eliminating instructor posts. 
3. Are the authors also the instructors? If yes then this should be mentioned in the limitations. Instructors seeking to "write up" CATs effectiveness would have a natural bias towards seeing the courses with CATs be more successful. Bias needs to be recognized in the paper. 
4. Is there any statistical signficance in your differences between your control group and your treatment group? If no statistical analysis was run then perhaps this should be mentioned in the limitations section.
5. Some parts of the Methods section and the Results section seem more suited for the Lit. Review. For example, the whole first paragraph of the Methods section reads this way. The whole idea of KWLs being used in an online classroom to assess students' prior learning and engagement with the content should appear somewhere in the Lit. review. 
6. Your methods section says that your CATs were "primarily" KWLs. The word primarily really seems to throw off the whole study almost. If they were not all KWLs then I would just stick with CATs in general or explain what the other CATs were. 
7. In the Methods you bring up a discussion of "church leaders and or pastors". This seems to come out of the blue. 
8. You have some GCU specific terms that are not defined. For example, you write "DQ1". Many of your readers would be uncertain about what you mean here. 
9. Was the time when the courses were taught a factor? Did an instructor teach two courses at the same time with one having CATs and one not? This would be the ideal. Otherwise the date when the courses were offered could have been a factor (ex. Courses over the holiday season tend to show fewer posts).
10. A couple words in your title should be capitalized.You also have a few other sentence structure errors throughout the paper. I recommend going over it a few more times to catch these.  

Hello Lori,

Thanks for reading and commenting on our paper. Responding to the KWLs is not mandatory but does count towards the student's participation grade. It can also be a great way to earn participation points while ensuring that you have understood the week's objective. Thanks for your feedback! Have a great day!

-John

Hello Gregory,

Thanks for kind words! Glad that you enjoyed the diagrams! Actually the courses that were discussed in this essay were seven weeks not five. Perhaps, I do not completely understand your question but I will answer it the best I can. As instructors it is our responsibility to do all we can to make the classroom as engaging as we can for our students being that we are subject matter experts. The more engaging and interesting the instructor can make things in the classroom the better the chance the students will be engaged as well. This may also spur the student interest in the subject matter. Thanks so much for your comments!

-John

Hello Kristen,

Thanks for comments and reviewing our essay! I agree that some students can get confused about whether or not the CATs count toward their participation posts. I also try to stress the KWLs the first week and make sure that students understand that it does count towards their participation posts.  I definitely do see how classes can often vary from one class to another. I would agree that it does seem that the more the instructor posts the more the students will usually post. It can only help to try to make the classroom more engaging for the students.  This could be an interesting follow up study if the sample size was increased and some of the limitations could be decreased or eliminated completely. Thanks for all your input Kristen! :)

-John

Hello Scott,

Thanks so much for your feedback as it is much appreciated. All suggestions and constructive criticism will be very helpful. I will address each of the issues/suggestions via number to correspond with your comments.

1. This is a great point Scott. We did have a breakdown that corresponded to the amount of posts in Table #2 that displayed the average posts per student making the class size relevant. However, the point of us including Table 3 was to show how participation increased even more during the three most crucial modules. Thanks for feedback as it makes a valid point that will be addressed as the table will be adjusted probably with the post per class ratio for the modules instead of the total posts.
2. While this could be considered a confounding variable it could also be argued that more instructor posts is increased participation as well. Hopefully with more instructor posts there is more instruction leading to more learning, student engagement, and student achievement. As we know, students typically take the instructor's lead in participation. Also, being that the instructors posts are counted in classes with no CATs and classes with CATs there should be no issue here.
3. This was a great point! This will be noted in the Limitations after the public review period. Would you suggest listing the authors being the instructors and also including the instructor posts with regards to limitations? Our hope was that the by having the same instructors with the same classes it would keep things consistent.
4. Should it just be stated that there was no statistical analysis was performed in the limitations? 
5. Would it make more sense for the Design/Method to be split up into separate sections? Then the part that was mentioned could be shifted into the design portion.
6. The CATS in that paragraph should be changed to KWLs to keep the verbiage consistent. Great point as it would be helpful to explain what other CATs are and keep consistent with our terms.
7. The "church leaders and or pastors" that are mentioned is actually in the Literature Review.
8. Thanks for this Scott as we did overlook this and need to give more explanation as to what DQs are.
9. Classes were not taught over the holidays but also did not coincide with each other. This could be added to the limitations as well.
10. There is still an abundance of work to be done on this so it will definitely get several read overs and adjustments!

Thanks for all your feedback as it will be very useful in the editing process for it! It was very much appreciated!  Thanks again Scott!

-John

John, I am excited about the final outcome. With some of these minor changes I feel you and Tom will have a quality paper.  You asked a few questions in your response so I have commented on them below. Each corresponds to the number above. 

#3 - Yes, I would suggest mentioning that the authors are also the instructors. I would also suggest including the number of instuctor posts somewhere in the paper as well. If there is no major difference in the number of instructor posts in CATs/No CATS courses then adding this will strengthen your paper. If there is a major difference then you should probably explain why this is the case. 

#4 If no statistical analysis was performed then I would leave it out. If you could add some statistical analysis to show statistical signifance then I think this would strengthen your paper greatly. 

#5 I think this is an excellent idea. 

#9 While classes were not taught over the holidays, Q4 data (Oct. to Dec.) has always been significantly lower in terms of GPA and SSR then the other three quarters. Well I do not know if this applies to participation as well, but I assume it would. Since there is seasonal variance, it is always better to have concurring treatment & control groups rather than sequential.

Blessings on your journey to see this research published!

Scott 

Hello Scott,

Thanks for your response and feedback on our changes! We will definitely incorporate these changes and really appreciate you taking time to help us with this! Thanks so much Scott!

Have a great weekend!

-John

One must be careful during the design and implementation of multimedia in the online classroom.  There are laws that prohibit video interaction with minors.  Some college level students fall into the category of minors.  This begs the question, how can one be sure that these laws are not broken?

Hi Karen,
Thank you for your feedback, however I am not sure what multimedia you are referring to. Would you mind elaborating on your question and how it fits the scope of our research?

Tom