This is a great topic.  Online classrooms provide many unique opportunities and challenges.  For example, both student and teacher can interact at convenient times and perhaps have more interaction.  However, it's difficult to know when a student will log on, and one can't simply call on a student to respond when asking a question.  Also one cannot see expressions nor non-verbal queues to gain additional insight into student progress. Consequently, CATs would seem a valueable tool in evaluating student progress.

Would you suggest that most CATs in an online environment would take the form of discussion questions?

You've done a great job of representing a conceptual model of CATs based on Angelo & Cross's initial work.  Are there some steps or tactics that are exclusive to the online environment?

Jason

I found this article quite interesting. I must say I do not have much experience with Classroom Assessment Techniques and found this study to be quite informative.

I would be interested in the results of this study using a masters or doctoral level course. I wonder if there would be differences between results from an introductory course for taking an online class and a doctoral level course.

I also agree there is significant importance in adding the limitations of this study in the discussion section and would be interested in reading that portion if it were available.

Hi Jason. Thank you for your message. We do use discussion questions almost exclusively for implementing CATs. We usually add an optional post within the forum that acts as a quick check for understanding. The main difference in an online course is the time table and the asynchronous nature. In most classes, the learning modules are one week in length. So, we do need to be strategic with what part of the week we implement the CAT. If it is implemented too late, the summative assessment may have already been addressed. Also, we must keep in mind that these courses are asynchronous. With this in mind, we must choose CATs that can be done individually. We have also found that if they are too complex, students do not take part in them. Thanks you for your questions and comments!

Rick

Hi Lindsay. Thank you for your comments. We have several different studeis in the works for undergraduate and Master's level courses. Hopefully we will have much more data in the near future.

I really enjoyed the paper and find it extremely relevant. As GCU tries to compete in the world of higher education, instructors need to make sure they are assessing student learning as it happens, rather than just after the fact. This method of assessment is extremely important in the online environment because oftentimes online learners are only assessed using summative measures. Using formative assessments in the online classroom is akin to classroom discussions in brick and mortar classrooms. Using these simple techniques and CATs helps everyone----from instructors to students. As an online instructor, it is important to me that I am meeting the students'' needs and guiding them throughout their learning. Without formative assessments, I am only measuring their learning, but not helping them with concepts and ideas that they might have missed. I truly believe formative assessments set GCU's online programs apart from other online programs. Full-time online instructors who are educated on the value of formative assessments make GCU a more inviting, accommodating, and educational environment. These Classroom Assessment Techniques are easy to implement and make learning more meaningful to students. Additionally, students who are teacher candidates are given the opportunity to see formative assessments in action; this is modeling at its finest!

Kimber Underdown

Thank you for your comments, Kimber. It is great to hear that you found relevance in this topic. CATs have been a great addition to our online learning by giving us the data we need to guide our instruction before it is too late.

Thank you, Emily and Rick for a fine explanation of CATs in the online classroom.  You are so right in that online learning is sometimes an isolated experience for students.  The need for CATs in online education seems even more necessary due to the lack of visual clues of comprehension.  

One area Angelo and Cross advocated was anonymity in some of their techniques.  Do you think some students would be hesitant to admit their muddiest points or lack of comprehension with their names linked to the postings?
 

Hi Cheryl,

Thank you for viewing our manuscript. You bring up a great point with anonymity. This can be achieved in CAT's that use an outside survey such as Survey Monkey or by adding Poll Anywhere. The CATs that we use most frequently do not have the anonymous element to it. At the beginning of the course, we mention to students that we will be adding these checks for understanding and try to create an atmosphere where they are comfortable participating. We also tie these to participation points for the week as an incentive. So far, we have found that most students are happy to participate in these. I hope this helps.

Rick

Hi Rick and Emily! Nice work regarding CATs! I think that formative assessment is crucial in both the traditional the onlline settings. In the traditional classroom, the instructor can issue pop quizzes to the students to assess their  knowledge about the objectives and see if the they completed the assigned readings. In the online classroom, the instructor can ask additional questions in the discussion forums; this may not have the same affect, but the basic idea is the same. Nice work!

Hello,

I am a huge fan of incorporating CAT's into the online classroom. The CAT that I most frequently use is the KWL. In the first DQ I will post what they know and what they want to know about the topic/objective for the week. Based on student responses I create further CAT's from this to check for understanding. I find that when I ask what students want to know; it would seem as if they did not do the reading as of yet because if they did, they would know the answer to their 'what they want to know' question. From here I recommend to the student to go back and do the reading and then respond with 3 to 5 things they have learned from the reading and then respond with any further questions they may have. The following week I will post a message in the DQ forum recapturing what we learned about last week's topic/objective. I find this formative assessment technique helpful to students and useful to myself for grading and providing feedback on the summative assessment. I find that most students who respond to these CAT's do better on their assignments than those that do not.

This is just one example of a CAT that I use within my online courses. An additional method that I use when responding and participating in the DQ forum is that every post I do, I somehow turn it into a CAT; most often by providing a resource and asking student to respond with three to five things they have learned from the resource and then ask them to post any further questions they have about the information.

As an instructor in the online environment I feel that it is imperative that we make each post we do count; meaning that the information we are providing will somehow be able to assess our students to identify the gaps so that we can create additional opportunities for the same day, tomorrow or throughout the following week, responses and questions that will help to close that particular gap; which can then be recognized by a summative assessment (their weekly individual assignment).

Moving forward I feel that it is important for all instructors teaching online to utilize CAT's within their classrooms for better student learning and understanding of course content and information.

Thank you for your comments, Kristen and Nicole. I agree that we can use every discussion post as a CAT if it is done properly. For instance, you can change the subject line in a response to address the entire class, or even a group of students within the class. I think that every post an instructor uses should be checking for student understanding in some way.

Thank you for all of the wonderful feedback everyone. It is great to hear that there are so many interested in this topic and implementation into the online modality. I will attempt to reply to each of you to the best of my abilities below:

Jason-I would certainly agree that the easiest way to incorporate CATs into the online modality is through the main question forums. This is a place where students already must go in order to respond to mandatory postings. However, the success of these CATs depends upon the explanation and delivery. At the end of the day, do students feel that responding or completing the CAT will be beneficial? This is an area that we are still working on and is in a state of infancy in our exploration.

Lindsay-You make a great suggestion in looking at a comparison between undergraduate, masters and doctoral levels. Each of these levels bring a very different type of student to the classroom. Would this improve interaction or be a cause for a decline? How might we approach each level if we are looking to formatively assess? These are great questions for Rick and I to reflect upon.

Kimber-Thank you for your insight here. I think that reminders about techniques like this would be fantastic for our future educators. There are times that a strategy is simple, but the most effective. In addition, despite our different modalities, some of these long standing, successful techniques are easily transferable. Great reflection and input. Thank you.

Cheryl- Great points here! I would say that there are certainly going to be the hesitant students. With online education, there is already a sense of anonymity, but I would agree that we will still have the student that is uncomfortable sharing a short coming. One way that we could possibly tackle this is use the individual forum for responses. This would allow students to personally communicate without sharing with the whole class. In addition, we could use tools like Poll Anywhere or Survey Monkey to implement certain CATs without knowing student identifications.

 Kristen and Nicole- I am so glad to hear we have some fellow advocates of using Classroom Assessment Techniques in the online classroom. Nicole, you make a great point about making every post count. Using CATs can certainly help us in achieving this. We do not want to post meaningless responses, but reply with informative comments that create cause for further exploration and thinking for our students. Thank you so much for your insight here.

-Emily

Hello Kristen,
Thank you for your comments in regards to CATS in the online classroom. Using CATS is very similar to a traditional classroom setting providing a pop quiz. Both scenarios provide an opportunity for students to prove their understanding of the weekly materials and objectives. Many times I utilize a CAT to check for understanding on a current assignment. This provides the students an opportunity to clarify the assignment instructions and/or ask any questions they may have. Thank you so much for your perception and communication here.

Excellent paper you have submitted for peer review.  I felt the literature was extremely informative, organized, and nicely referenced.  These actions you have conducted present a higher level of thinking in the educational field and CAT's.

The concept of assessment is tricky from student to student.  Since we are all aware that we all test different we find that there needs to be an assortment of options when it comes to seeing just where our students are.  With the on-line environment we find that the students seek to work at their own pace, mostly desiring to move quickly while the traditional classroom is a slower pace.  Both learning environments are wonderful places to learn and one can master their skills thoroughly when they are in the proper setting and complete assessments accordingly.

As an online professor and student I find that I prefer the online classroom.  However, I have completed more of y educaiton in the traditional setting and enjoyed this as well.  I would have to say that your information on assessments is quite useful to many of us, especially me :-)

Thanks for allowing us to read your work.  Nice job!

Thank you for the kind words, Michelle. It is our belief that online students deserve a high quality education, so the teaching and learning cycle must be met for all students. CATs provide us with a quick and easy informal assessment of student understanding. As this modality continues to grow in popularity, we must get better at giving our online students a high quality education while maintaining the fleixibility of an asynchronous platform. CATs are one way to provide our students with a meaningful learning experience.

I thoroughly enjoyed reading this paper and feel that the notion is very much alive and relevant for online instructors. I find it interesting to reflect on the different kinds of Classroom Assessment Techniques (CATS) that can be employed in the classrooms. Personally, I have experimented with CATS that involve personal reflection and real world application. In addition, the KWL approach evokes more engaging interaction with the students. Finally, I feel that a check for understanding is integral.

I have recently developed my own "pop-quizzes" for students as a formative assessment technique.  The quizzes focus on checking that the student understands the objectives of the week and how they are incorporated into the assignment. I have found this very helpful as online students sometimes struggle with comprehension of assignments without an extra "push." Overall, this paper allowed me to return my own thoughts back to how I can continue to implement and improve CATS in my own classes. Nice job Emily and Rick!

I agree that this article has merit through offering a model promoting a clear understand of the process of creating or developing and using Classroom Assessment Techniques within the online courseroom as compared to the face-to-face classroom.

Hi Emily and Rick!

Nice work!  Although intuitively formative assessments like CATs could be beneficial in any learning environment, your point about the accelerated nature of online courses explains the specific benefit of such assessments in online education.  This is further supported by your mentioning of both teaching presence and transactional distance.  There are many issues that your proposal touches upon within online education theory.  One area that you have not fully developed, which may be fruitful for future research, is the concept of social presence and CATs.  There is a tremendous dual opportunity when using CATs in online education, both as formative assessment and as a way to represent the instructor and student as real people online (Garrison, 2011).  Research has demonstrated that when students experience a higher sense of social presence online their satisfaction with the learning environment increases (Gunawardena & Zittle, 1997; Lowenthal, 2009).

As for suggestions to improve your paper, the sections that explain the conceptual model could benefit from having more CAT examples, especially steps two and four.   Providing CAT examples would not only help further define each step but also demonstrate your logic in constructing the conceptual model.  Good work Emily and Rick!  As an online instructor and online education researcher, I can say that your research has great potential to improve online education.

Dr. Greenberger

References
Garrison, D. R. (2011). E-Learning in the 21st century: A framework for research and practice (2nd ed.). New York: NY: Routledge.

Gunawardena, C., & Zittle, F. J. (1997). Social presence as a predictor of satisfaction within a computer-mediated conferencing environment. The American Journal of Distance Education, 11(3), 8-26.

Lowenthal, P. R. (2009). The evolution and influence of social presence theory on online learning. In T. T. Kidd (Ed.), Online education and adult learning: New frontiers for teaching practices (pp. 124-139). Hershey, PA: IGI Global.

Hi Beverly. thank you for your comments. We are all learning more about the use of CATs in the online classroom, but it appears that they have become quite useful for getting more student enagagement as well as giving instructors the opportunity to check for understanding.

Sherion, thank you for your thoughts as well. I think that our last step of retaeching/validation is much like your reframing step. Whenever possible, we try to direct students to a better understanding and mastery of the objectives. reframing would definitely be a part of this.

Thank you for your thoughts, Scott. We appreciate your thoughtful recommendations. I think we could loop in research about social presence and the use of CATs. That is an interesting thought for sure. I would agree that the CAts do ad a social element to the class and helps build presence for the instructor. Perhaps we have a study in the making? I also agree that we could give more detailed examples within the conceptual model descriptions.

Thank you all for your help with this!

Rick

Hi Emily and Rick,

You have developed a helpful and informative paper to assist instructors in the implementation of CATs in the online classroom. You concentrated on the use of CATs as formative assessments in gauging student comprehension prior to the weekly summative assignments. This, I feel, is a central issue and one of the reasons your article is vitally important for instructors to read and implement. Due to the pace and demands of the online classroom, instructors are often including substantive feedback a few days late - the student have moved on to the next topic mentally and the instructors feedback may receive only cursory consideration. The "Concept Model" you presented was helpful in assisting instructors in the steps to properly implement CATs. You put into words what we have learned but fail to always remember.

Thank you for your article.

Ron Steadman

Nice work, Emily and Rick. After I completed the CATS workshop a few months ago,  I tried a few different ways to implement CATS in several of my classrooms. I found that adding the CATS in the weekly discussion forum, and letting students know it would be counted as one of their responses, elicited more responses than when I posted a separate DQ. I appreciate the suggestions you included in your paper. Examples always help to understand the concepts.

Thank you for taking the time to present these techniques.

Kathleen

The KWLs seem like additional work for students, whereby increasing participation level, due to the three additional questions each week, may further inspire additional posts.  This study may pose an interesting question; does this technique support the addition of more discussion questions each week?  The normal numbers of discussion questions contain two for each module.  This is the case in most, if not all, online collegiate classrooms.  It seems as though the KWL is similar to weekly summary techniques used over past decades in the online classroom.  This technique has been eliminated as a requirement in most, if not all, online collegiate classrooms.  Based on my experience, the number of questions, in addition to the elimination of the summary, was to minimize extraneous material, which minimizes learner's working memory cognitive load. 

Could not locate these acronyms using http://www.acronymfinder.com/CAT.html

Could not locate these acronyms using http://www.acronymfinder.com/KWL.html

Hello to all!

   There is so much great insight here that is helping us dig deeper into this concept.

Michelle-I defenitely agree that assessment in general must be differentiated from student to student as they are all different. Another way that we want to look at our CATs is as an assessment for the class as a whole. For example, is the topic you are explaining in your lesson being received the way you had planned? Do the majority of the students "get it" or do you need to have a re-teach lesson?

Beverly-I love the idea of using a "pop-quiz". This is along the lines of using a quick survey to assess the class' knowledge on a particular topic. We will want to be sure that we keep these quizzes short and simple for both the student and teacher. In addition, are these quizzes anonymous? This can bring an added benefit at times.

Sherion- I am interested in hearing more about your reframing option. I was a bit unclear and would like to learn more. Can you provide a few examples?

Ron-Thank you for your feedback. I think there are often times that, as teachers, we are filled with so many techniques and strategies, that we sometimes forget some of the most basic and useful tools. I am in constant need for those little tips and reminders. I am glad we could do just that.

Karen-You bring up some interesting points. I do think that these techniques do open the door for additional quesitons and conversation. However, in some modalities, it might be posed as an optional DQ or question. The key is in explaining the process to students as well as the benefits. I would agree that KWL is similar to the summary piece; however, it does allow students and instructors to make a bit of a pre-assessment of their knowledge as it looks at what the class knows before the lesson is taught. Thank you for your feedback.

-Emily

Hello Rick and Emily,

I appreciate the efforts you both made to offer some valuable resources and tools in using CATs in the classroom. I appreciate that you discussed formative assessments as well. While many of us who have classroom experience realize that CATs are formative assessments others may see CATs as another "thing to do" in the classroom, specifically online when things move quite quickly. What I gained out of your writing is that formative assessments like CATs do what their intention is. They allow instructors to determine the student's prior knowledge regarding the objective and notate where they are throughout the week. In my opinion, CATs are especially important in the online classroom when we only have one week to "master" an objective. While agree with many of the concepts you discuss there are limitations to this type of assessment. Do you find that typically the students who participate in the CATs would do well regardless of this formative assessment? Will the CATs meet those learners who typically shy away from participation where they are and lead them to engage? I believe you attempt to discuss this in Step 2 when you talk about choosing the appropriate CAT. One thing I might add here for possible future research is the recognition of student learning styles. Although we are limited in some ways in the online classroom we can begin to understand how our students learn best possibly by offering a Learning Styles Inventory at the start of class. We then can create CATs that not only address the objective, but also address learning styles such as auditory and visual learners. We have an issue with Kinesthetic learners in the online classroom. Overall, I think you both did a fine job and I hope this has helped in some small way.

Tom