Using an Audience Response System (ARS) in a Face-to-Face and Distance Education CPT/HCPCS Coding Course

by Susie T. Harris, PhD, MBA, RHIA, CCS, and Xiaoming Zeng, MD, PhD

Abstract

We report the use of an audience response system (ARS) in an undergraduate health information management course. The ARS converts a standard PowerPoint presentation into an interactive learning system that engages students in active participation, and it allows instructors to display questions, surveys, opinion polls, and games. We used the ARS in a 2008 course, Health Services Coding, for lecture and test reviews.

The class consisted of 15 students; nine were on-campus students and six were distance education students. All of the responding students agreed that the ARS software facilitated learning and made lectures and reviews more interesting and interactive, and they preferred taking a class using an ARS.

Key words: audience response systems, classroom pedagogy, education technology, interactive learning

Introduction

Traditional lectures in a classroom are appropriate for disseminating information to a large number of students, but several issues diminish the efficiency and effectiveness of this teaching technique. For example, attention spans for adults begin to decline after sitting in a lecture for 15 to 20 minutes, and students generally recollect a mere 25 percent of the information presented.¹ Further, when students sit in a classroom and the instructor lectures to them, they do not have a chance to participate actively, and the instructor cannot get real-time feedback from the class. 

New and emerging information technologies offer many opportunities for changing how teaching is done in college and university classrooms, resulting in more efficiency and effectiveness in higher education.² For example, technology makes it possible to get immediate responses from students. Traditionally, instructors who ask questions in the classroom receive answers from a few students, while the other students do not respond. Students are often slow to respond and tend to shy away from answering questions out of fear of giving the incorrect answer. Many students depend on the quick-to-answer students to speak out. At times, instructors try to collect responses to questions by asking for a show of hands from students. However, students often change their answer choices in front of their classmates.³ Now, with an audience response system (ARS), instructors have the chance to ask a question, receive answers or responses from all the students, and review the answers instantaneously.

Background

In the mid- to late 1960s, ARSs were mainly used by businesses (for focus groups, employee training, meetings, etc.).⁴ Classtalk was the first widely used ARS in education, but this system was costly and not user friendly. Companies have now created a new generation of ARSs that use “clickers” (handheld response devices), and in recent years ARSs have been widely used in universities, particularly in large lecture classes.^5,6 Overall, undergraduate college students feel that ARSs are helpful in their learning.⁷ Students rate clickers as enjoyable and helpful and say that they should be used in classrooms.^8-15 Additionally, with ARSs, students perceive the instructor as more aware of students’ needs and the teaching style as more “immediate (warm, friendly, close)” and caring.^16-18 Students also like the anonymity of an audience response system along with the ability to strengthen their learning and compare answers with those of the rest of the class.^19,20 Instructors’ opinions of ARSs in the classroom are also favorable. ARSs are viewed by some as a convenient and expedient means to test students’ understanding.²¹

HIM educators strive to develop critical thinking skills in their students. With an ARS, students may be more likely to respond using these skills. Also, the educator receives concurrent, real-time answers from the students. Students also may acquire communication and teamwork skills from using the software. 

ARSs include both hardware and software that allow an instructor to give real-time questions to students. Students use clickers or similar devices to key in their answers without disclosing their identity, which encourages students to answer without fear of being embarrassed at giving an incorrect answer. Thus, with ARSs, instructors receive immediate responses from the entire class. This article reports the use of an ARS in a health information management (HIM) undergraduate coding course and gives the results of a survey of students on its usefulness.

The Course

In HIMA 3148, Health Services Coding, students learn how to code services and procedures according to Current Procedural Terminology (CPT) and the Healthcare Common Procedure Coding System (HCPCS) for documentation and for reimbursement. 

The online course content management system used is Blackboard (version 8.0; Blackboard, Inc., Washington, DC). We also use classroom-capture software that allows distance education students to view classroom lectures asynchronously and allows on-campus students to view missed classes. A recording may include a full-motion video stream, an audio stream, and a data video stream that contains a series of “slides,” such as PowerPoint or the output of the document camera. The full-motion video is compressed into Windows Media format for fast transmission and economical data storage. The recordings must be viewed using software that presents both video streams and the audio stream concurrently. 

In the course, there are four tests covering one to four chapters each and a final comprehensive examination. The ARS is used for test and final exam reviews and for lecture reviews on complicated chapters such as surgery. All on-campus students respond to each question, and discussion typically follows the result graph. Distance education students view the classroom presentation, and the review slides are e-mailed to all students. 

According to Brockenbrough (2008), “Questions used with ARS usually fall into three general categories: factual recall, conceptual understanding, and knowledge application.”²² Examples of factual questions used in the coding course are as follows:

1. Per CPT guidelines, anesthesia time begins when the anesthesiologist begins to prepare the patient for induction, and ends at what point
2. Should code 28445 be appended with a LT or RT HCPCS Level II modifier?

Examples of conceptual questions used in the coding course are as follows:

1. Correctly apply the anesthesia code for 19367, a breast reconstruction with TRAM flap.
2. A physician orders part of a Hepatic Function Panel: Serum Albumin, Total Bilirubin, Direct Bilirubin and SGPT, SGOT. Can these lab services be covered with one code assignment?

Examples of application questions used in the coding course are as follows:

1. A physician draws blood to test for levels of T3 on a non-Medicare patient. The blood is sent to an outside lab for analysis. When billing for the physician’s services, which of the following modifiers should be appended to CPT code 84480?
2. What is the correct code assignment for bilateral EMG of the cranial nerves?

Slides containing questions are created and formatted using the ARS software to make them into interactive media for the classroom. On-campus students answer the questions with the ARS, and distance education students can view the sessions via the class recording.

After-Class Survey

To assess the efficacy of ARS in the classroom, a 10-question survey was distributed at the end of the course in December 2008. The questions asked students

1. how useful the ARS was for learning;
2. the extent to which using the ARS made the class lectures and reviews more interesting and interactive;
3. the extent to which they would prefer to take a class that uses an ARS over one that does not use an ARS;
4. the extent to which the ARS assisted in their learning;
5. the extent to which using the ARS increased meaningful intellectual knowledge and exchange between students;
6. the extent to which they would want to participate in a similar future activity;
7. the extent to which they would want to see ARSs more widely used as a learning tool in future courses;
8. the extent to which they would want to purchase a clicker ($10–$20) for a course;
9. their reasons for liking or disliking the ARS; and
10. any additional comments or thoughts regarding the ARS.

A five-point Likert scale ranging from “strongly agree” to “strongly disagree” was used for the first eight questions except for the first question which used “extremely useful” to “not useful at all.” The last two questions were open ended.

Results

Data were collected from all the class members who were senior HIM majors. Seven of nine on-campus students responded to the questionnaire for a response rate of 77.7 percent; however, four students did not answer the last, open-ended question. All six distance education students responded to the questionnaire; however, three students did not answer the last, open-ended question. We report the survey results below.

1. ARS was useful in my learning.

When students were asked their opinions of the usefulness of the ARS for learning in the course, the majority said it was “extremely useful” or “very useful.” None of the students answered “not very useful” or “not useful at all.” Overall, more on-campus students than distance education students agreed that the ARS helped their learning in the class.

2. ARS made the lectures/reviews more interesting and interactive.

All the on-campus and distance education students felt that the ARS made class lectures and reviews more interesting and interactive, answering “strongly agree” or “agree” to this question. Notably, the on-campus students were more positive about the interactive aspect than the distance education students.

3. I would prefer taking a class that uses ARS over one that does not use it.

In response to the question about preference for taking a class that uses an ARS, three of the on-campus students and two of the distance education students answered “strongly agree,” and four of the on-campus students and four of the distance education students answered “agree.”

4. ARS assisted my learning in HIMA 3148.

Three of the on-campus students and one of the distance education students strongly agreed that the ARS assisted with their learning in the course. Also, four of the on-campus students and four of the distance education students agreed with this statement, while one of the distance education students (but none of the on-campus students) answered “neutral.” Clearly, on-campus students felt that the ARS assisted with their learning more than did distance education students.

5. ARS increased the level of meaningful intellectual knowledge and exchange between students.

When asked whether the ARS increased meaningful intellectual exchange, three of the on-campus students and one of the distance education students responded “strongly agree.” Also, four of the on-campus students and four of the distance education students responded “agree,” while one distance education student (and none of the on-campus students) answered “neutral.” Overall, on-campus students felt that the ARS increased the level of meaningful intellectual knowledge and exchange between students more than did distance education students.

6. Participation in a course using ARS or something similar to ARS in the future.

When asked their opinions of participating in a course using ARS or something similar in the future, three on-campus students and two distance education students answered “strongly agree.” Also, three on-campus students and two distance education students answered “agree.” However, two of the distance education students (but none of the on-campus students) answered “neutral” to this question, and one of the on-campus students (but none of the distance education students) answered “disagree.”

7. More use of ARS (or a similar means) as a learning/assessment tool.

This question asked students about their interest in more use of the ARS (or a similar means) as a learning/assessment tool;  four on-campus students and two distance education students responded “strongly agree,” while three on-campus students and three distance education students responded “agree.” However, one distance education student (but none of the on-campus students) answered “neutral” to this question. Overall, the on-campus students were more interested than distance education students in using the ARS (or a similar means) as a learning/assessment tool.

8. Requirement to purchase ARS clicker for $10–$20 for a class.

This question asked students’ opinions about an instructor’s requiring students to pay $10 to $20 for an ARS clicker for a class. Half of the distance education students and slightly more than half of the on-campus students responded “strongly disagree,” “disagree,” or “neutral” to this question. Overall, students were not in favor of having to pay $10 to $20 for an ARS clicker as a course requirement.

9. Reasons students liked or disliked using ARS.

Question 9, an open-ended question, asked students why they liked or disliked using the ARS. Table 1 gives some of the positive comments from the students. Students liked the positive interactions and immediate feedback and said that the ARS reinforced learning, kept students focused and interested, and was excellent for lecture and assignment reviews. The only negative comment, which was from a distance education student, was “it is harder for [distance education] students to hear and understand it as well.”

Table 1: Comments from Students

On-campus students:

Distance education students:

10. Additional thoughts or comments regarding ARS.

Question 10, the last, open-ended question, asked students for additional thoughts or comments regarding ARS. Three on-campus students stated, “I like ARS a lot and wish it were used in more classes. It is a great review resource and helpful for preparing for exams;” “It would be great if you could enter numbers on the remotes and not just letters. This would help in the CPT coding class. The only drawback is we can only choose the letters for responses;” and, “I have really enjoyed using this system.” Three distance education students stated, “ARS is good, but as a [distance education] student the recording is delayed somewhat, and sometimes we didn’t get to see the answer because the recording didn’t catch it. It was good that you had sent us the ARS presentations to ensure that we received the material;” “Every little bit helps when it comes to learning;” and, “I think the ARS would be extremely helpful in most of the classes I am taking. When you get an answer wrong, I think you are more likely to remember how to get the correct answer when you see it again (such as on a test.) It is a great study tool.”

Discussion

This article reports the use of an ARS as a learning tool for face-to-face students and distance education students in an HIM coding course. Like other authors, we found that the majority of the students agreed that the ARS could help them to learn and interact in the class.^23-25 All the students would like to use it in future classes. The most frequently cited reasons for liking the ARS were its ability to increase students’ interactions and the fact that it immediately notified them of the correct answers to questions. Many students liked the ARS because it facilitated interaction without embarrassment at giving incorrect answers. However, two of the six distance education students disliked the ARS because they were not able to interact and could only view the class discussion and not take part in it. This may indicate that although the distance education students received the same information-PowerPoint slides, classroom recordings, and e-mail communication with the instructor for questions, they still wish for spontaneous interaction with the instructor. Solutions to this need could include having students watch the classroom recording in real time and interact with the instructor through text messages or online chat, or preemptively providing distance education students a summary of questions and answers collected in class to give them more information before or during the time when they view the recording. 

These findings suggest that it is beneficial to integrate ARSs into classroom lectures for their enhancement of classroom interaction and participation. Instructors may also want to emphasize that the ARS is an interactive reviewing tool.

Limitations and Areas for Future Research

There were several limitations to this study. First, all participants were from the same university. Including students from other colleges or universities might reveal likes and dislikes not found here. A second limitation is that the sample predominantly consisted of white females. Finally, the sample was a convenience sample, and it was small. Additional research is needed to compare college students at various universities, using a large, diverse sample.

Although the results of the study may not be generalized due to these limitations, the findings suggest that ARSs can be considered an effective tool for supporting interactive learning. Thus, ARSs would be useful for HIM educators to include in their teaching.

Susie T. Harris, PhD, MBA, RHIA, CCS, is an assistant professor in the Department of Health Services and Information Management at East Carolina University in Greenville, NC.

Xiaoming Zeng, MD, PhD, is an associate professor in the Department of Health Services and Information Management at East Carolina University in Greenville, NC.

Notes

1. Brockenbrough, G. “New Methods, Lectures Help Attendees Get the Most out of the Latest AANA Meeting.” Orthopedics Today 28, no. 4 (2008): 74–75.
2. Mayer, Richard E., Andrew Stull, Krista DeLeeuw, Kevin Almeroth, Bruce Bimber, Dorothy Chun, Monica Bulger, Julie Campbell, Allan Knight, and Hangjin Zhang. “Clickers in College Classrooms: Fostering Learning with Questioning Methods in Large Lecture Classes.” Contemporary Educational Psychology 34, no. 1 (2009): 51–57.
3. Herreid, Clyde Freeman. “‘Clicker’ Cases: Introducing Case Study Teaching into Large Classrooms.” Journal of College Science Teaching 36, no. 2 (2006): 43–47.
4. Lowery, R. C. “Teaching and Learning with Interactive Student Response Systems: A Comparison of Commercial Products in the Higher-Education Market.” Paper presented at the annual meeting of the Southwestern Social Science Association, New Orleans, LA, March 23–26, 2005.
5. Beatty, I. D. “Transforming Student Learning with Classroom Communication Systems.” Educause Center for Applied Research, Research Bulletin 3 (2004): 1–13. Available at http://net.educause.edu/ir/library/pdf/ERB0403.pdf (accessed May 27 2009).
6. MacGeorge, Erina, Scott Homan, John Dunning, David Elmore, Graham Bodie, Ed Evans, Sangeetha Khichadia, Steven Lichti, Bo Feng, and Brian Geddes. “Student Evaluation of Audience Response Technology in Large Lecture Classes.” Educational Technology Research and Development 56, no. 2 (2008): 125–45.
7. Graham, Charles R., Tonya R. Tripp, Larry Seawright, and George L. Joeckel III. “Empowering or Compelling Reluctant Participators Using Audience Response Systems.” Active Learning in Higher Education 8, no. 3 (2007): 233–58.
8. Caldwell, Jane E. “Clickers in the Large Classroom: Current Research and Best-Practice Tips.” CBE-Life Sciences Education 6, no. 1 (2007): 9–20.
9. McDermott, Lillian C., and Edward F. Redish. “Resource Letter: PER-1: Physics Education Research.” American Journal of Physics 67, no. 9 (1999): 755.
10. Draper, S., J. Cargill, and Q. Cutts. “Electronically Enhanced Classroom Interaction.” Australasian Journal of Educational Technology 18, no. 1 (2002): 13–23.
11. D’Inverno, R., H. Davis, and S. White. “Using a Personal Response System for Promoting Student Interaction.” Teaching Mathematics and Its Applications 22, no. 4 (2003): 163.
12. Elliott, C. “Using a Personal Response System in Economics Teaching.” International Review of Economics Education 1, no. 1 (2003): 80–86.
13. Beekes, W. “The ‘Millionaire’ Method for Encouraging Participation.” Active Learning in Higher Education 7, no. 1 (2006): 25–36.
14. Bunce, D., J. R. Van den Plas, and K. L. Havanki. “Comparing the Effectiveness on Student Achievement of a Student Response System Versus Online WebCT Quizzes.” Journal of Chemical Education 83, no. 3 (2006): 488–93.
15. Simpson, V., and M. Oliver. “Using Electronic Voting Systems in Lectures.” Available at http://ucl.ac.uk/learningtechnology/assessment/ElectronicVotingSystems.pdf (accessed May 27, 2009).
16. Jackson, M., and A. Trees. “Clicker Implementation and Assessment.” Available at http://comm.colorado.edu/mjackson/clickerreport.htm (accessed May 27, 2009).
17. Nicol, David J., and James T. Boyle. “Peer Instruction Versus Class-Wide Discussion in Large Classes: A Comparison of Two Interaction Methods in the Wired Classroom.” Studies in Higher Education 28, no. 4 (2003): 457.
18. Knight, J. K., and W. B. Wood. “Teaching More by Lecturing Less.” Cell Biology Education 4, no. 4 (2005): 298–310.
19. Jackson, M., and A. Trees. “Clicker Implementation and Assessment.”
20. Bunce, D., J. R. Van den Plas, and K. L. Havanki. “Comparing the Effectiveness on Student Achievement of a Student Response System Versus Online WebCT Quizzes.”
21. Caldwell, Jane E. “Clickers in the Large Classroom: Current Research and Best-Practice Tips.”
22. Brockenbrough, G. “New Methods, Lectures Help Attendees Get the Most out of the Latest AANA Meeting.”
23. Beatty, I. D. “Transforming Student Learning with Classroom Communication Systems.”
24. Morling, Beth, Meghan McAuliffe, Lawrence Cohen, and Thomas M. DiLorenzo. “Efficacy of Personal Response Systems (‘Clickers’) in Large, Introductory Psychology Classes.” Teaching of Psychology 35, no. 1 (2008): 45–50.
25. Duncan, Douglas J. M., and Eric Michael Mazur. Clickers in the Classroom: How to Enhance Science Teaching Using Classroom Response Systems. San Francisco: Pearson Education, 2005.