Making the Switch to Online Evaluations

Recently the Provost announced that William & Mary had purchased an online evaluation application. As Provost announcements go, this one was particularly gratifying to me since I’ve been working on various pilots, experiments, proposals, and faculty meetings on this topic for more than 12 years. My first foray into this world came from working with Terry Meyers, who was then chair of the English Department, to launch the first pilot to evaluate changes to the evaluation process. We delivered our primitively coded survey to a group of courses and did paper evaluations for a matched sample, and the results from my perspective as a social scientist were encouraging. There was no significant difference in the scores that could be attributed to the mode of administration.

Paper evaluations may soon be a thing of the past at William & Mary.

Faculty in the Department were less impressed. (For them “statistical significance” was one of those weasel phrases that statisticians invent to elevate their status in the academy.) The English Department has continued to experiment over the years with different models of collecting quantitative and narrative data from their students, but still hadn’t found the right combination of tools to allow faculty to pull the trigger.

Why Online Evaluations Now?

My own sense is that the new system will address most of the concerns that departments might have about collecting good data about teaching in a much more humane and environmentally responsible way.

One of the major drivers for the initial interest in the Department of English in online evaluations was the incredible amount of manual work required by departmental administrators in supporting the process. The Department has lots of courses and high student enrollment, as well as used two forms to evaluate each class. Administering the evaluation, keeping the forms secure, preparing materials for scanning and then distributing results required hundreds of person-hours, cubic yards of file cabinets and boxes, and a custom-built lockbox.

As you can imagine, support for an integrated online evaluation system was very high among departmental administrators who believe they can support their departments more effectively through activities other than schlepping boxes of evaluation forms around. Another source of support has come from sustainability advocates who question the use of printing 50,000 forms every semester to collect minuscule amounts of data. No one that I know of has ever calculated the indirect costs of copying, storing, assembling tenure and promotion portfolios, but it’s safe to guess that the costs would be mind boggling.

Addressing Concerns about Online Evaluations

Much, if not all of that manual work goes away with online administration. However, the concerns about the system haven’t changed much in the twelve years since the English pilot study.

  • Low response rates — This is probably instructors’ most common concern about using online evaluations. If students aren’t given time in class to fill out evaluations, will they? We’ve come a long way since 1999, and students now are much more comfortable using their laptops (or even their phones). There are lots of strategies that faculty members can use to encourage higher responses rates. As other universities have moved from paper to online evaluations, they have found that the most important factor in increasing response rates is to communicate with students about the importance of the process and the impact that their evaluations have.
  • Quality of student responses — Many students actually spend more time filling out online evaluations than paper ones, providing more detailed feedback when they are able to use time outside of class. Instead of quickly filling out evaluations so that they can leave, letting students fill out evaluations when it’s convenient for them lets them take the time that they need to provide more detailed, and probably more helpful feedback.
  • Wireless and bandwidth technical difficulties — The wireless infrastructure can support a whole class of students logging on and filling out evaluations, should the instructor choose to have students fill them out in class.  It’s a low-stress application with regards to bandwidth, and the wireless can handle it.

All of that said, the system also offers flexibility in administering evaluations (although the extent of that flexibility is ultimately up to individual departments).  Instructors can have students fill out the online evaluations during class time set aside for it, or have students fill out their evaluations at home within a specific window of time during the last week(s) of class. The new system also integrates with Blackboard — this is another option that will be dependent on the decisions of individual departments. But, for departments that choose to allow this, towards the end of the semester Blackboard can be set up to remind students to fill out an evaluation every time they sign into the Blackboard course site.  These kinds of options make online evaluations easy to administer and can let instructors make decisions about how they want to encourage students to complete their evaluations.

Switching to online evaluations will help make the process more streamlined for department administrators, provide more helpful feedback for faculty, and be a more green alternative to paper evaluations.  Please share any comments or questions about the system in the comments field below!

About Gene Roche

Gene Roche is director of Academic Information Services with responsibility for assisting faculty in using technology effectively in their teaching, learning and research. He also has an academic appointment as Executive Professor in the School of Education where he teaches courses in educational technology planning, emerging technology, and adult education and works with with students on independent study, dissertations and comprehensive exams. Current projects include working with the SOE’s Executive EdD program, co-chairing William and Mary’s Survey Center, and serving as chair of the Electronic Campus of Virginia. Gene completed his AB degree at Hamilton College and his MS and EdD degrees at Syracuse University–all in the snow belt of upstate New York. Before coming to William and Mary, he was the Director of Career Services at Hamilton and taught in graduate programs in Adult Education at Syracuse University and Elmira College.