I don’t go through all of this hassle because it’s peaceful and relaxing. It is super stressful for students, but even more so for me. There is a lot that I have to do to prep, and I feel the weight of responsibility for each and every student on my shoulders. Here’s what’s involved behind the scenes.
At the start of term, I have to register the dates and times of my remote exams with eClass support so they can arrange to have support staff available. A couple of days before the exam, they’ll also check my Smart Exam Monitor (SEM) and ExamLock settings to make sure everything is okay. SEM is the proctoring software that uses AI to monitor students’ cameras and mics. It uses really, really bad AI. Shockingly bad. It flags almost every student for a violation. It takes an excruciatingly long time to go through each flagged student and make sure they are not actual violations. ExamLock is exam integrity software that takes screenshots of students’ screens to make sure they’re not running Google searches in another window. ExamLock is crashy, but seeing students’ screens can often help me troubleshoot problems--which are frequently caused by: ExamLock. It also flags around half the class for (nonexistent) violations, which means even more time clicking and clearing flags. It takes 2-3 hours overall to go through the software and clear all the flags.
A month before the exam, I have to coordinate with my wife. Yes, with my wife. She and I share parenting duties, including picking up and dropping off our kids at school. Well, schools actually--two different schools, which is why it takes both of us. This family coordination is required because some of my students have academic accommodations, most commonly that they receive extra time on exams. So even though an exam might start at, say, 2:00pm, it is possible that some students will still be writing their exam at 5:00. I’ve had a student who got 7 hours to write a 2-hour final exam. That’s right: 7 hours. This is why I have to coordinate schedules with my wife. She’s a family doctor, so she has to change her schedule and end her workday early enough to pick up both of our kids while I’m staring, sweating, at a computer screen. It’s hard enough for her patients to get an appointment to see her, but if I have a lot of exams, it’s even harder for them because of all the child-picking-up. So online exams require a lot of planning in my household. (Oh, and that 7-hour student? They didn’t show up for the final exam. So we had to rearrange our lives again around the 7-hour deferred exam. Which the student didn’t show up for either.)
First, I have to create an exam, for example a multiple choice exam, using my exam management software. I have to export the exam, then import it into eClass. For some reason, eClass does not keep the order of questions intact, but instead sorts the questions alphabetically. Yup, alphabetically. Questions that start with “A person experiences...” come first, and “Zookeepers demonstrate...” come last. (Why...just why?) I have to spend 20-60 minutes manually rearranging questions, depending on the number of questions on the exam.
Each student, though, gets the exam questions in a random order. Why do I have to rearrange them? In order to keep track of which questions are from chapter 1, chapter 2, etc. This lets another piece of software in eClass called ExamVis produce a detailed visualization of each student’s results. Without the proper sorting, nothing will make sense and ExamVis is useless.
Next, I have to set up a session for ExamLock, and another one for Smart Exam Monitor (SEM). Actually, I have to set up two SEM sessions: a regular one, and a backup in case someone encounters problems or accidentally closes out of the exam.
Then, I have to send out instructions to everyone on how to take the exam, even though I’ve made a video on how to use SEM with ExamLock. (This video is now used by eClass support in their knowledgebase article as a how-to. It’s my most popular video, with almost 5,000 views. I’m a YouTuber!) For those students who only have access to a ChromeOS device, I have to send out a different set of instructions, as ChromeOS cannot run ExamLock.
On the day of the exam, I have a whole checklist of things to do:
_ have phone ready (a shared, on-call phone--not my personal cell)
_ open SEM session (done manually, about 20 minutes before the exam is to start)
_ hide lecture videos in eClass (no cheating!)
_ remove lecture notes from my website (no cheating!)
_ check eClass status (see below)
_ check AWS status (Amazon Web Services is a cloud service that provides back-end storage and processing for eClass)
_ check Shaw status and outages
_ check Telus status and outages
Those last two are to see if there are any local internet outages. They are surprisingly frequent. Unfortunately, this only helps me see if there are local students who have problems. Some students are overseas with sketchy internet connectivity.
eClass never goes down, amirite? Cough. There was a major outage on December 11, 2020 that affected one of my classes. eClass was down for so long, the exam was eventually cancelled. Helpfully, every single student in the class emailed me to inform me that eClass was down. There is a protocol on what to do, given an exam disruption. Basically, the Faculty will move the exam to a Saturday. Needless to say, this is not a popular choice--with students, or with me. In the case of my class, each student was given an option to either retain their term mark and skip the final, or go ahead and write the final. In the end, 14 of 123 students actually decided to write the final exam.
When it comes to remote exams, some students are fully prepared--they’ve watched my explainer video, set up the software, and tested everything by running the proctoring trial (for which I typically give a 1% bonus mark). If these students encounter an issue, they know what to do and can handle it well. Other students are not so prepared: they haven’t watched the video, set up the software, or tested it out--but they manage to successfully complete the exam. The issue for me is the small number of students who don’t know what to do or how to do it, are not prepared--and run into problems. What do I click? Where is it? What software? Who do I call?
I want to ensure a smooth process for everyone--even if they are a mere 1%. So I have given IST/eClass support some pretty harsh criticism over the past two years. There are problems like the “Black Screen of Death” that they say affects only 1-2% of students, so it’s not a big deal (this issue was not even described in any of their troubleshooting documentation). They got quite an earful from me about that. If you can’t fix a known problem, at least acknowledge that it exists and how to handle it.
The recent (and, sadly, continuing) brutal budget cuts have had negative effects across campus; the IST/eClass support team has not been spared. They have lost a great number of people. I get it. To be fair, they have responded to my complaints, even going so far as to set up Zoom meetings with me, and reaching out to my students who were affected by software problems in order to understand them better.
I know students don’t like proctoring software. I don’t either. I hope we’re all done with it for good.
Why aren’t you studying?
0 comments:
Post a Comment