Hailey Schnorr has spent years peering into the bedrooms, kitchens, and dorm rooms of students via Webcam. In her job proctoring online tests for universities, she has learned to focus mainly on students’ eyes.
“What we look for is eye movement,” says Ms. Schnorr. “When the eyes start veering off to the side, that’s clearly a red flag.”
Ms. Schnorr works for ProctorU, a company hired by universities to police the integrity of their online courses.
ProctorU is part of a cottage industry of online proctoring providers that has grown in recent years as colleges and universities have set their sights on “nontraditional” students who want to earn degrees without leaving home.
The old biases against online education have begun to erode, but companies that offer remote-proctoring services still face an uphill battle in persuading skeptics, many of whom believe that the duty of preserving academic integrity should not be entrusted to online watchers who are often thousands of miles from the test-takers. So ProctorU and other players have installed a battery of protocols aimed at making their systems as airtight as possible.
The result is a monitoring regime that can seem a bit Orwellian. Rather than one proctor sitting at the head of a physical classroom and roaming the aisles every once in a while, remote proctors peer into a student’s home, seize control of her computer, and stare at her face for the duration of a test, reading her body language for signs of impropriety.
Even slight oddities of behavior often lead to “incident reports,” which the companies supply to colleges along with recordings of the suspicious behavior.
Rebekah Lovaas, 24, served as a proctor at Kryterion, another such company, for three years before being promoted to operations analyst. When she first started, Ms. Lovaas said, the company’s methods struck her as “almost intrusive.”
She was not alone. Teresa Fishman, director of the International Center for Academic Integrity, a leading advocate for reliable safeguards against cheating and a former police officer, said she favored the approach of asking online students to come to a physical testing center for exams. “To watch somebody in their room—that seems a little invasive to me,” she said.
Each online-proctoring company has developed its own approach. Some monitor live feeds; others record students via Webcam and watch the recordings. Some require students to share a view of their computer monitor, and empower a proctor to override their cursor if necessary; others simply make students install software that makes it impossible to use Web browsers or chat programs while the exam is in progress.
The companies make bold claims about their effectiveness, arguing their services are not just equal to but better than in-person proctoring. “The level of supervision over the Web is much more intense,” said William Dorman, chief executive at Kryterion. “Frankly,” he said, “we can spot any cheating.”
Kryterion notes “aberrant behavior"—a test-taker leaves his seat, or answers the phone, or some similar breach—in about 16 percent of the exams it monitors, said Mr. Dorman. This does not always mean the students are cheating, but it does mean the university will be notified.
Software Secure, another company that works with universities, classifies such “incidents” into three tiers. The company’s subcontractor in India, Sameva Global, said it notes “minor suspicions” in 50 percent of exams; “intermediate” suspicions in 20 to 30 percent; and “major” incidents in 2 to 5 percent.
Creating Standards
The availability of these options raises a question for all universities: How much proctoring is enough?
Higher-education institutions are expected to certify academic achievement. But how they do that has been left largely unregulated.
Federal officials, when drafting the Higher Education Opportunity Act of 2008, specifically avoided detailing proctoring requirements for online education, said Mollie McGill, a deputy director at the Western Interstate Commission for Higher Education’s Cooperative for Educational Technologies. When it came to policing online exams, the rule-making committee elected to avoid regulatory language that would favor any specific technology or practice, said Ms. McGill.
The result was a minimum standard for compliance—a secure login and password—that has left online programs largely to their own devices, she said.
The emergence of massive open online courses has brought new attention to ensuring integrity in a global online classroom. The American Council on Education, a Washington-based group that advises college presidents on policy, recently put ProctorU’s protocols under the microscope as part of its review of five MOOCs from Coursera.
“In general our standard was that we wanted to see something that was at least as good or better than what you would see in a large lecture class,” said Cathy A. Sandeen, vice president for education attainment and innovation at the council. That was the basic guidance the council gave the professors it enlisted to judge whether students who succeed in the Coursera MOOCs should be awarded transfer credit from degree-granting universities. All five courses earned a seal of approval, in what was a big moment for ProctorU and for online proctoring in general.
The Proctoring Life
At ProctorU’s office in Livermore, Calif., Ms. Schnorr and her colleagues report to work wearing color-coded polo shirts: black for managers, blue for proctors, white for trainees. The proctors’ workspaces are identical, she said, each with a computer and two monitors, and bear none of the family photos or other accouterments that adorn a typical lived-in cubicle; employees do not have regular workspaces, says Ms. Schnorr, they just take whatever workspace is open. The shifts typically last four hours, including a 10-minute break, although proctors sometimes work double shifts.
Watching people take tests can be dull work. Three proctors interviewed by The Chronicle said most incidents were routine, often the result of a misunderstanding. But occasionally a student will try to outwit the system—or simply throw proctors for a loop.
One student tried to fool ProctorU by attaching a sticky note just below his Webcam, so that the proctor couldn’t see it. But the proctor caught the student’s eyes drifting to the note and made him hold up a mirror to his monitor, busting him. Now the mirror check is part of the company’s regular protocols.
Michael Malicia, professional-services manager for Software Secure, said he had caught a student pretending to read questions aloud to herself when she was really dictating to a co-conspirator in the next room, who would then relay an answer. But no dice: In addition to watching the students, Software Secure, like other proctoring services, also listens in.
Other incidents are downright weird. Ms. Lovaas, the former Kryterion proctor, said one student appeared on screen wearing a chicken costume, and proceeded to take the test. The gesture was mystifying, but “we never detected any aberrant behavior,” she said, until the student made a move to put on a chicken mask to match the outfit—a breach of Kryterion’s policy against headgear that might be used to disguise a listening device. At that point, Ms. Lovaas said, “We were like, OK, this has gone too far.”
Remote-proctoring services rely on the same technology that has made it possible for people to earn college degrees without having to report to a campus.
It’s ironic, then, that most remote proctors are not allowed to work from home.
Why? Because they might slack off—or cheat.
The process requires “a lot of attention,” said Nilofar Nigar, a marketing manager at Sameva Global, the company that oversees Software Secure’s operations in India, where 80 percent of its proctors are located. Proctors who work from home are liable to get distracted, she said, “and we can’t afford to have any fault in this process.”
ProctorU and Kryterion also require their proctors to work from a central office.
One of Kryterion’s goals is to turn the job of proctoring from an incidental or part-time gig for professors and other educators into a bona fide “career path,” said Mr. Dorman, the chief executive.
Mr. Dorman said the 70 or so proctors at the company’s headquarters in Phoenix, Ariz., run the gamut from young adults in their first jobs to more-experienced workers, often with customer-service or tech-support backgrounds.
The company requires 100 hours of training before its proctors can begin monitoring live tests. The proctors get paid hourly—between $15 and $25 per hour, depending on whether they are also qualified to troubleshoot technical difficulties—with opportunities for advancement within the company.
At the same time, Kryterion is sensitive to the possibility that a proctor might try to collude with a test-taker to cheat, or jot down the content of a particular exam with the intention of selling it to future students taking the same course.
And so, as they review video recordings of students taking exams, Kryterion’s online proctors are sometimes under surveillance by their supervisors, who are, as Mr. Dorman put it, “proctoring the proctors.”
