海角大神

Online exams raise concerns of racial bias in facial recognition

As students take exams remotely, universities around the world are adopting proctoring software that uses facial recognition systems. Some complain the systems are racially biased or overly invasive.

|
Szilard Koszticsak/MTI/AP
Two students learn from their home in Budapest, Hungary, Nov. 11, 2020, after tighter restrictions came into force to contain the spread of the coronavirus pandemic. School officials across the world are split on how to monitor cheating during remote exams.

When law student Areeb Khan tried to sign into the online portal to take his practice bar exam, he was met with聽a strange message: 鈥淒ue to poor lighting we are unable to identify your face.鈥

Additional lighting did not solve the issue. Mr. Khan even tried to sign in from the brightest room in his New York apartment 鈥 the bathroom.

Mr. Khan began to suspect that it was his dark skin tone that rattled Examplify, a test proctoring platform adopted by New York state鈥檚 law exam board during聽the COVID-19 pandemic. It took days of back and forth with customer service before he could sign in.

鈥淭here are so many systematic barriers preventing people like me from obtaining these degrees 鈥 and this is just another example of that,鈥 he told the聽Thomson Reuters Foundation.

As COVID-19 restrictions force students to take remote exams, universities around the world are relying on proctoring software like Examplify. But many聽students are wary of the technology, not just because of bias in facial recognition but also because of the potential for data collection.

鈥淪tudents are already under tremendous pressure because of the global pandemic,鈥 said Hye Jung Han, a researcher at advocacy group Human Rights Watch who聽specializes in technology and education.聽鈥淎nd now we have this invasive and unfair surveillance pushing the envelope, invading their private lives.鈥

Industry leaders maintain their platforms are a critical part of the infrastructure that allows students to continue learning.

鈥淲e believe that many lives have been positively impacted by being able to continue their education and careers,鈥 said Nici Sandberg, spokeswoman for ExamSoft,聽which makes the Examplify platform.

鈥淓xamSoft maintains a non-biased identification and exam delivery process to ensure that individuals of color are not disproportionately affected.鈥

Global expansion

More than 90% of countries have instituted some form of remote learning since the start of the pandemic, according to a report in August by UNICEF.

This, in turn, has created a booming business for companies dealing in educational technology 鈥 or edtech 鈥 including firms that specialize in ensuring聽remote exams are free from cheating.

One firm, Proctorio, reported that it was proctoring more than five times the number of exams this year compared to last year.

The remote proctoring industry offers a range of services, from basic video links that allow another human to observe students as they take exams to algorithmic聽tools that use artificial intelligence (AI) to detect cheating.

But asking students to install software to monitor them during a test raises a host of fairness issues, experts say.

鈥淭here鈥檚 a big gulf between what this technology promises, and what it actually does on the ground,鈥 said Audrey Watters, a researcher on the edtech industry聽who runs the website Hack Education.

鈥淸They] assume everyone looks the same, takes tests the same way, and responds to stressful situations in the same way.鈥

Facial recognition systems 鈥 which some proctoring platforms use to confirm the identity of the test taker 鈥 are less accurate with dark-skinned people,聽noted Shea Swauger, a researcher who tracks the industry at the University of Colorado, Denver.

And algorithms designed to detect suspicious movement will inevitably flag disabled students and others who do not move in the way the platforms expect,聽he added.

Students also are balking at allowing third-party software access to their devices, with some services requiring that students give them permission to聽read their computer files, monitor their keystrokes, and analyze their biometrics.

鈥淚t鈥檚 not just [about] racial bias,鈥 said Miguel Bishop, a member of the student senate at California State University in Chico, which uses the platform聽Proctorio for exams.

鈥淭here鈥檚 the unaccountable data collection and the damage to the student-teacher relationship,鈥 he said.

Proctorio CEO Mike Olsen said in a phone interview that the platform is the most convenient way for schools to deal with cheating in the COVID era.

鈥淭aking an exam in the comfort of your own home, on your own schedule, is less invasive,鈥 he said.

Critics of the technology often misunderstand how it works, he added, emphasizing that the tools do not directly identify cheaters, but simply flag suspicious聽behavior to universities.

Students rebel

Students in various countries have been demanding their schools reconsider the use of remote proctoring software, with mixed results.

In September, the Supreme Court of India canceled a remotely proctored admissions exam at the National Law School of India University, after a parent of聽a prospective student and a former university official filed a lawsuit.

They argued that the exam 鈥 which was slated to use AI to detect cheaters 鈥 was unfair to students with 鈥渓esser means and from marginalized areas鈥 who聽might not have high speed internet or fast enough computers to run the exam.

Students at the University of Queensland in Australia have been unsuccessfully petitioning the university to rein in what they consider to be the most invasive and potentially discriminatory aspects of ProctorU, an American proctoring service the school uses.

鈥淲e鈥檝e asked them to do things like in-house identity verification, rather than having [the platform] read our ID and do biometric analysis on our faces,鈥澛爏aid Rowan Evans, a student representative at the school.

Jennifer Buckley, an engineering professor at the University of Delaware, decided not to use ProctorU after she heard from her students that proctors had聽interrupted remote exams and asked students to show they weren鈥檛 cheating.

鈥淣o thank you,鈥 she said. 鈥淚鈥檇 rather my students not feel like they鈥檙e in a police state.鈥

She and her assistants use Zoom to proctor exams themselves.

ProctorU chief executive officer Scott McFarland said that 鈥渟tudents should be reassured that all testing data is owned by their schools, not the proctoring聽provider.鈥

鈥淪chools set the rules about what data is collected, how it鈥檚 retained and for how long,鈥 he said in an emailed statement.

In the United States and Canada, students at dozens of universities have sent petitions criticizing exam proctoring technology, according to a tally by聽the Electronic Frontier Foundation, a digital rights non-profit.

Patrick Sullivan, a sophomore at the University of Massachusetts, Lowell, created one such petition after a math professor asked him to install聽the remote proctoring service Respondus for an exam in September.

He said the software, which is designed to block access to certain functions on a computer while a student takes an exam, would gain access to files on聽his personal device. 鈥淕iving software this much access is playing with fire,鈥 he said.

Jodi Feeney, the chief operating officer of Respondus, said that聽鈥渙ur applications have also been closely analyzed by hundreds of customers along the way 鈥 university security teams, IT staff, students, third-party companies.鈥

After 1,200 students signed Mr. Sullivan鈥檚 petition demanding the school ban the software, the university announced that professors
could not mandate its use.

As the pandemic makes remote learning a long-term state of affairs for many schools, Mr. Swauger, in Denver, recommends that schools take some time to do their research聽and only roll out systems that are proven to work.

鈥淗igher ed [institutions] like to perceive themselves as serious, deliberative, evidence-based research institutions,鈥 he said. 鈥淲hen it comes to these systems,聽that鈥檚 all been going out the window.鈥

This story was reported by the Thomson Reuters Foundation. Additional reporting was contributed by Rina Chandran.

You've read  of  free articles. Subscribe to continue.
Real news can be honest, hopeful, credible, constructive.
What is the Monitor difference? Tackling the tough headlines 鈥 with humanity. Listening to sources 鈥 with respect. Seeing the story that others are missing by reporting what so often gets overlooked: the values that connect us. That鈥檚 Monitor reporting 鈥 news that changes how you see the world.
QR Code to Online exams raise concerns of racial bias in facial recognition
Read this article in
/Technology/2020/1117/Online-exams-raise-concerns-of-racial-bias-in-facial-recognition
QR Code to Subscription page
Start your subscription today
/subscribe