Clouded judgment: Ethics, bias, and fairness in the age of AI

Why did so many of the experts who signed the “Statement of AI Risk” call their life’s work an “existential threat”? In part, it may be because they’ve released something into the world without fully understanding how it works. 

“When you look at large language models (LLMs), like what powers ChatGPT, there are billions of parameters,” says Eric Baumer (pictured), an associate professor of computer science and engineering (CSE). “Once the model is trained, it’s so complex that it becomes difficult to interrogate and understand why any one of those billions of parameters influences the output.” 

So while the creators of these systems are knowledgeable about algorithms and the training process, he says, they don’t have a complete understanding. “And I think that’s what leads to these claims about them being an existential threat.”

Baumer’s concerns, however, don’t quite rise to that level. He’s much more worried about how this lack of understanding can lead to bias in these systems.

“Because of our inability to interrogate, inspect, and understand the models, we’re not going to know what biases are being ingested or how they will manifest until they actually do,” he says. “I don’t know that this technology is going to destroy humanity, but it might destroy our ability to be humane to one another.”

Professor Larry Snyder, Harvey E. Wagner Endowed Chair in the Department of Industrial and Systems Engineering (ISE) and Lehigh’s Deputy Provost for Faculty Affairs agrees. It’s not the prospect of living in The Matrix that worries him—but the potential for serious harm if generative AI technologies are “running rampant.” Generative AI (e.g., ChatGPT and Bard) refers to the algorithms that can create new content, including text, code, images, audio, video, and simulations.

“Deep fakes, online trolling, misinformation, increasingly hostile rhetoric about minoritized groups—basically all the bad stuff on the internet and in the world of computing—just gets accelerated and magnified now because of these technologies,” he says.

They also infringe on intellectual property. Because the systems are trained on what’s available across the internet, when they generate art, for example, the images are based on the work of others—artists who have not consented to having their intellectual property train an algorithm. 

What’s more, the training data for ChatGPT and similar technologies hardly represents all corners of the web; instead, the dataset pulls from the most easily accessible parts of the internet, such as Wikipedia and Reddit, says Snyder, pointing out that those sources are primarily written by white men.

“So the outputs are reflecting only that slice,” he says. “And they can’t necessarily make the training data more diverse because that content is less prevalent on the internet for all the systemic, historical reasons why white men are overrepresented on the internet to begin with.”

If he could wave a magic wand, Snyder (pictured) says, ChatGPT and similar technologies would have been developed more slowly and thoughtfully, to allow for more consideration around their safety and societal implications, and for more guardrails to be put up around their use. 

But it’s not just LLMs that are problematic, of course. AI, machine learning, and computing more broadly are replete with issues of ethics, bias, and fairness. 

The examples given by Lehigh researchers are seemingly endless: 

• Many of the algorithms that power search engines are trained on biased data that perpetuate and create stereotypes about certain groups of people. Such systems also reflect the biases of those who create the data, such as the tendency among English speakers to refer to doctors as men, which creates bias around gender and occupation.
• Smart appliances trained on data that doesn’t represent all potential users haven’t functioned properly for people with certain skin colors. 
• On social media platforms, algorithms continually feed us what we want, which essentially puts us inside an echo chamber that lacks diversity of thoughts and ideas and exacerbates polarization. 
• Algorithms used to predict criminal recidivism have been found to be inherently unfair, performing worse for certain racial categories. When facial recognition was found to be more accurate for white male faces than for other demographic groups, the response was to train the model on more people with darker skin, which posed complicated ethical questions around which groups of people are having their likeness used—without their consent—to train models used by police. 

“We have so many examples of systems that were created to do something cool, innovative, and groundbreaking, yet were built without consideration of the ethical or societal impact,” says professor Brian Davison, chair of the Department of Computer Science and Engineering. “It’s no longer enough to be a highly skilled, technical person. You also need to understand society and how the technology you’re designing will impact it.”  

There is a real effort to address these societal impacts. Baumer, who calls himself a “hopeful skeptic” when it comes to our ability to fix such issues, points to the Association for Computing Machinery’s Conference on Fairness, Accountability, and Transparency (ACM FAccT). This annual meeting brings together researchers and practitioners across a range of professions who are interested in addressing and reducing the harms inherent in algorithmic systems. Increasingly, corporations and corporate researchers are developing AI design guidelines around best practices to accompany trained models. 

And there’s even a move to embed philosophers trained in ethics within computer science teams, he says, “so you have someone who can go beyond the what-if question and who knows the literature and how to reason about these problems. We shouldn’t assume that every computer or data scientist is also going to become an ethicist, but we should help them become familiar enough with such considerations to converse meaningfully and productively with ethicists.” 

At Lehigh, newly created and revised courses for both undergrad and graduate students are preparing the next generation of professionals to do just that. Snyder and Suzanne Edwards, an associate professor of English and Women, Gender, and Sexuality Studies in Lehigh’s College of Arts and Sciences, developed and co-taught Algorithms and Social Justice, an interdisciplinary class for engineers and humanists that was offered for the first time last fall (and wrapped up just as ChatGPT was starting to take off). It’s being offered again this fall. 

“This generation of students is well aware of the societal harm baked into these technologies,” says Snyder. “It’s not an afterthought to them. They’re thinking about the ethical pieces in ways that are natural and inherent, and that’s encouraging.” 

Baumer teaches ethics to both undergraduates and master’s students. He gives programming assignments to the latter that demonstrate the difficulty in trying to optimize multiple metrics simultaneously.

“They learn that it’s mathematically and practically speaking, impossible,” he says. “And then it’s like, ‘Okay, you built this model that’s biased—now what do you do?’ Part of the benefit of having an ethics course is for students to make mistakes in a safe environment where you don’t have people, for example, denied parole. These classes prepare them to make decisions when they’re creating real-world models, so they can avoid those very real consequences.”