Cheng Chen, with colleagues Sundar and Eunchae Jang, just completed an expansive study. Their research exposes deep racial prejudices baked into artificial intelligence technologies, particularly within the AI training data. This ground-breaking five-year research has illustrated that AI systems can reproduce and even amplify societal biases. The team focused on AI’s ability to reliably identify emotional expressions. They included images of both Black and White people in their study.
To test this hypothesis, the study developed 12 distinct iterations of a prototype AI system. This system is purpose built to read humanity’s most complex expressive tool – our users’ faces. The training data for these systems was often incredibly unbalanced in terms of race representation. This problem was particularly apparent in certain classifications, such as happy or sad faces. This initial experiment helped the researchers identify a glaring bias in the AI’s training data. Specifically, black defendants’ sad faces were much more often ruled as Black.
The Research Overview
To test this hypothesis, the researchers ran three experiments with 769 total participants. Participants in these first two experiments were still predominantly but not exclusively white, sometimes overwhelmingly so, depending on the racial composition of the area. Unlike the first two experiments, the third experiment was constructed to have a balanced design with equal numbers of Black and white research participants. This was an important methodological decision, because it enabled these researchers to test how awareness of racial bias might change perceptions of AI performance.
As the final experiment, participants were given the stimuli from the previous experiments with additional counterexamples. This configuration was designed to test whether participants could identify biases in AI systems. They confronted striking proof of weighted representation just to see how well they are paying attention. Perhaps more surprisingly, the findings revealed that a majority of participants failed to identify any racial bias in the national scenarios they confronted.
In the second unique scenario of the experiment, the AI is forced to contend with racially biased performance. For instance, it was unable to correctly identify the facial expressions in photos from under-served communities.
“In one of the experiment scenarios—which featured racially biased AI performance—the system failed to accurately classify the facial expression of the images from minority groups,” – Cheng “Chris” Chen
Findings and Implications
The findings underscore a troubling trend: many individuals are unaware of how deeply entrenched racial bias can be within AI training data. The study’s results demonstrated that participants largely failed to detect the bias present in the AI’s performance. This failure is important, as it indicates a risky trust in AI systems without interrogating the biases they’re built to reflect.
This unawareness, Cheng Chen highlighted, can make users overly trusting of AI evaluations. To that point, he noticed that folks find it hard to spot the racial confound in the training data. In turn, they rely on AI performance for making these evaluations.
“That is what we mean by biased performance in an AI system where the system favors the dominant group in its classification.” – Cheng “Chris” Chen
The real world implications of these findings extend well past an academic case study. AI is now quickly taking over various sectors, from hiring to law enforcement. It’s very important to acknowledge and combat the racial bias that permeates these systems. The researchers call for more accountability and testing before deploying AI. To this end, they underscore the need for transparency to ensure these new technologies do not exacerbate societal inequities that already exist.
The Role of Race in Emotion Recognition
The research further explores the mechanics of AI systems—particularly how they learn to classify emotions, often in a racially discriminatory manner. First Amendment to Intellectual Curiosity S. Shyam Sundar raised a central question. Regardless of our intentions to create race-neutral systems, AI appears to learn race as a primary contributor to its understanding of facial expressions portraying indicators of happiness or sadness.
“In the case of this study, AI seems to have learned that race is an important criterion for determining whether a face is happy or sad,” – S. Shyam Sundar
This powerful insight makes it imperative to ask more meaningful questions about how the development processes behind AI systems breed vulnerability to societal biases. Sundar was especially surprised at participants’ inability to identify the connection between race and emotion in the training data.
“We were surprised that people failed to recognize that race and emotion were confounded, that one race was more likely than others to represent a given emotion in the training data—even when it was staring them in the face,” – S. Shyam Sundar