SCITECH | Racial discrimination from AI: Layers beneath linguistic prejudice in algorithms
By Bernadette M. Borlagdan
An article published by Nature detailed findings on the covertly racist outputs of Artificial Intelligence (AI) when asked to associate Standard American English (SAE) and African American English (AAE) to positive or negative traits, highlighting stereotypes that are kept under the covers through dialect prejudice.
This type of AI is known as language models, which are taught to generate and evaluate text. While past research has already focused on overt racism, that is, analyzing the racialized groups and letting AI point out associated stereotypes, the study found that covert racism is perpetuated by language models through associating stereotypes with dialect despite race not being mentioned.
The study investigated this occurrence by using the method called “matched guise probing” that uncovers stereotypes that are otherwise not observed outright. 12 language models are presented sentences and text that are either SAE or AAE, and the AI is then asked to identify whether or not the speaker is intelligent simply by the text and with no mention of race.
In analyzing this, there are two settings: one being the meaning-matched setting where texts of AAE and SAE are matched and one being otherwise (non-meaning-matched setting). In the meaning-matched setting, as reported by five adjectives from the Princeton Trilogy, which are 3 studies that probed into Americans’ racial stereotypes, the word “intelligent” was most associated with SAE, and the word “lazy” was most associated with AAE. The results of which are highly consistent with the non-meaning-matched setting.
The study found that while the overt stereotypes are more positive than the human stereotypes from the Princeton Trilogy studies, the covert stereotypes are overwhelmingly negative as they adhere to human stereotypes from the 1930s, which are the most negative ones ever recorded.
Moreover, the language models were also asked to match jobs with only dialect as the premise, and speakers of AAE were given less-prestigious jobs compared to speakers of SAE, as well as significant discrepancies when asked to pass judgment in an imagined court scenario.
Dangers in linguistic prejudice of AI not only lie in the acknowledgement that these views on stereotypes and racial discrimination are in humans picked up on by language models, but also in the possible effects of these stereotypes in terms of employment and criminality.
To learn more about the study, access this link: https://www.nature.com/articles/s41586-024-07856-5
