THEORY-864: When a generative model reproduces a prejudiced narrative learned from its training data, the resulting testimonial injustice runs through a credibility excess assigned to the model, so that hearers believe the amplified account over the testimony of those it discredits, rather than through a hearer's prejudiced judgement of a present speaker

Source

Kay, Kasirzadeh and Mohamed (2024), Proceedings of AIES 7: 684–697, LIT-1338, read in full in NOTE-1049 from the publisher’s PDF. The statement is the paper’s, in §4 under “Amplified Testimonial Injustice”.

What was actually shown

Nothing was measured. The paper argues that a model trained on web data “inherit[s] and reproduce[s] the prejudices and biases embedded within those sources”, that deployment scale and “the perceived objectivity and authority of AI-generated content can lend credence to these narratives”, and that the wrong therefore takes a different form from Fricker’s interpersonal case: “In the conventional human-only environment, testimonial injustices involve a credibility deficit assigned to someone’s account of truth based on the prejudices of the listener. In the algorithmic setting, the injustice requires a credibility excess assigned to the algorithm; that is, humans believe the account amplified through the technology over the individual or group who is discredited.”

Its illustration is NewsGuard’s test of GPT-3.5 and GPT-4 on 100 false narratives (80% and 100% followed), which shows that models produce the narratives on request. It does not show that anyone believed them over the people they concern, which is the step the account needs.

What could have come out otherwise: the account predicts that attributing a claim to a model raises belief in it relative to the same claim from a human source of the discredited group. If model attribution lowers credibility, the excess the account requires is absent.

What this does not say

• That users do overtrust models. That is the premise the promote_when asks for.
• That the prejudice has gone. The prejudice is carried by the training data. The account moves it, from a hearer judging a speaker to a corpus shaping what the model says. It still needs it, which is why it does not supply what CLAIM-406 says the loop lacks.
• That deliberate fabrication with a model is testimonial injustice. The paper says so for its “manipulative” configuration. Fricker (THEORY-327) treats deliberate manipulation of credibility as a distinct wrong.
• That the amplification is a self-reinforcing loop. The paper says it “creates a feedback loop” and gives no condition. CLAIM-277’s loop and CLAIM-330’s condition are the record’s, not this paper’s.