Study reveals influence of Chinese censorship on AI models worldwide
Research indicates that AI models, including those from American companies, may reflect Chinese censorship practices. A peer-reviewed study found that Chinese state media influences the training data of various AI systems, raising concerns about the neutrality of these technologies in political contexts.
Regulators and researchers have warned that Chinese AI models incorporate Beijing's censorship, and new studies suggest that American models may not be immune to similar influences.
A study published in Nature found that Chinese state-controlled media is present in AI training data, affecting how models respond to questions about China. It highlighted that models from companies like Anthropic, OpenAI, Google, and Meta were more likely to avoid criticism of governments in countries with restricted political speech.
The research identified over three million Chinese-language documents in the CulturaX dataset used for training AI, showing that models could reproduce phrases from Chinese state media. Further training on state-scripted news examples led to significant changes in model responses, with retrained models providing more favorable answers regarding China's political system.
The study also revealed that AI models tend to produce more favorable descriptions of countries with lower press freedom when queried in the local language. This phenomenon raises concerns about the potential for AI to perpetuate state media control.
Additionally, an Oversight Board report found that American AI models sometimes applied political restrictions from authoritarian countries even to users outside those jurisdictions, a situation termed 'censorship-by-proxy.' Testing showed that refusal rates for political criticism requests were higher for models queried about restrictive countries compared to those about freer ones.
The results indicate a risk that AI models could further entrench the restrictive speech norms of repressive regimes, prompting calls for greater scrutiny of AI training practices.