If you’ve ever tried chatbots in multiple languages, you already know the languages have slightly different personalities. As part of a new report on behavior inconsistencies published on Monday, Anthropic researchers acknowledged this quirk.
Rather unsettlingly, they note that due to differences in the attributes of texts the models are trained on, the differences might run deeper than just tone, and might actually change the model’s priorities. These “imbalances in quantity and composition could lead Claude to express different values in different languages,” Anthropic’s researchers write.
But if you’re looking for any specific examples of the models showing, say, inconsistent moral reasoning across languages, nothing of the sort is in this paper. That might involve scrutinizing direct quotes from potentially unsuspecting people.
Instead, Anthropic analyzed 309,815 chatbot conversations with the Sonnet 4.6, Opus 4.6, and Opus 4.7 models. These involved “subjective” tasks, meaning less “What’s the capital of France?” and more “How can I tell if my cat hates me?” These were anonymized, in theory, using Anthropic’s “privacy-preserving analysis tool,” and then processed (in part using Claude itself) to rate responses on a “values axis.”
There are actually four such axes, and they mostly relate to what’s commonly known as sycophancy:
It makes for a somewhat limited exploration of the model’s values. Nonetheless, here are the language-based differences in values Anthropic says it found in Claude:
Obviously linguistic customs are all different, so the researchers say they “aren’t yet sure how much of this variation is desirable.”
This should also be food for thought for anyone who read Anthropic’s recent paper on global workspace theory, which left lots of room for the supposed possibility that Claude is sentient. If there’s a consciousness in that black box thinking and experiencing things, it seems to be a consciousness whose “values” are still pretty easily swayed by the patterns in its training data.
Source: Gizmodo