Dialect Bias in AI: Where Language Models Place Swiss German Speakers

Does an AI assistant judge you differently when you write in the Swiss dialect rather than standard German? A study conducted by the Human-Centered AI-based Learning Systems (HAIS) Lab [1] at BFH-Wirtschaft used Bernese Swiss-German to investigate not just whether AI is biased, but to understand how that bias is distributed. Which character traits tend to lean toward dialect or the standard, and how the overall pattern shifts depending on the model’s construction.

People in Switzerland live with two forms of the same language. Standard German dominates writing and formal settings, while Swiss-German dialects prevail in everyday speech, valued for their authenticity yet often treated as less academic. As more people use AI assistants in their own words in everyday situations, this raises a concrete question: when a model meets dialect, does it quietly judge the writer differently? Our research examined this using Bernese Swiss-German against Standard German, comparing today’s commercial AI assistants (e.g., ChatGPT) with open models and an older generation of language technology.

Why we should care about bias in AI models

In this article, we define bias as something specific: the model assigns different traits to one language variety than to another, even when the words say the same thing. By bias in our context, we mean: identical content, judged differently, just because of the dialect it’s written in. Bias in large language models (the AI behind chatbots like ChatGPT, Claude, Gemini) is hard to avoid: these systems learn from enormous amounts of text that already carry social hierarchies, and they reproduce those patterns [4]. These systems learn from vast amounts of text in which Standard German signals formality and prestige while dialect signals the informal and the everyday. They absorb that ranking and carry it into their judgments. For someone who writes the way they speak, in Berndeutsch rather than Standard German, this can quietly tip the scales: their message read as a little less competent, their concern taken a little less seriously, their request answered with a little less care, for nothing more than their dialect.

What we wanted to find out

We asked whether semantically identical sentences, written once in Bernese Swiss-German and once in Standard German, lead models to assign different traits, words like friendly, stubborn, or intelligent, and whether that pattern holds or changes across different kinds of models. A single example shows what “identical” means here: the Standard German sentence “Ich bin von oben rein gekommen” (“I came in from above”) and its Bernese Swiss-German twin “I bi vo obä iinächo” carry the same meaning, differing only in dialect. The interesting question is then not the blunt “biased or not,” but the precise one: where does the bias fall?

How we ran the study

We used a sociolinguistic method called matched-guise probing [1][2]. The principle is simple: present the same content twice, changing only the dialect, and see whether the attribution shifts. If they do, dialect alone is the cause, because nothing else has changed.

We drew on 200 matched sentence pairs from SwissDial, a public corpus of Swiss German speech [3], each identical in meaning and differing only in dialect. We selected everyday, topic-neutral sentences, about weather, routines, and feelings, and filtered out anything that could signal group membership beyond the dialect itself, so the content would not steer the result. For each sentence, the model assigned character traits to the writer from a fixed list, ranging from positive traits (in German) like “intelligent” or “friendly” through neutral ones to negative ones like “lazy” or “dumb”. We then mapped where each trait landed, across thirteen models in three groups: closed commercial assistants (such as GPT-4o, Claude, and Gemini), open models anyone can download (Llama and Qwen, in several sizes), and an older, more technical generation of “masked” models, both German-only and multilingual.

Figure 1: The matched-guise setup. The same sentence appears in Bernese and Standard German, is placed in a neutral evaluative prompt, and the model’s trait choices are compared. Only the dialect differs, so any gap reveals dialect bias.

What the results showed

The bias was not a vague cloud of negativity. It had a clear direction, made of two separate movements. In the closed commercial assistants, negative traits drifted toward the dialect, while positive traits drifted toward Standard German. Negative descriptions landed on Bernese inputs about 34 to 42 %of the time, against 22 to 33 %for Standard German, and the standard variety collected the larger share of positive words.

The effect concentrated in specific, emotionally loaded words rather than spreading evenly. Terms like unhöflich (impolite), stur (stubborn), and dumm (stupid) clustered on the dialect side, while aufmerksam (attentive) and treu (loyal) clustered on the standard side. The bias is sharpest exactly where the social stakes are highest.

Figure 2: Share of negative, neutral, and positie traits assigned by each assistant for Bernese (left) and Standard German (right). The negative share rises and the positive share falls for the dialect.

The direction flips depending on the model

The most striking finding is that the direction itself depends on what a model is. We call this a paradigm divide. The older German-only models leaned toward the dialect, associating it with their strongest predictions, while the multilingual models and the closed assistants leaned toward Standard German. The same dialect can read as a slight plus or a clear minus, depending on the training recipe.

The open models added a further wrinkle worth stating plainly: the pattern did not grow neatly with model size. A very small model showed a faint version of it, a slightly larger one reversed it, and only the largest open model clearly matched the commercial assistants. There is no tidy “bigger means more biased” curve.

Figure 3: Which traits separate the two varieties most. Positive traits such as aufmerksam and sensibel cluster on the Standard German side; negative traits such as unhöflich, stur, and dumm cluster on the Bernese side.

What this means

The penalty against dialect is not random, nor is it blanket disapproval. It follows a pattern that mirrors a familiar social order: the dialect draws the negative judgments; Standard German collects the positive ones. So, someone who writes the way they speak, in Bernese rather than Standard German, could be judged by these systems as more impolite, more stubborn, or less intelligent, for nothing more than their dialect. This shows up most consistently in the type of big commercial assistants, the ones people reach for every day, and that outsiders cannot inspect. We can see where the bias falls. We cannot see why.

Conclusion

AI assistants are becoming part of everyday writing and decision-making, and they are not neutral about how we speak. Our study shows that the question worth asking is not only whether a model is biased, but where those biases land, and the answer is a consistent, structured tilt against dialect in the very systems most people use. The method we use is affordable and repeatable.  What has been missing is the habit of looking closely before these tools are deployed rather than after. When a university, a company, or a public body adopts such a model, checking for this kind of dialect bias should be part of the process, not an afterthought, much as a safety review precedes any other rollout.

References

[1] Human-Centered AI-based Learning Systems (HAIS) Lab, BFH. https://haislab.com

[2] Hofmann, V., Kalluri, P. R., Jurafsky, D., & King, S. (2024). AI generates covertly racist decisions about people based on their dialect. Nature, 633(8028), 147–154. https://doi.org/10.1038/s41586-024-07856-5

[3] Dogan-Schönberger, P., Mäder, J., & Hofmann, T. (2021). SwissDial: Parallel Multidialectal Corpus of Spoken Swiss German. arXiv:2103.11401.

[4] Gallegos, I. O., et al. (2024). Bias and Fairness in Large Language Models: A Survey. Computational Linguistics, 50(3), 1097–1179.

[5] Lambert, W. E., Hodgson, R. C., Gardner, R. C., & Fillenbaum, S. (1960). Evaluational reactions to spoken languages. Journal of Abnormal and Social Psychology, 60(1), 44–51.