Opens in a new tab
English

Useful Links

Why Multilingual Data Quality Is Becoming AI’s Biggest Blind Spot

Organizational AI adoption has reached 88%. Market growth data from Slator estimates the global Data-for-AI market at approximately USD 9.3 billion, growing at a predicted CAGR of 18% to USD $21.5 billion by 2031, suggesting the trajectory is likely to continue. Unfortunately, these numbers only tell half the story of how multilingual data services perform. Even as AI capabilities accelerate, improving performance and reaching more people than ever, traditional benchmarks fail to reveal critical gaps in cross-language effectiveness.

The Underlying Numbers That Show a Crucial Gap in AI Quality

The need for high-quality data is growing. However, nearly half (46.7%) of Slator readers polled in early 2026 say their companies aren’t participating in the data-for-AI space. Only 19% describe their companies as “very active” in the data-for-AI market. When the conversation turned to language technology in the AI boom, polling data revealed:

  • 28% of companies use between two and four different AI models for translation in production.
  • 16% use 10+ different models.
  • 12% don’t know how many AI translation models their company uses. 

In July 2026, the European Commission launched EU MMLU. This benchmark assesses whether LLMs perform fairly and effectively across the EU’s linguistic diversity. Their findings illuminated clear discrepancies. Most AI datasets are in English and, as a result, perform very well on English data. However, they underperformed significantly in other languages, including French, Hungarian, and Maltese.

Standard benchmarks rate multilingual data on the ability to produce a translation, rather than whether the translation is effective for the user in the intended situation. Success is then rated by the number of languages a model produces, leaving meaningful evaluation out of the equation. Models built on English-heavy data carry this imbalance into every market. When multilingual AI is introduced after a build is complete, the supporting evidence behind the output lacks authority and can even be inaccurate. Multilingual data services need to prioritize multilinguality from the start.

The Real-Life Consequences of Gaps in Multilingual Data Services Quality

Many LLMs are built with and tested on English datasets. When a model is built in a single language, other languages serve more as a surface-level mask than as an integral part of the tool.

Language is more than the words that make up sentences and paragraphs. It carries tone, character, meaning, relevance, and purpose. Changing the use of a verb can turn a suggestion into a command (or vice versa). A misplaced adjective can reduce the efficacy of safety warnings. A direct translation of an icon can become meaningless because the source word doesn’t have the same connotation in the target language. 

A model learns from a map of language derived from its data and training process. The model’s map contains texts, recordings, labels, translations, and evaluations. In society, people speak differently at home and at work. They use slang words that deviate from a word’s conventional meaning or borrow words from other locales. Forms used in government proceedings, court decisions, classrooms, health initiatives, and casual conversations use different vocabularies and depend on precise meanings for accuracy. If the model’s map doesn’t accurately align with the language used in society, then the LLM fails to produce the intended results.

When multilingual data services fail, the user absorbs the cost. From finding a translator who can complete a task to repeatedly communicating with customer service teams, users face excessive friction due to language barriers. When it affects a purchase, the cost is usually a lost sale for the company. But the impact can be much more severe when the process involves legal compliance or critical safety information.

AI Governance Gaps in Linguistic Performance

AI systems are expected to work across languages. However, current governance frameworks fail to define what “working” means.  The 2026 International AI Safety Report is authored by over 100 experts and backed by more than 30 countries. It acknowledges that AI risks remain unevenly distributed. It identifies risks such as malicious use, security gaps, reliability challenges related to false information, and risks to human autonomy. However, linguistic performance disparity is not identified as a risk. 

Current AI governance frameworks include:

  • EU AI Act: Sets risk-based rules for AI developers and deployers regarding specific uses of AI
  • South Korea’s AI Basic Act: Focused on transparency and safety, with requirements to provide notice when AI is used and for performance levels in critical sectors
  • OECD AI Principles: Calls for inclusive growth of AI, including advancing inclusion of underrepresented populations

None of these frameworks includes regulatory mechanisms to prevent high-risk AI systems from being deployed in multilingual contexts where the model performs reliably in one language but unreliably in others.

Why Human-Centered Multilingual Data Is the Solution

Benchmarks for AI translation cannot hinge on counting languages. They must consider how each language responds within the model’s infrastructure. An effective model does more than repeat words. It allows people to seamlessly access services using their native language and make their voices heard. This can’t be achieved by adding more data to AI training sets. The solution is to refine the data so the model knows how to respond in specific use cases. 

EU MMLU criteria revealed language discrepancies that were completely invisible in other benchmark tests because unique testing mechanisms guided the process. Traditional multilingual benchmarks primarily rely on machine translation and use thousands of multiple-choice questions. The EU MMLU (Massive Multitask Language Understanding) used nearly 240 human translators from 21 universities to adapt over 1,000 questions to ensure cultural relevance rather than a word-for-word translation. The goal is to go beyond translating questions to ensure they retain their meaning, difficulty, and testing value across languages. The results clearly illustrate the value of human input to improve how multilingual AI performs in real-world situations.

Vistatec Data Introduces Expert Multilingual Data Services at a Foundational Level

A reliable LSI combines multilingual expertise, cultural intelligence, and localization-grade quality with advanced AI data services.

Vistatec Data helps organizations build, evaluate, and improve AI systems using data that reflects how people actually communicate, behave, and interact across languages and markets.

The next challenge for multilingual AI is not simply broader language coverage, but greater consistency of performance. Treating linguistic and cultural context as a core part of AI quality will be essential if systems are to work as effectively across languages as users expect.

Contact Vistatec today to speak to an expert.

Do you want to know more?

Talk To Us