How AI Performs in Translation Across European Languages
Claude, ChatGPT, Gemini and DeepL compared across high-resource and lower-resource European languages
Executive summary
AI translation is no longer just a faster version of old machine translation. The best current systems – including ChatGPT, Claude, Gemini and DeepL – can produce fluent, context-aware translations that are often good enough for drafting, internal communication, website copy and first-pass professional workflows.
But the results are not equal across languages. For high-resource European languages such as German, French, Spanish, Italian, Dutch and Portuguese, AI translation is now very strong. DeepL remains one of the most reliable dedicated translation engines for many European pairs. ChatGPT, Claude and Gemini have become serious competitors because they can follow instructions, handle context and adapt tone.
For lower-resource European languages such as Slovenian, Croatian, Serbian, Bosnian, Macedonian, Icelandic, Latvian, Lithuanian and Maltese, the picture is more mixed. AI can still be very useful, but errors are more likely to be subtle: wrong case endings, unnatural syntax, inconsistent terminology, incorrect register, gender mistakes or mistranslated technical terms.
The most important conclusion is simple: AI translation is now excellent for many European languages, but it is not equally reliable for all languages, domains or use cases. It performs best when it is combined with professional review, good terminology and clear instructions.
1. Background: public testing evidence used
This case study compares four major AI translation options – ChatGPT/GPT models, Claude, Gemini and DeepL – using public benchmark and testing sources rather than marketing claims alone. The key references are:
- WMT25 General Machine Translation Shared Task evaluated systems across 30 language pairs and includes detailed rankings for Gemini 2.5 Pro, GPT-4.1, Claude-4 and other systems.
- WMT24 General Machine Translation Shared Task reported very strong overall performance for Claude 3.5 Sonnet and GPT-4.
- WMT24++ expanded the WMT24 benchmark to 55 languages and dialects and found frontier LLMs among the strongest systems.
- EU20 automated quality assurance study compared DeepL, ChatGPT and Google/human-edited translated benchmark data across European languages.
- DeepL 2026 Translator announcement reports tests across 16 high-priority language combinations.
2. Who performs best?
There is no single winner across all European languages. A practical summary looks like this:
System | Strongest use case | Main strength | Main risk |
DeepL | Fast first drafts for major European languages | Fluent, idiomatic output; easy to use | Can miss document-level context or specialized terminology |
ChatGPT | Translation with instructions, terminology and QA | Good at following detailed prompts and explaining choices | May over-edit, normalize or silently “improve” source meaning |
Claude | Long-form, natural and style-sensitive text | Readable output; strong tone adaptation | Benchmark position varies by model and language pair |
Gemini | Multilingual benchmark performance and long-context tasks | Very strong recent WMT25 rankings | Still requires review for terminology, formatting and domain accuracy |
According to WMT24, Claude 3.5 Sonnet was the best overall system, winning in 9 language pairs, while GPT-4 won in 5 language pairs. Human references ranked first in 5 language pairs and were in the winning cluster for 8 language pairs.
In WMT25, Gemini 2.5 Pro and GPT-4.1 were often near the top, especially in European directions such as English→Italian, English→Czech, Czech→German, English→Serbian Cyrillic, English→Ukrainian and English→Icelandic.
DeepL is harder to compare directly in WMT tables because public online systems are often anonymized in WMT. However, it remains highly relevant because it is a dedicated professional translation platform, and its 2026 material reports 48,000 blind tests across 16 language combinations.
3. High-resource European languages: strong results from all major systems
High-resource European languages have the best AI translation results. These include German, French, Spanish, Italian, Dutch, Portuguese, Polish and Czech. These languages benefit from large quantities of training data, many bilingual corpora, strong online presence and mature professional translation markets.
Example: English → Italian
WMT25 shows how close the top systems can be in a high-resource European language pair. For English→Italian, the leading systems were tightly grouped:
Rank cluster | System | Human evaluation score |
1–4 | Gemini 2.5 Pro | 79.4 |
1–4 | GemTrans | 79.4 |
1–4 | GPT-4.1 | 79.0 |
1–4 | Shy-hunyuan-MT | 78.7 |
6–10 | Claude-4 | 72.1 |
The difference between Gemini 2.5 Pro and GPT-4.1 was only 0.4 points. That is a very small gap in practical terms. Claude-4 was lower in this specific ranking, but still produced a competitive score. Source: WMT25 findings.
For a business translation team, English→Italian is now a language pair where multiple AI systems can produce strong first drafts. The best choice may depend less on raw quality and more on workflow: DeepL for a fast polished draft, ChatGPT for instruction-following, Claude for tone and readability, and Gemini for benchmark strength.
Example: Czech → German
Czech and German are both important European languages, but Czech is morphologically more complex than English, French or Spanish. That makes Czech→German a useful test of whether AI can handle more than the largest Western European pairs.
Rank cluster | System | Human evaluation score |
1 | Gemini 2.5 Pro | 90.7 |
2–4 | GPT-4.1 | 89.5 |
2–4 | Claude-4 | 88.8 |
2–6 | DeepSeek-V3 | 88.1 |
Here, all three major LLMs – Gemini, GPT and Claude – performed very strongly. Gemini led, but GPT-4.1 and Claude-4 were close behind. Source: WMT25 findings.
4. Lower-resource European languages: quality drops faster and less predictably
Lower-resource European languages are not all the same, but they share one practical issue: there is less high-quality bilingual training data and less benchmark coverage. This group includes Slovenian, Croatian, Serbian, Bosnian, Macedonian, Albanian, Icelandic, Latvian, Lithuanian, Maltese and Estonian.
Example: English → Serbian Cyrillic
WMT25 shows very strong AI performance for English→Serbian Cyrillic:
Rank | System | Human evaluation score |
1 | Gemini 2.5 Pro | 94.2 |
2–3 | GPT-4.1 | 92.5 |
5 | Claude-4 | 90.0 |
6 | Human reference | 88.7 |
Gemini 2.5 Pro scored 5.5 points above the human reference in this benchmark, while GPT-4.1 scored 3.8 points above the human reference. Claude-4 also scored above the human reference by 1.3 points. Source: WMT25 findings.
This does not mean that AI is generally better than human translators. Benchmark references can vary in quality, and evaluation methods can favor certain system outputs. But it does show that for some lower-resource or medium-resource European languages, current LLMs can perform extremely well.
Example: English → Icelandic
Icelandic tells a different story. In WMT25, the human reference was clearly ahead:
Rank | System | Human evaluation score |
1 | Human reference | 87.5 |
2 | Gemini 2.5 Pro | 77.6 |
3–4 | Erlendur | 68.3 |
3–4 | GPT-4.1 | 68.0 |
8–10 | Claude-4 | 47.8 |
Gemini 2.5 Pro was about 9.9 points below the human reference. GPT-4.1 was about 19.5 points below the human reference. Claude-4 was much lower in this particular test. Source: WMT25 findings.
This example is important because it shows why we should not generalize too much. The same AI model that performs extremely well for Serbian may be much weaker for Icelandic.
5. ChatGPT: best when the task includes instructions
ChatGPT is especially useful when translation is not just translation. Professional translation often involves instructions such as preserving tags and placeholders, keeping line breaks, using client-approved terminology, avoiding certain words, translating formally or informally, checking only major errors, or explaining whether a term is technically correct.
This is where ChatGPT can be very strong. It can translate, revise, explain, classify errors and act as a QA assistant.
Language pair | GPT-4.1 result |
English→Italian | 1–4 rank cluster, score 79.0 |
Czech→German | 2–4 rank cluster, score 89.5 |
English→Serbian Cyrillic | 2–3 rank cluster, score 92.5 |
English→Icelandic | 3–4 rank cluster, score 68.0 |
English→Ukrainian | 6–7 rank cluster, score 87.9 |
These results show that GPT-4.1 is consistently competitive, although not always the top system. Source: WMT25 findings.
The practical advantage of ChatGPT is flexibility. A translator can ask it to produce three options, explain a term, check consistency, identify mistranslations or rewrite the target text for a specific audience. The main risk is that ChatGPT may be too helpful: it can smooth over ambiguity, improve the source, add implied meaning or choose terminology that sounds right but does not match the client glossary.
6. Claude: excellent in WMT24, more variable in WMT25
Claude has a strong reputation for natural, readable language. It often performs well with longer text, corporate communication, marketing copy and style-sensitive adaptation.
The strongest public benchmark result for Claude comes from WMT24. There, Claude 3.5 Sonnet was described as the best performing system overall, winning in 9 language pairs, while GPT-4 won in 5 language pairs.
That is a strong result. The same WMT24 report also notes that human references were in the winning cluster for 7 out of 11 language pairs, about 64%. Source: WMT24 findings.
However, WMT25 shows a more mixed picture for Claude-4 in selected European directions:
Language pair | Claude-4 result |
Czech→German | 2–4 rank cluster, score 88.8 |
English→Serbian Cyrillic | Rank 5, score 90.0 |
English→Italian | 6–10 rank cluster, score 72.1 |
English→Czech | 7–10 rank cluster, score 79.6 |
English→Icelandic | 8–10 rank cluster, score 47.8 |
Claude remains useful, especially for tone and long-form readability, but WMT25 suggests that Gemini 2.5 Pro and GPT-4.1 were stronger in several tested directions. Source: WMT25 findings.
7. Gemini: the strongest recent benchmark performer
Gemini’s recent translation results are impressive. In WMT25, Gemini 2.5 Pro ranked at or near the top in several important European language pairs:
Language pair | Gemini 2.5 Pro result |
English→Serbian Cyrillic | Rank 1, score 94.2 |
Czech→German | Rank 1, score 90.7 |
English→Czech | Rank 1, score 88.7 |
English→Italian | 1–4 rank cluster, score 79.4 |
English→Ukrainian | 1–3 rank cluster, score 90.3 |
English→Icelandic | Rank 2, score 77.6 |
This makes Gemini 2.5 Pro one of the clearest winners in the WMT25 European-language evidence. Source: WMT25 findings.
The WMT25 results are especially relevant because they include both high-resource and more difficult European directions. Gemini did not only perform well for Italian; it also performed strongly for Czech, Serbian, Ukrainian and Icelandic.
8. DeepL: still one of the safest dedicated MT choices for Europe
DeepL remains highly important because it is built specifically for translation. It is not just a general chatbot that can translate. It is a dedicated translation platform with glossaries, formality settings, document workflows and a strong reputation for European languages.
DeepL is especially strong for German, French, Spanish, Italian, Dutch, Polish and Portuguese. Its main advantage is speed and predictability. For many professional users, DeepL produces a fluent first draft with less prompting than an LLM.
DeepL’s own 2026 announcement says that its latest model was tested in 48,000 blind tests with language experts across 16 high-priority language combinations. Because these are DeepL-commissioned tests, they should be treated as vendor evidence, not independent academic proof.
The EU20 study is also useful because it compared translation sources including DeepL, ChatGPT and Google/human-edited translations in benchmark datasets for European languages such as German, Spanish, French, Italian and Romanian. The study found that lower COMET scores were associated with more mistranslation and accuracy errors, supporting the practical idea that automated metrics can help prioritize human review.
9. Language group analysis
Group A: High-resource Western European languages
Examples: German, French, Spanish, Italian, Dutch and Portuguese.
Expected AI quality: high. For these languages, AI usually produces fluent and usable output. Errors still occur, but they are often easier to detect. In general business, marketing and website translation, the output may require light to moderate post-editing.
Recommended workflow: use DeepL, ChatGPT, Gemini or Claude for the first draft; apply terminology rules or glossaries; have a professional reviewer check accuracy, tone and brand style.
Group B: Central European languages
Examples: Polish, Czech, Slovak, and Hungarian.
Expected AI quality: medium to high. Czech performs very well in WMT25, with Gemini 2.5 Pro scoring 88.7 for English→Czech and ranking first. GPT-4.1 and Claude-4 were lower in that direction, with scores of 80.8 and 79.6 respectively. Source: WMT25 findings.
Group C: South Slavic and Balkan languages
Examples: Slovenian, Croatian, Serbian, Bosnian, Macedonian, Bulgarian and Albanian.
Expected AI quality: variable. The strong English→Serbian Cyrillic WMT25 result is encouraging: Gemini 2.5 Pro scored 94.2, GPT-4.1 scored 92.5, Claude-4 scored 90.0, and the human reference scored 88.7. Source: WMT25 findings.
However, this does not mean that all South Slavic languages are equally well handled. Slovenian, Croatian, Bosnian and Serbian are related but not interchangeable. AI may accidentally transfer wording, grammar or terminology from one language into another.
Group D: Baltic, smaller Nordic and lower-resource languages
Examples: Latvian, Lithuanian, Estonian, Icelandic and Maltese.
Expected AI quality: variable to risky. Icelandic is a clear example. In WMT25, the human reference scored 87.5, while Gemini 2.5 Pro scored 77.6, GPT-4.1 scored 68.0, and Claude-4 scored 47.8. Source: WMT25 findings.
This does not mean AI is unusable for Icelandic. It means that quality differences between systems can be large and that human review is more important.
10. What percentages tell us
- In WMT24, Claude 3.5 Sonnet won 9 language pairs, while GPT-4 won 5 language pairs. Claude therefore had almost 1.8 times as many wins as GPT-4 in that benchmark summary. Source: reference.
- In WMT24, human references were in the winning cluster for 7 out of 11 language pairs, or about 64%. Source: reference.
- WMT24 also reported that ESA evaluation produced 37% more clusters than DA+SQM while using only half the number of human annotations. This matters because evaluation method can change how clearly systems are separated. Source: reference.
- WMT24++ expanded the benchmark to 55 languages and dialects and found that frontier LLMs ranked highest across the tested languages according to automatic metrics. Source: reference.
- DeepL reported 48,000 blind tests across 16 high-priority language combinations for its 2026 model update. Source: reference.
These numbers point in the same direction: LLM translation quality has improved dramatically, but evaluation still depends heavily on language, domain and method.
11. Practical recommendations for companies
For high-resource European languages
Use AI confidently, but not blindly. For German, French, Spanish, Italian, Dutch and Portuguese, AI is usually good enough for first drafts, internal communication, customer support drafts, product copy drafts, SEO localization drafts and quick multilingual summaries. For publication, a human reviewer should still check tone, terminology and brand consistency.
For lower-resource European languages
Use AI as a productivity tool, not as the final authority. For Slovenian, Croatian, Serbian, Bosnian, Macedonian, Latvian, Lithuanian, Icelandic and Maltese, AI can save time, but the risk of subtle errors is higher. Native-speaker review is strongly recommended.
For legal, medical and technical translation
Always use expert review. Even when AI output looks fluent, regulated or safety-critical content needs human control. The most dangerous AI translation errors are often not obvious spelling mistakes; they are plausible mistranslations.
Examples include wrong dosage or measurement wording, incorrect legal obligation, mistranslated warning, wrong UI command, inconsistent product terminology, incorrect formal/informal address and wrong country-specific terms.
For marketing translation
AI is very useful, especially when combined with human editing. ChatGPT, Claude and Gemini can be excellent for adapting slogans, headlines and tone. DeepL can provide a clean base translation. A human linguist should still decide whether the final text sounds natural and persuasive in the target market.
12. Final conclusion
AI translation is now strong enough to change professional translation workflows. For high-resource European languages, DeepL, ChatGPT, Claude and Gemini can all produce excellent drafts. DeepL remains one of the best dedicated MT platforms for European languages. ChatGPT is especially strong when detailed instructions, terminology and QA are needed. Claude is strong for readability and tone. Gemini currently has some of the strongest public benchmark results, especially in WMT25.
For lower-resource European languages, AI is useful but less predictable. Serbian performed very well in WMT25, while Icelandic showed a much larger gap between AI and human reference quality. This proves that language coverage is uneven, even among advanced systems.
The best approach is not to ask whether AI translation is “good” or “bad”. The better question is: which AI system works best for this language pair, this domain, this client terminology and this risk level?
For many projects, the winning workflow will be hybrid: AI produces the draft, a professional linguist checks accuracy, terminology and formatting are verified, and the final text is adapted for the real target audience.
That is where AI translation performs best: not as a replacement for linguistic expertise, but as a powerful productivity layer on top of it.
Need a translation?
Contact us and we'll make you a personalized offer.
Fill out the form and we will prepare a customized offer for you. For more information you can also call us at +386 1 620 26 16.
Not sure yet?
Before the first order, we perform a free test translation so that you can see if our translations meet your expectations.
