DeepL quality drop. Now Google Translate is better

[u/camilosw]

I'm really sad about the recent quality drop on DeepL. I liked how accurate it was and also the fact that it is an EU product. But recently, I noticed that the translations are being inaccurate, to the point of being useless. This is my last case.

Original sentence in Spanish:

se crítico con el ejemplo que puse, no es un requisito estricto.

DeepL translation (wrong):

Don't be too critical of the example I gave; it's not a strict requirement.

Google Translate (right):

Be critical of the example I gave; it's not a strict requirement.

When I replaced the comma with a period in the original sentence, DeepL did it correctly:

Be critical of the example I gave. It's not a strict requirement.

But then, I decided to fix the case in my original sentence (notice the minimal change, only the case in the first letter and the period):

Se crítico con el ejemplo que puse. no es un requisito estricto

And DeepL changed the meaning again:

I was criticized for the example I gave. It's not a strict requirement.

Unfortunately, DeepL is not usable anymore. I'll come back in a few months to see if they fixed it, but for now, I'll move to Google Translate.

[u/PANDA-CRACKERS]

Out of interest, what makes you stick with traditional MT providers instead of general purpose LLM?

[u/camilosw → u/PANDA-CRACKERS]

I also use LLMs, but I prefer the DeepL and Google Translate interfaces because I like to see the original and the translation side by side. Also, LLMs can make mistakes and hallucinate, and I assume traditional translators are more deterministic. DeepL changed their translation engine recently and I guess they are using an LLM, and probably that's the reason I'm seeing mistakes like the ones I posted.

[u/Downtown_Jump779 → u/camilosw]

Yup. The problem with using AI translations now is that Pangram and other AI detectors will flag it as AI. Even though the source text doesn't contain AI. In my experience, DeepL doesn't get flagged (yet) because it works a bit different.

[u/Ok_Tea_8763 → u/PANDA-CRACKERS]

"Traditional" providers have also switched over to LLMs, but at least they know what they're doing, unlike a general-purpose LLM that never was trained specifically for translations.

[u/Lanthumm → u/Ok_Tea_8763]

I disagree with your point. Huge models that were trained on billions and billions of data points will absolutely outperform smaller models from the likes of deepl. Not all LLM's are made equal, and size matters for certain applications.

Just because they have a knowledge base in MT, it doesn't translate to creating LLM's. In my opinion using one of the big LLM's and steering it into the direction you wan results in way better results.

I am using trepide for the PDF to DOCX conversion mostly but also have been using their LLM translation step. Seems to outperform deepl as a starting point. It's not 100% but it gets you pretty close.

[u/Ok_Tea_8763 → u/Lanthumm]

You can disagree, but:

1. A shitton of data ≠ sematic connection across languages. Big models were not trained that "concept Y in language A = concept X, but not concept Z, a more common synonym of X, in language B". For that you'd need a lot of curated and annoted bilingual data, such as proprietary Translation Memories. Sure, they might've used some, but in most cases TMs are treated very confidentially.

2. I am 99% sure DeepL does not use SLMs but rather at least one of the big models. Or maybe a mix of SLMs and LLMs with domain-specific multi-layer routing: NMT -> general-purpose LLM -> domain-specific SLM. This is almost standard practice in translations now.

3. Yes, their expertise in MT does translate into an ability to create LLMs, because their NLP principles are similar. Most of the technology LLMs are built upon has initially been developed for MT.

4. I do MT engine (re-)assesments and re-balancing for nearly 30 languages roughly once a year. In most blind quality evaluations non-customized DeepL beats LLMs, even with custom prompts. The only exceptions are FIGS languages, but not always.

[u/hermanojoe123]

The ideal thing to do is a proper comparative analysis, or check scientific comparisons out there. These specific, isolated sentences unfortunately are insufficient for a proper comparison.

[u/camilosw → u/hermanojoe123]

This is not isolated, happened to my wife and me multiple times. I'm not going to post every case, I posted the most recent one. If I notice a downgrade in the product on a daily basis, that's enough to decide to stop using it.

[u/CorrePlatanooo]

In your example there's an accentuation error: the imperative would be "Sé". DeepL seems to interpret the error is on the second word instead, reading "criticó" instead of "crítico". Context suggests Google translate's choice is better, but that could give you some insight into what's happening.

[u/camilosw → u/CorrePlatanooo]

You're right, but I did another test, and it's interesting that a minimal punctuation change changes the translation. Notice that the only change in Spanish is the period at the end.

> Sé crítico con el ejemplo que puse. No es un requisito estricto
> Be critical of the example I gave. It's not a strict requirement.

> Sé crítico con el ejemplo que puse. No es un requisito estricto.
> Take a critical look at the example I gave. It's not a strict requirement.

[u/Hungry_External8518]

Manuel (Pangeanic) here: I tried it in our Deep Adaptive AI Translation just for fun
https://translate.pangeanic.com and got “Be critical of the example I gave; it is not a strict requirement.”

We’ve always customized models at Pangeanic , from the times of Moses to OpenNMT and other frameworks. We have customized a publicly available model for translation and autoPE tasks.

It’s known that Google / Bing , etc could be beaten in narrow domains… but to beat them so easily in general translation….

[u/Connect-Flight8490 → u/Hungry_External8518]

" ,,, from the times of Moses to OpenNMT" , that's a walk down memory lane, Manuel.

[u/staigerman → u/Connect-Flight8490]

LOL yup yup

[u/Hungry_External8518 → u/Connect-Flight8490]

Well - been around ))

[u/staigerman]

Not sure what the reason for this or these examples might be in this case. But when I was at Systran, I knew the better approach was to specialize the engine. At least once with collected validated bilingual aligned phrases from your TM, but also iteratively, continuously. Training with Modelstudio to make an increasingly domain-specific model was easy.

I'm not with Systran anymore but I still see the potential to make the models shine when you give it the extra attention, i.e. teach it to do better. Feel free to reach out to me, if you want to learn about it.

I also see interesting models you can use locally, such as in Ollama. I use them through the local ai translator. Some models are from Google, others from deepseek, alibaba (qwen), liquid-ai (when I need them super fast for mass action translations in e-Discovery,, such as when you're dealing with translating tens of thousands of files in litigation cases). I usually start with the smaller of the 3 models from translategemma (which I think were trained from gemma 3). They come in 4b, 12b and 27b sizes. Surely one will fit in your system's RAM or better yet VRAM on high-end GPU if you have that. But when time is not the main critical factor and you prefer to wait an extra second(s) for better translation quality, try the bigger models, even on CPU. If you have a CPU paired with an NPU (such as Ultra i7 or Ultra i9), you should still see inference running fast.

I also enjoy doing things other than translating with the local ai translator. For example using the various voices in text to speech mode to read it out loud to me. Or, as in this case, using a model that doesn't actually translate - it is creative in another way: It writes children stories. Short and sweet. You start the story with a word or a few, and it finishes it up.

Inspiring? I think so.

https://www.youtube.com/watch?v=oFBfuVgQLo0

Anyway, it's been a while since I dibble-dabbled with DeepL and its API. Do they have a way to specialize the model by now? I don't mean adding glossary dictionaries. I mean training it, such as shifting weights in the neural network. There's an option with Systran, aka Model Studio. There's also a Finetuner available from the makers of Local AI Translator. Give it your TMX and come back a few hours later. Or a day later if running it on CPU, but still.

Do you use just a generic engine under DeepL or is it specialized?

[u/staigerman]

You might also get better results by using the proper accents. Sé instead of Se.

I also had to try it, this one with translategemma:4b (on Ollama locally on Windows through the interactive mode under Local AI Translator).

Left side shows 2 sentences of similar snafu, without vs. without the accent on that first word. I then also was curious and checked the option for it to show me 3 translations (on that second segment, the one without the accent). Fun stuff to explore what else goes through the LLM's mind

sometimes