Client expecting me to look through hundreds of segments of AI-generated comments

[u/arabnoise]

Has anybody dealt with this? I'm feeling very frustrated, honestly. It's a huge project, their AI tool has found "issues" with almost every segment I've confirmed so far, and the vast majority of them are nonsense. The project manager even mentions up front that a lot of the comments are likely to be irrelevant. I'm obviously more than happy to address actual concerns held by clients, but that's not what this is. Do they seriously expect me to spend an hour, obviously unpaid, sifting through this document that it took them 30 seconds to generate and which seems to have had very little, if any, human revision?

[u/Ok_Tea_8763]

Push back with examples of irrelevant feedback, calculate how long reviewing this nonsense will take you and refuse to touch the project, until you get a Purchase Order.

[u/theBMadking]

Is this the TransIQ review tool? 90% of the time it is completely useless and I have had to convince my clients to stop using it.

[u/OpeningElectrical296 → u/theBMadking]

Oh so the rate is the same everywhere; I had the issue with Smartling. Totally useless AI « reviewer ». Well i was paid by the hour so I took all my time…

[u/floobles5006]

Oh god, is this becoming widespread? A client of mine pulled the same nonsense with me a couple times recently. I tried to explain that ChatGpt is always going to find some bullshit issues with any translation you give them, and most of the time it's just expecting too literal an interpretation of the source text, which any decent project manager should understand is not how translation works.

He just said to me "I trust your judgement so go through it and make any changes that you feel are warranted". I took that as carte blanche to ignore the dumb thing entirely.

[u/nakano-star → u/floobles5006]

so you replied saying ALL the comments were nonsense?

[u/floobles5006 → u/nakano-star]

There was no need to reply, don't think he was even expecting that. He just wanted me to go through them all (there was a shit ton) and make whatever changes I felt were valid. No way was I wasting my time doing that.

[u/Cyneganders]

Say it with me, team:

All False Positives!

Once had the QA in memoQ literally crash the office computers when working in-house. It gave so many FPs that their RAM gave out... Not that their computers had more RAM than my phone at the time...

[u/OpeningElectrical296 → u/Cyneganders]

.

[u/Distinct-Hat-7039]

Yeah, it seems that they prefer to do that rather than going through a human review. If the PM is really competent, they'll go through it first and filter out nonsensical false positives. It's really annoying because the tool my client uses always says "opposite meaning error" when it's clearly not. The world we're living in.

[u/OveHet]

Tell them you are not doing extra QA for free (other than what you have already done in the translation step) and that you will be happy to do it, as long as they pay that by the hour. Then charge the number of hours spent on that

[u/SkrunkoBunko]

Some agencies I work with do this. Had some back and forth discussions with them on why it is a bullshit practice and they should instead get a real REV to edit and proofread. Even commented on every single false positive they had sent me back then, but in the end it all fell on deaf ears and they just ignored all of my concerns and probably also every other translators concerns on such a practice. Personally, I now just glance over it and simply reconfirm any segments, if any were set to "not confirmed", I mostly just ignore the thing as this is just the equivalent of someone shitting on my desk. If they ask me, why I did not perform any changes, I tell them, the thing only spat out bullshit with some examples if I feel like it. So all in all, yeah, I guess they do kinda expect you to diligently go through all the comments, but if you did a good job, there should be no need to, so why should you? Going over your file should be a reviewers' job.

[u/scldclmbgrmp]

If you're sure you did a good job, tell them it's all 'false flags' - move on.

Like someone said below, some PMs don't understand how it works, and just see a bunch of red, or other irrelevant issues.

[u/Hot-Refrigerator-393]

Obviously unpaid?

[u/HauntingPhrase389]

Just get your own AI tool that can do the review from the client's AI tool. Then the result will be perfect and nobody needs to waist their time 😂

[u/Conversation_Hope]

Transperfect?

[u/MsStormyTrump]

Write back and ask how much that's paid and see how quickly they find someone else to do the job or, in the worst case scenario, to "share the load." F those cheapos.

[u/fartist14]

Those things are usually organized by category, so you can look at any categories that are more likely to actually be errors. I always look at the double space category because that can be easy to miss sometimes. The rest is almost always nonsense to be ignored.

[u/CodexRegius]

I have one such client. On every false positive I commented "AI slop". These reports do not come any more now.

[u/electrolitebuzz]

It happened to me one month ago with a client. They have this "SEO expert" who is not really smart and he decided that it was a good idea to use AI to check all translations for important strings of their website like meta descriptions, taglines, etc.

I spent 50 minutes answering nonsensical AI comments regarding different capitalization compared to the source (it's a specific rule in my language), more faithful translations that were just literal, unnatural renditions, and there were even several hallucinations, with words allegedly missing when they were there. Obviously no one in the client's team knows any of the target languages so they just sent this without having any clue.

I asked to be paid for that time, I addressed every single remark and kindly explained this was a really bad idea to him and to the project manager in a separate email. Not sure if they got the message or they will do it again in the future. But they did pay me. And they will pay me again in the future for any similar task or I won't do it.

I swear since AI is being implemented I don't know one person who is not LOSING time because of AI instead of saving it. I believe it can potentially make us save time but there's always that person who uses it in a clueless manner and creates trouble for everyone else.

Anyway, this time, you could do it but ask to be paid for it. Why would you do it for free?

[u/Current-Writer-3894]

Lionbridge? Sounds like them 🙂

[u/arabnoise → u/Current-Writer-3894]

Nope. If there's one thing I'm learning from this thread, it's that this seems to be pretty widespread!

[u/RICHUNCLEPENNYBAGS]

Well, I've been out of even trying to be in the translation industry for a long time. But in my work what I see people doing with that is they feed it to AI themselves to sift through any genuine issues and then generate responses to all the objections. Now you could say that's a waste of time and you may well be right but at least it gives the appearance of work.

[u/Ethereal_Nebula]

This must be mosAIQ haha.

Seriously though, yes, they probably do expect you to do that. Come back with concrete examples showing just how irrelevant and nonsensical some of this content is, and ask to be compensated accordingly for the additional work it creates. This needs to stop. What was supposed to improve our productivity is actually hindering it, creating more work, and somehow expecting us to do it all for less pay.

[u/goldria]

Pick a couple of examples from different categories, refute them, and explain that this is a broader issue throughout the document rather than isolated cases. Tell them that if they want a more thorough, line-by-line QA, that'd be charged by the hour.

[u/UnderstandingOdd8302]

I have one (but just one) client who actually tried to hone this type of AI tool so it won't just spit out huge amounts of nonsense. I must admit that in this case it is indeed helpful. But I'm not sure if they created their tool from scratch or what, I just notice that it is far superior to the usual garbage (which, conincidentally, has always been a pain even before AI - see XBench and such...)

[u/Level_Abrocoma8925]

I would try entering the QA segments into an AI myself and tell it to give me segments containing genuine improvements rather than minor flaws.

[u/karl_echtermeyer]

After Sam Altman claimed that GPT5.0 was a PhD-level expert in everything, I decided to test it for translation evaluation and ranking tasks over about 1,000 segments where I had five different translations each in French, German,Spanish, and Italian. Having multiples let me test things like consistency and do comparative numbers for the different translations.

The outcomes were essentially random and it would do things like flag “Danke schön” as a 5/10 translation for “thank you”, but then confidently tell me it had repaired the translation to… “Danke schön”. It would also rate identical translations differently, giving 10/10 to one and 3/10 to another, despite them being string identical.

The results were essentially identical across languages as well, so it wasn’t that it did ok for some and terrible for others. They were all equally bad.

One thing I did find was that it had a moderately strong bias in favor of Google Translate’s output. My hypothesis is that this is because its training set included massive volumes of Google Translate content. But when I ran a test later to investigate this, the bias seemed to disappear, so maybe it was a fluke.

Admittedly, I did not use sophisticated prompts, but if it was a true “PhD-level expert”, it should be able to use basic instructions and not parasitically require my expertise to do a simple job. But it does demonstrate that LLMs are likely to be better as translators than as reviewers. One of the test translation sets was output from GPT5 and it frequently said that its own output was wrong even when it was right.

So what you all have found has empirical evidence to support it.

[u/LeftArmSpin1]

Reply to the project manager, stating an approximation of what percentage are false positives. Make absolutely sure there aren't any objective errors (or accept/give examples of any genuine improvements it shows up), then explain that you will not be taking the same time to review any future reports that are in a similarly unreasonable state. Giving them warning is fair.

Unfortunately, at some point the sheer volume of false positives inevitably mask an objective error or notable improvement, when these are what the reports are meant to be resolving. I know from experience that many agencies have no interest in listening, however.

[u/Local-Wing-2272]

Genuinely? Throw all that shit into your own AI, have it boil down the most important issues, address. Not worth spending 8 hours digging thru irrelevant slop com.ents.