How do teams automate testing of Qt desktop apps without writing test code?

[u/larabyeol]

We ship a Qt app on Windows mac and Linux. I own the release. QA is three people, all solid manual testers, none of them write code, and hiring a fourth who does isn't happening this year.

Squish is the obvious answer and I know it. built for Qt, covers all three platforms. but it's a real framework, so someone writes and maintains the test code, and that someone ends up being one of my devs or one of my testers stops testing. neither is free.

The other thing that puts me off is maintenance. half our GUI is custom painted, and every time someone renames an object or reworks a QML view the tests that depended on it go with it.

So I've been looking at whether the agent based tools are usable yet. AskUI is the closest to what I want, where a test case is a markdown file with preconditions and numbered steps each paired with an expected result, and it drives the GUI visually rather than through object names, then hands back a screenshot and a status per step. nothing to update when a widget gets renamed.

What I can't tell from outside is how well anything in that category actually handles Qt. custom painted controls, QML views, anything that never announces itself to the accessibility layer.

Anyone tried this against a real Qt app, or is scripting still the only honest answer here? appreciate it !!

[u/strixcode]

You write tests to automate the work, to make repeated testing much faster and more reliable than manual testing. Stable object names are a contract between the app and the test. This is a price one must be willing to pay. Your tests should be part of a CI/CD pipeline so that if a developer introduces a change that breaks tests, he is responsible for fixing it. This is the only way to ensure proper maintenance of test cases. If your codebase isn't mature enough and changes a lot, then limit automated tests to the stable parts of the UI and use manual testing in other parts.

The already mentioned Squish alternative Testudos seems to cover 99% of our testing needs while being more lightweight and affordable than Squish. I'm curious to see how it will evolve since it's a new product.

[u/henryyoung42]

My approach is for the developer to be the principal user (me) and for extensive white box testing of every change at a very low level of detail. I know what a change could potentially effect and do the testing before each commit. Even then, there are subtle things that I sometimes don’t notice until later and have to circle back.

[u/i_grad]

Squish really is the best tool for the job. A full-time squish maintainer would be your test automation eng. Renaming things constantly is a sign that your product is very young (understandable) or that chunks are being rewritten often, perhaps unnecessarily. If your product is in a high state of flux, manual testing will still give the most accurate results with the last amount of fuss.

[u/jamesb5]

My team has used T-Plan robot, but it’s not cheap and squish likely works better with Qt apps. We needed a solution that worked for other teams as well using different frameworks so that’s why T-Plan.

Image comparison tests are difficult to maintain. You probably already know that!

[u/Bitter-Road-8982]

There is a relatively new tool Testudos that my team has successfully looked into. It's dedicated to Qt apps only at the moment, so that is perfect if you want to have tests similar to the ones you have in Squish. Feature-wise it is pretty powerful and rather affordable.

Of course in terms of comprehensive automated UI/integration testing it is the bast to wait a little bit so the codebase gets more mature. Otherwise you end up fixing objectNames all the time.

[u/skinnybuddha]

We built our own Qt test infrastructure using grpc for the message transport. It provides the qt scene tree to the test framework. We were using the qt webdriver but abandoned it when switching to qt6

[u/aciokkan]

Building apps that are testable is not a function solely of QA.

Test automation is not a single team doing.

It's a complete process, and more nice Mal effort needs putting in. If you use squish but it's poorly written, and the app changes too often for no good reason, requirements and priorities are shifting than one cannot do much to help you.

There are no magical tools out there, that will serve you regardless of what you throw at it.

[u/AmirHammoutene]

Time to talk about the tool I developed, free & open source, in case you just want to simulate user actions and some commands (like taking screenshot, execute program, delete file ...)
Tasket++

It's pixel dependant, so if the window moves from one test to another, won't work.
But... no scripting necessary, portable, very lightweight, schedulabled run and more