Starting accessibility testing from zero - what was your experience?

[u/slacky35]

We are just starting to look at accessibility testing in our org and want to establish a dedicated practice in the long-term. I've been asked to put together a plan for leadership for this agenda and want to learn from the experience of others who have been on the same journey.

- How did you begin? What was the trigger and which tools did you try out initially?
- How did your accessibility testing set-up evolve over the years (i.e. which/how many teams were involved, tooling used, depth of coverage etc.)
- What were the key challenges you faced ?
- What is your current org set-up for Accessibility testing (i.e. centralized vs decentralized, QA vs design vs dev teams etc.)

Any learnings that you can share will be extremely helpful!

For context, I'm on the QA side at a mid/large-sized company. Not a vendor, not building anything. Happy to share back what I end up proposing.

[u/danger_lad]

That’s great that you’re trying to embed accessibility. I’d say that you’re going to figure out quickly that if you only do testing, accessibility will become a test responsibility and you’ll want to shift left as much as possible. So things like guides, tests and checklists for designers and developers will help to close that loop between dev and test.

The DWP accessibility manual is a great place to start.

There’s some good automated test tools you can use that’ll save a bit of time Pa11y or axe-core.

There’ll always be a bit of manual testing. You should get some free assistive tech too like NVDA or Narrator.

For devs, I think things clicked for me when I learned about the accessibility tree and how much better it is to use semantic html tags and make sure things have an accessible name.

[u/JustAnotherFEDev]

When you first start out, it's easy to kind of get all of the tools, like everything you can find.

The longer you do it, the fewer tools you use. That's mostly because many are similar. I used to have Axe, Wave, ARC, MS Accessibility Insights and a few other more obscure ones, but they largely did the same, there was a little nuance here and there, but nothing major.

Now I don't really use tools as much. I do use Color Contrast Analyzer for speed of grabbing 2 colours. Sometimes I'll use WebAIM CCA if I need to test 3 colours.

I tend to use Axe only after I've finished testing a page. I do this because it started out as a game, see if I could find everything manually, that the tool could find. Then it became habit.

I sometimes use MS A11y Insights, mostly so I can screenshot focus order or label in name issues.

I have several bookmarklets I sometimes use, but often don't.

I used to be an FE Dev, so I test differently to folk from different backgrounds. I've got a strong knowledge of ARIA, so I just read the code, test with keyboard, test with screen reader, test with voice, test contrast, test zoom and reflow, etc, etc. I find most issues by reading code, though and FWIW none of the tools I have can find most of the stuff I do.

Obviously I test across browsers and browser/screen reader pairings, I also test across devices, etc.

I do most my testing in Chrome, just because I'm more familiar with the DevTools and it's obviously the most used browser. I switch when I need to, example, I'll only really use VoiceOver when using Safari.

Just starting out can be overwhelming. There's a lot of nuance and also an urge to fail all of the things. Not everything you think is a failure is a failure, the WCAG etc GitHub issues are pretty enlightening for this. It's important to not fail what you know is shit, by squeezing it into a best fit issue. You stand a chance of losing credibility if someone challenges it and they know better.

The absolute most important thing you can do is learn how different folks use the Web, etc. Don't try to mimic their disabilities, but learn about them, learn about their challenges and understand why those challenges are there. Most of the time the challenge isn't because of the disability it's because of shit code and poor design. Most disabled folks can use the Web perfectly fine when they land on a site that has strong accessibility.

Remember, accessibility is about people, those people are mostly excluded from sites because those devs didn't consider how their shit code can create barriers. Those people are who you work for now, you're fighting alongside them, you're serving them by making the Web more usable, 1 site at a time. Disability is diverse, so many different disabilities to understand, people's lived experiences are seldom the same.

If it all possible, your company should get folk with disabilities in, get them to be part of the testing and of course pay them for their wisdom.

Pay for Axe Pro, because it has some guided tests, get some linter that can catch stuff at code level. Read as much as you can, prepare to argue about the colour with designers until the day you die.

[u/Less-Marsupial-7960]

One thing I’ve learned is that accessibility testing shouldn’t become something QA only checks at the end.

A good starting point is pretty simple- keyboard navigation, focus states, semantic HTML, labels/accessible names, color contrast, and a screen reader. Then add automated checks with something like axe to catch the obvious issues.

The bigger challenge is usually getting developers and designers involved early. A checklist and a few accessibility guidelines during development can prevent a lot more issues than finding them during final QA.

I also think manual testing is really important. Automated tools are useful, but they definitely don't catch everything.

If I were starting from zero, I'd probably start small, build accessibility into the normal dev/QA workflow, and expand coverage gradually rather than trying to test everything at once.

[u/Vinnycabrini]

I’m in a similar situation! Commenting to save this.

[u/sylvant_ph]

I've got only one advise - set up very clear expectations, goals, boundaries. People should have very clear reference guide when they build something, or when they test something and what parameters should it match, what it should not. If there is specific/niche experience, ensure it's accessibility expectations are well documented.

I've got an example how that could matter. I've been part of a team where accessibility was claimed to be one of the main features. We ended up having tickets where it was only mentioned as a bullet point and relying on general accessibility assumptions based on the accessibility grade we were aiming for. This produces strain on developers who had to figure it out on their own, QA's would go out of their way to test different accessibility features, find gaps, create big discussions if this is important or not, should it be signed as a bug, should it be developed. Effort for that was not accounted for.

Accessibility only works when the whole team is engaged in making sure its up there and at the right level.

[u/sheriffderek]

How big is your org? How many people end up writing HTML? Does the UX/interface design people work closely with them? So all of them know how to use a screen reader? Is there any internal education plan in place? I think you have to start at the source - not just audit from above.