How to help your developers manually test accessibility with screen readers

Not long after you publish a new page or add a new feature, someone will turn on a screen reader and use it to navigate what you shipped. The million-dollar question is: who will be the first? One of your developers? Or one of your users? Wait for users, and odds are they’ll turn up all kinds of issues. Everyone wins when it’s your team that jumps in first.

Manually testing your product’s accessibility, and fixing the bugs you find along the way, is the only way to build a truly accessible app or site. Automating testing and “shifting left” won’t cover everything.

So whether your organization is working toward compliance, creating a dedicated internal accessibility team, or making their way through W3C’s Accessibility Maturity Model, manual testing is a critical part of the workflow.

It sounds tough when your organization has no prior experience using screen readers. But getting your developers to test accessibility using screen readers doesn’t have to be an uphill battle. Making the transition boils down to four steps:

1. Address misunderstandings about manual testing
2. Make manual testing an expectation (and reward it!)
3. Provide great accessibility training
4. Reduce friction during manual testing

Let’s take a look at each in detail.

Address misunderstandings about manual testing

Plenty of developers are happy to learn a new skill or make their code more accessible. Others may need convincing. In the words of Assistiv Labs co-founder Andrew Hedges, “Developers are so busy that it's hard enough to get them to test in Safari, much less screen readers.”

Developers’ hesitance to test manually may be because:

• They are so busy with their existing tasks, they haven’t had time to think about accessibility testing.
• They may think testing should be someone else’s job (QA, third-party auditor, etc.).
• They may not realize how many people rely on assistive technology.
• They may think accessibility is a nice-to-have addition that can be worked in later, when they’re less busy.
• They may not realize how unusable their software is without accessibility optimization.
• They may be intimidated by the scope and complexity of accessible development.

Perhaps the biggest misconception stems from the idea that you can automate accessibility away. Search for accessibility testing solutions, and you’ll find dozens of tools that promise to find accessibility issues. The reality is that the average automated testing tool tends to miss well over half of the most common accessibility issues.

Addressing misunderstandings that surround manual testing is all about change management. In the axe-con 2022 session Change Management for Accessibility, accessibility-focused customer support manager Heidi Kelly-Gibson said, “When you’re trying to implement a process change – like including accessibility testing in your software delivery life cycle – if you aren’t thinking about the positive and negative ways the group and individuals might see the change affecting them, and you aren’t addressing those concerns, you are risking success in the adoption of change.”

Before you announce manual testing as an expectation, before you decide how it will be rolled out, meet one on one with developers who will be affected. Get their honest, unfiltered opinions about the change. Listen with an open mind, dispel misconceptions (automation isn’t enough, and screen readers aren’t that hard to learn how to use), and adjust your plan based on legitimate concerns.

Make manual accessibility testing an expectation (and reward it!)

Want your developers to evaluate the accessibility of your product with a screen reader before anyone else? Then you need to make it clear that it’s no different than any of their other job requirements.

First, update managers from design, development, and product management teams that accessibility testing will become a required part of their team’s workflow. They need to be in the loop and on board.

After that, announce to each of the teams that manual accessibility testing is a new expectation and schedule a meeting to discuss it.

Start the meeting off on the right foot by focusing on benefits. There will be fewer bugs down the line, better user experiences, faster load times, and, well, it’s just the right thing to do.

Emphasize that manual testing will be included as acceptance criteria for every new feature or page. That means support from designers and PMs as well. Manual testing should even be a competency metric for HR reviews. If your teams access screen readers and other assistive technology with a SaaS solution like Assistiv Labs, you can generate reports on how much time they spend testing to inform performance reviews.

Ideally, recognition shouldn’t be limited to infrequent performance reviews. Deque Systems Program Consultant Katie Olson recommends rewarding initiative in other ways, “When teams or individuals are doing good things in the accessibility space, be sure to celebrate wins. Congratulate them publicly, post it in your company newsletter…celebrate the positive things that are happening with this transformation.”

Beyond recognizing manual testing rockstars, you’re reinforcing the idea that accessibility is a collective goal. A group norm. Humans naturally gravitate toward group norms, we can’t help it. So the more often that developers see accessibility mentioned positively in company meetings, recruitment, procurement, etc. – the more interested they’ll be in “joining the club.”

Provide great accessibility training

Developers need a thorough introduction to accessibility before being trained to evaluate software with screen readers.

“Accessibility can initially feel daunting, especially for those who may not have been exposed to disability growing up,” says Jiaxin Zheng, a program manager at Asana. “By providing a primer on the global disability landscape, digital accessibility terminology, and assistive technology demos, teams feel a lot more grounded in the why.”

Along with their primer, another integral part of Asana’s training is accessibility awareness labs, opportunities for “dev teams to observe how people who are blind or have low vision use a screen reader to navigate the web,” Jiaxin explains. “We’ve found it’s much easier for teams to interpret accessibility issues having been exposed to users firsthand.”

Whether you develop your own training program or rely on external resources (Frontend Masters has a fantastic collection of free and paid accessibility courses), someone inside your organization should be fielding accessibility questions from trainees.

Dylan Barrell author of Agile Accessibility Handbook says, “You’ve probably heard of agile coaches being instrumental to helping teams and organizations adopt agile practices, values, and concepts. We recommend a similar approach with accessibility coaches. The coach will work with the team until they reach certain milestones. And ultimately, become self-sufficient.”

Like agile coaches, accessibility coaches are the ones who “teach your team to fish.” Coaches are not the ones picking up the slack.

For example, Yura Zenevich, a systems engineer at Slack, recommends “recordings of screen reader walkthroughs and design requirements/specifications that describe expectations around screen reader experiences.” Accessibility coaches should work with developers directly whenever possible but asynchronous resources are a great way to get the ball rolling.

Whether it’s you in the role or a volunteer developer with enough experience, the coach will help developers build up team momentum and accessibility inertia. They’ll ensure job postings include accessibility responsibilities and requirements. Maintain manual testing documentation (on GitHub, Notion, etc.). Track the team’s accessibility training levels, milestones, and overall maturity (this Google Sheets template for tracking team training is a good starting place).

Putting training basics in place frees you or your coach up to focus on other, weaker links in the manual testing workflow.

Reduce friction during manual testing

One of the biggest challenges when testing new features and content with screen readers is providing access to the most important assistive technology software. At Slack, Yura says, “The vast majority of our developers are using Apple hardware for engineering. However, most of the accessibility users we have use Windows and screen readers that are distributed on that platform (NVDA, JAWS, etc).”

“If developers want to test their work manually against a configuration that most of our users have, they have to set up an elaborate environment with a VM and an appropriate screen reader as well as deal with potential security requirements of being able to test production code in the VM.” That adds a ton of operational friction.

One alternative is to test your app or site with in-browser access to Windows screen readers, something Jiaxin says Asana’s team did that “significantly cut down the set up cost for jumping into testing. Especially after implementing single sign-on, any developer can now reach out to our IT team and get access to assistive technology within a few hours instead of a few days or weeks.”

What else can you do to reduce manual accessibility testing friction?

• Help developers understand the differences in functionality and UX between JAWS, NVDA, and VoiceOver.
• Create keyboard shortcut cheat sheets for each screen reader (i.e., set up a Slackbot that shares shortcuts when someone types in “JAWS keys”).
• Provide thorough user stories and acceptance criteria (like these templated acceptance criteria from T-Mobile) to guide developers through testing.
• Make sure your design team provides wireframes with accessibility annotations to developers.

The screen readers your developers need to manually test with are mostly set in stone. But how you make those tools available and support the people using them has an enormous impact on testing quality.

This is all you need to get developers testing manually

You could do a million things to improve your product’s accessibility. But all you have to do today is start a dialog with developers about making manual accessibility testing an expectation, begin outlining a training program, and make screen readers as easy to access as possible. These are critical steps towards making your software and sites accessible for everyone.

Get your team testing accessibility on NVDA, JAWS, Narrator, and other essential tools from inside a browser. No installs and minimum configuration necessary. Sign up for a free trial of Assistiv Labs’ tools and start testing today.