Discover how usability tests with users who have disabilities can enhance your product’s accessibility, going beyond WCAG compliance. Our post provides friendly, practical tips for creating more inclusive experiences. Dive in to learn more!
When people talk about testing accessibility, they often talk about accessibility audits and compliance with WCAG.
However, compliance is not enough to guarantee a pleasant experience for users with disabilities. To find the users’ real struggles, it’s essential to conduct usability tests and observe them interact with your product.
Don't know how to start? We have summarized all you need to know in this article!
Users with disabilities are users like everyone else! So, a usability test with people with disabilities is precisely like a standard usability test.
This blog post is not about a different type of test but just some tips and tricks for focusing the test on accessibility and guaranteeing a better experience for everyone involved.
If you don’t know how to conduct a usability test, at Buildo we are fans of the Steve Krug methodology, which is well explained in his book Rocket Surgery Made Easy!
There are two things to test during a usability test for accessibility: usability and accessibility.
When testing usability, recruiting people from your product's target audience is crucial. This will provide invaluable insights into how your target audience perceives your product and help identify specific problems they may encounter.
To test the accessibility, it’s important to involve people with different disabilities and limitations, especially people who use assistive technologies.
People who fit both categories are the best candidates for your test. However, they are not always easy to find, so just include people from both categories, even if they match just one.
These are the most common users to be involved in accessibility tests:
The number of people involved depends on your budget, but we strongly suggest including at least one user who uses a screen reader and a user who uses an assistive keyboard. Choose the others participants based on your target audience. Assistive technology users are the ones who usually face more challenges using digital products.
Finding the right people can be difficult. Some recruiting companies offer the option of including disabilities as participant requirements. If you recruit your participants yourself, a good suggestion is to contact disability support associations and ask for help. And often, a participant can refer you to other good fits.
For a usability test, we usually prepare:
There are numerous resources available on how to conduct a usability test. Therefore, we will focus solely on providing accessibility tips.
In the introductive questions, don’t focus only on the participant's disability. It’s important to know precisely what their struggles are and what technology they use to navigate. Still, it’s also important to understand their profession, their digital level, their experience with competitors, and so on. Feel free to ask all the questions typically included in standard usability tests.
During a test with assistive technologies, issues may be impossible to solve. For this reason, instead of writing a few long scenarios, write a lot of short scenarios. Short scenarios give you the flexibility to change tasks if one is impossible to finish.
Try to complete all the scenarios, once using only the keyboard and once using a screen reader. It is better to test various scenarios to determine their feasibility beforehand. Additionally, understanding how assistive technologies function is important for comprehending the user experience during the testing phase.
Using assistive technology can be exhausting, especially if the tested product is not accessible. Don’t plan activities for more than an hour, and be ready to cut off the session in advance if you see your participant frustrated.
Remember to design the test expecting your users to need more time to complete the scenarios than you think. But consider also that your users can surprise you. They are more used than you to use assistive technology. So, have some backup scenarios in case they are faster than expected.
Check that everything you need to share with your participants is accessible, too. For example, if you send a consent module to record the session in advance, check that the PDF is accessible. Be sure to plan in advance how to show the scenarios to your participants.
In our experience, reading the scenarios aloud may be easier for blind participants. Older individuals often prefer printed scenarios with a large font size. Those with paralysis are unable to handle a printed sheet, so they prefer a digital document. For individuals with dyslexia, it is recommended to use a font specifically designed for their needs.
Once your test is ready to be executed, it’s time to set up the session!
Some people with disabilities can have issues traveling. Always ask if they prefer to have the session at their house or in a place they know (for example, the location of the association that put you in contact).
If the session will take place in your office, ask the participants if they need help reaching it. Also, check carefully that your office doesn’t have mobility barriers that can prevent a participant in a wheelchair from reaching the room for the test.
People are more confident in using their own devices, and this is particularly true for people who use assistive technology and for older people. So, ask your participants to use their own devices. This will allow you to see their common setup and struggles.
Attention! When blind people use a smartphone, they often set it up so that nothing shows on the screen. This means that you won’t be able to see where they are on the page and can’t help them. If the application or website being tested doesn't have excellent accessibility and you are not thoroughly familiar with it, it’s better to use a computer.
If you use a tool to transcribe the audio after the test, bring a good microphone. Some of your participants may have pronunciation problems or low voices, which can impact the accuracy of the transcript.
In particular, people who use a screen reader often talk over the screen reader's voice. A good microphone near the person can reduce the audio overlapping.
Remember to alert the user when you start recording!
It’s always better to conduct accessibility tests in person. If it’s not possible and you need to conduct the test remotely, verify that the participants have experience with video calls or have someone who can help them, especially older people.
Ask them about their preferred video tool. Zoom and Microsoft Teams have better accessibility features than, for example, Google Meet. But even here, working in an environment known by the participant is more important.
During the session, the observer's primary focus should be collecting insight. Instead, the facilitator's primary focus should be the participant's wellness.
Using assistive technology often is tiring, even if the product is really accessible. Reassure your participants that they can interrupt the session when they want, and be careful about their level of frustration.
If you encounter a very frustrating scenario, consider skipping other scenarios that you know are difficult with assistive technology. In this case, having a lot of short scenarios helps.
The standard technique in usability tests is to ask the participants to think aloud. Talking while executing the tasks can be challenging for some people, for example, if they use a screen reader or have reading or cognitive issues.
If the participants don’t talk during the task, don’t force them, but ask questions at the end of the task. Even in this case, short scenarios help.
As said before, the audio transcript can be sub-optimal in accessibility tests. So, consider taking more time to analyze the sessions.
When implementing a tagging system for your product, it is recommended to categorize feedback into two main categories: accessibility and usability. This division allows for a more focused approach when addressing specific issues. Additionally, consider creating tags that are tailored to different assistive technologies. This will help in efficiently organizing and addressing feedback related to specific tools and technologies used for accessibility.
Expect to dedicate some time during your analysis phase to investigating the bugs that emerged from the test. It’s an added value to include not only the issues encountered but also the technical reasons behind them in your report.
Testing accessibility with users can be challenging at first, but it's essential to transition from a compliant product to one that is truly accessible and inclusive. By conducting usability tests with a diverse range of participants, we can identify and address issues that may not be captured by standards alone. This approach yields valuable insights that can elevate good design into exceptional experiences for all users.
Sara is a designer at Buildo, specializing in User Research and UX. Her passion for research enables her to gain a deep understanding of users' needs and translate them into intuitive and user-friendly designs.
Are you searching for a reliable partner to develop your tailor-made software solution? We'd love to chat with you and learn more about your project.
