A11yJam Roundtable Report: Inclusive & Ethical Usability Testing
Usability testing offers the opportunity to observe and learn from real users, enabling governments to iterate design to meet user needs. In this session Rua Williams, Assistant Professor in the School of Applied and Creative Computing from Purdue University, shared more about their research and practical steps government practitioners can take to conduct more inclusive and ethical usability testing. Attendees discussed partnering with disabled users in ways that are respectful, effective, and sustainable.
Inclusive & Ethical Usability Testing
On April 15, 2026, the Beeck Center hosted Accessibility Jam, which featured four interactive roundtable discussions. This is an anonymized report out of the discussion, showcasing key insights, questions, and topics.
Show & Tell Demo Video
Key Roundtable Insights
- Usability testing alone comes too late
By the time a system reaches participant testing, most of the decisions that determine whether it excludes people have already been made. Relevant users need to be involved earlier and in framing the problem itself.
- Standard research criteria quietly screens out disabled people
Common inclusion/exclusion rules (e.g., requiring corrected vision, verbal communication, or keyboard-and-mouse use) can unintentionally exclude many people with disabilities from the very testing meant to serve them.
- Cognitive access is chronically underprioritized
Accessibility conversations still focus mostly on plain language, but cognitive access also depends on how information is organized and presented. This oversight disproportionately affects people navigating government forms and social services.
- Commitment to testing outpaces the resources behind it
Usability testing is widely described as the gold standard, but there is usually little to no budget, training, or dedicated time to actually carry it out. This shortfall shows up most clearly in recruiting: restrictions on incentives and low awareness among leadership about the need to pay for specialized expertise make it harder to recruit and compensate disabled participants, even when agencies want to.
Roundtable Questions & Summary Answers
How are you empowered to do this testing in your day to day work?
- Using third-party accessibility research platforms, such as Fable Testing, to reach end users with disabilities
- State-provided training on user-centered design, paired with a dedicated product team focused on user needs
- Embedding community engagement (e.g., user interviews, focus groups) as a core part of the agency’s management system
- Usability testing is treated as the “gold standard” in name only, without the budget, training, or support to actually carry it out
- Partnering directly with disabled staff, such as working with a blind colleague in another agency, to test with a screen reader has helped.
Resources & Examples
How to conduct a usability test
Usability tests can help teams develop products that are user-centered, accessible, and inclusive. This guide will help you conduct a successful usability test, from coordinating with participants to analyzing your findings.
Usability Starter Kit
The DigitalGov Usability Starter Kit offers a comprehensive collection of tools and templates designed to assist in creating user-centered digital experiences.
What are your experiences with inclusive and ethical usability testing?
- Accessibility is treated as a design priority with technical implementation remaining difficult.
- Neurodivergent internship programs create an ongoing channel for working alongside and soliciting feedback from disabled staff.
- Partnering with existing disability networks, local disability centers, and referral relationships (e.g., VA offices) to reach participants.
Resources & Examples
User Experience Toolkit
City of Boston’s User Experience Toolkit to support City employees in learning how to execute a UX project.
Usability testing kit
A collection of guidelines and templates to help you check the usability of election materials
What has made recruiting disabled participants easier or harder for your team?
- Limited budget often forces reliance on commercial usability testing platforms, which can still be costly and have questionable recruitment quality.
- Compensation is a recurring obstacle. In many agencies, incentives aren’t permitted and leadership doesn’t always understand the need to pay disabled participants for specialized expertise. Some agencies have tried to work around this by using gift cards in place of monetary compensation.
- Since many disabled users don’t identify as disabled, both internal and public-facing recruitment run into self-identification barriers.
- Limited visibility into testing opportunities may limit participation.
- Building an ongoing recruitment pool through partnerships with nonprofits and state agencies is a promising alternative to one-off recruitment.
Resources & Examples
CivicActions’ Accessibility Site
This open-source guide and playbook offer practical strategies, tools, and best practices for government employees and contractors to improve digital accessibility and inclusion.
Embedding Disability Equity into Efforts to Advance Upward Mobility
This article emphasizes the need for local leaders to prioritize disability equity in advancing upward mobility, addressing systemic barriers that hinder disabled individuals’ escape from poverty.
Where or how did user testing shift your assumptions about accessibility?
Repeated user resting with the same participants can give staff a deeper understanding of how inaccessible design creates real barriers.
Resources & Examples
Seattle Executive Order 2025-03: Making Information Inclusive in Writing and Design
This Executive Order mandates citywide actions to make government information more inclusive, accessible, and understandable through digital accessibility and plain language initiatives.
Hennepin County Writing Guide
Hennepin County’s (MN) writing guide for County employees.
