A11yBoard Seeks to Make Digital Artboards Accessible to Blind and Low-Vision Users

Just about everybody in business, education, and artistic settings needs to use presentation software like Microsoft PowerPoint, Google Slides, and Adobe Illustrator. These tools use artboards to hold objects such as text, shapes, images, and diagrams. But for blind and low vision (BLV) people, using such software adds a new level of challenge beyond keeping our bullet points short and images meaningful. They experience:

High added cognitive load
Difficulty determining relationships between objects
Uncertainty if an operation has been successful

Screen readers, which were built for 1-D text information, don’t handle 2-D information spaces like artboards well.

For example, NVDA and Windows Narrator would only report artboard objects in their Z-order – regardless of where those objects are located or whether they are visually overlapping – and only report its shape name without any other useful information.

From A11yBoard video: still image of an artboard with different shapes and the unhelpful NVDA & Windows Narrator explanation as text.

To address these challenges Zhuohao (Jerry) Zhang, a CREATE Ph.D. student advised by Jacob O. Wobbrock at the ACE Lab, asked:

Can digital artboards in presentation software be made accessible for blind and low-vision users to read and edit on their own?

Can we design interaction techniques to deliver rich 2-D information to screen reader users?

The answer is yes!

They developed a multidevice, multimodal interaction system – A11yBoard – to mirror the desktop’s canvas on a mobile touchscreen device, and enabled rapid finger-driven screen reading via touch, gesture, and speech.

Blind and low-vision users can explore the artboard by using a “reading finger” to move across objects and receive audio tone feedback. They can also use a second finger to “split-tap” on the screen to receive detailed information and select this object for further interactions.

From A11yBoard video: still image showing touch and gesture combos that help blind and low vision users lay out images and text.

“Walkie-talkie mode,” when turned on by dwelling a finger on the screen like turning on a switch, lets users “talk” to the application.

Users can therefore access tons of details and properties of objects and their relationships. For example, they can ask for a number of closest objects to understand what objects are near to explore. As for some operations that are not easily manipulable using touch, gesture, and speech, we also designed an intelligent keyboard search interface to let blind and low-vision users perform all object-related tasks possible.

Through a series of evaluations with blind users, A11yBoard was shown to provide intuitive spatial reasoning, multimodal access to objects’ properties and relationships, and eyes-free reading and editing experience of 2-D objects.

Currently, much digital content has been made accessible for blind and low-vision people to read and “digest.” But few technologies have been introduced to make the creation process accessible to them so that blind and low-vision users can create visual content on their own. With A11yBoard, we have gained a step towards a bigger goal – to make heavily visual-based content creation accessible to blind and low-vision people.

Paper author Zhuohao (Jerry) Zhang is a second-year Ph.D. student at the UW iSchool. His work in HCI and accessibility focuses on designing assistive technologies for blind and low-vision people. Zhang has published and presented at CHI, UIST, and ASSETS conferences, receiving a CHI best paper honorable mention award, a UIST best poster honorable mention award, and a CHI Student Research Competition Winner, and featured by Microsoft New Future of Work Report 2022. He is advised by CREATE Co-Director Jacob O. Wobbrock.

Zhuohao (Jerry) Zhang standing in front of a poster, wearing a black sweater and a pair of black glasses, smiling.

Zhang’s dissertation: A11yBoard: Making Digital Artboards Accessible to Blind and Low-Vision Users
Complete Introducing A11yBoard video:

Postdoc Research Spotlight: Making Biosignal Interfaces Accessible

The machines and devices we use every day – for example, touch screens, gas pedals, and computer track pads – interpret our actions and intentions via sensors. But these sensors are designed based on assumptions about our height, strength, dexterity, and abilities. When they aim for the average person (who does not actually exist), they aren’t usable or accessible.

CREATE Post-doctoral student Momona Yamagami seeks to integrate personalization and customization into sensor design and the resulting algorithms baked into the products we use. Her research has shown that biosignal interfaces that use electromyography sensors, accelerometers, and other biosignals as inputs provide promise to improve accessibility for people with disabilities.

In a recent presentation of her research as a CREATE postdoctoral scholar, she emphasizes that generalized models that are not personalized to the individual’s abilities, body sizes, and skin tones may not perform well.

Momona Yamagami presenting her biosignal research, with a slide noting that biosignals fluctuate and are higher on the neural circuitry. — Momona Yamagami presenting her biosignal research, with a slide noting that biosignals fluctuate and are higher on the neural circuitry and a smartwatch as an “always on” sensors for continuous health monitoring.

Individualized interfaces that are personalized to the individual and their abilities could significantly enhance accessibility. Continuous (i.e., 2-dimensional trajectory-tracking) and discrete (i.e., gesture) electromyography (EMG) interfaces can be personalized to the individual:

For the continuous task, we used methods from game theory to iteratively optimize a linear model that mapped EMG input to cursor position.
For the discrete task, we developed a dataset of participants with and without disabilities performing gestures that are accessible to them.
As biosignal interfaces become more commonly available, it is important to ensure that such interfaces have high performance across a wide spectrum of users.

Momona Yamagami is completing her time as a CREATE postdoctoral scholar, advised by CREATE Co-director Jennifer Mankoff. Starting summer 2023, Yamagami will be an Assistant Professor at Rice University Electrical & Computer Engineering as part of the Digital Health Initiative.

Accessible eSports Showcase 2023: Event Recap

In April 2023, CREATE hosted its first ever Accessible eSports Showcase event, bringing together members of the CREATE community, local community organizations, tech and games Corporate Partners, and folks from all over the Seattle area looking to learn about and celebrate ongoing strides being made in making video games more inclusive and accessible to people with disabilities.

Zillow Commons in the Bill & Melinda Gates Center was transformed into a gamer’s playground with big-screen projections of racing and party games, a VR space, and stations where users could customize their own adaptive gaming tech.

UW CREATE Presents: Accessible eSports Showcase 2023 with a colorful digital background.

CREATE’s Community Partners had showcase tables, demoing the latest advances in accessible gaming technology. And UW graduate students, undergraduates, and postdocs highlighted the many creative ways they’ve worked to make games accessible:

Event co-organizers Jesse Martinez (Ph.D. student, CSE) and Momona Yamagami (Postdoc, UW CREATE) opened with an overview of the many accommodations and community access norms they established for the event.
Emma McDonnell (Ph.D. Student, HCDE) live-narrated a round of Jackbox Games’s Fibbage, followed by a competitive mixed-ability showdown in the Xbox racing game DiRT 5, in which Martinez, taking his turn as emcee/color commentator, highlighted the many techniques being used to make Xbox gameplay accessible.
Rachel Franz (Ph.D. Student, iSchool) let attendees try out her latest work in accessible VR research.
Jerry Cao (Ph.D. Student, CSE) showed attendees how to use custom 3D-printed input devices for computer accessibility.
A brilliant team of undergraduates from HuskyADAPT, including Mia Hoffman, Neha Arunkumar, Vivian Tu, Spencer Madrid, Simar Khanuja, Laura Oliveira, Selim Saridede, Noah Shalby, and Veronika Pon, demoed three fantastic projects working to bring improved switch access to video games.

Momona Yamagami and Jesse Martinez open the Accessible eSports Showcase in front of a large screen with a dedicated screen showing a sign language interpreter.

Corporate and Community Partners connected with the CREATE community and engage directly with our many attendees.

Solomon Romney, of Microsoft’s Inclusive Tech Lab, showcased the brilliant design of the Xbox Adaptive Controller (XAC), the state-of-the-art tool in accessible controller design, and guided attendees through setting up and playing with their own XACs.
Amber Preston of Seattle Adaptive Sports described the work SAS does to make all sorts of games and recreational activities more accessible and inclusive in the Seattle area.
Other corporate and community partners, including researchers from Meta, Google, and Apple, were on hand to meet and connect with attendees around other exciting developments in the accessible gaming space.

Three student members at the HuskyADAPT table, sharing information and video about the program.

The Seattle Adaptive Sports table, with the different size balls used in games and a screen showing video of disabled athletes playing.

The organizers thank all attendees, partners, volunteers, and organizers for making the event such a success! As gaming accessibility continues to blossom, we’re looking forward to doing more events like in the future – we hope to see you at the next one!

Pre-event announcement

Who should attend?

Anyone is welcome to attend this event! In particular, we extend the invitation to anyone who has an interest in video game accessibility, who works in the games industry, or who is a member of the Seattle-area disability community.

More information about the event will be available here soon! In the meantime, if you have any questions, please feel free to reach out to our event co-organizer Jesse Martinez at jessejm@cs.washington.edu. We hope to see you there!

Stipend and paid parking for non-UW-affiliated attendees

For our attendees with disabilities who are not affiliated with UW, we will have a $50 stipend to cover local travel and time spent at the event. You will receive a gift card link within 10 business days after the event. We will also pay for event parking. We hope that will be helpful in covering some of the costs of attending this event.

Activities

Mainstage gameplay

Attendees can go head-to-head in our accessible esports tournament that will include Forza Horizon 5 and Rocket League.

Spotlight tables

Engage with CREATE corporate and community partners around game accessibility, including Seattle Adaptive Sports, Microsoft XBox, HuskyADAPT, and UW CREATE. Participate in accessible gaming tech demos, and more!

Currently, spotlight organizations include:

If you or your organization would be interested in reserving a free showcase table at the event, contact Jesse Martinez at jessejm@cs.washington.edu.

Non-competitive gameplay

In addition to the mainstage gameplay, there will be various accessible video games available to play, ranging from cooperative games to streamed large-audience party games. We’ll also have a VR station available! Games will include

Jackbox Party Pack Games
Keep Talking and Nobody Explodes
Beat Saber

Socializing, networking and food

We will also have designated spaces for attendees to socialize with each other and make new connections in the accessible gaming space. Dinner will be provided.

Accessibility & logistics

Wheelchair-accessible space & accommodations

The building entrance is level from Stevens Way and Zillow Commons is wheelchair-accessible via the elevator and wide doorways. A volunteer will be at the building entrance to help guide you to the event.

We will have the following accommodations in place:

Live gameplay commentary on Mainstage gameplay
Captions and ASL interpretation for all Mainstage content
Quiet room with ample seating and a silent livestream of Mainstage gameplay
Complimentary food and beverages
Screen reader-accessible online event program/guide

Adaptive devices

For those interested in playing games, we will have the following devices:

Xbox Adaptive Controllers with customizable switches, joysticks, and foot pedals
Additional specialty gaming equipment provided by industry partners (TBD)

If you have any additional accommodation requests, please include them in your event registration, or reach out to Jesse Martinez at jessejm@cs.washington.edu.

Considerations to keep in mind

During the event, attendees can support each other with the following considerations:

Introduce yourself by name in a conversation.

Keep pathways clear, and be mindful of others when navigating the space.

DO NOT touch other attendees, their assistive devices, or their mobility devices without consent.

Please keep conversation family-friendly as there are children at the event.

Please wear a mask and keep your hands clean (hand sanitizer is available throughout the venue).

Questions?

Please reach out to Jesse Martinez (event co-organizer) at jessejm@cs.washington.edu with questions about this event.

International Disability Rights: Past, Present, & Future – A Must-See Public Lecture with Senator Floyd Morris

International Disability Rights: Past, Present, & Future
Public Lecture with Senator Floyd Morris

Wednesday, April 19, 2023, 2:30 p.m.
HUB 340
Free and open to the public

Floyd Morris, Ph.D., is the Director of the Centre for Disability Studies at the University of the West Indies, a current Member and Past President of the Senate of Jamaica – where he was also their first Blind member – and Special Rapporteur on Disability for the Caribbean Community (CARICOM). He has researched inclusion of persons with disabilities in several aspects of Jamaican life and published numerous books and articles.

Senator Floyd Morris, a black man wearing a grey suit and red tie, seated in front of the Jamaican flag.

Senator Morris is a member of the UN Committee on the Rights of Persons with Disabilities, which is the treaty body charged with the responsibility of overseeing the implementation and interpretation of the Convention on the Rights of Persons with Disabilities.

Sponsored by the UW Center for Global Studies, UW Disability Studies, and Law, Societies, and Justice programs to welcome visiting lecturer Floyd Morris.

Advisory Board: Welcome ChrisTiana ObeySumner

CREATE welcomes our newest Advisory Board member, ChrisTiana ObeySumner.

ObeySumner (they/them) is the CEO and principal consultant of Epiphanies of Equity LLC (https://www.christianaobeysumner.com/), a social equity consulting firm specializing in change management, social and organizational psychology, intersectional equity and liberation, and disability justice.

ChrisTiana ObeySumner: a Black, queer, non-binary, and multiply disabled researcher in front of a bright red background

For two decades, they’ve dedicated their life and career to exploring and practicing innovative approaches to achieving social equity – in other words, how to sustainably and effectively bring parity to areas of disparity so “humans can human with other humans” equitably, collectively, and intersectionally.

ObeySumner joins board members Mary Bellard, Amy Hurst (who also joined recently), and Jonathan Lazar. Rory A. Cooper and Juan E Gilbert are concluding their board memberships this spring; we thank them for their perspective and expertise over the past two years.

Faculty essay: Signs of Disability

By Stephanie Kerschbaum, CREATE faculty member, Associate Professor of English, and Director of the Program in Writing and Rhetoric

I’ve just returned to Seattle from a trip to a conference in Chicago. On that trip, I pointed out to a friend a billboard in a downtown Chicago bus stop inviting viewers to learn to recognize the signs of a rare medical condition. And just a bit later, at my gate in O’Hare, I found myself sitting across from a large advertisement sorrowfully highlighting the signs of autism.

I can’t go very far in my everyday experience without encountering scores of material signs that point in various ways to disability. They’re not always giant billboards trumpeting the signs of a particular disability, but collectively, despite the fact that these signs are often well-intentioned, they nevertheless reinforce problematic orientations to disability that circumscribe how, where, and when disability can be noticed.

Billboards like these signs had a great deal to do in motivating my work in Signs of Disability. One of the stories I tell in the book has to do with a yellow diamond-shaped “Deaf Person in Area” road sign that appeared in my neighborhood and which I subsequently spent a lot of time talking about with people around me. At first, when I pointed the sign out to my husband, we laughed about it. “Good thing we live in the neighborhood,” I joked to him, “so that there actually is a Deaf Person in the Area.” But when I shared this story with other people, they didn’t always think it was funny.

The tension I felt between my own sense that the Deaf Person in Area sign did very little to meaningfully support attention to deafness and other people’s strong defenses of the sign (and others like it) was key to helping me build one of the core conceptual terms in Signs of Disability. That term, dis-attention, is an intentionally clunky neologism that points to the ways that disability is simultaneously hyper-perceptible and invisible in everyday life. The Deaf Person in Area needed a bold yellow sign warning others about her presence, but of course no one passing the sign has any idea who this Deaf Person in Area actually is, nor would they likely be able to identify the Deaf Person in Area.

As a middle-aged white deaf cisgender woman in a heteronormative relationship, how deafness appeared on my body and became available for others to notice was a complicated question, and deafness was never legible outside of its entanglements with race, gender, sexuality, age, environment, and more. I started by pointing to my behind-the-ear hearing aids or the ways that others might interpret the sound of my voice or encounters where I am signing with other people or encounters where I announce or name my deafness in an interaction. But the list of ways that someone might identify cues pointing to deafness kept growing, and were interactionally contingent as well as tied up in others’ perceptual practices which are themselves shaped by and through cultural milieus often saturated with ableism and dominant orientations to disability that limit how and when it can be perceived.

Learning to recognize the signs of disability involves building a conceptual and practical vocabulary, to have embodied and en-minded experiences that make such signs available for noticing, and the central project of the book is to invite readers to consider not only how dis-attention shapes perception of disability, but also the ways that disabled ways of knowing and being can help change how disability comes to be noticed and to matter in the world.

Teach Access Study Away program

There is a wonderful, free short course, Teach Access Study Away, run by the Teach Access organization each year.

The 2023 Teach Access Virtual Study Away is a two-part series. Part 1 focuses on the basics of accessibility and disability; Part 2 covers more advanced topics so completion of Part 1 or a previous Study Access Away session is required.

The two parts combined total 7 virtual sessions, each 1 1/2 hours on Wednesdays starting on April 5th. Signup for 2023 closes on March 25th.

CSE course sequence designed with “accessibility from the start”

The CSE 121, 122, and 123 introductory course sequence lets students choose their entry point into computer science and engineering studies, whatever their background, experience, or confidence level. And, as part of the effort to improve diversity, equity, inclusion, and accessibility (DEIA), the courses were designed with “accessibility from the start.”

A member of the course development team was a dedicated accessibility expert, tasked with developing guidelines for producing accessible materials: using HTML tags correctly, providing alt text for all images, and ensuring accurate captions on all videos. The team audited both content and platforms — including the course website — for accessibility concerns.

In CSE’s DEIA Newsletter article, author Brett Wortzman, Associate Teaching Faculty, points out that “many of the guidelines followed are good universal design, helping all students, not just those with disabilities, and at the same time reducing the work for instructors needing to comply with many DRS [Disability Resources for Students] accommodations.”

Excerpted from article by Brett Wortzman, Associate Teaching Faculty, in CSE’s DEIA Newsletter.

Honoring Judy Heumann’s outsized impact

Judy Heumann — disability activist and leader, presidential advisor to two administrations, polio survivor and quadriplegic — passed away on Saturday, March 4. Heumann’s family invited the community to honor her life at a memorial service and burial that is now available on video with ASL, captioning, and English interpretation of Yiddish included.

Who was Judy Heumann?

Judy Heumann fought for disabled rights and against segregation. She led the “longest nonviolent occupation of a federal building in American history,” according to the New York Times. When communications were cut off by the government, she passed messages to supporters using sign language and received support from the Black Panthers and the Mayor of San Francisco. The protest led to successful action on section 504 of the Rehabilitation Act.

Heumann did not stop there. According to President Biden, “Her courage and fierce advocacy resulted in the Rehabilitation Act, Individuals with Disabilities Education Act, and the Americans with Disabilities Act – landmark achievements that increased access to education, the workplace, housing, and more for people with disabilities.” Heumann talked in a 2020 article in Ability Magazine about the importance of “helping people to identify and understand the scope of disability that the ADA and other laws cover.” She believed in an expansive view of disability, and wanted to see more people included in the disability rights movement, stating “… I think it’s a combination of shame and fear that we may not be talking about if we have diabetes or epilepsy or cancer or anxiety or depression or bipolar or whatever. …I think expanding our circle is one of the big issues that we need to be dealing with over the next five to ten years.” Heumann went on to talk about the power of knowing that you have rights under the law, recognizing and fighting discrimination, and the importance of diversifying the disability movement by race, religion and sexual orientation: “We need to have an understanding, for example, of the fact that [disparities] may exist in various communities based on race and socio-economic status, how there are people within the U.S. who are not benefiting from laws because they don’t have the resources to hire an attorney or an advocate. The government, in my view, is not always enforcing laws as they should be.”

Heumann’s auto-biography, Being Heumann came out in 2020 (co-authored with Kristen Joiner). Crip Camp documents the 1970s birth of Heumann’s and other activists’ advocacy for people with disabilities. Ability Magazine inter-view touching on the ADA, the increasing diversity of the disability community, and the pandemic.

CREATE Co-director Jennifer Mankoff noted that, “Personally speaking, her influence on my career has been indirect but important. In my first years as a faculty member, at UC Berkeley, the Center for Independent Living (which Heumann founded and called “the first organization in the world to be run for and by the disabled” according to the NY Times) reached out to educate me about disability activism, through another leader, Scott Leubking. At the same time, I began to learn about disability studies work with the help of Berkeley’s nascent disability studies program and specifically academic and activist Devva Kasnitz. These early encounters directly influenced the direction of my research and advocacy and are visible today in CREATE’s emphasis on disability studies and disability justice and entwining them in its accessibility work.”

Learn more about Judy Heumann

**Heumann’s auto-biography, Being Heumann: An Unrepentant Memoir of a Disability Rights Activist, published in 2020 and co-authored with Kristen Joiner.**

**Crip Camp documents the 1970s birth of Heumann’s and other activists’ advocacy for people with disabilities.**

**Ability Magazine interview touching on the ADA, the increasing diversity of the disability community, and the pandemic.**

UW and CREATE extended family shine in SIGCHI Awards

We’re so proud to have learned from and collaborated with these shining stars! UW and CREATE faculty and our extended family are prominent in the recently announced 2023 SIGCHI Awards. All three winners of the SIGCHI dissertation award, which recognizes “the most outstanding research contributions from recently graduated Ph.D. students within the HCI community” are associated with the UW, CREATE, and/or our campus partner, UW DUB (Design. Use. Build).

Congratulations to all!

Screenshot from SIGCHI '23 awards page labeled "Outstanding Dissertation" with photos of Megan Hofmann, Dhruv Jain, and Kai Lukoff.

Dhruv Jain, ‘22 Ph.D. UW Computer Science & Engineering

Makeability Lab alum advised by CREATE associate directors Leah Findlater and Jon Froehlich

Jain’s dissertation was honored for “advancing the design and evaluation of interactive systems to improve sound awareness for people who are deaf or hard of hearing (DHH). This research, drawing on his own experience as a person who is hard of hearing, has two goals: first, to better understand how DHH people feel about technology-mediated sound awareness and how these feelings manifest across contexts; and second, to design, build, and study new technical solutions for sound using iterative, human-centered design.” Jain is now an Assistant Professor in Computer Science and Engineering at the University of Michigan.

Kai Lukoff, ‘22 Ph.D., UW Human Centered Design & Engineering

UW DUB alum

Lukoff’s dissertation Designing to Support Sense of Agency for Time Spent on Digital Interfaces addresses the problem that mobile devices are omnipresent in many people’s lives, and yet many people are dissatisfied with how much and when they use them. They adopt various devices and apps for their promise to connect with others, to accomplish tasks, and to be entertained, but may then find that their use–or others’ use–of those very same apps and devices gets in the way of connection, productivity, and meaningful entertainment. Lukoff is now an assistant professor in computer science and engineering at Santa Clara University, where he directs the Human-Computer Interaction Lab. Read more about Lukoff’s research on HCDE.

Megan Hofmann, ‘22 Ph.D. Human-Computer Interaction, Carnegie Mellon

CREATE and UW DUB alum

SIGCHI recognizes Hofmann’s research on “Optimizing Medical Making” for taking “a strong interdisciplinary approach, both with improved understanding of an important domain, and substantive technical contributions, using methodologies ranging from systems and programming language contributions, to ethnographic methods. This has allowed the work to make contributions in multiple areas such as accessibility, software tools, and digital fabrication.” Hofmann was advised by CREATE co-director Jennifer Mankoff at Carnegie Mellon University. Hofmann is currently an assistant professor of computer science and mechanical engineering at the Khoury College of Computer Sciences at Northeastern University.

Screenshot from SIGCHI '23 awards page labeled "Societal Impact" with photo of Nicola Dell.

Nicola Dell, ’15 Ph.D. UW Computer Science and Engineering

Dell received a Societal Impact Award, cited for exemplary work that represents an unusually ‘full stack’ model of intervention and social impact. “She has been the driving force in putting tech-related Intimate Partner Violence abuses on the radar of companies, government, and HCI as a field; has offered direct and meaningful support to survivors; and has produced real-world changes that have begun to combat this pervasive and insidious problem.” Dell is currently an associate professor of information and computer science at Jacobs Technion-Cornell Institute.

CREATE Co-director Jennifer Mankoff received the Social Impact Award in 2022.

Screenshot from SIGCHI '23 awards page labeled "Lifetime Research" with photo of Gregory Abowd.

Gregory Abowd, ’91 Ph.D. Computation, University of Oxford, UK

Mentor and advisor to some of the most influential leaders in accessibility, usability, and HCI at the UW and beyond.

And we’re appreciating the influence of Lifetime Research Award recipient Gregory Abowd, ’91 Ph.D. Computation, University of Oxford, UK. A leader in disability research with a focus specifically on autism, Abowd has mentored or advised CREATE co-director Jennifer Mankoff and many CREATE members and DUB faculty, including Anind Dey (Dean of the UW iSchool), Julie Kientz (Chair of UW HCDE), and Shwetak Patel (professor in UW CSE and ECE). SIGCHI described Abowd as, “A world leader in the invention and application of ubiquitous computing technologies, his work has defined the field over the past three decades, and his intellectual contributions have shaped two major themes in ubiquitous computing: context-aware computing and automated capture and access of live experiences. He has shown how a variety of application areas—the classroom, the home, autism, and health care—benefit from innovations in mobile and ubiquitous technologies.

Jacob O. Wobbrock awarded Ten-Year Technical Impact Award

January 5, 2023

The Association for Computing Machinery (ACM) has honored CREATE Co-Director Jacob O. Wobbrock and colleagues with a 10-year lasting impact award for their groundbreaking work improving how computers recognize stroke gestures.

Jacob O. Wobbrock, a 40-something white man with short hair, a beard, and glasses. He is smiling in front of a white board.

Wobbrock, a professor in the Information School, and co-authors Radu-Daniel Vatavu and Lisa Anthony were presented with the 2022 Ten Year Technical Impact Award in November at the ACM International Conference on Multimodal Interaction (ICMI). The award honors their 2012 paper titled Gestures as point clouds: A $P recognizer for user interface prototypes, which also won ICMI’s Outstanding Paper Award when it was published.

The $P point-cloud gesture recognizer was a key advancement in the way computers recognize stroke gestures, such as swipes, shapes, or drawings on a touchscreen. It provided a new way to quickly and accurately recognize what users’ fingers or styluses were telling their devices to do, and even could be used with whole-hand gestures to accomplish more complex tasks such as typing in the air or controlling a drone with finger movements.

The research built on Wobbrock’s 2007 invention of the $1 unistroke recognizer, which made it much easier for devices to recognize single-stroke gestures, such as a circle or a triangle. Wobbrock called it “$1” — 100 pennies — because it required only 100 lines of code, making it easy for user interface developers to incorporate gestures in their prototypes.

This article was excerpted from the UW iSchool article, iSchool’s Wobbrock Honored for Lasting Impact by Doug Parry

Rory Cooper, CREATE Advisory Board member, receives IEEE Biomedical Engineering Award

Congratulations to CREATE Advisory Board member Rory Cooper on receiving the 2022 IEEE Biomedical Engineering Award!

For more than 25 years, Cooper has been developing technology to improve the lives of people with disabilities and his inventions have helped countless wheelchair users get around with more ease and comfort.

Rory A. Cooper, a white man with salt-and-pepper hair, dressed in a suit and tie.

Cooper’s first innovations in mobility were a modification to the back brace he wore after a spinal cord injury left him paralyzed from the waist down, then a better wheelchair, then an electric-powered version that helped its user stand up. After earning his Ph.D. in electrical & computer engineering with a concentration in bioengineering at University of California at Santa Barbara, he focused his career on developing assistive technology.

Cooper (second from the left) and his colleagues—David Constantine, Jorge Candiotti, and Andrin Vuthaj (standing)—at the University of Pittsburgh’s Human Engineering Research Laboratories working on the MEBot. Photo: ABIGAIL ALBRIGHT

Since 2013, Cooper and his team at the University of Pittsburgh ’s Human Engineering Research Laboratories have been working to develop advancements including a wheelchair that can travel on rough terrain.

The most common cause of emergency-room visits by wheelchair users is falling from the chair or tipping over. “This often happens when the individual’s wheelchair hits thresholds in doorways, drives off small curbs, or transitions from a sidewalk to a ramp,” Cooper said.

The team hopes that the Mobility Enhancement Robotic Wheelchair, known as the MEBot, can minimize such injuries.

The MEBot, can climb curbs up to 20 centimeters high and can self-level as it drives over uneven terrain. It does so thanks to six wheels that move up and down plus two sets of smaller omnidirectional wheels in the front and back. The wheelchair’s larger, powered wheels can reposition themselves to simulate front-, mid-, or rear-wheel drive.

This article is excerpted from the IEEE Spectrum’s award announcement.

Carl James Dunlap Memorial Scholarship

University of Washington student Carl James Dunlap had a powerful impact on the UW community with his vibrant personality and persistent advocacy for students with disabilities. To honor his legacy, the Dunlap family established the Carl James Dunlap Memorial Endowment. The Dunlap Memorial Endowment seeks to support students with disabilities encountering unique challenges when attending and completing higher education. The D Center is grateful to further Carl’s legacy by awarding two $2,000 Carl James Dunlap Memorial Scholarships to UW students for Winter 2023.

The Dunlap Memorial Scholarship selection criteria is a UW student who identifies as having a disability and is currently receiving financial aid.

Apply no later than January 31

If you have any questions, please contact the D Center at dcenter@uw.edu.

The Carl James Dunlap Memorial Fund is accepting donations to further help students with disabilities.

Flyer for the Carl James Dunlap Memorial Scholarship with a link to contact dcenter@uw.edu for details and a picture of the UW Seattle campus in fall.

UnlockedMaps provides real-time accessibility info for rail transit users

Congratulations to CREATE Ph.D. student Ather Sharif, Orson (Xuhai) Xu, and team for this great project on transit access! Together they developed UnlockedMaps, a web-based map that allows users to see in real time how accessible rail transit stations are in six metro areas including Seattle, Philadelphia (where the project was first conceived by Sharif and a friend at a hackathon), Chicago, Toronto, New York, and the California Bay Area.

screenshot of UnlockedMaps in New York. Stations that are labeled green are accessible while stations that are labeled orange are not accessible. Yellow stations have elevator outages reported.

Shown here is a screenshot of UnlockedMaps in New York. Stations that are labeled green are accessible while stations that are labeled orange are not accessible. Yellow stations have elevator outages reported.

Sharif, a UW doctoral student in the Paul G. Allen School of Computer Science & Engineering advised by CREATE Co-Director Jacob O. Wobbrock, said the team also included nearby and accessible restaurant and bathroom data. “I think restaurants and restrooms are two of the most common things that people look for when they plan their commute. But no other maps really let you filter those out by accessibility. You have to individually click on each restaurant and check if it’s accessible or not, using Google Maps. With UnlockedMaps, all that information is right there!”

Adapted from UW News interview with Ather Sharif. Read full article »

CREATE Contributes to RFP on Healthcare Accessibility

The Agency for Healthcare Research and Quality (AHRQ) requested public comment about comprehensive, longitudinal, person-centered care planning for people with Multiple Chronic Conditions (MCC). CREATE contributed to a disability justice-focused response that highlights nine recommendations:

Account for medical trauma.
Meet basic standards for accessibility.
Value individual and community knowledge about MCC.
Treat accessibility as a first-class component of patient care.
Prioritize community.
Look beyond “care.”
Remove financial barriers.
Include people with MCC in planning.
Enable people with MCC to enter clinical roles

Read the full response (PDF).

Wobbrock Co-leads ACM UIST Conference, Brings Accessibility to the Conversation

CREATE founding Co-Director Jacob O. Wobbrock served as General Co-Chair for ACM User Interface Software and Technology (UIST) 2022, held at the end of October.

Nearly 500 people traveled to beautiful Bend, OR to share their latest innovations in user interface software and technology from fabrication and materials, to VR and AR, to interactive tools and interaction techniques. UIST showcased the very best inventive research in the field of human-computer interaction. “Attending UIST is like attending an exclusive preview of possible tomorrows, where one gazes into the future and imagines living there, if only for a moment,” said Wobbrock.

Two photos from UIST 2022 Conference: A table of attendees chatting animatedly and a photo of Jacob O. Wobbrock and closing keynote speaker Marissa Mayer

Bringing accessibility into the conversation, Wobbrock’s opening keynote questioned the assumptions made in statements we often see, such as, “Just touch the screen” assumes the ability to see the screen, to move the hand, and so on.

For the closing keynote, available on YouTube, Wobbrock interviewed Marissa Mayer, former CEO of Yahoo and an early employee at Google. She studied Symbolic Systems and Computer Science with a focus on artificial intelligence at Stanford, along with Wobbrock. Mayer answered audience questions, including one about making design choices through a combination of crowdsourcing, an abundance of data, and strong opinions.

CREATE Leadership at ASSETS’22 Conference

CREATE Associate Director Jon Froehlich was the General Chair for ASSETS’22, the premier ACM conference for research on the design, evaluation, use, and education related to computing for people with disabilities and older adults. This year, over 300 participants from 37 countries engaged with state-of-the-art research in the design and evaluation of technology for people with disabilities. UW CREATE was a proud sponsor of ASSETS’22.

Keynote speaker Haben Girma is the first Deafblind graduate of Harvard Law School and a leading human rights advocate in disability. Girma highlighted systemic ableism in education, employment, and tech and opportunities for change in her speech.

“There is a myth that non-disabled people are independent and disabled people are dependent. We are all interdependent. Many of you like drinking coffee; very few of you grow your own beans,” she pointed out.

ASSETS’22 was held in Athens, Greece. “The birthplace of democracy, we were surrounded by so many beautiful antiquities that highlighted the progress and innovation of humanity and served as inspiration to our community,” said Froehlich.

“Perhaps my favorite experience was the accessible private tours of the Acropolis Museum with conference attendees—hearing of legends, seeing the artistic craft, and moving about a state-of-the-art event center all in the shadow of the looming Acropolis was an experience I’ll never forget,” he added.

Artifact awards

CREATE Ph.D. student Venkatesh Potluri, advised by CREATE Co-Director Jennifer Mankoff in the Make4All Group, and his team tied for 1st place for the Artifact Award. Potluri presented their work on CodeWalk, Facilitating Shared Awareness in Mixed-Ability Collaborative Software Development.

Third place went to Ather Sharif‘s team, advised by Jacob Wobbrock, UnlockedMaps: Visualizing Real-Time Accessibility of Urban Rail Transit Using a Web-Based Map.

Future of urban accessibility

As part of the conference, Froehlich, Heather Feldner, and Anat Caspi held a virtual workshop entitled the “Future of Urban Accessibility” More here: https://accessiblecities.github.io/UrbanAccess2022/

A Ph.D. Student’s Promising Research in Mobility in Cerebral Palsy

Whether she’s researching how biofeedback systems can guide gait training in children with cerebral palsy or leading toy adaptation events, Alyssa Spomer is committed to advancing accessible technology.

A Ph.D. student in UW Mechanical Engineering (ME) and advised by CREATE Associate Director Kat Steele, Spomer is the student chair of CREATE-sponsored HuskyADAPT. Her studies have been multidisciplinary, spanning ME and rehabilitation medicine. She uses her engineering skills to understand the efficacy of using robotic devices to target and improve neuromuscular control during walking.

“Delving into how the central nervous system controls movement and how these systems are impacted by brain injury has been such an interesting aspect of my work,” Spomer says. “My research is a mix of characterizing the capacity for individuals to adapt their motor control and movement patterns, and evaluating the efficacy of devices that may help advance gait rehabilitation.”

In her dissertation work, Spomer is primarily evaluating how individuals adapt movement patterns while using a pediatric robotic exoskeleton paired with an audiovisual biofeedback system that she helped design. The Biomtoum SPARK exoskeleton works to sense and support motion at the ankle during walking, using motors worn on a hip belt to provide either resistance or assistance to the ankles during walking. The audiovisual system is integrated into the device’s app and provides the user with real-time information on their ankle motion alongside a desired target to help guide movement correction.

Collage of two images. Left: Man walking on a treadmill wearing a robotic exoskeleton device around his hips and legs while a female researcher monitors the process through a tablet. Right: A closeup of the researcher's hands holding the tablet showing the real-time information. — The audiovisual system that Spomer helped design (shown on a screen in the right photo) provides the user with real-time information on their ankle motion alongside a desired target to help guide movement correction.

Inspired by CREATE’s Kat Steele and the Steele Lab

Spomer was drawn to ME by the Steele Lab’s focus on enhancing human mobility through engineering and design. Working with Kat Steele has been a highlight of her time at the UW.

“I really resonated with Kat’s approach to research,” Spomer says. “The body is the ultimate machine, meaning that we as engineers can apply much of our foundational curriculum in dynamics and control to characterize its function. The beauty of ME is that you are able to develop such a rich knowledge base with numerous applications which really prepares you to create and work in these multidisciplinary spaces.”

This winter, Spomer will begin a new job at Gillette Children’s Specialty Healthcare. She’s excited to pursue research that aligns with her Ph.D. work. Her goal remains the same: “How can we advance and improve the accessibility of healthcare strategies to help promote independent and long-term mobility?”

Excerpted from UW Mechanical Engineering, written by Lyra Fontaine with photos by Mark Stone, University of Washington. Read the full article

Increasing Data Equity Through Accessibility

Data equity can level the playing field for people with disabilities both in opening new employment opportunities and through access to information, while data inequity may amplify disability by disenfranchising people with disabilities.

In response to the U.S. Science and Technology Policy Office’s request for information (RFI) better supporting intra- and extra-governmental collaboration around the production and use of equitable data, CREATE Co-director, Jennifer Mankoff co-authored a position statement with Frank Elavsky, Carnegie Mellon University and Arvind Satyanarayan, MIT Visualization Group. The authors address the three questions most pertinent to the needs of disabled people.

They highlight the opportunity to expand upon the government’s use of accessible tools to produce accessible visualizations through broad-based worker training. “From the CDC to the Census Bureau, critical data that is highly important to all historically underrepresented peoples and should be available to underrepresented scholars and research institutions to access and use, must be accessible to fully include everyone.”

Reiterating “Nothing about us without us,” the statement notes that when authoring policies that involve data, access, and equitable technology, people with disabilities must be consulted. “Calls for information, involvement, and action should explicitly invite and encourage participation of those most affected.” In addition to process notes, the response addresses roles, education, laws, and tools.

Just having access to data is not enough, or just, when power, understanding and action are in the hands of government agents, computer scientists, business people and the many other stakeholders implementing data systems who do not themselves have disabilities.

The statement identifies access to the tools for producing accessible data, such as data visualizations, as low-hanging fruit and concludes with a call for funding of forward-thinking research that investigates structural and strategic limitations to equitable data access. More research is needed to investigate the ways that various cultural and socio-economic factors intersect with disability and access to technology.

Read the details in the full response on arxiv.org (PDF).

Large-Scale Analysis Finds Many Mobile Apps Are Inaccessible

Mobile apps have become a key feature of everyday life, with apps for banking, work, entertainment, communication, transportation, and education, to name a few. But many apps remain inaccessible to people with disabilities who use screen readers or other assistive technologies.

Any person who uses an assistive technology can describe negative experiences with apps that do not provide proper support. For
example, screen readers unhelpfully announce “unlabeled button” when they encounter a screen widget without proper information provided by the developer.

iStockPhoto image of several generic application icons such as weather, books, music, etc.

We know that apps often lack adequate accessibility, but until now, it has been difficult to get a big picture of mobile app accessibility overall.

How good or bad is the state of mobile app accessibility? What are the common problems? What can be done?

Research led by Ph.D. student Anne Spencer Ross and co-advised by James Fogarty (CREATE Associate Director) and Jacob O. Wobbrock (CREATE Co-Director) has been examining these questions in first-of-their-kind large-scale analyses of mobile app accessibility. Their latest research automatically examined data from approximately 10,000 apps to identify seven common types of accessibility failures. Unfortunately, this analysis found that many apps are highly inaccessible. For example, 23% of the analyzed apps failed to provide accessibility metadata, known as a “content description,” for more than 90% of their image-based buttons. The functionality of those buttons will therefore be inaccessible when using a screen reader.

A bar chart showing the percentage of application icons and images missing labels. Out of 8,901 apps, 23 percent were not missing labels, 23 percent were missing labels on all elements. The rest of the apps were missing labels for 6 to 7 percent of their elements. — Bar chart shows that 23 percent of apps are missing labels on all their elements. Another 23 percent were not missing labels on any elements. And the rest were missing labels on 6 to 7 percent of their elements.

Clearly, we need better approaches to ensuring all
apps are accessible. This research has also shown that large-scale data
can help identify reasons why such labeling failures occur. For example, “floating action buttons” are a relatively new Android element that typically present a commonly-used command as an image-button floating atop other elements. Our analyses found that 93% of such buttons lacked a content description, so they were even more likely than other buttons to be inaccessible. By examining this issue closely, Ross and her advisors found that commonly used software development tools do not detect this error. In addition to highlighting accessibility failures in individual apps, results like these suggest that identifying and addressing underlying failures in common developer tools can improve the accessibility of many apps.

Next, the researchers aim to detect a greater variety of accessibility failures and to include longitudinal analyses over time. Eventually, they hope to paint a complete picture of mobile app accessibility at scale.