A new tool developed for use with the Charles B. Wang Center, called A11yShape, is improving the 3D modeling experience for users with visual impairments. Liang He, an assistant professor of computer science at the University of Texas at Dallas, created A11yShape. Explore Tactile City. This programming and design tool developed by Gallaudet students and faculty aims to improve accessibility for blind and low-vision people. The project took root when He conversed with a classmate who faced challenges in 3D modeling due to their low vision.
The A11yShape platform features a web interface that consists of three interconnected panels: a code editor panel equipped with programming capabilities, an AI assistance panel that delivers contextual feedback, and a model panel that showcases the hierarchical structure and rendering of the resulting 3D model. By synchronizing these components, A11yShape allows users to understand how modifications in code influence design elements, fostering independent exploration of 3D modeling.
User-Centric Design and Functionality
A11yShape’s creative design emphasizes consonant components in each to connect the dots across all three panels. Once a user clicks on a line of code or model object, the system goes to work. It rewrites these descriptions to improve clear, natural flow in the user’s workflow. As you can imagine, this level of synchronization is very important for users with visual impairment. They typically rely on auditory feedback to work through challenging programming tasks.
The new AI assistance panel is key to flowing through this process. It offers a simple, trustworthy way to describe the models you’re building. Early evaluations indicate that the prompts rated 4.1 to 5 on the 1–5 scale for geometric fidelity, clarity, and hallucination avoidance. These findings indicate that the AI is robust enough for use in a real-world setting.
As a designer, Liang He is committed to producing functional tools for the community. His response was, “I want to create something really valuable, really actionable for the cohort. Software participants who tested the software developed a profound appreciation for its usefulness. One person even learned about modeling for the first time while using A11yShape. Their input showed its value in proving that even the simplest of structures can be built.
Testing and Feedback from Participants
A11yShape was tested with four participants having varying degrees of visual impairment and different programming backgrounds. These trials were crucial in establishing what does or does not work with the tool, including what needs to be improved. Iterate with the AI generated descriptions, participants found that the AI-generated descriptions really deepened their understanding of the modeling process.
Another participant shared how A11yShape’s creative approach to 3D modeling inspired new ideas for their own project. They insisted that this provided the blind and low-vision community with a new outlook, showing that we are able to build relatively simple structures. This changing sentiment, paired with the power of A11yShape, can help users regain control and freedom through accessible technology.
While we consider it fairly strong, some challenges arose throughout testing. A failed attempt to construct a 3D helicopter showed that even the accuracy of the outline shape and the placement of the elements were problematic. Such cases are a testament to the need for constant improvement as A11yShape grows and develops.
Future Developments and Aspirations
Looking ahead, Liang He and his team hope to add more capabilities to A11yShape. Just imagine future iterations with tactile displays, real-time 3D printing capabilities and even more compact AI-generated audio descriptions. Over time, these improvements have made the user experience even more engaging and broadened the tool’s use among the visually impaired community.
Stephanie Ludi, a collaborator on the project, remarked on the broader implications of accessible technology: “People like being able to express themselves in creative ways… using technology such as 3D printing to make things for utility or entertainment.” A11yShape makes this vision a reality. It furthers user creativity and artistic expression, initiating more passionate individuals of all backgrounds into the world of 3D design and production.