A new web interface called A11yShape is changing things by making 3D modeling tools accessible to programmers with visual impairments. Developed by a dedicated research team led by Liang He, A11yShape aims to empower blind and low-vision users to independently explore the impact of code modifications on their designs. This innovative tool incorporates an AI assistance panel, a code editor, and a model display, all synchronized to enhance user experience.
The Stone Soup project got started when he was in dialogue with a low-vision classmate who was learning 3D modeling. He saw an important hole in the available tools for the visually impaired. Thus, he and his team endeavored to produce a user-centric solution that put the user experience first and made it simple to enter the world of 3D modeling. The research team recruited four participants with varying degrees of visual impairment and programming backgrounds to test the tool and provide feedback.
Features of A11yShape
A11yShape consists of three main panels: a code editor, an AI assistance panel, and a model display panel. The code editor unlocks your creativity, allowing you to easily write and edit your program. Alongside this, the AI assistance panel offers important contextual feedback on the models being created. From detailed descriptions of each 3D model, UX is always in the know as to what’s being worked on at all times.
The model display panel features the internal hierarchical hierarchy and 3D real time rendering of the output model. Once the user highlights a given code segment or model part, A11yShape kicks in. Specifically, it helps to illustrate the similarity between the elements all three panels exhibit in common. This feature allows for a fluid understanding of how code changes will affect the design, maintaining uniformity and synchronicity between each facet.
“I want to design something useful and practical for the group,” – Liang He
We found that the AI assistance panel has been a very powerful mechanism. It produces accurate geometric description satisfactory to a mean score of 4.1-5 with a 1-5 scale along with geometric accuracy, clarity of expression, and hallucination generation. This indicates that the AI piece is robust enough to be used in day-to-day usage by visually impaired programmers.
Empowering Creativity and Independence
Beyond technical learning, A11yShape inspires creativity among users. Participants commented that the tool allowed them to see 3D modeling in a new light. As one participant put it, it showed off their talents in building relatively complex structures with extreme visual simplicity.
As Stephanie Ludi, an innovative leader in the field of inclusive technology, once said, creative expression is fundamental to the human experience. She stated, “People like being able to express themselves in creative ways… using technology such as 3D printing to make things for utility or entertainment.” This sentiment resonates deeply within the blind and low-vision communities, where A11yShape opens doors to creative exploration and practical applications.
The development team’s commitment to inclusivity is clear, not only in their dedication to making A11yShape better through users’ feedback. Future iterations of the tool could use tactile displays. They can include real-time 3D printing which traditionally involves greater latency, revolutionary hands-on experience for users. In addition, we’re working to deliver shorter AI-generated audio descriptions to make for a more efficient user experience even further.
A Bright Future for Accessible Technology
Even the broader impact of the potential, transformative A11yShape implementation can’t be overstated. It’s a real leap forward in accessible technology. A11yShape connects the worlds of coding and 3D design for people with visual disabilities. It remains our field’s lodestar for innovation in advancing technology.
Liang He, pictured above, and his team are all focused on creating this revolutionary tool. They’re committed to ensuring that everyone, regardless of ability, can interact with technology in powerful and impactful ways. Yet the adventure from idea to production has been transformative for everyone involved. More than that, it has laid the groundwork for future innovations that might completely change the way visually impaired people interface with digital design.
“On a 1–5 scale, the descriptions earned average scores between about 4.1 and 5 for geometric accuracy, clarity, and avoiding hallucinations, suggesting the AI is reliable enough for everyday use.” – research team