This is a system designed for Blind & Visually Impaired(BVI) users using which they can access services like Yelp and Uber through simple hand gestures. There are 3 major sections in this project
- Designing Simplified Aural Navigation Patterns
- Designing Hand Gesture Grammar
- Technical Implementation Of The System
The Problem
Mobile applications have not reached their potential among the Blind and Visually Impaired(BVI) users due to the following contraints :
- Complex navigation patterns
- Social & Privacy concerns with speech input
- Unacceptable hands-busy condition.
The Solution
The above mentioned constraints arise due to limitations in input and output mechanisms. To overcome these limitations, we have developed a new navigation system using hand-gestures and aural cues which simplies services like Yelp & Uber for the BVI users.
Concepts Developed
- Binary Aural Navigation
- Free Flow
- Dynamic menu items
Advantages of Hand gestures
- Natural & Intuitive
- Quiet Technology
- Wide Interaction Space
Interaction Design Decisions
Ballistic and fine movement
The aural free flow is designed to launch the user to specific interest point quickly and then provide finer navigation through the hand gestures. This also reduces the gesture fatigue as the number of gestures is reduced.
Earcons as gesture feedback
Every gesture is associated with an earcon. This feedback improves the learnability and error handling of the system.
Contextual Help
At any given point of time, the user can perform the "fist" gesture to get contextual help. This provides 2 types of information - "where are you? what can you do from here?".
Since the system is aural and not visual, helping the users regain context was very important.
App Drawer
The concept of binary search in programming is used to navigate apps in this system through hand gestures and aural cues. The apps are split into 2 nodes based on their origin. Native apps to the left and from-store apps to the right of the center cursor.
Next Steps
- I demonstrated the concept at BOSMA Enterprises. They are helping us recruit participants to test the conceptual fit of the system with BVI users.
- The screenless access concept is efficient at browsing content. We are working on a model to help BVI users input text through hand gestures.
This is an ongoing project. This project will be updated with more progress soon. In the meantime, reach out to me with any questions you have.
Here’s my e-mail address - jdara@iupui.edu
Other Projects
-
Circle Around Me
So many devices? So many notifications? Here is our attempt to optimize notifications for the "wearable future".
Click to read more -
10-foot-design
This is a design exploration for creating affordances on large screens for collaboration in classrooms.
Click to read more -
Together
Together is a ubiquitous solution to community exploration and organization.
Click to read more