Protected: Thesis

This content is password protected. To view it please enter your password below:

Inclusive design for VR – Accessibility project with Microsoft

As a part of our UX class project, Microsoft gave gave us the following problem statement:

“Design a product, service, or solution to solve for exclusion in a deskless workplace. The prompt of this challenge is to integrate inclusive design to build a solution for people with disabilities working in deskless workplace.”

Inclusive design was a new concept to most of us and we were excited about all the ideas that we could come up with for an inclusive desk less workspace.

Since we are all designers and we are in a program that involves various artists, we connected with the idea of solving for artists with limited hand mobility and trying to create content in VR. Our professor — Dana Karwas and Nick Katsivelos from Microsoft encouraged us to work in this area, since not much has been done.


Cherisha Agarwal, Joanna Yen, Pratik Jain, Raksha Ravimohan, Shimin Gu, Srishti Kush


Approaching this idea was not an easy task and it involved us talking to a lot of designers at IDM and understanding their workflow. We spoke to them about how they use softwares and the potential problems that people with limited hand mobility would run into. We were very fascinated by the idea of creating a multi-modal application that uses eye tracking and voice commands to enable people to draw in VR. To dive deep into the technology and understand its use, we read a few papers listed here and got great insights from professors at NYU like Dana Karwas, Todd Bryant, Claire K Volpe, experts in UX research like Serap Yigit Elliot from Google and Erica Wayne from Tobii. Every person we spoke to gave us more insights to tackle the challenge at hand from a user experience, VR technology and accessibility perspective. Here are some pictures:

Interview with Mayukh, a student at NYU IDM

VR guidance from Professor Todd Bryant

VR guidance from Professor Todd Bryant
Discussing with Serap Yigit Elliot from Google

Getting our hands dirty

After an extensive research, we decided to get our hands dirty and build our very first prototype. How do we make a prototype for an art tool in VR? We figured out that the best way was to convey our interaction is through a role play video. We did some rapid prototyping using some paper, sharpies and clips to bring our interface to life. Watch the video here.

Next, we went out to meet potential users from the Adapt community. Adapt formerly known as — United Cerebral Palsy of New York City, is a community network is a leading non-profit organization in providing cutting-edge programs and services that improve the quality of life for people with disabilities. There we got to meet Carmen, Eileen and Chris who were very interested in painting and loved the idea of drawing in VR. This kind of experience was new to them and to our surprise we got good feedback from them. All three of them expressed interest in trying out a tool that enabled them to draw using eye movements and voice commands.

A user from the ADAPT network trying VR

Once we got insights and whole lot of motivation from our users, we decided to give a structure to our interface. This time, we printed our tools on paper and stuck them together in an organized way. We used a laser pointer as the “eye tracker” and we were all ready for our second round of user testing. The whole process of prototyping for VR was very interesting and this process made us think clearly about details of the interactions and potential issues.

Our second round of user testing opened us to two kinds of scenarios based on prior experience. The users who had experience with using design software could complete the tasks well and pointed potential issues with the concept. Some of the issues were with the figuring out the z-axis, feedback for interactions, adding stamps etc. The users from the ADAPT community who did not have prior experience wanted the tools to be less ambiguous and some of the tools felt unnecessary.

Taking cues and learning from responses, we went to make the hi fidelity prototype using the Unreal engine. Unreal is a a good tool to build Virtual Reality content. To make a working prototype, there are several tasks we needed to accomplish which are as explained below –

Screen Shot 2018-04-30 at 5.54.55 PM

Final prototype

The final prototype was designed using Unreal and had the functionality for drawing using head movements, changing the stroke, changing the environment and teleporting.


The most basic function of a eye tracking painting tool in virtual reality was realized in our prototype by using head movements. User could choose tools, paint, teleport and change environment with our final prototype but it was still constrained by some technical limitations. Few functions like erase, undo, redo could not be realized by Unreal for now but we hope we can make them work by using other softwares and hardwares. We also hope to look into the technology to enable eye tracking and make it possible to select the tools using the gaze movements to draw. In our current prototype, the voice instructions are manually monitored. We would like to include this functionality as well to make a multi-modal solution for our users.

To view the complete details of this project, checkout the project booklet here.

Designing Justice

What words can’t say, design can — nothing could explain this better than a visit to Luba Lukova’s exhibition. Luba, a New York based Bulgarian graphic artist uses metaphors, juxtaposition of symbols, very few lines and text to bring out the most basic essence of humanity. I happened to stumble at one of her exhibitions at MODA in Atlanta last month.

Her posters provoke you to think and expose the injustice worldwide. Luba’s posters finely reflect on the human behaviour, hypocrisy and injustice. Something that really struck me were her series of posters on women inequality. Luba designed these posters when one of her Iranian followers asked her to address the issue of women rights in Muslim world. In this series she created images which showed how the everyday situations are different for men and women. The artist stayed away from the image used in western feminist posters, depicting women fighting for equality. I am sharing pictures from this particular series –

Another design called — Sudan, was designed by her when she watched a documentary about the impoverished country. Right after the documentary there was a commercial break promoting low calorie food. This made Luba design the poster below, throwing light on the striking contrast –

Another one depicting income gap –

Lukova’s powerful and brilliant posters are minimalistic but highlight the critical social justice issues that currently dominate us, including health care, women’s rights, LGBT rights, immigration, corruption and gentrification.

Check out more of her work here and here.


Putting People before Product

A lot of times, while building products, designers, developers & product managers tend to think of the product before the people they are building it for. In short — they are building for their convenience and not the user’s.

Almost 2 years into working on software products, here is the most important lesson I have learnt:

Do not blame people when they fail to use your product properly

What we call an error could be bad communication and interaction — There is always a reason why the user is reacting to your product in a certain manner.

Introspect, talk to users, test and figure out what could be the reason of the error. This is a simple formula we use at SHEROES, deep diving into problems users face. Here is an example:

86% of our users reported that they never received a response to the jobs that they were applying to. Now, this is clearly a problem arising because of lack of communication from a company’s end. But to think of it — the user reported this issue because they expected a reply. Why did they expect this reply? They were making an effort in filling their profile and applying to jobs.

Nowhere did the product communicate to them that they will receive a reply only if their profile is shortlisted or liked by a company. On the other hand, companies aren’t obliged to respond to the candidates who aren’t suitable.

The result — lack of replies from companies frustrated users looking for jobs and question the platform’s credibility.

Simple fix — A notification to the user when they applied, they will hear from the company only if their profile is relevant. And giving an option to companies to shortlist or reject application. While this is a very ad-hoc solution and we are experimenting with better ways to improve the interaction and communication for this particular solution.

Difficulties people face are signifiers of where you can improve. What might seem like a simple task to a developer or designer, can be a cumbersome task to perform for the user. Another example from SHEROES

We designed a system to upload events in our backend. Everytime someone uploaded an event, they got confused and lost. The developers assumed it was fairly simple and easy to navigate. We designed it according to what we thought was the correct method whereas it was not a simple to use system for the users. Based on feedback from everyone, it was concluded, what we had earlier accomplished in five screens was possible in one.

Interaction between product and a person can be a collaboration. Machines are machines and human are humans. But behind machines there are humans — if building a product think human not machine. A small example — replace error messages and warnings with help text. Error and warnings can put down the user. Build a product that understands your user and not the other way round.

While this is something we are working on at SHEROES and still feel we are far from building a product that fully understands our user. But the only way to get there is — research your user.

Got any thoughts on how we can get better? Leave a comment below.