Graphic Design
Overview
ROLE: Researcher & UI/UX Designer
CLIENT: Mount Sinai Hospital, New York
TIMELINE: 12 weeks, 2021
TEAM: 3 Developers, 1 3D Artist, 1 Game Designer
TOOLS: Illustrator, Figma, Unity 3D
Challenge
Allow prospective Brain Computer Interface(BCI) users to become proficient in the skillsets that they require to operate a BCI before undergoing an invasive implantation surgery.
-
Create a fun eye tracker game, paired with a click or keystroke command.
-
Progressively train for precision. accuracy and speed with eye-tracker and click command.
-
Design a dashboard back end that allows the science team to view eye tracking proficiency statistics for each patient.
Background
The Strentrode is a novel BCI technology that has the potential to restore functional computer use in people with severe paralysis. Stentrode technology pairs with Tobii eye tracking technology in order to enable cursor control with eye movements and multiple brain-controlled click functions.
Much of this required training could easily be performed in a gamified environment online, allowing prospective BCI users to become proficient in the skillsets that they require to operate a BCI before undergoing an invasive implantation surgery.
Assumptions
Upfront, I wanted to take note of any assumptions I had going into the project, which I could then prove or disprove based on research and testing, helping to steer the design direction.
-
Quadriplegic people are reluctant to play video games.
-
Players with disability have different experiential goals from the wider population of players.
-
Quadriplegic people will prefer easy levels of challenge and simplistic visual information.
Research
SECONDARY RESEARCH
I began the research phase by conducting secondary research, first by completing a market research to understand the existing advancements in the field of eye tracking and BCI, continuing with research in accessible player experience, and a competitive analysis.
Market Research
Market research helped me in understanding the existing range of assistive technologies, and how they have impacted the lives of people with severe paralysis. I especially wanted to get a sense for what effect games have on people's emotions, intelligence, creativity and just a person’s well-being.
“The eye mouse isn’t just an IT device,
but arms and legs for a patient with advanced disease”
- Hyung-Jin Shin of Yonsei University, who was born quadriplegic and helped in testing the Samsung EYECAN+
“For our users, it means so much more than being able to read and write,
it’s about independence."
- Fredrik Ruben, president of Tobii Dynavox
Accessible Player Experience
Through this research I better understood what it means to design something inclusively, from basic access where users can perceive and operate aspects of the system, to enablement where they can achieve goals, which then yield different experiences.
I gained insights on players with disabilities: who they are as players, the types of games they play and the prevalence of use of assistive technologies and accessibility settings in games. It was intriguing to learn that people with disability were playing games of large formats just as the wider gamer population, and that the accessibility options are also being utilised by the population of abled gamers to enhance their experience.
Images sourced from APX:The Players published by Department of Computer Science, University of York & AbleGamers Foundation, WV
Competitive Analysis
The competitive analysis showcases the strengths and weaknesses of existing Tobii eye-tracker games. I analysed the games for features that engaged or frustrated me and then evaluated them on the physical limitations of our target demographic of severely paralysed patients.
Problem statement:
Current eye tracker experiences are not engaging and tiring which makes people not want to interact with the eye tracking technology making it difficult for the researchers to train the patients with the same.
PRIMARY RESEARCH
The main portion of the research phase were the user interviews because they allowed for a direct insight from users with disabilities. To understand the interests of patients with severe paralysis who couldn't interact verbally, I interviewed neurologists, BCI researchers and the medical attendants who worked with the patients on a daily basis.
User Interview
Overall the interviews discussed the experiences of users with disability playing video games, what makes games important for them, what are their interests in games, and how games enable them.
" It's frustrating if the tracking isn't responsive."
"I can play for a short duration because I get tired soon."
"I'd love to play multi levels if I get the controls right."
"I want to be able to play games just like everyone else."
Along with quadriplegic patients, I also interviewed abled users from 15-60 years of age who play games, to understand people's connection with games in general because getting behind that reasoning could really steer the direction of engaging quadriplegic patients of all ages with diverse interests.
User Empathy
Upon completion of the interviews, I synthesized the results into various categories based on the different kinds of feedback in order to identify insights that would help to better the user experience. I focused on the users' interests in gaming and their experiences with the eye tracker interfaces.
Insights
I was able to gather key insights from the empathy map findings. The biggest insights I discovered were related to players frustrations with unresponsive controls that interrupt an enabling experience.
-
Users enjoy playing games and find it a fun way to spend time.
-
Users get frustrated by complex mechanics and unresponsive controls.
-
Users with no neck mobility avoid long game sessions as they get tired.
-
Users prefer control personalisation like alternative inputs and subtitles.
Needs
Rounding back to the empathy map by referencing the insights, I built a list of user needs that would dictate many decisions moving forward in the creation of a highly engaging experience.
-
Users need to be engaged and boost their mood.
-
Users need easy to learn mechanics and intuitive controls.
-
Users need short play sessions.
-
Users need to be able to personalise the controls.
Define
Persona
Using the insights and feedback from the empathy map, as well as the research gathered to this point, I was able to build a persona that would represent the user base for designing the game moving forward - Enzo. He loves to play games with minimal eye-tracking effort.
Problem Statement
Current eye tracker experiences are not engaging and tiring which makes people not want to interact with the eye tracking technology making it difficult for the researchers to train the patients with the same.
HMW Questions
The “How Might We” questions intend to foster discussions for potential solutions that could better the user experience.
.png)
Business, User, and Technology Goals
I began to set the design pillars for the game by first defining the goals of the project, from different perspectives, as well as how they overlap. The mutual goals are dictated by much of the research that's been done this far.
Game Design Pillars
I began to set the design pillars for the game by first defining the goals of the project, from different perspectives, as well as how they overlap. The mutual goals are dictated by much of the research that's been done this far.
High engagementfor all players
-
Grabs attention and immerses
-
Removes action-interaction dissonance
-
Attracts broad demographic
Simple to pick up,hard to master
-
Low barrier of entry
-
Designing objectives for mastery
-
Encouraging replayability
Provide analytics to researchers
-
Accuracy, Precision, and Reaction Time
-
Guests explore the edge of the screen
-
Multiple moments of explicit evaluation
Ideate
Building design hypothesis:
I planned, mediated and participated in the brainstorming sessions where the team came up with design hypothesis based using our design pillars as the foundation. We kicked off brainstorming with 80 or so ideas, which we consolidated and ranked into what excited us the most.
Following in faith of the playtesting spiral we refined our ideas in whatever simple way we could, including writing out designs, making storyboards, and discussing with clients to get their initial reactions.
Playtests and Iterations
Playtesting was our decision maker. I was responsible to design the weekly design sprints, and plan, conduct and execute weekly playtests, and provide actiontable insights to the team. We used methods like A/B testes, direct observation, surveys and interviews to conduct playtests with the BCI researchers and users. I also led the creation of research materials like the survey and interview guides.
With the help of a few willing participants, I was able to lead and contribute to a group brainstorming session. I first started with the question “How might we bring back the 90s?” to get everyone comfortable with the process and kick up some nostalgia. Backstreet's back, alright?
Playtest limitations:
Due to the physical limitations of the quadriplegic patients, we got an opportunity to playtest with them only once the product was close to finishing where we used likert scale surveys to get their responses by the assistance of their doctors.
To overcome this challenge and be able to conduct playtests every week during the project lifecycle, I recommended we tie the abled played to the chair and use an unintuitive large enter button triggered by foot, to see if they were still able to access the functionality of the game intuitively and with ease. This was as close as we could get to the real users' experience in the given situation. The feedback from the BCI researchers also played a key role throughout our process.
.png)
Mechanics Prototyping
Playtest to validate design hypothesis:
We took our best ideas and the developers turned them into low-fidelity prototypes that showcased a variety of game mechanics like gazing, chasing, flinging, and shooting.
We designed around animal and Space themes to avoid any action or interaction that could be offensive to people with disability and give them a world full of imaginations and fun.
We presented our demos to the BCI researchers and our clients to access which prototypes met the users' requirements and were easy to use and enjoyable for the quadriplegic patients. We also conducted playtests with our friends and colleagues to understand how these game mechanics were experienced by the abled population.
Playtest analysis:
Following the responses from the researchers and our clients, we analysed each prototype for how well it aligned with the clients' requirements, needs of the users, and production opportunities for the team. We trimmed our scope to the 3 ideas, and identified the major concerns with each prototype.
Result- 3 core prototypes:
After validating the training and fun aspects of each of our early 5 prototypes with the BCI researchers, we went on to iterate them into 3 new prototypes that addressed the concerns like motion sickness and depth analysis of 3D space, exploring the scope for training for precision, accuracy and reaction time. We evaluated and combined the mechanics that were enjoyed by the playtesters like eye lasers and looking away/restful controls, and themes that appealed a wider demographic.
Prototype 1 Spaceship
Prototype 2
HeroBull Runner
Prototype 3
Endless Runner
Playtest to dicover:
Utilising direct observation, survey and interviews, we tested the new prototypes with the users' to answer-
-
which game engaged players for a longer duration for training purpose ?
-
which mechanics provided an easy and effortless eye tracking experience?
-
which worlds and characters the players enjoyed the most?
Link to survey I was responsible to design for this playtest.
.png)
.png)
.png)
.png)
.png)
.png)
.png)
Prototype 1 Spaceship
Prototype 2
HeroBull Runner
Prototype 3
Endless Runner
Playtest analysis:
Prototype 1
Pros-
+ Freedom/Exploration
+ Tracking the enemy
+ Wider range appeal
Cons-
- Confusion about enemies / targets
- Disorientation
Prototype 2
Pros-
+ Flinging is awesome
+ Minigames
Cons-
- Sharp turning feels bad
- Get lost w/ the objective
Prototype 3
Pros-
+ Rewarding Combos
+ People felt excited about it
+ Very replayable
Cons-
- Easy to get tired by the fast pace
Result- 2 game modes:
The playtest results suggested how the players enjoyed parts of all 3 prototypes which led us to combine those into our final product SpaceBull Nova. Upon several playtests and design iterations we divided the game into 2 modes campaign and arcade.
Campaigh Mode:
-
Introduction to the Fantasy & Mechanics (Gaze, Chase & Fling)
-
Introduces Shooting MiniGame (provides necessary analytics)
Arcade Mode:
-
Gradually introducing more challenge by level design
-
Reinforcing a sense of mastery so players want to keep playing
Playtest to refine:
Utilising direct observation, survey and interviews, we further tested our final direction with the users' to refine and scale the game by questioning-
-
did they understand the goal?
-
did they understand the game mechanics?
-
did they understand the visual elements including the art assets and UI?
-
would they be interested in playing more levels of this game?
Link to survey I was responsible to design for this playtest.
UI Design
User Flow
Goals
Based on the users' needs and limitations that I learnt from the user research, I defined a set of design goals that guided my design process and tested the mockups of my designs weekly with the surrogate users to ensure the designs met the intended goals.
Simple and Intuitive
For easy understanding and readibility
Centred to the Screen
For ease of access in patients with no neck mobility
Visually Rich and Juicy
For bringing the game to life and more engagement.
Design
Game View
Distance from enemy
Speed of spaceship
Speed of enemy
Wireframe
Mockup
Game Screens
Distance from enemy
Speed of spaceship
Speed of enemy
Wireframe
Mockup
Iterations
# Non-distracting UI for uninterrupted game interaction
Redundant Distance from enemy
Speed of spaceship
Speed of enemy
Before testing
After testing
# Big & widely distanced buttons for easy eye-tracking
The users found the side to side placement and less distance between the buttons difficult to navigate using the eye-tracker. They ofter accidentally clicked the adjacent button that they did not intend.
For some users the level of effort required to navigate from one button to another was more than expected or a great deal.
Before testing
After testing
.png)
# Visual clarity for intuitive control
Increase the contrast between the background and the objects by simply changing the background color.
Before testing
.png)
After testing
Hue
.png)
Saturation
.png)
.png)
.png)
.png)
.png)
Final Designs
.png)
.png)
.png)
.png)
UI Kit
Visual theme:
The game is based in space where the player plays the role of a pilot driving a spaceship chasing away the enemies. Supportive of this theme, the UI is intended to be look like controls in a spaceship to enhance the users' experience of driving a spaceship and make them feel more immersed in the experience.
-
Campaign mode UI generates an intense feeling and is aimed for people who like more elegant and simple visuals.
-
Arcade mode generates a more playful carnival like feeling for people who enjoy stylised and vibrant visuals.
From the playtests I learnt that the style of visual for the campaign mode attracted more of the older population while the younger population preferred the arcade mode more for its playfulness.
Visual Effects
To fulfil our experience goal of engaging the users for long to be able to increase the training session lengths, the visuals effects played a key role. They allowed our game to be a visual treat for our players by adding visual juiciness and a satisfying feeling.
I enjoyed creating all the visual effects utilising the graphic nodes in Unity 3D.
Making of the visual effects:
Impact
" We hope to use this game to introduce quadriplegic patients to assistive technologies including eye-tracking itself and brain-computer interfaces that might benefit from the addition of eye-tracking while learning to use the technology. Our expectation is this fun game will increase patient engagement, which is so important to achieving successful outcomes. "
- Dr. Adam Fry, Mount Sinai Hospital
- Dr. Dev Sharma, Phd Researcher at CMU
Team Branding Design
I was responsible to design the branding for team Eyedeal, a team of 6 students from Carnegie Mellon University responsible for this project for Mount Sinai Hospital.
The logo is inspired from a gaming controller that is made of neurons that depict the neuroscience genre of the game, with an eyeball as the only button on the controller depicting the utilisation of the eye track and a single click command.
The poster extends the theme of neurons and depicts a brain with an eye ball representing the BCI that utilises the eye tracking technology. The elements hanging from he brain represent the gamified and digital nature of the product.
.png)
Team logo
Team poster
