A neuroscience game that trains prospective brain computer interface users in the eye tracking skills and provides the researchers with proficiency statistics for each patient.
Mount Sinai Hospital, New York
15 weeks, Sep-Dec 2021
with Ryan Eckert, Arianna Gong, Ebrahim Karam, Leo Lin, Rui Xi
Tobii Eye Tracker
Daily Scrum Master
The logo depicts a gaming controller made of neurons with an eyeball as the only button on the controller. The neurons depict the brain computer interface and the eye ball depicts the eye tracker.
Team Website: https://projects.etc.cmu.edu/eye-deal/
Team Branding Design
A brain computer interface (BCI) have been identified as a feasible technology for enhancing independence and participation in various activities for individuals living with severe paralysis. Many BCI technologies must be paired with accessory adaptive technologies such as eye-tracking in order to be truly functional in the intended population. Much of this required training could easily be performed in a gamified environment online, allowing prospective BCI users to become proficient in the skillsets that they require to operate a BCI before undergoing an invasive implantation surgery.
Create a fun eye tracker game, paired with a click or keystroke command that can eventually be controlled by brain signal.
The game environment progressively trains precision and speed with the eye-tracking technology and click command.
Design a dashboard back end that allows the science team to view eye tracking proficiency statistics for each patient.
Quadriplegic patients of all ages like people suffering from severe paralysis.
Abled people of all ages interested in eye tracking games.
Are people with disabilities playing games?
What makes games important to the quadriplegics?
What is an inclusive digital game?
Interviews with BCI researchers at Putrin Labs and accessible design consultants at the AbleGamers Foundation.
Players with Disablities
There are at least 46 million potential players who identify as having a disability within the US. platform manufacturers and game studios are responding to this potential market by adapting both hardware and software to accommodate diversity.
Source - Research papers provided by the AbleGamers Foundation
• Connecting: a way of bringing people together both as friends and family but also to build communities.
• Diverting: a distraction from problems and a way to relax and unwind from day-to-day stresses and to enter different worlds.
• Beneficial: games can bring about benefits to players outside of the world such as developing skills or learning about the world.
• Art: games are of intrinsic value to players because they are a form of creative expression for both developers and players.
• Fun: games are to be enjoyed.
• A way of life: players play games because that's something that they have always done and always want to do.
• Universal: players felt that games have something for everyone.
• Enabling: for the players with disabilities games were a way to be on an equal footing with everyone else.
• Autonomy: the need for personal agency under voluntary control
• Competence: the need to feel effective
• Relatedness: the need to be connected to other people
Inclusion in Digital Games
First, users need access. They need to be able to operate systems, and consume information that is presented to them.
Players need to be able to shape the challenge so that it better suits them and have options around the challenges presented in games.
Developing deep understanding of how players experience as digital games provide opportunities for diverse experiences for people with disabilities, beyond 4 walls.
Mechanics should define the limits, not the disability
Short play sessions for more engagement and wider appeal
Control personalisation like alternative inputs and subtitles
Playtest! Analyse! Iterate! Repeat!
Frustration of wonky controls in adaptive technology
Design all the actions and UI centred to the screen
Current eyetracker interfaces require some neck movement
Keep the characters and actions abstract to make them non relatable.
Some fantasy worlds and content may not be appropriate
Give incentives and rest to eyes without interrupting the experience
Long term engagement can be tiring for the eyes
Current eye tracker experiences are not engaging and tiring which makes people not want to interact with the eye tracking technology making it difficult for the researchers to train the patients with the same.
for all players
Grabs attention and immerses
Removes action-interaction dissonance
Attracts broad demographic
for all players
Simple to pick up,
hard to master
Accuracy, Precision, and Reaction Time
Guests explore the edge of the screen
Multiple moments of explicit evaluation
Simple to pick up,
hard to master
to BCI researchers
Low barrier of entry
Designing objectives for mastery
Provide analyticsto BCI researchers
5 mechanics prototypes
Playtest to discover
3 core prototypes
Final Product SpaceBull Nova
Playtest to refine
We kicked off our brainstorming with a dump of 80 or so ideas, which we consolidated and ranked into what excited us the most.
Following in faith of the playtesting spiral we refined our ideas in whatever simple way we could, including writing out designs, making storyboards, and discussing with clients to get their initial reactions.
We took our best ideas and turned them into extremely simple prototypes. We explored mechanics like gazing, chasing, flinging, and shooting
We designed around animal and Space themes to avoid any action or interaction that could be offensive to people with disability and give them a world full of imaginations and fun.
Throughout week 2 & 3 we were able to show our demos to clients and they were able to tell us which prototypes met their requirements.
Following our client responses we analysed each prototype for how well it aligned with their requirements, interests of the players, and production opportunities for the team. We trimmed our scope to the 3 ideas, and identified the major concerns with each prototype.
Addressing concerns like motion sickness, depth analysis of 3D space, training for precision, accuracy and reaction time, and artistic, and narrative scope and combining the fun mechanics and other pros like simple UI, fun eye lasers, ease of attaining analytics, wider demographic appeal, and look away/restful mechanics we built them into 3 new prototypes which were later realized into our final product SpaceBull Nova.
3 Core Prototypes
+ Tracking the enemy
+ Wider range appeal
- Confusion about enemies / targets
Hero Bull Runner
+ Flinging is awesome
- Sharp turning feels bad
- Get lost w/ the objective
Endless Runner Bull
+ Rewarding Combos
+ People felt excited about it
+ Very replayable
- Easy to get tired by the fast pace
Playtest to Discover
To decide our final direction we playtested the 3 prototypes with the paralysed patients at Putrino Lab. We also playtested with the abled players using an unintuitive large enter button triggered by foot. We wanted to discover which of the games provided-
We used Google survey and observation methods to collect data.
which game engaged players for a longer duration for training purpose
which mechanics provided an easy and effortless eye tracking experience
which worlds and characters the players enjoyed the most
Final Direction - SpaceBull Nova
The playtest results suggested how the players enjoyed parts of all 3 prototypes which led us to combine those into our final product SpaceBull Nova. Upon several playtests and design iterations we divided the game into 2 modes campaign and arcade.
SpaceBull Nova - Campaign Mode
Introduction to the Fantasy & Mechanics (Gaze, Chase & Fling)
Introduces Shooting MiniGame (provides necessary analytics)
SpaceBull Nova - Arcade Mode
Gradually introducing more challenge by level design.
Reinforcing a sense of mastery so players want to keep playing.
Playtest to Refine
Playtesting was our decision maker. We conducted playtests with the quadriplegics assisted by the doctors on a monthly basis and received feedback through quantitative surveys. We also conducted weekly playtest with surrogate playtesters and recieved feedback through interviews, observation and surveys.
For easy understanding and readibility
Simple and Intuitive
For ease of access in patients with no neck mobility
Centred to the Screen
For bringing the game to life and more engagement.
Visual Rich and Juicy
The UI is designed to look like a part of a futuristic spaceship.
Campaign mode UI has a more intense feel as it is visually clean and subtle.
Arcade mode is stylised and vibrant with a carnival like feeling to attract the younger population.
Game View Design
Hard to identify objects due to lack of visual clarity.
Normal State - 66ffff
Hover State - ffffff
Game Screen Design
Game Screen Design - Campaign Mode
Game Screen Design - Arcade Mode
(Hover on the button images)
To make our game a visual treat for our players, I was took up the task to create juicy and satisfying visual effects in Unity.
" We hope to use this game to introduce quadriplegic patients to assistive technologies including eye-tracking itself and brain-computer interfaces that might benefit from the addition of eye-tracking while learning to use the technology. Our expectation is this fun game will increase patient engagement, which is so important to achieving successful outcomes. "