The signage existed, it was just hard to find; but for signage to be effective, it needs to be seen.
‍
I thought the best way forward was to employ an eye-tracking research study. This approach would be cheaper and faster than A/B testing signage configurations, and would also uncover hidden user behavior and allow us to understand how testers perceive existing UI. Returned metrics could also give insight into how we could move signage to be in the best position for consumption.

My hypothesis on optimal signage configuration, which would be discovered with the help of eye-tracking hardware, was a first in the world of transit station design. Instead of placing signage in areas with the highest amount of passenger traffic, which on paper should return the highest levels of interaction, signage should be placed in “ocular magnets”, or where users naturally gaze as they navigate a station. These hotspots could be uncovered using eye-tracking glasses, possibly eliminating the task of information searching altogether.

Experimental Design

I decided to run alpha test trials as a single-blind study. I was concerned about testers inadvertently interacting with signage in an unnatural manner if they understood research context and outcomes, so testers were simply instructed to navigate towards "Exit A". Testers were sourced with a number of demographic stipulations, such as having no prior exposure to the test station, further solidifying behavior performance validity.

In addition to uncovering ocular hotspots, an objective of the study was to determine specific areas of the station that testers encountered difficulty negotiating. Part of the reason why existing signage was poorly placed was due to an imperfect understanding of user journey decision points, which was based on decade-old research with little scientific reasoning. If we could identify these specific pain points, findings could inform future design projects and streamline signage retrofitting in other stations. To solve this, I turned to Python.

Beta tester blink behavior

Eliminating user report bias

The problem with concurrent thinkaloud methodology is the inherent delay in task execution roadblock verbalization; the time it takes for a participant to explain their thought process and where they're getting stuck is longer (and comes with more delay) than the actual process itself.

This lag in self-reporting roadblocks may seem negligible, but considering the test environment circumstances and proximity of stimuli, this form of data collection would generate incorrect pain point locations.

My solution to this was to determine pain points using blink rate. Rapid blinking is commonly associated with increased mental cognition and confusion, which became the flagship metric for identifying pain points along a user's journey.

Conveniently, our eye-tracking hardware contains sensors that measure the frequency at which a wearer blinks, so a quick round of beta testers and python scripting identified these cognitively difficult sections. Isolating problematic parts of the station also sped up the environment point-cloud model scan process, which are needed by testers to generate adequate motion parallax for ocular biometrics collection. While I was running unit tests on my Python script, I was also coordinating alpha tester sourcing and scheduling.