How can we measure the impact of our work as designers in a scalable way that also communicates our progress clearly to external audiences?
Part of my job as the UX & Design manager is to socialize the successes of the team. Everyone in the organization sees our final designs, but I wanted to make sure stakeholder partners thoroughly understand our process—research included.
In the past, we had run ad-hoc findings presentations after key research project. I wanted to supplement these with something that:
- Doesn't take a lot of time to do
- Is always available and up-to-date
- Provides a meaningful level of detail for non-designers
- Allows us to measure improvement over time
Something quantitative seems to fit the bill. The organization had used periodic consumer satisfaction surveys in the past, but that wasn't going to be sufficient for our stated goals.
Instead, I made the case to our Product and Software Dev leaders, and eventually integrated a survey and usability testing tool (UserZoom) into our software, allowing me to launch studies and surveys whenever I needed to. This is how we now do most of our usability testing, but it is great for intercept surveys as well.
To establish meaningful KPIs, we licensed SUPR-Q, a generalized measurement of user experience across 4 categories: usability, credibility, loyalty, and appearance. We combined this with a question to gather the user's intended task and general consumer satisfaction to build a 10-question survey that gives us (subjective) quantitative metrics. Combining this with behavioral data from Analytics gives us a detailed way to talk about the user experience in general, as well as a diagnostic tools for areas where we should follow up with qualitative research and design.
I would like to let a research program like this run for at least a year before analyzing its success or accuracy or usefulness, but so far the results from our UX KPI studies tell a reasonable story. The KPIs reveal general strengths and weaknesses in the experience that are in-line with anecdotal evidence. We've seen our Usability metric rise with the release of a navigation-focused redesign. We can segment consumer satisfaction by user intention (i.e. people who come to the site to do X have a much worse perception of the site than others). As an added bonus, this ability to measure the experience continuously helps our healthcare-related products meet accredidation requirements more efficiently.
- Study Build
- Research Plan
- Business Case
Cover photo by Peter Heeling