I am a connector. With research, I aim to bridge the gap between designers, engineers, and their users.
usability test in mockup 2.png

Swedish Express Care Usability Study

Swedish Express Care Usability Study

Usability study conducted for a client at Providence Health & Services for their Swedish Express Care site as part of a Usability Studies course

 

Overview

Timeline

8 Weeks
Jan 2018 - Mar 2018

 

 

Roles

Usability Researcher
Project Manager

 

Team

Sonnet Lauberth
Alice Lin
Corynne Umeda
Mandy Xu

Company and Website Overview

Providence Health & Services is a health care system headquartered in Washington state. Swedish Medical Group is one of Providence's subsidiaries. Swedish Express Care is a collection of medical services that provides express medical services for low-acuity health problems, such as a colds, skin irritation, and sprains, as opposed to high acuity, emergency problems like breaks and heart attacks.

 
 Swedish Express Care Homepage

Swedish Express Care Homepage

 

There are 4 services, as shown above, and the Express Care website allows patients to learn more about each service and schedule appointments.

Project Goal

Our client at Providence was a Senior UX Researcher on the UX team and asked us to conduct a usability test on Swedish Express Care's website. Specifically, he wanted to find out if users could use the site to determine which Express Care service they wanted to use.

 

Research Questions

In order to fulfill the client's research goal, we identified 3 research questions to guide our usability study.

01 Do users comprehend what the different Express Care Options are?
 

 

02 How easily and successfully do users discover which conditions are treatable by Express Care services?

 

03 How easily and successfully do users discover what insurance Express Care services take?

 

Methods

Participants

All participants were recruited through User Research International based on the client's criteria. Participants were to have never interacted with the Express Care site before, have booked a medical appointment in the last 6 months, and not be afflicted with chronic high-severity conditions. In addition, our client wanted to look specifically at 3 segments of people, Millennials, Gen Xers and Parents with children under 17 years old. Full screener can be found here

P5a. Presentation.png

Study Overview

Each session was one-on-one and involved a participant, moderator, and notetaker. The sessions consisted of a Task Observation where participants undertook 3 tasks on the Express Care website, and a Semi-Structured Interview.

Task Observation

The first part of our study had participants completing 3 tasks using the Express Care website. We chose to do a task observation to see exactly where the breakdown (if any) of using the site was. These tasks were designed to address our research questions 2 and 3, as they let us look at how users went about looking for information like insurance accepted or problems the different services could treat which might inform their decisions in choosing a specific service.

 Participant completing usability study

Participant completing usability study

The participants were given a scenario that asked them to complete the following tasks (pictures are of the target sections of the website for each task):

Task 1 - Treatable Condition

Determine which of the services, if any, could treat pink eye.

Task 2 - Untreatable Condition

Determine which of the services, if any, could treat a broken leg.

Task 3 - Insurance

Determine if the services take "Premera" insurance.

Participants were instructed to think aloud during the tasks. We observed participants' navigation of the site in completing their tasks and also recorded the screens and participants using Morae. The specific task protocol can be found here.

Semi-Structured Interview

After the task observation portion of the sessions, we asked 4 questions, some of which addressed the overall usability of the site. The 3rd question aimed to uncover if users comprehended the differences between each Express Care service (research question 1). We thought a participant-provided explanation of each of the services would best show that particular user's comprehension of the services rather than a task, as tasks often do not show the user's mental models even if thinkaloud is encouraged. 

 
 Protocol for semi-structured interview

Protocol for semi-structured interview

 
 

Data Analysis

We conducted data analysis by reading over our notes from sessions and rewatching the videos from Morae. We used RealTimeBoard to create affinity diagrams and userflows for each task.

ml2rZ5z (2).jpg
affinity diagram example.PNG
user flows.PNG
 

Findings

  1. Do users comprehend what the different Express Care Options are?
    • Yes, somewhat. A lot of it is through pre-existing assumptions.
  2. How easily and successfully do users discover which conditions are treatable by Express Care services?
    • Most could discover the list of treatable conditions, but with difficulty (extra steps taken, thrown into loops).
  3. How easily and successfully do users discover what insurance Express Care services take?
    • Most participants discovered Premera was taken, but with difficulty (extra steps taken, thrown into loops).

We found 3 main reasons for the usability issues our participants experienced, explained below. A comprehensive detailing of our findings can be found in the full usability research report.

1. Information Overload

Participants felt that many of the pages, especially the home page, were too busy and led to confusion of where to seek wanted information.

Actually figuring out which one of these I need to go to first felt like it took too long and felt kind of frustrating. It was frustrating because it felt like it took me too long to figure out which one to go to.
 
 There are at least 13 interaction points just on the home page.

There are at least 13 interaction points just on the home page.

 

 

2. Confusion Over Site Structure

Participants had trouble completing each task because at many points, a link on the homepage would lead to anchor points at the bottom of a new page. It was also difficult to find information about insurance and took many different types of steps between each service. Information about insurance for Virtual Care, for example, was in the FAQ section, while it was on a separate page for At Home Care.

I think it was a little klutzy. It was kind of hard to find. I think they could organize it better. There was really nothing on the main page about insurance. Like this [the “conditions we treat” and “insurance accepted” navigation bar] should be on the homepage.

 

3. Page Format Inconsistencies

Another point of confusion was the inconsistent way the same information was presented across the different pages for the different services.

When I click on virtual care, it has a different layout versus clicking on express care [clinic]. I have to look around and find information for each.
 The "Conditions we Treat" sections for 3 of the Express Care Services. Similar information, but very different locations and formats.

The "Conditions we Treat" sections for 3 of the Express Care Services. Similar information, but very different locations and formats.

 

Impact + Feedback

We presented our high-level findings to the UX, finance, marketing, and engineering teams at Providence. The presentation deck can be seen here.

One of the UX team's goals for the year is to redesign the Express Care website for the rest of the year and indicated they will be considering our findings when undergoing redesign efforts.

This is great. I will be getting to work on these when we start the design reworks.
— Jonathan M. (Providence UX Designer)

The study also started conversation among the different teams, especially the ones more removed from design, about the importance of the website's layout in customer experience and satisfaction. For example, the finance, marketing, and UX teams agreed to set up a future meeting to discuss the findings further and what it meant for the workflow for the next few months.

 

Takeaways

  • Spend time thinking through how data is going to be used before undertaking data analysis: We first thought to do a quantitative analysis comparing the number of clicks and scrolls each person completed while doing each task. However, many of the tasks actually fed into each other and a mere quantitative comparison would not be meaningful. The process to count each action by rewatching each session wasted time, and in the end, the data was not valuable for our client.
  • Standardize protocols and have each moderator practice multiple, or just have one moderator to ensure consistency: We had very inconsistent moderation due to wording and different styles of encouraging participants to think aloud. We feel many results and different ways of approaching the tasks resulted because of the moderation difference. As this was a class, and every team member was required to moderate, we could not stick to one moderator, but in an industry situation, I would opt to have a single moderator, or standardize protocol down to the prompts to get participants back on track.
  • Do pilot runs of tasks before testing on real participants: We found that one of the tasks (finding that a broken leg was not treatable at any of the Express Care services) was a sort of "trick" task, as this information was never stated. However, the wording of the task, "where can you treat a broken leg" made participants assume it could be treated at one of the services. This confusion over the task was only discovered after we had already tested 4 out of 10 participants as we scheduled back-to-back sessions and did not have time to reflect between. Doing a pilot would have easily allowed us to discover this high-level problem in our protocol and may have led to different results for the task.
  • Many times design decisions are not autonomous to the UX and design teams: Our team was surprised when one of the Providence team members interrupted the presentation to say that they were aware of the inconsistencies with the Virtual Care website, but could not do anything about it because the page was standardized to the video service they used for virtual appointments. Despite obvious usability issues in some sites, there are often bureacratic matters design teams must face in redesigning a website.