Healthcare Device Evaluations

home healthcare devices
Home-healthcare devices: Koogeek and Littman Stethoscope.

Background

This project was completed for a course at UNC Chapel Hill in Human Factors in System Design (INLS 418) with one other student. The goal of our project was to compare the usability and functions of three healthcare devices:

  • Clinicloud: a stethoscope and non-contact thermometer integrated device and mobile application, for home healthcare use.
  • Koogeek Smart Thermometer: a digital thermometer and mobile application, typically for home healthcare use. We used this to compare the Clinicloud thermometer function.
  • Littmann Electronic Stethoscope: a digital stethoscope with accompanied learning mobile application. While not typically used for home healthcare, it was a good tool to compare Clinicloud’s stethoscope functions.

While this was a comparative evaluation of the 3 devices, the goal was to analyse where Clinicloud may have succeeded or failed in delivering a device that integrates more than one healthcare function for use at home. The desired result of this comparative evaluate was a set of design recommendations for home healthcare applications that integrate several functions.

Phase 1: Heuristics and Device Evaluation

The first phase of this project was first performing a heuristic evaluation (i.e. “expert” usability evaluation) of the devices and applications and researching the intended users of the devices to develop personas and task scenarios to inform how to conduct usability research. Researching users was largely done by reviewing what the intended users for each device are from the manufacturers, as well as looking at user reviews on websites like Amazon. Determining the heuristic evaluation to use was more complicated because we needed a set that covered both the physical device issues and mobile applications. We used Zhang’s 14 Heuristics for evaluating medical devices, as it incorporated elements of interface heuristics such as Nielsen Norman’s along with additional considerations for physical devices and healthcare. Myself and my colleague rated the each devices separately and then compared results. Each device was given a rating from 0 to 3, where 0 meant there were no issues and 3 represents issues that must be addressed to be useful/functional. Below is a sample of the results for the first 4 heuristics:

Heuristic
ClinicloudKoogeek Thermometer Littmann Stethoscope
Consistency0 – theme remains consistent in app and aligns with device1 – each app page has a different appearance but still aligns with device function0 – theme remains consistent in app and aligns with device
Visibility0 – uses color coding to distinguish features2 – user knows where they are at, but no feedback to explain what they are looking at0 – has many words and directions to alerts user of what they are looking at
Match0 – application has images and colors to match. Icons signify  real world representations1 – has images for the user to have to assume what it is, limited description.0 – has images and colors to match word descriptions and real world representations
Minimalist0 only has 8 features organized in buttons2 – only has 4 buttons but it is hard to assume what each are and has many pop up features…..
Table of Zhang’s Heuristics 1-4 for each of 3 devices.

We also did some initial user testing sessions with all three of the devices and their applications. We had 4 tasks: (1) use Clinicloud to record temperature, (2) use Clinicloud to record heart rate, (3) use Koogeek to record temperature, and (4) use Littmann’s Stethoscope to examine heart rate. Data was collected on task performance (completion, screens visited, errors), post-task questionnaire responses, post-session open-ended questions as well as observations.

From the heuristics and user testing, some of the key issues we found overall was that instructions to do task were difficult to find in most systems, especially Clinicloud as it offered two separate functions. Aside from Littmann which is marketed towards practitioners, the home healthcare devices don’t really offer enough details or feedback to troubleshoot when results don’t appear correct. One issue for Clinicloud specifically is that it lacked a meaningful hierarchy to its functions on the mobile application, It had several buttons for recording data from devices such as heart rate monitors that the system did not have, which were given the same emphasis as their core thermometer and stethoscope functions. The results from user testing supported these findings, as users were not very confident they were doing the right thing and did not find enough information or helpful feedback.

PHASE 2: Prototype and Testing

Based on our analysis in the first phase, we wanted to test some of the more problematic aspects of the devices by creating a prototype in Adobe XD for a single application to be used with a device like Clinicloud. The prototype changes largely involved changing the hierarchy of tasks and better informing users about their results. For an example, see the homepage comparison below:

Original Clinicloud homepage
Clinicloud homepage
Prototype clinicloud homepage
Prototype homepage

We tested the prototype using a similar script to what we used on the original devices. We found that confidence in task completion and ease of use increased, likely due to having improved feedback and clearly showing which tasks were most essential and which were extra.

Recommendations

Based on the findings of this study, we came up with 3 key recommendations for designing home-healthcare devices that integrate different functions:

Ease of Use vs. Accuracy of Use: Often applications designers push for simple diagrams or sets of instructions in order to not overwhelm users with details, but when too many details are missing the device is rendered ineffective. Device designers should aim to offer users all necessary details to operate the device with accuracy, and only hide detailed information that is not necessary.

Answer the User’s Key Questions: Users purchase home healthcare applications primarily to answer one question: Am I healthy? Home healthcare applications should do their best to offer users feedback that compares their test results to established data on what is considered a healthy reading based on the user’s profile.

Don’t over-rely on applications: Users purchase a home healthcare device for the physical device, so the application should be a bonus to assist the users in tracking past information, easily inputting a user profile and other features such as connecting the device results to other health applications.