The danger of unconscious bias
Unconscious bias can affect the design of any product, including many of the ones we use daily. For example, social media platforms are used by billions of users however, the way they function is often decided by only a small number of software developers. When the infamous ‘like’ feature was introduced to Facebook in 2009, the developers viewed the feature as simply an innovative way to determine the interests of different users, in order to better tailor the content on their newsfeed. With the designers being predominantly young, male engineers, they did not predict the immense social and mental impacts this small feature would go on to have for the millions of users. It was not long before social media appeared to correlate with depression and anxiety among young people, with many users being negatively impacted in the event they receive especially few 'likes' on a post.
The issue of unconscious bias can occur in medical device development as well, without appropriate oversight. For example, designers might introduce their own unconscious biases in the physical design of a device, by not sufficiently testing it on a wide enough variety of users. Such design bias has recently been brought to light in the U.S. for example, with senators calling on the FDA to launch a post-market study of pulse oximeters and skin colour, following reports that these devices have a lower accuracy for patients with darker skin. Though more information is needed to confirm both the accuracy and potential cause of these reports, issues such as these likely point to a lack of appropriate testing or consideration for different user groups during the device development.
Gender bias and AI
New technologies such as Artificial Intelligence (AI) and machine learning used in the diagnosis and treatment of conditions is another cause for concern around design bias. As highlighted by Team Consulting’s Thorbjorg Petursdottir in her article 'Is AI Sexist?', there have already been a number of instances where software developers have inadvertently coded bias against certain genders into their algorithms.
Such bias could have serious consequences in the medical industry, especially when AI is used to help diagnose illnesses or symptoms. A poor data set, such as one based on historical data that favours one gender, could lead to an algorithm diagnosing the favoured gender more accurately, or failing to diagnose the other entirely. If we wish to continue exploring the use of AI and machine learning in healthcare, it will be essential to account for such errors to ensure patient safety.
Considerations in design for different user groups
More research is needed in this area, though studies have also found some potential physical differences between different ethnic groups which may need to be accounted for in the development of medical devices. For example, a study conducted on four different ethnic groups used computational fluid dynamics to determine differences in nasal spray deposition. The study was conducted on a small number of subjects, though the findings revealed that the Caucasian and Latin American subjects had a lower patent nasal cavity and lower consistency in particle deposition patterns compared to African Americans and Asians. Differences such as these could well need to be accounted for in the development of drug delivery devices such as nasal sprays, to ensure patients of different ethnicities get the appropriate dosage and treatment they need.
Finding a way forward
Without the right checks and balances in place, unconscious bias can become a significant issue in the design of medical devices. Medical device manufacturers must be aware to avoid this by drawing on inputs from a wide variety of stakeholder groups throughout a device development, from the engineers creating the device to those testing it. A diligent design team should incorporate human factors approaches throughout the device development additionally, employing user studies, design research and analysis to gain insights into how different users interact with a product, as well as their understanding and expectations of it. User studies are essential to identify the strengths and weaknesses of a device, as well as an important way of ensuring that different users are placed at the heart of the design process and preventing discrimination.
