Can medical devices become discriminatory?
Risking unconscious bias
The issue of unconscious bias can arise during the design of any product, including many of the ones we use daily. For example, social media platforms are used by billions of users, however the way in which they function is often dictated by just a small number of software developers. When the infamous ‘like’ feature was introduced to Facebook in 2009, the developers behind it viewed the feature as merely an innovative way to determine the interests of different users, in order to better tailor the content on their newsfeed. With the designers being predominantly young, male engineers, they did not predict the immense social and mental impacts this small feature would go on to have for the millions of users. It was not long before social media appeared to correlate with depression and anxiety among young people, with many users experiencing negative impacts in the event that they receive especially few 'likes' on a post.
Gender bias and implications for artifical intelligence
The introduction of new technologies such as Artificial Intelligence (AI) and machine learning for the diagnosis and treatment of conditions presents an additional cause for concern around design bias. Team Consulting’s Thorbjorg Petursdottir highlighted in her article 'Is AI Sexist?', that there have already been a number of instances where software developers have inadvertently coded bias against certain genders into algorithms.
Bias such as this could have serious consequences in the medical space, notably when AI is used to help diagnose illnesses or symptoms. An insufficient data set, such as one based on historical data that favours one gender, could lead to an algorithm diagnosing the favoured gender more accurately, or failing to diagnose the other entirely. If we want to continue exploring the use of AI and machine learning in healthcare, it will be essential to account for such errors to ensure patient safety.
Accounting for different user groups
While more research is needed in this field, studies have also found some potential physical differences between different ethnic groups which may need to be accounted for in the development of medical devices. For example, a study conducted on four different ethnic groups used computational fluid dynamics to determine differences in nasal spray deposition. Though conducted on only a small number of subjects, the study found that the Caucasian and Latin American subjects had a lower patent nasal cavity and lower consistency in particle deposition patterns compared to African Americans and Asians. These differences could indeed need to be accounted for in the development of drug delivery devices such as nasal sprays, to ensure patients of different ethnicities get the appropriate dosage and treatment they need.
The way forward
Without sufficient checks and balances in place, unconscious bias can become a serious issue in the design of medical devices. Medical device manufacturers must work to avoid this by drawing on inputs from a wide variety of stakeholder groups throughout a device development, from the engineers creating the device to those testing it. A conscientious design team should incorporate human factors approaches throughout the device development process as well as, employing user studies, design research and analysis to gain insights into how different users interact with a product, as well as their understanding and expectations of it. User studies are imperative in the identification of the strengths and weaknesses of a device, as well as an important way of ensuring that different users are placed at the heart of the design process and preventing discrimination.