Member Article

Lack of transparency is hindering AI adoption in the healthcare sector

A new study investigating the reasons behind slow AI adoption in the healthcare sector has revealed users are reluctant to place the same levels of trust in machines as they are in human healthcare providers, because they do not understand how the technology works.

The research, conducted by Dr. Romain Cadario at the Rotterdam School of Management Erasmus University, alongside Dr Chiari Longoni and Professor Carey Morewedge at Boston University Questrom School of Business, reviewed the importance of technology transparency when urging consumers to adopt AI-enabled healthcare technologies, such as healthcare apps and other monitors, into their day to day lives.

Dr Cadario says: “Artificial intelligence is revolutionising the healthcare industry, offering services at a cost and scale that makes healthcare more accessible and affordable to more patients in both developed and developing countries. Many AI-driven diagnosis can perform comparably or even better than human specialists, as well providing easier access to information for patients outside of clinical settings via smart devices. However, the clarity of these benefits is often matched by the opacity with which they are delivered, which presents a major barrier to their adoption.”

By conducting five online experiments with a sample of 2,699 American citizens, the study revealed that consumers wrongly believe that they can better understand decisions made by human healthcare providers than any comparable decisions made by their AI-led healthcare counterparts, even when they lack any level of insight into how algorithmic and human providers make medical decisions.

Dr Cadario says: “In one experiment, for instance, we randomly assigned a group of participants to self-assess how well they felt they understood the process by which human provider or an algorithmic provider triages potentially cancerous skin lesions through visual inspection. We then assigned another group to take a test measuring their actual understanding of his process and compared the responses of the two groups. We found that participants consistently claimed lower subjective understanding for medical decisions made by algorithmic providers—even though their objective understanding of both AI and human practitioner decision making was the same.”

The researchers say this lack of transparency has led to consumers viewing healthcare AI as an indecipherable “black box”, making them less likely to make use of such services. Concerningly, the researchers say this perspective has also hindered AI adoption and implementation by decision-makers within the healthcare sector as they too are commonly in the dark about how such systems work, thus remaining unsure of its benefits and unlikely to invest.

Dr Cadario and his colleagues advocate for AI providers and the healthcare services that use them to do a better job of providing clarity to consumers, to help boost trust and improve uptake.

A further experiment conducted by the team indicated that such simple steps can hold significant benefits. Using Google Ads, the researchers discovered that an advertisement for a skin cancer diagnosis application that explained how its algorithm worked had a higher click-through rate than the same ad ran without the explanation received.

The study’s findings are particularly significant as ethicists and policymakers argue that patients have a right to understand how decisions are made on their behalf, and as bodies such as the UN, the American Medical Association and the UK’s National Health Service all debate the ways and means for which AI adoption can meet ethical standards in healthcare.

Dr Cadario says: “Transparency provides multiple benefits to patients, providers, healthcare systems, and firms delivering healthcare services. Our results illustrate how transparency facilitates patient adoption and can reduce patient resistance to medical artificial intelligence, critical psychological barriers to meeting patient needs given the current surge in healthcare demand.”

This was posted in Bdaily's Members' News section by RSM Rotterdam School of Management, Erasmus University .