ECRI states that their annual hazards report “reflects our judgment about which risks should be given attention now to help care providers, device manufacturers, and others prioritize their patient safety efforts.” The organization points out that the information in the report can help avoid or minimize risks through careful management of technologies.
Beyond the impact of bad health information, that can affect everyone, this article explores some of the hazards that might involve HTM with the help of ECRI experts.
IT’S NOT WITHOUT ITS FAULTS
While AI may be a significant leap forward in technology, concerns about the misuse of AI chatbots can leave patients with misinformation that could result in life-or-death decisions.
One AI developer even runs commercials boasting about a chatbot’s ability to guide patients with medical decisions, but patients may get a false sense of security from information that is not always reliable. Should warnings be required to inform patients that they should not rely on AI information? If yes, how could this be achieved? Should this warning be directed at the patient? Where would it originate?
“ECRI advocates for both approaches – responsible use and appropriate expectations by the end user (whether a patient or healthcare professional) and developers including as many safeguards as possible. There may be technical differences, as some vendors provide development environments (such as NVIDIA), while others deliver a final product (such as OpenAI or Google). Regardless of the delivery method, warnings should be placed appropriately,” says Christie Bergerson, Ph.D., consultant and AI subject matter expert at ECRI.
She says that some large language models (LLMs) have begun including warnings that they should not be used for medical advice and that users should consult a physician; however, similar to cigarette warnings, these are likely often ignored.
“From ECRI’s perspective, relying primarily on end-user warnings is not an effective way to manage AI-related risk in healthcare, although it is better than having no risk-mitigation strategies in place. The FDA has long recognized that written warnings and disclaimers – referred to as “labeling” – are among the weakest forms of risk mitigation and should not be relied upon when more robust design or system-level controls are feasible,” Bergerson says.
She says that this is especially relevant for AI systems, where fluent, authoritative-sounding outputs can easily override generic disclaimers, particularly in high-pressure clinical settings. Warnings to users may be necessary, but they are insufficient on their own; safety must be designed into AI systems rather than delegated to end-user vigilance.
Should physician-centric organizations or the AMA be tasked with originating these warnings?
“Multiple national organizations and associations are actively addressing the issue of artificial intelligence from a range of perspectives and have produced publications offering recommendations. ECRI has repeatedly flagged concerns related to AI over the last few years precisely because we want everyone in the healthcare ecosystem – patients, providers and technology developers – to balance risk and reward in the use of AI,” Bergerson says.
INFORMATION DISRUPTION
Technology remains in the spotlight as ECRI warns of “digital darkness events” and another scenario wrought by technology, and our reliance on it, that may not have existed in the past. The danger involves losing access to electronic systems and patient information. This can occur because of a natural disaster or a cyberattack. How can HTM be prepared for one of these events?
“Cyberattacks are not the only entry point for a digital darkness event. Cyberattacks, natural disasters and other unaccounted for outages all pose equally high risk for a digital darkness event. Many existing safeguards in place solely focus on cyberattacks. While cyber incidents are often the first scenario considered, preparation must also address events such as natural disasters, power outages or other events that cripple critical infrastructure, technology and communication systems,” says Scott Luney, cybersecurity consultant, lead at ECRI.
He says that for healthcare technology management specifically, the cause of the event is less important than proactive preparedness to ensure timely and successful recovery. HTM departments, in coordination with other leadership in IT, cybersecurity, and other stakeholders, should have well-defined disaster recovery and business continuity plans in place, including clear instructions for how to enact business recovery efforts.
Luney says that components of disaster recovery planning should include, but are not limited to:
1. Alternate communication systems between staff, technicians, and leadership. If phone, cellular or Internet service is down, satellite communications (phones or Internet) can be a viable interim solution.
2. Redundant cloud data locations. Many organizations rely on cloud solutions, which are very susceptible to digital darkness events. As a result, many cloud providers such as AWS, Google, and Microsoft all offer redundant hosting in alternate regions.
3. Alternate care sites. In hospital settings, cost may be a factor, but the ability to operate an additional triage or treatment “hot site” could help maintain some level of patient care capacity.
4. Data integrity. Disaster planning should include regular intervals of data backup and then tests of data restoration ability. If a darkness event destroys a physical data center, having that data backed up/replicated elsewhere is critical to continued operations.
5. Data availability. Along with potential “hot site” clinics, establishing “hot site” technology locations supports access to organizational data in an event, allowing staff to access systems such as cloud-based EMRs from a remote site.
MAINTAINING COMMUNICATIONS FOR DIABETIC HOME CARE
ECRI points out that technological advancements have improved home diabetes management. However, the organization warns that if there is a product recall or update for a device in home use, it may not reach patients and caregivers in a timely manner.
This can present dangers such as an “integrated insulin pump could overdeliver insulin” or “sensors on a CGM could yield incorrectly high glucose readings.”
“Sometimes, important product safety information does not reach patients and caregivers in a timely manner; and sometimes the information they receive is not sufficiently clear, leaving them unsure how to respond. ECRI’s report encourages providers and equipment suppliers to work toward improving the notification process. That said, there are steps that patients and caregivers can take as well,” says Bradley Bonnette, senior project engineer at ECRI.
Bonnette provides these examples: Register your medical devices with the vendor. If you get the devices/supplies from a third party (typically a DME), reach out to them to understand how recall notifications will be communicated and stay informed. Get involved with communities of other users, who often share recall and safety information.
UNPATCHED AND VULNERABLE
Cyberthreats continue to be a concern for healthcare systems and those tasked with defending against them. ECRI points out that legacy medical devices are a particular concern. These older devices, that aren’t updated with the latest security updates, pose a vulnerability.
In recent years, with the advent of hybrid roles in the HTM department, the role of reviewing legacy equipment and possibly removing it from the network, has been a focus. What information can HTM use to address these vulnerabilities?
“Legacy devices can include aging technology, but also encompass any device, regardless of age, that cannot be successfully patched or updated by the vendor or manufacturer,” Luney says.
He says that guidance around legacy devices has remained largely consistent around making sure the devices are not “attackable” in their degraded state.
Luney says that if a device can be taken off the network and still operate to full capacity, that would be the preferred approach.
“If a device must be on the network, but has open vulnerabilities, it should be segmented so that only the core functionality is enabled, and it cannot communicate with outside networks,” he says.
Luney says that if a device can be upgraded or replaced, whether on a predictive replacement schedule or other opportunity to replace, that is desirable. However, budget constraints and the possible lack of suitable replacement devices on the market may limit this option.
“Physical security of devices is also a risk consideration. If a legacy device is in an insecure state, it must be physically protected by storing securely in a locked location, and any data must be extracted and deleted from the device prior to storage,” Luney adds.
PURE WATER FOR PATIENT SAFETY
ECRI has identified a risk in the ability to maintain sufficient water quality during instrument disinfection or sterilization. This risk could expose patients to dangerous pathogens. Is there a role that the biomed department plays in maintaining water purity standards?
“Maintaining appropriate water quality in sterilization and reprocessing is an interdepartmental endeavor, and not something that biomedical engineering departments can achieve on their own. Rather, the biomedical engineering department should team up with infection prevention, sterile processing, facilities, surgical suite, and senior leadership members to ensure that they have the needed expertise and resources,” says Mairead Smith, principal project engineer at ECRI.
She says that if biomed departments find that their organizations are not working to adhere to ANSI/AAMI ST108, they can take the lead in establishing a multidisciplinary team and can continue supporting the team’s efforts to adhere to the standard over time.
“Members of the biomed department may have an advantage in that they are more likely to be members of AAMI – the organization that developed the ANSI/AAMI ST108 standard on Water for the Processing of Medical Devices. Through AAMI, they are likely to have access to guidance, resources, and community support to help them navigate water quality issues and adherence to the standard,” Smith says.
As technology aids clinicians and HTM professionals in improving workflows, identifying problems and working more efficiently, it also introduces new concerns that must be addressed. ECRIs annual health technology hazard report brings many of these concerns to light to protect patients and alert caregivers.