By AAMI
As artificial intelligence (AI) continues to dominate the public eye and regulators increasingly look to get involved, there’s new work afoot at AAMI.
In late 2023, AAMI hosted its Machine Learning Cyber Threat Taxonomy Consensus Report Workshop at its Arlington office. The goal? To jointly identify, understand and define machine learning (ML) enabled cyber threats. After a full day of collaborative sessions, Pat Baird, senior regulatory specialist at Philips and co-chair of AAMI’s AI Committee, stepped into the AAMI Studio to discuss the goals and outcomes from the workshop.
What necessitates an AAMI Consensus Report document, or CR, on this issue? Baird shared that cybersecurity is a “very, very hot topic lately” and attention on AI has also “really exploded.” But as these new tools are developed and adopted, “there’s new kinds of threats and new kinds of vulnerabilities when it comes to artificial intelligence systems that are a little different than what we’re used to in health care.”
Some of the new vulnerabilities are to be expected, others are shocking. Machine learning already relies on a massive amount of data, and Baird said, “that data is touched by multiple people form multiple organizations before it goes into the products for either training or testing.”
The sheer amount of data, the need for transfer and its private contents all make these kinds of databases very attractive targets for malign actors. Specifically, Baird noted the possibility of outside actors tampering with datasets in ways that result in ML-enabled tools producing dramatically incorrect outputs. He cited a case he observed at a conference several years ago, where researchers presented an ML-enabled tool that could correctly identify a picture of a cat. However, “they changed 13 pixels on the photo, and now with an even higher confidence level, the software said, ‘that’s a picture of a toaster.’ ”
This was Baird’s tipping point. Although the anecdote is amusing, it shows the danger of data poisoning, or the introduction of new data into a model to intentionally derail an AI or ML-enabled tool’s ability to generate an accurate output. While the medical device industry already has controls for more traditional issues with ML-enabled tools, Baird and his colleagues are concerned with new threats that won’t just be used to turn cats into toasters.
Thus, the Machine Learning Cyber Threat Taxonomy Consensus Report Workshop was convened at AAMI to provide the first step toward drafting forthcoming AI-related consensus-based resources. Establishing common language and goals, including a shared taxonomy between medical device stakeholders, will be essential.
“As much as possible I want to take and reuse what we already know and what we already call things in medical device security,” Baird added. Because existing knowledge from other fields is not universally known in the medical device industry, “we need to move quickly.” The experts involved at the workshop included members of the AAMI Cybersecurity Committee, and AI experts from the AAMI AI Committee in order to bring in different types of expertise and avoid siloing.
What’s next? The workshop’s resulting CR is well on its way, and AAMI volunteers have already signed up to draft sections, review the academic literature and adjust the project’s scope. In the long term, the new consensus report will likely follow the route of AAMI CR34971:2022, Guidance on the Application of ISO 14971 to Artificial Intelligence and Machine Learning, which was recognized by the U.S. Food and Drug Administration in January, 2023. CR34971 was followed by AAMI TIR14971, Application of ISO 14971 to machine learning in artificial intelligence—Guide, published as a dual logo document that is both an AAMI Standard and British Standard via the British Standards Institution.
But for now, speed is of the essence. The new cyber threat consensus report will eventually be succeeded by a TIR and be used in international harmonization efforts, but Baird’s primary goal is to establish a foundation for additional documentation and expert consensus as quickly as possible.
For more information & to watch a video of this interview, visit AAMI’s YouTube channel @aamiconnect.