By Steven Hughes, FAC-COR FACP/PM VHA-CM, VISN 21 Biomedical Engineer, VA Sierra Pacific Network
Always fail forward not backward. Failure is not a negative thing, but a place for improvement and an opportunity to learn. Failure is a part of life. We all learn by failing. You are constantly making iterative changes in everything you do from the last time you performed it. Failing forward forces improvement on how things are currently done, provides transparency of processes, takes responsibility of situations, and creates new workflows and processes that makes things better by reducing time, money and resources. In health care the margin for error is very small and any critical mistake can literally mean life or death.
One area where all healthcare delivery organizations (HDOs) have a significant failure is vulnerability management, which is mostly due to no fault of their own. HDOs all have inherited risks and vulnerabilities, and you can never reach the elusive “zero vulnerabilities” because a new one is always arriving or has already arrived as you are reading this sentence.
Medical devices in addition to waiting for operating system or software patches must also wait for medical device manufacturer (MDM) testing and approval of some patches or the creation of a mitigation strategy to a vulnerability before remediating thus increasing their risk. Many MDMs are designing and allowing for real time patching of the underlining OS and third-party software via information given in Manufacturer Disclosure Statement for Medical Device Security (MDS2) and Software Bill of Materials (SBOMs), but even then, builds of operating systems and software can creep into an unsupported state if not monitored. For example, there are several different builds of Windows 10 – some are supported, and some aren’t. Many MDMs aren’t aware of end of life (EOL)/end of service (EOS) dates for software and operating systems in their own products due to unforeseen changes made later in the software/operating system design and life cycle and that risk is passed onto the HDO.
INVENTORY
Knowing what you have makes it easier to patch and manage your current and future vulnerabilities. This could be a simple spreadsheet of your inventory or an integrated system with your CMMS that scans your network traffic and sniffs packets to determine system OS, latest patch installed, antivirus version with latest update, third-party COTS software with version, etc. All of this is important to keep track of in case something needs to be patched or upgraded to circumvent a new known vulnerability. If possible, work with your MDM and see if they allow credentialed scanning and approve the installation of agents to automatically track operating system and software updates as well as provide an inventory of your networked medical devices and give a “heartbeat” of information of the “current status” of the medical device. This provides a huge improvement in automating your inventory. If agents can’t be installed there are several solutions available that can integrate with your current networking equipment and/or IT software to analyze network traffic and provide this information.
EVALUATE
Perform a security risk assessment of your current vulnerability management program and look for areas for improvement that will reap the most ROI of that change with minimal effort or it may even be maximal effort if the “juice is worth the squeeze” which creates huge, monumental change and improvement. Look at what devices require the most work to maintain/update, have the highest risk to patient safety, devices that are no longer supported, etc. and formulate a game plan to address these issues and bring it up to senior management. Make sure you document this plan and ensure everyone is aware of the risk of what the current state is and where it could be. Some organizations use a plan of action and milestones (PoA&M) to track these risks with a plan to rectify the issue given, known circumstances and accepting the current risk of using the device. Always isolate your devices with VLANs, ACLs and/or firewalls with rulesets allowing only minimal allowed traffic (given by your MDM) in and out of medical devices on the network to mitigate and reduce risk. Keep track of EOS and EOL dates of your devices, operating systems, software, etc. to compare with your current inventory on a reoccurring basis and formulate strategies accordingly.
COMMUNICATION
Subscribe to CISA updates of known vulnerabilities and remediate as soon as approved by your MDM. Reach out to your MDM for patching information, upgrades and updates about your current inventory as well as new and future products. Communicating what you have in your inventory helps your MDM help you create an equipment replacement/upgrade strategy as well as allow time to request any necessary funds before a device reaches EOL/EOS. Be sure to establish a good working relationship and communication pathway to ensure information can readily be obtained when needed. Create a regularly updated list of points of contact (POC) for your MDM and links to sites for your system support. This is also great to have during after-hours support as well, which can also start out as a simple shared spreadsheet and eventually grow into full blown hosted web-based database. At the VA we have implemented a national patch database where HTM staff can look up POCs and share the latest patching approvals, disapprovals and information by MDMs for the systems they maintain with direct links to their vendor/service portal, product security updates, bulletins, security advisories, product software download center, validated patches, etc.
REFLECTION
Keep track of where you are and where you have been. This benchmark helps to measure if your efforts are paying off and how much more work lies ahead. Most organizations set reachable thresholds as a goal or key performance indicator (KPI) for an improvement in something that needs drastic change and through the implementation of process improvement it becomes a regular state of being rather than an unattainable goal. This goal or KPI can be as simple as say patching and or upgrading/replacing half of all networked devices and slowly increment each month, quarter, year, or timeframe you deem reasonable to achieve. The next goal could be 75% or even 100% given new processes in place, time, money, and resources, but remember this must be achievable and temper expectations as well as communicate that there may not even be path forward for an upgrade or replacement for some medical devices.
Realize that if you don’t fail, you may not be taking enough risks to make progress. We must change our current culture and recognize that failure is a stepping stone to your next opportunity. In order to learn from mistakes, correct them, improve and succeed.
– Steven Hughes, FAC-COR FACP/PM VHA-CM, is a VISN 21 Biomedical Engineer at the VA Sierra Pacific Network.
