By Joseph E. Fishel, CBET, MBA
It seems that the biomed field has been hit with more disasters in the past 10 years than in the previous 30 years. Many of these have been natural disasters such a hurricanes, tornadoes, snowmageddons, flooding, landslides, tsunamis, earthquakes and wildfires. Each is easily capable of taking out utilities and entire hospitals. Then, there are the manmade disasters.
Many of these disasters can impact your cyber network. Are you ready should your facility be hit with one? A master disaster recovery plan is actually a whole lot of individual plans that together create a comprehensive playbook. These individual plans can be in various areas, including servers, wired networks, wireless networks, applications and equipment.
What should be included in a disaster recovery plan? Contact your IS/IT departments to see what format they are using. You may want to integrate into their plan. An Internet search provides many different formats, but the content of the plan should include the following:
- Application/device including manufacturer, function, description of what the application does and tier level
- Last drill date/next drill date
- Minimum return to operations hours
- Minimum hours recovery point objective hours
- Accountable manager(s) and contact information
- Responsible manager(s) and contact information
- Subject matter experts and contact information
- Individual and department business stakeholder(s)
- Does this support acute care?
- Is PHI transmitted?
These are just the headings for the disaster plan. The service (disaster) recovery plan should describe who does what and what is done to put the device, app or service back into operation. This can be broken up into different sections.
The service description describes what it is and its function. Included in this are notes regarding communications and updates to who and by whom. This should include IS/IT, clinical customers and clinical engineering/biomed.
Routing and support is broken up into the various groups that may support the recovery process. These can include service desk escalation, vendor support and biomed support.
Which servers are part of the support? DNS, DHCP, network servers and application servers are just a few. What upstream or downstream dependencies can affect the service? Recovery design diagrams can be helpful to map out and provide a visual of the entire system – both upstream and downstream and it’s interconnected abilities.
Finally, what are the procedure steps for recovery using backup? What are the options? Defining this up front can prevent a disaster from becoming a catastrophe. This gives the opportunity to pass down to others what was done before when this sort of thing happened and how you were able to work around it. This is extremely important when running a 24/7 operation.
In some cases, the recovery may require a rebuild or reloading of software. Identify where that software or a backup hard drive can be found. Also, if you can, identify where the latest version can be found.
Finally, what is your testing protocol to identify that everything is back up and working? List this out in detail. You aren’t always going to be the one doing this. It might be someone else and what is normal for you may not be common knowledge to someone else.
Plans should be reviewed on an annual basis. This can be done on a routine basis. Review a few plans every month. One of the best ways is in conjunction with a disaster drill. It can be as simple as a water pipe breaking at a server center and flooding the area. How would the device ready for review be affected and if affected how does your plan cover the recovery? If it doesn’t, then the plan needs to be expanded.
These are just some of the topics to include in a recovery plans. As every institution is different, every plan will be different.
Joseph E. Fishel CBET, MBA is the Healthcare Technology Systems Manager for Sutter Health eQuip Services.
