What you will learn from this tip: This tip discusses the impact of change control (or lack thereof) on disaster recovery and offers some best practice information.
Does your disaster recovery (DR) plan documentation still refer to recalling 9-track tapes from vault or outline recovery procedures for Windows NT 3.51 server? While most plans are not that far out of date, a majority of IT practitioners involved with disaster recovery planning pause and smile when asked how current their organization's disaster recovery plan is. This is usually followed by an admission that it needs attention. There are many reasons why DR plans become out-of-sync with the production IT infrastructure; below are among the most common.
The plan was developed only to meet an audit requirement and is now collecting dust.
The plan is not tested or maintained on a regular basis.
The plan is maintained on a fixed schedule only (i.e., once a year or after a test).
Change control (or change management) does not take disaster recovery into consideration.
Absence of a formal change control process.
A recent Gartner Inc. survey published in the Disaster Recovery Journal shows that approximately 51% of respondents made changes to the disaster recovery environment on a predetermined maintenance cycle while only 35% relied on the production change management process.
The absence of change control is, by far, a disaster recovery plan's or disaster recovery environment's worst enemy. In fact, nonexistent or insufficient change management can be the cause of disaster. There are many documented cases of major outages resulting from poorly planned, yet routine software upgrades. Small and midsized businesses (SMB) can be particularly challenged when it comes to change control due to a lack of resources and funding. This often results in more cycles being spent resolving issues rather than preventing them.
Following is a summary list of change control related items that any size business with IT dependencies should consider with respect to disaster recovery:
Implement a change control process if one does not currently exist. It does not have to be elaborate but should not allow any IT changes to proceed without prior review.
Ensure change control takes into consideration the disaster recovery environment and/or plan documentation when evaluating the impact of a change. This must be beyond simple questions and include some form of verification.
Disaster recovery must be part of the systems development lifecycle and not an afterthought.
Disaster recovery planning must become an ongoing process rather than be considered a project with an end.
The disaster recovery coordinator (or person responsible for disaster recovery) should take part in the change review process.
Do not wait until the next disaster recovery test to update the disaster recovery environment to reflect changes to the production environment.
The disaster recovery plan and disaster recovery environment should be considered as critical as the production systems they were designed to cover and should be maintained accordingly.
Organizations with a hot-site subscription with a vendor must be reminded that, although a disaster recovery environment does not physically exist until needed, it is their responsibility to keep the vendor informed of any configuration changes to the production environment that might affect the disaster recovery environment. This should not be done only at contract renewal time but before changes are actually made to ensure that any arising technological or contractual issues are addressed during the planning process.
Just as we test our disaster recovery plans and procedures on a regular basis (technically), it is also advisable to test your change management process. The mere existence of change control does not automatically guarantee efficiency of the process. This can be as simple as conducting a post-change review to make sure every aspect and possible impact of a change were considered prior to its implementation.
IT organizations wanting to help their change management process reach maturity may elect to follow IT industry best practices and guidelines, such as those outlined in the ITIL framework (IT Infrastructure Library).
Whether world class or entry level, change control must not only protect your organization from the impact of poorly planned IT changes; it must help keep both the production and disaster recovery IT environments fully synchronized and ensure all associated documentation and procedures are in good maintenance.
About the author: Pierre Dorion is a certified business continuity professional for Mainland Information Systems Inc.
