The Afognak Native Corp. IT team faces the triple crown of disaster threats -- earthquakes, volcanoes and tsunamis....
But the Alaska-based company copes by having an alternative data center in Chesapeake, Va., and other geographically dispersed failover sites.
The Afognak Native Corp. began as a timber development company in Alaska in 1977, and now has 36 offices and project sites throughout the U.S. It performs security, IT, construction and other services for the federal government.
The corporation's main data center at its Anchorage, Alaska headquarters is surrounded by four active volcanoes and the area has a history of earthquakes. The company's IT security manager, Fred Berestoff, is based on Kodiak, an island off the coast of Alaska that was devastated by a 1964 tsunami. That history of natural disasters makes Afognak's DR plan a large part of the IT staff's job.
"We have disaster on the brain all the time," Berestoff said. "This is a highly geographically active area. We have an interesting perspective on natural disasters. We've always had the perspective that, at any moment, weather can come in and throw a monkey wrench into what we're trying to do."
Afognak has Dell EqualLogic iSCSI storage area networks (SANs) in five locations -- the two major data centers and alternative data centers in Charleston, S.C., Huntsville, Ala. and Denver, Colo. The SANs store the company's crucial file and application data. Afognak has about 160 virtual servers and 35 physical file servers.
Berestoff said Afognak revamped its data protection setup three years ago, switching from running Symantec Backup Exec at each remote office to centralized CommVault Simpana implementation. It also uses Quest Software's vRanger to back up some of the virtual servers.
"We had 18 different installs of Backup Exec," he said. "We wanted to centrally manage our backups and back up efficiently across the WAN. Of course, DR [disaster recovery] was a huge piece of what we set out to accomplish. Our board of directors told me a major requirement was establishing business continuity and the ability to recover from a major disaster that would take down an entire office."
Chesapeake and all the satellite offices back up locally using Simpana, and critical data is replicated across the WAN to Anchorage. Then a second copy of all the critical data is sent to the alternative data center in Chesapeake. That includes user files, databases and the corporation's major financial application.
"If we go down in Anchorage, my goal is to provide a duplicate of our financial package in Chesapeake in two hours," Berestoff said.
The tricky part is pushing that data through narrow bandwidth. Berestoff said bandwidth is more expensive in Alaska than in most U.S. states, and Afognak recently upgraded its main data center from 1.5 Mbps to 20 Mbps. "We knew we had to run whatever failover process we had over some pretty small pipes."
Simpana's deduplication and replication helps squeeze data through those pipes. Berestoff said he backs up and replicates his main financial application in a SQL database every hour. "Simpana can do snap backups, transactional backups every hour, and push only one or two megabytes down to Chesapeake because of its dedupe," he said.
He said it only takes two minutes to move that data. At night, Afognak replicates virtual machines from Anchorage to Chesapeake. "The first time, it takes 20 hours, then after that it only sends differentials and takes about an hour," Berestoff said.
He said a full restore of the financial data takes about 75 minutes.
Afognak's Microsoft Exchange 2010 and its Microsoft Lync communications platform are other key applications for its DR plan. Exchange fails over to Huntsville, and Lync fails over to the Kodiak office. "They fail over in any type of outage," Berestoff said. "If Anchorage goes down, we fail over to Huntsville for Exchange services and Kodiak takes Lync and voice services."
Berestoff said the company runs annual DR tests, and regularly initiates restores of virtual machines from local sites to Anchorage. Only one site still uses tape, but, "We haven't had anybody need to restore anything from tape in over two years."