At 5:12 a.m. on April 18, 1906, the Pacific and North American tectonic plates slipped past one another, tearing a rupture in the earth almost 300 feet long in under a second. The epicenter of the earthquake, one of the strongest on record in the U.S, was almost directly under the city of San Francisco.
The first of at least 700 deaths caused by the quake occurred when buildings, jarred off their foundations by the tremor, collapsed. But the majority of the deaths and damage actually came following the quake, as fires swept through the vulnerable city and disease ravaged the surviving population.
Today, San Francisco once again sits on the shore of the Pacific, as economically viable -- and geographically vulnerable -- as ever. Experts say it's not a matter of whether one of the eight major fault lines that surround the city will repeat the cataclysm of 1906, but when.
But a report issued this week by the U.S. Geological Survey shows that in some ways, San Francisco is still ill-equipped to survive an earthquake like the monster that struck in 1906. Much of the downtown area is still built on reclaimed land, created by the pouring of landfill into the bay. In a major earthquake, that land will undergo a process called liquefaction, essentially becoming like quicksand. It will topple even reinforced buildings like dominoes, experts say. Many of the area's buildings are still wood-frame structures and uncommonly vulnerable to fire.In any event, any earthquake of the same magnitude as the 1906 disaster is all but certain to kill thousands; render roads, bridges and highways impassable; sever gas and water lines; and isolate survivors in the city from rescue.
So how does one conduct business in such a place? What does disaster recovery (DR) mean in the face of this kind of history?
'You have to base your operations outside of the city.'
"There are definitely safer places to live," said Rahul Bakshi, west region district manager for VeriCenter Inc., a national DR services provider with a number of clients in the Bay Area. "But it's a Tier-1 city, socially and economically. The culture here is very proud of what they've built, and to be candid, they'll be damned if it goes under."
Still, Bakshi said, his company recommends that any business that chooses to have a presence in San Francisco not base its primary operations there -- and that's exactly the kind of infrastructure he said his company provides for blue chip clients in the Bay Area, including Yahoo! Inc., Kodak, Clorox.com, and NASA.gov.
"It's not nearly as hard as it was 10 or even five years ago -- certainly not as hard as it was 100 years ago -- to either replicate your data or get your whole infrastructure out of the area," Bakshi said. "That's our first recommendation in all cases -- the bottom line is, if you have any hope of continuing your business, you have to base your operations outside of the city."
VeriCenter itself has its primary data centers in Houston and Atlanta. "We eat our own dog food," Bakshi said. "All of our San Francisco data infrastructure is actually hundreds of miles away."
Bakshi said that for clients, VeriCenter implements different replication and failover plans according to level of service and price. The company uses products, including Veritas Software Corp.'s Volume Replicator, Oracle Corp.'s RHC and Double-Take Software, depending on the customer. Bakshi said generally high-end customers with high availability demands get Veritas or, where applicable, the Oracle product; smaller or predominantly Microsoft shops get Double-Take.
And while the blue chips are the clients he mentions first, Bakshi said his company has a number of midsized customers as well.
"The fact is that the technology is more and more efficient, and more and more accessible," Bakshi said. "It's more accessible and possible to do than people think."
VeriCenter's process for working with customers also includes the development of a business continuity plan, and Bakshi said customers are often surprised when they hear of the options that are open to them.
"A lot of customers get caught in the trap of trying to understand their servers and their storage," he said. "We try to get them to look at their business processes first, and apply technology last."
'We're fairly old-fashioned.'
But some users, like Craig A. Holt, senior systems engineer for Sacramento, Calif.-based Health Net Inc., aren't convinced that the latest replication and hotsite technology are really necessary. Holt said Health Net backs up its 10-plus terabytes of data on tapes in a mix of Hewlett-Packard Co. 9595, 9195 and StorageTek SL8500 libraries and sends them to an Iron Mountain Inc. facility approximately 30 miles away once a day. As an extra precaution, the company also leases a hot site from IBM in New York and runs a DR test annually.
Holt said he "absolutely agrees" that companies in the Bay Area need to have some form of off-site storage if they expect to do business in the area, but said, "That isn't just limited to the Bay Area. It applies to any facility, any company.
"I'm not really sure we do anything different than we would if we were in a different area," he said.
Steve Mays, chief technology officer of X2 Technologies Inc., said he also uses removable media to provide DR services for his 20 Bay Area small and midsized business clients, but he prefers to use large Universal Serial Bus (USB) drives, which the company stores in a bank vault, rather than tape.
"We think that in the event of a disaster, it would be almost impossible to get a tape drive and connectivity up and working to restore data," he said. "But we could go to any store and get a laptop, hook up a USB drive and get access to the data almost immediately."
'Computer systems will be the least of our problems.'
According to Mays, though his company's own internal documents are replicated to sister sites in Florida and New York, and a few of his clients have small amounts of data replicated over the wire through EVault, the majority of the small shops he works with "are prepared for a local disaster, like theft or a fire in their building. But when it comes to a regional disaster, they say, 'if something destroys everything in the area, we have bigger problems than our computer systems.' "
Steve Perry, IT director for Costello and Sons, an insurance brokerage with just two IT employees (himself included), also said that as a small business with very little infrastructure, disaster recovery is limited to planning for more common events, like accidental file erasure or a system crash.
The company does nightly backups, which Perry takes home with him and also uses Maxtor Corp.'s OneTouch USB drives for on-site recovery. The company's customer relationship management and document management systems are replicated using Double-Take to an off-site data center maintained by SonicNet -- but the secondary data center, in fact, all of the company's DR infrastructure -- is located in the Bay Area.
"If we're talking about a scenario like the 1906 earthquake, someplace you'd have to be hundreds of miles away to avoid," he said, "We're also talking about widespread loss of life and the absence of outside infrastructure like electricity. We wouldn't be able to get back up and running for several days anyway -- and we figure our computer systems will be the least of our problems in an event like that."
But what about the long-term viability of the business in the wake of such a disaster? Perry said in some ways, as a small business, it is covered in ways the big guys aren't -- by the big guys themselves.
"We're an insurance brokerage, but the insurance policies for our customers are held by the big insurance firms," he said. "When it comes down to piecing our data back together, our business partners could also help us."