David Cagigal now has lived every CIO's worst business continuity nightmare. But flooding that hit Alliant Energy Corp.'s Cedar Rapids, Iowa facility in June, when the Cedar River crested at 32 feet, gave Cagigal's IT department a chance to demonstrate its value.

While the electric and gas utility still has some recovering to do, IT passed the test thanks to a well-defined business continuity plan, a workhorse backup generator, and a partially completed data migration.

Alliant avoided the 20-cent per share financial hit it had announced early in the crisis, and information technology is viewed in a new light. Cagigal said IT rose to a new level of importance in the organization, and CEO Bill Harvey spread praise around during a recent investor conference call.

“In spite of the fact that five of our facilities in the Cedar Rapids area were evacuated as a result of the flooding, including our corporate offices downtown, our planning and execution ensured that our operations continued,” Harvey said.

Still, Cagigal can point to a number of IT lessons learned, including the importance of data replication. In this look at Alliant's approach to business continuity, WTN found several important CIO take-aways.

Madison backstop

The storm swept across Iowa on Wednesday evening, June 11. While flooding caused 75 percent of the counties in Alliant's three-state territory to be declared federal disaster areas, the most severe damage was in Cedar Rapids, Alliant's largest load center.

Cagigal was in Madison receiving storm updates via cell phone when he dispatched an IT team leader to Cedar Rapids for a first-hand account. When she arrived at midnight, water was flowing over the riverbank and toward Alliant's 21-story Cedar Rapids tower.

By Thursday, the first floor was under water, as were 1,700 city blocks. Service to 35,000 customers was disrupted. Alliant's 185-megawatt and 55-megawatt generating stations were knocked out; the utility still is working to bring them back.

A myriad of steps were taken just to keep Alliant running: the relocation of infrastructure services, a virtual server environment for metering applications, special phone and data systems for customer billing, and temporary work locations.

The main data center and network operations center, both located on the tower's fifth floor, were inaccessible. Alliant had been using the Iowa data center as a redundant backup for Madison, and was setting up the reverse when the storm hit. Information technology systems were restored within five days; during those five days, the data replication project helped avoid service disruptions.

It also enabled Alliant to restore e-mail and other communications in a matter of hours, not days. As equipment was being shipped from Madison to the tower and temp sites in Cedar Rapids, Alliant conveyed from Cedar Rapids to Madison network shared drives, e-mail servers, and voice-mail servers. They were needed in the Madison data center to make possible communications in Iowa. By Monday, Alliant established power in Cedar Rapids with a diesel-powered generator, and the drives and servers were transported back to Iowa.

“Otherwise, all the other data was replicated and sent over an OC-48 [broadband] line from Cedar Rapids to Madison,” Cagigal explained.

Alliant expects to complete data replication in October. Without Madison as a live backup, Iowa customers would have faced prolonged outages and, now, all of Alliant's single points of failure are being reviewed.

Some luck was involved. Workers found an entire bank of modems and work stations on the 18th floor, and the modems were used for meter reading. The modems since have been consolidated into the data center, expanding redundancy.