Case Study: Optimum Bank
OptimumBank Improves Backups, Bolsters Client Security
After evacuating for two weeks in the aftermath of Hurricane Wilma, Florida-based OptimumBank knew the importance of protecting its critical data. “Disaster recovery definitely became less of a hypothetical scenario,” said Seth Goldstein, the bank’s vice president of Information Systems. “It wasn’t soon after that we began to look at co-location facilities in North Carolina.”
The bank’s backup and recovery processes were much like many other community banks of its size. Its handful of branches relied heavily on tapes, and the bank’s non-IT employees were often tasked with monitoring backups, changing out tapes, and labeling each tape for couriers who shuttled them and other bank paperwork between locations. Restores from tape were slow and problematic.
Scaling a tape backup model as the company grew seemed to invite more issues. “We would have needed to invest in new tape formats and more complex equipment while maintaining backward compatibility, not to mention paying staff to ship and swap cartridges for me at the colocation facility,” said Goldstein. It was time to apply OptimumBank’s own core belief to the issue: find quality partners who can provide proven innovative technologies to address the needs of our stakeholders—this time regarding the ability to bounce back quickly from disaster.
Goldstein, and OptimumBank’s CTO and Executive Vice President Tom Procelli, both subscribe to what Goldstein calls the “LOCKS (Lots Of Copies Keep’em Safe)” philosophy: “We like having lots of copies of the data to keep things safe,” he said.
“We wanted to make sure all the data we needed would be in a safe place and recoverable,” added Procelli. “You can always rebuild the infrastructure and put in new equipment, but the data is the part that will make or break the recovery. That’s the most critical piece and the most important. So, we were striving to have it available not only in one location, but multiple locations.” Affordability was also a factor. The key elements they hoped to achieve in a new backup and recovery solution included:
- An affordable hot site out of the region for rapid recovery of bank operations, plus the ability to minimize the cost of WAN bandwidth required
- Faster, more flexible data restores to various locations
- Encrypted backups for increased security
- Centralized backup operations that eliminate the use of non-IT staff for tape management
- Disk-to-disk technology for primary backups; use of tape for archival storage
- Easily modified backup routines that accommodate changing priorities or increased backup requirements
Describing their prior backup process as a “very standard program,” Goldstein said the bank found itself following a traditional tape-based backup routine that involved rotating tapes, using couriers to move tapes between branches, and relying on branch office personnel to manage the process.
While this worked up to a point, Goldstein readily admitted it wasn’t ideal. “The servers in each branch location were being backed up with a local tape drive. This created work for the staff. We were also dependent on the staff remembering to switch tapes, report when things weren’t right, ensure the tapes got to the courier and that they were labeled properly.”
As the bank added more branches, it acquired more tape drives that weren’t always fully compatible with tapes at other branches. That made it more difficult to recover the data from another location. Tapes stored at branches created a challenge if someone needed access outside of normal business hours. “Once they were locked up in the vault, the tapes would not be accessible until morning. If you needed it on a Saturday, you had to wait until Monday,” said Goldstein.
The bank also used single tape drives attached to servers, which restricted the number of backups that could be performed. Custom backups meant performing separate jobs and, often, different sets of physical tape. Once again, branch office staff had to oversee the job. Because the tapes were reused frequently, Goldstein said he was also concerned they might not perform when you really needed them. “You tend to push these tapes in terms of their age a bit longer than you should. And, we never knew at what point our luck was going to run out.”
To help them find a better solution, OptimumBank turned to its trusted technology partner, InfoSight, Inc. Based in Miami Lakes, Florida, InfoSight had built a reputation for assisting financial institutions with setting up, monitoring and troubleshooting their day-to-day network operations, business applications, and data.
According to InfoSight’s Regional Sales Director Tom Grady, the choice of EVault was straightforward. “EVault had a pretty good financial focus and a good list of banks that were already using its services. It just seemed to us that EVault was the only thing that could fit all of OptimumBank’s criteria at the time-like compliance issues, the ability to speed backups/restores as well as manage the backup process centrally through the EVault interface. All of those factors led us right down the path to recommend EVault,” said Brady.
InfoSight performed careful due diligence before recommending EVault. “If we bring a solution to our customers, they know it’s already been scrutinized by us to determine whether or not it’s going to work-from both a compliance side and a technology side. We were comfortable putting the EVault solution on the table and seeing how it could help OptimumBank achieve what they wanted.”
Especially important in the decision to go with EVault was the way it minimized the amount of data crossing the WAN. Its DeltaPro technology sends only the new and changed data blocks since the previous backup.
EVault’s efficient use of WAN bandwidth hit a positive note with Procelli. “If there is a major problem-regardless of what it is-you need to have the data in multiple places and it has to be efficiently pushed to the location where you’re going to store it without driving your communication costs sky-high,” he said. “EVault helps us do that.” Goldstein also liked that he could send backup data across the WAN via his Internet DSL connection during normal business hours if necessary, and the software is able to accommodate traffic variations on the line.
The EVault installation was painless. “It plugged right into our existing network. We didn’t have to change it,” said Goldstein. “It helped that the EVault representative had done a lot of these types of installs before, and had actually come from a community bank like ours, so he knew exactly what we were trying to do.”
Since implementing EVault, OptimumBank has put EVault InfoStage to the test for its local data recovery issues. Each time, EVault passed. “We had a branch server go down last year in the middle of the night. I actually recovered the files from the DR site while in my pajamas from home. The files were ready to use with the new equipment first thing in the morning” said Goldstein. “It had been so long since I’d looked through the EVault manual that I needed a quick refresher. But, I was able to get someone from EVault on the phone right away. They held my hand all the way through it. It really helped with the speed of our recovery the next day.”
Goldstein also likes the fact that backups are now centralized-with no more need to swap tapes or rely on others at the branches. “I can actually prioritize backups now for some of the more critical applications and take [disk-based] snapshots more frequently for those,” he said. “I can also access EVault remotely and restore it to other locations. Sometimes I even use it to grab copies of things from the vault because it’s almost easier than figuring out where else to get the data from.”
Facing a more active than usual hurricane season again this year, Goldstein appreciates that he can quickly build a snapshot of the data if there’s a potential threat on the horizon and take a copy with him. “I can actually back up to a portable USB drive and put it in my briefcase-knowing it’s safe, secure, encrypted and with me.” Disaster recovery testing has also been a lot less painful. “For me to run a test with a bunch of tapes, it can take a couple of hours. I have to make special arrangements to bring all the tapes in one place. With EVault, I can restore in just minutes. It’s simply not a fair fight,” he said.
Now that EVault is in place to help the bank recover from a worst-case scenario, he already has plans to develop a remote branch backup strategy using EVault InfoStage LocalRecovery to create local branch backup vaults. This will decrease the backup window and improve recovery times because data can travel at LAN speeds. He explains this approach as follows: “We’ll maintain our offsite storage with one schedule that works for us and also set up local vaults where we can have an even more frequent retention and backup schedule between locations.”
Beyond the flexibility, cost savings, and faster restores, the bank’s IT staff is now certain about its data protection. “Peace of mind is really what it’s about,” said Goldstein. “That’s why we partnered with InfoSight. That’s why we have EVault. That makes us sleep well at night. We have a lot of confidence in both organizations. We just know in the event we need them, they’ll be there.”
“Peace of mind is really what it’s about,” said Goldstein. “That’s why we partnered with InfoSight. That’s why we have EVault. That makes us sleep well at night. We have a lot of confidence in both organizations. We just know in the event we need them, they’ll be there.”