Between the Lines

Migrating to Amazon Web Services: The blueprint

Posted in:

Amazon
General
Hardware Infrastructure
IT Management
SaaS
Software Infrastructure
Storage
Web Technology

Moving your infrastructure to Amazon Web Services--or any other cloud platform--sounds like a no-brainer in many respects. You can scale up or down as needed and pay only for the computing and storage you use. The big question, however, is this: How exactly does a company migrate its infrastructure? Helpstream, a software as a service customer service and relationship management company, recently completed a migration to Amazon Web Services. Bob Warfield, executive vice president of products at Helpstream, recently disclosed the company's move on his blog, his corporate blog and to the Enterprise Irregular mailing list. On Tuesday, Warfield, along with Helpstream CTO Dan Hardy and other executives, hosted a call walking me and Tom Foydel (see Tom's take) through the migration to Amazon Web Services (see all articles). Helpstream's service has a customer service portal, community module, solutions management and checklists. The company is a big Oracle partne

r and has about 140 clients and 90,000 users. In other words, Helpstream (right) isn't a just-hatched startup, but it's not exactly General Electric either. Here's a look at Helpstream's blueprint, which took about six months to implement. In those six months, Helpstream moved its infrastructure to Amazon Web Services--primarily the Elastic Compute Cloud (EC2) and Simple Storage System (S3). The migration is notable given that Helpstream has never talked to an Amazon person--everything is done online--and billing is handled via a corporate credit card (an interesting quirk of dealing with an e-tailer. Before we get into the step by step, there are a few high level takeaways:

Helpstream's previous infrastructure was operating at 10 percent utilization in most cases unless there was a spike. It had a co-located data center. With Amazon it can add an incremental server in three minutes. Amazon works well for testing, betas and other experiments. For instance, Helpstream can use Amazon to add capacity so customers can get a sneak peak at an upcoming deployment.
Amazon's prices were a big selling point. The margins are narrow for SaaS companies and Amazon's prices were attractive. Warfield reckons that the top half of public SaaS companies growth at a 35 percent clip, but spends 27 percent on service cost. In other words, the margins stink relative to traditional software. Helpstream reckons it will cut its IT infrastructure costs by about 59 percent. Meanwhile, Amazon's margins on Web Services are much better than e-tailing. Translation: Amazon doesn't have a burning desire to gouge you since the margins on Web services are infinitely better than its core business. It's a lot like the guy that lives on Long Island and decides to buy a house in North Carolina. You're not going to sweat a few pennies given you're shack on Long Island equates to a 5 acres and a lake in the Carolinas.
Helpstream was very cognizant of conflict of interests and Amazon was about as Switzerland-like as you could get. Amazon is primarily an e-tailer and that was appealing to Helpstream for a few reasons. For starters, Amazon isn't going to start making SaaS apps--something that could be a worry if you decided to put yourself in Salesforce.com's cloud for instance. Warfield noted that vendor lock-in was a concern, but Amazon was more advanced than other cloud providers and is "very neutral like Switzerland" compared to Helpstream. A company like eBay obviously wouldn't host apps on Amazon Web Services due to potential conflicts and competition.
If you are going with something like Amazon Web Services make sure you have analogs with other vendors just in case. Avoid anything proprietary. For instance, the services Helpstream uses are offered by other companies so "we could move to another cloud if we had to," said Warfield.
Helpstream was big on transparency, security and other key features that Amazon needs just to run its business. "Amazon has done a lot more than most small companies can do," said Warfield. One key point: Helpstream has a reporting tool outside of Amazon's systems so it can notify customers in the event of an outage.
There are cultural issues to ponder. I couldn't help but wonder how this migration would work out for larger companies with legacy applications and a rat's nest of systems (essentially most of you reading this). Helpstream's approach lined up with Amazon Web Services well. To wit: Helpstream's infrastructure is open source based--Apache, MySQL etc.--and lines up with Amazon's. Culturally, Helpstream is OK with not talking to a human. In addition, Helpstream is a SaaS company and understands the cloud. It's unclear whether other companies could build their infrastructure without any handholding let alone move terabytes of data to Amazon's cloud.

With that background out of the way let's go to the phases. Helpstream's goals were pretty straightforward. The company wanted to cut risks, avoid a move in one big swoop and improve performance.

Phase 1: Backups and test servers

This phase was really a "get to know Amazon" stage. Helpstream's first step was to move its backup copies from tape to Amazon's S3 storage service. The perks were fast backups, redundant physical copies and Helpstream didn't have to pay anyone to visit the datacenter cage. From there, Helpstream began experimenting with Amazon's EC2 service. Hardy noted that it took less than an hour to bring up a pod, essentially a data center cage. Helpstream began using EC2 for test servers and trials. This phase took about two months and the main benefit was practicing on Amazon's infrastructure.

Phase 2: Move most of storage to S3

In the second phase, Helpstream's goal was to move its BLOBS (binary large objects) such as attachments and knowledge base documents from MySQL to S3. Warfield noted that Helpstream kept the client code unchanged to minimize bugs. It didn't serve documents from S3 initially. Ultimately, Helpstream moved 85 percent of its data to S3 and found that its MySQL and servers became more efficient. By moving most of its storage to S3 it set the stage to move to EC2 without as much data. One big help in this phase was Amazon's move to offer elastic block storage. The primary benefit: You could mount storage to EC2 and have it mirrored. The process took about a month.

Phase 3: Moving to EC2

With the first two phases under Helpstream's belt it was all about planning for the final move. Here's where Helpstream's size helped. Although Helpstream had a detailed plan, it didn't have hundreds of terabytes to move. A larger company would have had more steps. "There's always something to tease apart," said Warfield. "Having talked to large enterprises, the scary thing to them is the big terabyte data cluster. You have to do it a step at a time." Helpstream's last phase came down to five hours over a weekend. Here were the steps:

Shutdown the service for maintenance;
Configure EC2 servers;
Move remaining 15 percent of data to EC2 servers;
Test;
Point DNS at new servers;
Leave proxy running in data center.

Fortunately for Helpstream, it went well. Hardy noted that the biggest risk was getting to step 5 and then having to roll back the changes. What moved the process along was migrating most of Helpstream's data ahead of time. Simply put, the more data you need to move the higher the risks.

The payoff (projected):

Since Helpstream has only been on Amazon Web Services for two weeks, its ROI figures are essentially projections. Here's a look at the monthly cost breakdown for Helpstream's data center:

It will save 100 percent on servers, switches, VPNs and other infrastructure;
21 percent on bandwidth, cage space and other Amazon charges;
59 percent on monitoring, server administration and in-cage work.

The biggest pop is clearly hardware costs. Helpstream reckons it will deliver savings of 59 percent overall by moving to Amazon. Anecdotally, Warfield noted that customers have seen a performance improvement. Going forward, Helpstream is looking to install more automation and control panels to scale up infrastructure and manage assets better. More reading:

posted by Larry Dignan
December 18, 2008 @ 2:25 am