December 20th, 2005
Scalability
I’ve been thinking about scalability a
lot today. The thought process actually kicked off with the Typepad
outage from last week…even though
in the end scalability
wasn’t exactly the problem there.
Yet it seems that “growing pains” is a common compliant
about many Internet social software tools — everything from Technorati
to Orkut have been criticized for straining under increased load.
I’ve never really been a coder, but
I’ve watched lots of different systems have to adjust — or rearchitect
– due to scale challenges. Back in 1990, I worked for FTD when they
wanted to get into an early version of e-commerce. Their approach
was to develop an in-house system which used an ASCII text colon-delimited
flat file for product records…which was fine for fifty items but not
for 5000. Needless to say, that system didn’t last very long (though
there were hundreds of additional reasons for that, which would make an
interesting case study one of these days).
In my world, cc:Mail was definitely
a system which eventually burst at the seams under scale issues. As
a cc:Mail administrator back in 1992, I only had solid performance when
my post offices were each about 200 MB. Now that’s 1/3 the size of
my own mailbox. cc:Mail had great hardcoded limits like 200 users
on a mailing list, or 20 attachments per message. Obviously, Hubert
and company designed an incredible system for its day, but its file-share
architecture eventually hit the wall for modern-day messaging requirements.
Given all of that context, it’s somewhat
amazing that Notes has been able to evolve from 75 users per server over
NETBIOS in Notes R3 to 150,000 user scalability with Domino 7, all without
a fundamental rearchitecture. It may not be sexy work, but the plumbing
done in Domino 7 to get 40-400% scalability improvements on existing hardware
is especially impressive when viewed this historical context.
Originally by Ed Brill from Ed Brill on December 19, 2005, 1:15pm
