February 19th, 2008

Another take on Vista vs. XP benchmarks

Posted by Ed Bott @ 10:02 pm

Categories: Windows Vista

Tags: Disk, Performance, Microsoft Windows XP, Service Pack 2, Problem, Microsoft Windows Vista, Microsoft Windows Vista SP1, Microsoft Windows Vista (Longhorn), Microsoft Windows XP Service Pack 2, Performance Management

As anyone who’s ever worked in a PC performance lab knows, the #1 rule of benchmarking is: Your mileage may vary.

I remembered that rule when I read my colleague Adrian Kingsley-Hughes’ epic account of his benchmark tests of Windows Vista SP1 versus Windows XP SP2 (Part 1 and Part 2). The first thing that struck me was how far apart his numbers were from those I was getting. In fact, I went back and redid all my tests to confirm that I hadn’t missed anything along the way. They checked out completely. On my test bed, with only one exception, Vista SP1 was consistently as fast as or faster than XP SP2, a result markedly at odds with Adrian’s findings.

I have no doubt that Adrian’s tests and timings were accurate, just as mine were. So what’s the difference?

For starters, our test beds were very different:

• Adrian chose a desktop system with a first-generation Intel dual-core processor, the 3.4 GHz Pentium 950D. I chose a Dell Inspiron 6400 notebook with an Intel T2050 1.6 GHz Core 2 Duo processor. (I bought this system in December 2006, a few weeks after Vista was released to business customers. It originally came with XP SP2 installed on it, and I upgraded the system to Vista almost immediately.)
• I chose to use a dual-boot configuration, designing my tests carefully so that file copy operations with each OS were done between the same source and destination volumes to minimize the effects of disk geometry on performance. Adrian used separate hard drives for each OS and each file copy operation.
• I used a wireless 802.11g network connection. Adrian used wired Gigabit Ethernet connections.
• Neither of us performed any special optimizations to either configuration except to ensure that drives were defragmented.

For my test files, I chose the same two groups of files I had used in previous rounds of performance testing last year. The first consisted of two large ISO files, each containing the contents of a ready-to-burn DVD, with a total size of 4.2 GB. The second group is a collection of music files, just over 1 GB in size, consisting of 272 MP3 files in 16 folders.

As it turns out, the test bed I chose is one that matches nicely with a lot of real world business-class systems. Notebooks represent the majority of the PC market these days, and the 802.11g connection in this one is by far the most popular networking option on portable PCs. From a performance standpoint, it’s neither a speed demon nor a slug. More importantly, this system’s specs match those that Microsoft’s engineers had in mind when they reengineered the file copy engine with Vista RTM and then with SP1. As Mark Russinovich notes in his detailed description of these changes, copies over high-latency networks such as WLANs are especially likely to benefit from the changes in Vista.

I ran each test multiple times and took the average of at least three tests. The graphs shown here are normalized, with Windows XP SP2 set to 100 and the results for Vista SP1 and Vista RTM charted proportionally.

Next –>

Pages: 1 2 3