February 28th, 2008

XP SP2 vs. Vista RTM vs. Vista SP1: Gaming benchmark

Posted by Adrian Kingsley-Hughes @ 10:39 am

Categories: Admin

Tags: Game, Microsoft Windows XP, Service Pack 2, Microsoft Windows Vista, Microsoft Windows Vista SP1, Microsoft Windows Vista (Longhorn), Microsoft Windows XP Service Pack 2, Operating Systems, Microsoft Windows, Software

Here are the benchmark results that many of you have been waiting for - a look at how well ten popular games work on XP SP2, Vista RTM and Vista SP1.

It’s been nearly a year since I took a look at the state of gaming under the Vista OS.  The last time I compared XP SP2 to Vista, I was disappointed with Microsoft’s new OS as a gaming platform because I could get far more bang for my bucks out of XP than I could out of the new OS.  Partly I put this down to immature drivers, but on the whole I was convinced that at the core of the problem was Vista. 

So, is Vista worth bothering with as a gaming platform?

The tests

The tests will be carried out on the AMD Spider platform that I have set up in the lab (Phenom 9700, Radeon 3850 graphics card, 2GB of RAM …).  I’ve used this system as the platform for a number of benchmarks I’ve run over the past week (for a full spec, see this post).

Here’s the deal.  I set up three images for the system - one based on XP SP2, one on Vista RTM and one on Vista SP1 (all 32-bit flavor).  On each of these images all the updates provided by Microsoft were installed.  The installation of drivers in addition to those supplied by Microsoft was kept to a minimum (XP SP2 required more drivers than Vista).  On each of the images the latest ATi Catalyst drivers (version 8.2) were installed.  Then, in order to be able to measure and record frame rates, the latest version of the FRAPS utility was installed.

Finally, the games we were going to were then installed and each one updated to the latest version.  Here’s a list of the games used:

• BioShock
• Call of Duty 4
• Company of Heroes
• Crysis
• Doom 3
• F.E.A.R.
• The Elder Scrolls IV: Oblivion
• Supreme Commander
• Unreal Tournament 3
• World in Conflict

For these tests we chose not to rely on any of the canned benchmark tools supplied with some of these games.  Instead we relied on real-world play - that is, playing the same bit of the game on each of the platforms (a technique that HardOCP use when benchmarking).  While this method is subject to variations, if you’ve played games for long enough and are familiar with the map, you can pull this off quite well.

Note: Why not used canned benchmarks?  Well, by not doing so we are ruling out the possibility that the graphics card drivers that we use (or even the OS) has in any way been optimized to produce better results for the canned benchmarks that you might see in the game. 

To come up with workable graphics settings we first experimented with each of the games in turn.  When we found graphics settings that worked well (all the tests were run at 1280 x 1024 resolution in full-screen mode with the games set to either medium or high quality graphics) these settings were carefully recorded replicated them across the three platforms.

So to come up with the average frame rate (measured in frames per second) we play the same bit in each game with FRAPS recording, repeat this four times.  If there was any wild variation in the data we went back and took another stab at it (we only had to do this once, with Supreme Commander).  Once we were happy with the data we averaged it out to get a final frame rate score. 

We were very pleased with the end data because we ended up with runs where the average frame rate only varied by one or two frames per second.  This gave us confidence in the final results we achieved.

The results –>

Pages: 1 2 3