March 25th, 2008

Analyzing 3 months of Vista reliability data

Posted by Adrian Kingsley-Hughes @ 9:07 am

Categories: Hardware, Vista

Tags: Microsoft Windows Vista, Microsoft Internet Explorer, Crysis Reliability, Microsoft Windows, Web Browsers, Microsoft Windows Vista (Longhorn), Operating Systems, Software, Internet, Adrian Kingsley-Hughes

A Windows Vista feature which I make use of regularly is the Reliability Monitor.  While being far from an ideal metric for measuring reliability, it does offer users a simple way to tell if their system is behaving or misbehaving.  The other day I looked at the data I have collected for my main quad-core system and realized that I had just over three months of data collected since I wiped the system and upgraded to 64-bit Vista Ultimate.  With that much data I thought that it would be interesting to take a closer look at this data and see what kind of issues my system has experienced over the past three months.

In case you’ve not come across the Reliability Monitor, here’s how you get to it:

• The quick way:
  Click Start and type Perfmon into the Start Search box, and click the Perfmon shortcut when it appears.  then click on Reliability Monitor.
• The long way:
  Click Start > Control Panel > System and Maintenance > Performance Rating and Tools > Advanced Tools > Open Reliability and Performance Monitor > Reliability Monitor.

The Reliability Monitor gives you access to a lot of varied and useful information. 

• A System Stability Chart which shows you a stability rating between 10 and 0 (the higher the number the better).
• The chart also shows if any of the following occurred on a particular day:
  - Software (Un)Install
  - Application Failures
  - Hardware Failures
  - Windows Failures
  - Miscellaneous Failures
• There is also a system stability report for each day which gives a breakdown of each of the above categories for each day.

Because it’s difficult to extract the data from the Reliability Monitor I have extracted the data manually so that I can present it here.

Here’s the data:

Here’s a breakdown of the failures that I encountered over the 133 days (note that multiple failures can occur during a single day):

• Application failures: 22 days (16.5%)
• Hardware failures: 0 days (0%)
• Windows Failures: 5 days (3.7%)
• Misc failures: 14 days (10.5%)
• Total failure days: 37 days (27.8%)

Let’s now take a look at each of the categories in detail.

• Application failures:
  - Internet Explorer 7 (iexplore.exe): 25
  - Crysis (crysis64.exe): 10
  - Foxit Reader (foxit reader.exe): 4
  - Explorer (explorer.exe): 3
  - SnagIt (SnagIt32.exe): 2
  - ATi Catalyst 8.1 (MMLoadDrv.exe): 2
  - Microsoft management Console (mmc.exe): 1
  - Windows Host Process (rundll32.exe): 1
  - RivaTuner (RivaTuner.exe): 1
• Windows failures:
  - OS stopped working: 5
• Miscellaneous failures:
  - Disruptive shutdown: 14

Here is an updated chart containing system upgrade details and information on some of the crashes.

From examining this data you can see that the two applications most prone to misbehaving are Crysis and Internet Explorer 7.  Crysis reliability has improved dramatically with the application of the game patches, and with each update to the Catalyst drivers (I play other games on the system regularly - Call of Duty 4, Company of Heroes, Silent Hunter IV, Orange Box games, Battlefield 2142 - but these behave themselves well).  I’m a firm believer in updating the graphics card drivers regularly as I’m certain this has a knock-on reliability benefit.  As for Internet Explorer, a number of these crashes I’ve traced back to a faulty Flash player, and updating the plug in seemed to return IE7 back to the sort of reliability which I expect.

As for the Windows failures, I’m confident that these were down to system crashes resulting from graphics cards driver.  Up until lately my system was experiencing a fair number of TDR - Timeout Detection and Recovery - warning, and while most of these were caught by Windows and a crash avoided, some ended up in the system crashing.  I traced this issue down to having dual monitors running at different resolutions and as soon as I put two identical monitors on the system the problem solved itself (gone from at least one a day to none in over a month).

I don’t put too much weight to the actual reliability index of my systems but I do keep an eye on the chart, because even though a couple of crashes in a day can drop the rating down significantly, I do then like to see a period of recovery, which shows that I’ve dealt with the issue.  What I’m aiming for as drivers become more robust and Vista SP1 settles down is to see the chart settle down a bit so there are fewer dips.  Well see.

Thoughts?  Questions?  Anyone else want to share their Vista reliability index trends?