February 26th, 2008
Vista RTM vs. Vista SP1 - Office 2007 benchmarking
Thoughts on benchmarking
I’ve been benchmarking systems for long enough to know that no matter how many questions I think that my results answer, what I’m really doing is creating about three new questions for each question I solve. This is what happened with my earlier run of Vista benchmarks - I’d run some tests, you’d then come back and offer different scenarios that you’d like to see done and different platforms for those scenarios.
Benchmarking is an artificial activity. The goal is to eliminate as many variables as possible and achieve some consistent metric. Problem is, by removing the variables you’re actually shifting the process out from reality and into a make believe land which only exists on the PCs being benchmarked. Before you carry out your daily PC tasks I bet that you don’t take elaborate steps to ensure that you get consistency. Hence my enthusiastic use of the phrase “your mileage WILL vary.”
Another fundamental problem with benchmarking is that neither the tests nor the results are exactly what people want. Ultimately, what everyone wants to see is a benchmark of their daily tasks carried out on their PC. That, I’m afraid, is something I cannot provide.
Thoughts?
<< Home >>
Adrian is a technology journalist and author who has devoted over a decade to helping users get the most from technology. He also runs a popular blog called The PC Doctor. See his full profile and disclosure of his industry affiliations
Want to get in touch? Got a tip? Feel free to drop me a note! I ALWAYS respect anonymity. I'm also on Twitter (@the_pc_doc)
Right to Reply: Should any industry representatives wish to comment on any posts on Hardware 2.0, I will be happy to publish their reply verbatim on this blog.
Subscribe to Hardware 2.0 via Email alerts or RSS.
| 02/27/08
