April 4th, 2007
Strike 3 for AMD hypocrisy on benchmarking
AMD swung hard at Intel for what it deemed as ethics violations, but since then, it has pulled the same stunt it's been criticizing Intel for three times in a row. I called out AMD's hypocrisy on strike one; David Berlind called it out on strike two; and now AMD is doing the same thing by cherry-picking the benchmarks and omitting superior Intel scores. Intel's Nick Knuppffer was rather upset with these omissions today and called AMD's benchmark postings "rubbish."
AMD, in this latest round, showed three sets of benchmarks comparing three products. It shows the latest AMD Opteron 2222 SE, which was released only this month, going against an Intel 5160 from mid-2006 and against an Intel 5355 released November 2006. The problem is that AMD omits the scores on two of the benchmarks and shows only the Intel 5355 in its worst light with SPECompMbase2001, which is a relatively obscure and less often cited benchmark to begin with. AMD selectively omits Intel's 5355 numbers for the more common SPECint_rate2006 and SPECfp_rate2006 benchmarks and then claims victory on all three sets of benchmarks. Here is the chart reconstructed with the best available scores posted by AMD, but I've also included the omitted 5355 scores highlighted in yellow.
| Published scores on SPEC.org | |||
| AMD 2222 SE | Intel 5355 | Intel 5160 | |
| SPECompMbase2001 | 13275 | 11822 | 10689 |
| SPECint_rate2006 Peak | 56.6 | 84.8 | 55.2 |
| SPECfp_rate2006 Peak | 52.1 | 60.2 | 45.1 |
| Using dual-core 5160 as baseline performance | |||
| AMD 2222 SE | Intel 5355 | Intel 5160 | |
| SPECompMbase2001 | 24.2% | 10.6% | 0% |
| SPECint_rate2006 Peak | 2.54% | 53.6% | 0% |
| SPECfp_rate2006 Peak | 15.5% | 33.5% | 0% |
So even without Intel's latest CPU in the lineup, Intel wins the two more common benchmarks by a significant margin. The biggest margin of victory goes to the integer performance of Intel's 2.66 GHz XEON 5355, and that gap will get even wider once Intel's just-released 3 GHz quad-core XEON gets included in this lineup. Earlier today, I explained the benefits of quad-core computing and what lies ahead for the two chip companies.
Cherry-picking benchmarks to show one's own product in its best light is fairly common in the industry, but not after publicly slamming one's opponent for unethical benchmarking. If AMD insists on calling Intel unethical for cherry-picking benchmarks, reasonable people have no choice but to call AMD hypocritical when it does the same thing. Since this is strike three on AMD hypocrisy in barely over a month, this appears to be a pattern that won't stop anytime soon.
George Ou is Technical Director of ZDNet. See his full profile and disclosure of his industry affiliations.
