AMD to BAPCo: We Quit

Nigel Dessau, AMD senior vice president and chief marketing officer, said in a statement that SysMark 2012, one of the consortia's latest system benchmarks, doesn't reflect real-world usage scenarios.

"Technology is evolving at an incredible pace, and customers need clear and reliable measurements to understand the expected performance and value of their systems," he said. "AMD does not believe SM2012 achieves this objective."

In the CRN Test Center, we agree; we gave up using SysMark years ago, and for some of the same reasons. In a blog post dated Tuesday, Dessau wrote that many of today's applications "use JavaScript and HTML5. We use applications that have high video content and tools that can be accelerated with GPUs. Off the shelf, Microsoft’s Internet Explorer 9, Office 11 and Movie Maker all use GPU acceleration – they come with it turned on. So does Adobe Flash......we need benchmarks that are relevant for those applications."

Dessau asserts that SysMark apps and scripts do not take adequate advantage of these capabilities, and give unfair advantage to AMD arch rival Intel, which remains a BAPCo member.

"The heart of our complaint is this: the SYSmark benchmark is not only comprised of unrepresentative workloads (workloads that ignore the importance of heterogeneous computing and, frankly, favor our competitor’s designs), but it actually generates misleading results that can lead to very poor purchasing decisions," he wrote.

This is not a new position for AMD, which has complained for nearly a decade about how benchmarks emphasize the wrong things and that clock speeds don't tell the whole story.

The company further stated today that it "will only endorse benchmarks based on real-world computing models and software applications, and which provide useful and relevant information. AMD believes benchmarks should be constructed to provide unbiased results and be transparent to customers making decisions based on those results. Hence AMD cannot endorse or support SM2012 or remain part of the BAPCo consortium."

The company is evaluating other benchmarking alternatives, including the creation of "an industry consortium to establish an open benchmark to measure overall system performance."