Benchmark health considered harmful

Publisher Website
Google Scholar
Abstract
In the past couple of years, a number of software and architectural techniques have been proposed for improving the performance of linked data structrues. These research ideas are often evaluated using the Olden benchmark suite [1]. Frequently, in such experients, the largest speed-up is attained for the benchmark called health . This article demonstrates that this benchmark is a micro-benchmark for enormous linked lists traversals, and not a good one at that. Given that linked lists of such size are not an efficient data structure, it is unlikely that this benchmark corresponds to any real program. Hence the benchmark should not be used. To demonstrate the inherent inefficiency in its use of linked data structures, the health program was modified algorithmically to generate the same output, while improving the execution time by over a factor of 200 on a 500Mhz Pentium II Xeon.

This publication has 1 reference indexed in Scilit: