Benchmark (computing)

In computing, a benchmark is the act of running a computer program, a set of programs, or other operations, in order to assess the relative performance of an object, normally by running a number of standard tests and trials against it.^[1]

The term benchmark is also commonly utilized for the purposes of elaborately designed benchmarking programs themselves.

Benchmarking is usually associated with assessing performance characteristics of computer hardware, for example, the floating point operation performance of a CPU, but there are circumstances when the technique is also applicable to software. Software benchmarks are, for example, run against compilers or database management systems (DBMS).

Benchmarks provide a method of comparing the performance of various subsystems across different chip/system architectures. Benchmarking as a part of continuous integration is called Continuous Benchmarking.^[2]

^ Fleming, Philip J.; Wallace, John J. (1986-03-01). "How not to lie with statistics: the correct way to summarize benchmark results". Communications of the ACM. 29 (3): 218–221. doi:10.1145/5666.5673. ISSN 0001-0782. S2CID 1047380.
^ Grambow, Martin; Lehmann, Fabian; Bermbach, David (2019). "Continuous Benchmarking: Using System Benchmarking in Build Pipelines". 2019 IEEE International Conference on Cloud Engineering (IC2E). pp. 241–246. doi:10.1109/IC2E.2019.00039. ISBN 978-1-7281-0218-4. Retrieved 2023-12-03.

[1] Fleming, Philip J.; Wallace, John J. (1986-03-01). "How not to lie with statistics: the correct way to summarize benchmark results". Communications of the ACM. 29 (3): 218–221. doi:10.1145/5666.5673. ISSN 0001-0782. S2CID 1047380.

[2] Grambow, Martin; Lehmann, Fabian; Bermbach, David (2019). "Continuous Benchmarking: Using System Benchmarking in Build Pipelines". 2019 IEEE International Conference on Cloud Engineering (IC2E). pp. 241–246. doi:10.1109/IC2E.2019.00039. ISBN 978-1-7281-0218-4. Retrieved 2023-12-03.

[1]

[2]