What is a benchmark?

A benchmark program, in the context of TOPBENCH’s intended audience, is a program that runs a section of code and times how long it takes that section to execute.  Most benchmarks will run a small section of code in a loop until a period of time goes by, and then note how many loops were achieved.  These code sections, coupled with how long they are allowed to execute in a loop, are referred to as metrics in the research section of this website.

Benchmarks are completely arbitrary programs; they serve no purpose other than being a more convenient tool to measure relative performance between systems.

The most accurate benchmark you can run is one that is tailored to your specific interest or need.  For example, if you want to build a new PC just to run the latest Call of Duty, then running Call of Duty on several machines is the best way to get useful information.  Since it’s not always possible/practical to do that (Call of Duty is not a free program), that’s where benchmarking programs come in:  They are freely available programs with repeatable test sequences that are easy to use.

Charles Bloom, one of my favorite compression authors and raving/rant-y bloggers, recently shared some thoughts on metrics where he also warns that metrics, if not properly understood, can steer you in the wrong direction.

Advertisements
  1. Leave a comment

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: