Benchmark Test
From Unofficial BOINC Wiki
Contents |
[edit] General
A synthetic program that attempts to quantify the capabilities of a computing system. The intent of the Benchmark program is to create a program that will provide an objective and reproducible metric. The downside is that the saying "There are lies, statistics, and benchmarks" is very true. The biggest problem is that all of these programs are artificial work loads and they do not match real world problems and usage patterns. However, in the case of the BOINC Software this is less of an issue in that the point of the benchmark is to calculate a rough metric of contribution and not an exact quantity for qualitative comparisons.
The best benchmarks have always been the actual workload itself. The best way to simulate reality is to do reality. However, real workloads tend to be chaotic and cannot be reproduced on demand which makes comparisons difficult.
If you look on the "Computer Summary" Page on the Project Web Site you will see:
- "Measured floating point speed" shows the value output by the Whetstone Benchmark, and
- "Measured integer speed" shows the value output by the Dhrystone Benchmark.
When we talk about these types of numbers,
- Floating point numbers are like: 1.48283 or 3.141592,
- Integer numbers are like: 1 or 2 or 938283 or 2004.
- Floating Point Numbers always have a decimal point
- Integer numbers are also called whole numbers, and they never have a decimal point.
- Numbers of either type can have a positive or negative value.
With that in mind, the following should be kept in mind:
- Whetstone Benchmark
- The Whetstone Benchmark does 8 different groups of tests (repeatedly of course), times how long they took to finish, and pops out a number. [ops performed]/[time].
- These tests all use floating point math operations of the CPUs being tested. Some of them are simple math plus, minus, times, divide and others check trigonometry features sine, cosine, tangent, exponent.
- Dhrystone Benchmark
- The Dhrystone Benchmark checks repeated integer operations, and several operating system file handling operations.
Neither of the tests really checks how well/fast a system can access memory, and the SETI@Home Science Application, for example, accesses memory a lot.
Here is an example of memory introducing a delay. A Pentium 4 CPU of any speed, can calculate the sine of an angle in approximately 170 tics of its internal clock. It could have performed 170 regular integer additions in this time.
But if it wanted to do an integer addition on a number somewhere out in memory (say it was working on a table of numbers), the CPU might have to wait for this memory integer to be delivered to the CPU as much as 260 tics. So a badly timed integer+memory operation would take far longer than a sine calculation.
This is where Celeron CPUs can really slow down. Pentium has many more features to predict when the CPU might be getting memory, and begins getting it long before the CPU actually calculates with it. Thus much less delay for most memory operations = faster.
Predicted time vs. actual
- Each Work Unit delivered to your machine includes an estimated number of Floating-Point Operations (FLOPS) calculations. BOINC divides this by the FP benchmark number to estimate completion time. SETI@Home's Work Units estimate number is currently always 27.9 trillion (American), however the actual number of Floating-Point Operations varies greatly which is why they don't all take the same amount of time.
- A better estimate might be calculated on the servers, but it would probably take so long looking through the WU data that much fewer Work Units would be generated and host machines wouldn't get Work Units.
SETI@Home vs. Benchmarks
- SETI@Home uses almost all "single" precision floating point math, whetstone is all "double" precision math. On Intel x86 processors the speed difference in calculating single vs. double isn't very large.
- SETI@Home uses mostly add, sub, multiply and divide. About 20% of its time is spent in trigonometry. Almost all the time in the Whetstone Benchmark is used for trigonometry.
- Memory access speed and trigonometry are the two major reasons that the benchmark results and SETI@Home processing speed don't match up on many systems.
Courtesy of: Benhur
When are Benchmarks run:
There are a number of conditions that will invoke the performance of the test:
- 1. Doing a new Client Install
- 2. When the Agent Version is Upgrade or Downgrade
- 3. Every 5 days on core0 (from 5.8.15)
- 4. When the system time changes significantly, forward or backward
- 5. If the number of cores change, thru e.g. the device profile or local preferences (latter from 5.10)
- 6. (Not confirmed), When new CPU is detected.
[edit] Also See
- Calibration
- Dhrystone Benchmark
- Cobblestone
- Reference Computer
- Whetstone Benchmark
- Example Benchmark Report
- Example Benchmark Report
- Paul's Lecture Notes
- More About Benchmarks
[edit] UCB Source
[edit] Copyright ©
- 2005 University of California
- 2005 Paul D. Buck
Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation.
RSS Feeds
