304 lines
13 KiB
Plaintext
304 lines
13 KiB
Plaintext
Version 4.1.0 -- 1997.08.26
|
|
|
|
================================================================
|
|
To use Unixbench:
|
|
|
|
make
|
|
Run
|
|
|
|
================================================================
|
|
You may run individual benchmark programs and groups of programs.
|
|
There are also a few options.
|
|
|
|
Run [<bench> ...] [<group> ...] [-qvd]
|
|
|
|
-q quiet no output to standard out
|
|
-v verbose extra output to standard out
|
|
-d debug Run is displayed as it is run
|
|
-<number> Use <number> as maximum number of iterations
|
|
(e.g. Run -1 index)
|
|
|
|
GROUP BENCHMARK EXPLANATION SOURCE
|
|
-------- ----------- -------------------- -------------
|
|
index only those need for
|
|
generating the BYTE
|
|
index
|
|
|
|
dhry2reg dhrystone 2 with regs src/dhry_1.c
|
|
src/dhry_2.c
|
|
src/dhry.h
|
|
whetstone-double double-prec. whetstone src/whets.c
|
|
execl execl call src/execl.c
|
|
src/big.c
|
|
fstime filesystem throughput src/fstime.c
|
|
fsbuffer filesystem throughput src/fstime.c
|
|
fsdisk filesystem throughput src/fstime.c
|
|
pipe pipe throughput src/pipe.c
|
|
context1 pipe context switch src/context1.c
|
|
spawn process creation src/spawn.c
|
|
shell load system with pgms/multi.sh
|
|
concurrent shell pgms/tst.sh
|
|
scripts src/looper.c
|
|
syscall system call overhead src/syscall.c
|
|
|
|
arithmetic
|
|
whetstone-double double-prec. whetstone src/whets.c
|
|
arithoh arithmetic overhead src/arith.c
|
|
register register arithmetic src/arith.c
|
|
short short arithmetic src/arith.c
|
|
int int arithmetic src/arith.c
|
|
long long arithmetic src/arith.c
|
|
float float arithmetic src/arith.c
|
|
double double arithmetic src/arith.c
|
|
|
|
system
|
|
syscall system call overhead src/syscall.c
|
|
pipe pipe throughput src/pipe.c
|
|
context1 pipe context switch src/context1.c
|
|
spawn process creation src/spawn.c
|
|
execl execl call src/execl.c
|
|
fstime filesystem throughput src/fstime.c
|
|
fsbuffer filesystem throughput src/fstime.c
|
|
fsdisk filesystem throughput src/fstime.c
|
|
shell load system with pgms/multi.sh
|
|
concurrent shell pgms/tst.sh
|
|
scripts src/looper.c
|
|
|
|
misc
|
|
C compile and link /bin/cc
|
|
src/looper.c
|
|
dc calculations with dc testdir/dc.dat
|
|
src/looper.c
|
|
hanoi recursion src/hanoi.c
|
|
src/looper.c
|
|
|
|
dhry
|
|
dhry2 dhrystone 2 w/o regs src/dhry_1.c
|
|
src/dhry_2.c
|
|
src/dhry.h
|
|
dhry2reg dhrystone 2 with regs src/dhry_1.c
|
|
src/dhry_2.c
|
|
src/dhry.h
|
|
shell load system with pgms/multi.sh
|
|
concurrent shell pgms/tst.sh
|
|
scripts
|
|
** Most C programs also include src/timeit.c
|
|
|
|
===================== RELEASE NOTES =====================================
|
|
|
|
v4.1.0
|
|
|
|
Double precision Whetstone put in place instead of the old "double" benchmark.
|
|
|
|
Removal of some obsolete files.
|
|
|
|
"system" suite adds shell8.
|
|
|
|
perlbench and poll added as "exhibition" (non-index) benchmarks.
|
|
|
|
Incorporates several suggestions by Andre Derrick Balsa <andrewbalsa@usa.net>
|
|
|
|
Code cleanups to reduce compiler warnings by David C Niemi <niemi@tux.org>
|
|
and Andy Kahn <kahn@zk3.dec.com>; Digital Unix options by Andy Kahn.
|
|
|
|
======================== Jun 97 ==========================
|
|
|
|
v4.0.1
|
|
|
|
Minor change to fstime.c to fix overflow problems on fast machines. Counting
|
|
is now done in units of 256 (smallest BUFSIZE) and unsigned longs are used,
|
|
giving another 23 dB or so of headroom ;^) Results should be virtually
|
|
identical aside from very small rounding errors.
|
|
|
|
======================== Dec 95 ==========================
|
|
|
|
v4.0
|
|
|
|
Byte no longer seems to have anything to do with this benchmark, and I was
|
|
unable to reach any of the original authors; so I have taken it upon myself
|
|
to clean it up.
|
|
|
|
This is version 4. Major assumptions made in these benchmarks have changed
|
|
since they were written, but they are nonetheless popular (particularly for
|
|
measuring hardware for Linux). Some changes made:
|
|
|
|
- The biggest change is to put a lot more operating system-oriented
|
|
tests into the index. I experimented for a while with a decibel-like
|
|
logarithmic scale, but finally settled on using a geometric mean for
|
|
the final index (the individual scores are a normalized, and their
|
|
logs are averaged; the resulting value is exponentiated).
|
|
|
|
"George", certain SPARCstation 20-61 with 128 MB RAM, a SPARC Storage
|
|
Array, and Solaris 2.3 is my new baseline; it is rated at 10.0 in each
|
|
of the index scores for a final score of 10.0.
|
|
|
|
Overall I find the geometric averaging is a big improvement for
|
|
avoiding the skew that was once possible (e.g. a Pentium-75 which got
|
|
40 on the buggy version of fstime, such that fstime accounted for over
|
|
half of its total score and hence wildly skewed its average).
|
|
|
|
I also expect that the new numbers look different enough from the old
|
|
ones that no one is too likely to casually mistake them for each other.
|
|
|
|
I am finding new SPARCs running Solaris 2.4 getting about 15-20, and
|
|
my 486 DX2-66 Compaq running Linux 1.3.45 got a 9.1. It got
|
|
understandably poor scores on CPU and FPU benchmarks (a horrible
|
|
1.8 on "double" and 1.3 on "fsdisk"); but made up for it by averaging
|
|
over 20 on the OS-oriented benchmarks. The Pentium-75 running
|
|
Linux gets about 20 (and it *still* runs Windows 3.1 slowly. Oh well).
|
|
|
|
- It is difficult to get a modern compiler to even consider making
|
|
dhry2 without registers, short of turning off *all* optimizations.
|
|
This is also not a terribly meaningful test, even if it were possible,
|
|
as noone compiles without registers nowadays. Replaced this benchmark
|
|
with dhry2reg in the index, and dropped it out of usage in general as
|
|
it is so hard to make a legitimate one.
|
|
|
|
- fstime: this had some bugs when compiled on modern systems which return
|
|
the number of bytes read/written for read(2)/write(2) calls. The code
|
|
assumed that a negative return code was given for EOF, but most modern
|
|
systems return 0 (certainly on SunOS 4, Solaris2, and Linux, which is
|
|
what counts for me). The old code yielded wildly inflated read scores,
|
|
would eat up tens of MB of disk space on fast systems, and yielded
|
|
roughly 50% lower than normal copy scores than it should have.
|
|
|
|
Also, it counted partial blocks *fully*; made it count the proportional
|
|
part of the block which was actually finished.
|
|
|
|
Made bigger and smaller variants of fstime which are designed to beat
|
|
up the disk I/O and the buffer cache, respectively. Adjusted the
|
|
sleeps so that they are short for short benchmarks.
|
|
|
|
- Instead of 1,2,4, and 8-shell benchmarks, went to 1, 8, and 16 to
|
|
give a broader range of information (and to run 1 fewer test).
|
|
The only real problem with this is that not many iterations get
|
|
done with 16 at a time on slow systems, so there are some significant
|
|
rounding errors; 8 therefore still used for the benchmark. There is
|
|
also the problem that the last (uncompleted) loop is counted as a full
|
|
loop, so it is impossible to score below 1.0 lpm (which gave my laptop
|
|
a break). Probably redesigning Shell to do each loop a bit more
|
|
quickly (but with less intensity) would be a good idea.
|
|
|
|
This benchmark appears to be very heavily influenced by the speed
|
|
of the loader, by which shell is being used as /bin/sh, and by how
|
|
well-compiled some of the common shell utilities like grep, sed, and
|
|
sort are. With a consistent tool set it is also a good indicator of
|
|
the bandwidth between main memory and the CPU (e.g. Pentia score about
|
|
twice as high as 486es due to their 64-bit bus). Small, sometimes
|
|
broken shells like "ash-linux" do particularly well here, while big,
|
|
robust shells like bash do not.
|
|
|
|
- "dc" is a somewhat iffy benchmark, because there are two versions of
|
|
it floating around, one being small, very fast, and buggy, and one
|
|
being more correct but slow. It was never in the index anyway.
|
|
|
|
- Execl is a somewhat troubling benchmark in that it yields much higher
|
|
scores if compiled statically. I frown on this practice because it
|
|
distorts the scores away from reflecting how programs are really used
|
|
(i.e. dynamically linked).
|
|
|
|
- Arithoh is really more an indicator of the compiler quality than of
|
|
the computer itself. For example, GCC 2.7.x with -O2 and a few extra
|
|
options optimizes much of it away, resulting in about a 1200% boost
|
|
to the score. Clearly not a good one for the index.
|
|
|
|
I am still a bit unhappy with the variance in some of the benchmarks, most
|
|
notably the fstime suite; and with how long it takes to run. But I think
|
|
it gets significantly more reliable results than the older version in less
|
|
time.
|
|
|
|
If anyone has ideas on how to make these benchmarks faster, lower-variance,
|
|
or more meaningful; or has nice, new, portable benchmarks to add, don't
|
|
hesitate to e-mail me.
|
|
|
|
David C Niemi <niemi@tux.org> 7 Dec 1995
|
|
|
|
======================== May 91 ==========================
|
|
This is version 3. This set of programs should be able to determine if
|
|
your system is BSD or SysV. (It uses the output format of time (1)
|
|
to see. If you have any problems, contact me (by email,
|
|
preferably): ben@bytepb.byte.com
|
|
|
|
---
|
|
|
|
The document doc/bench.doc describes the basic flow of the
|
|
benchmark system. The document doc/bench3.doc describes the major
|
|
changes in design of this version. As a user of the benchmarks,
|
|
you should understand some of the methods that have been
|
|
implemented to generate loop counts:
|
|
|
|
Tests that are compiled C code:
|
|
The function wake_me(second, func) is included (from the file
|
|
timeit.c). This function uses signal and alarm to set a countdown
|
|
for the time request by the benchmark administration script
|
|
(Run). As soon as the clock is started, the test is run with a
|
|
counter keeping track of the number of loops that the test makes.
|
|
When alarm sends its signal, the loop counter value is sent to stderr
|
|
and the program terminates. Since the time resolution, signal
|
|
trapping and other factors don't insure that the test is for the
|
|
precise time that was requested, the test program is also run
|
|
from the time (1) command. The real time value returned from time
|
|
(1) is what is used in calculating the number of loops per second
|
|
(or minute, depending on the test). As is obvious, there is some
|
|
overhead time that is not taken into account, therefore the
|
|
number of loops per second is not absolute. The overhead of the
|
|
test starting and stopping and the signal and alarm calls is
|
|
common to the overhead of real applications. If a program loads
|
|
quickly, the number of loops per second increases; a phenomenon
|
|
that favors systems that can load programs quickly. (Setting the
|
|
sticky bit of the test programs is not considered fair play.)
|
|
|
|
Test that use existing UNIX programs or shell scripts:
|
|
The concept is the same as that of compiled tests, except the
|
|
alarm and signal are contained in separate compiled program,
|
|
looper (source is looper.c). Looper uses an execvp to invoke the
|
|
test with its arguments. Here, the overhead includes the
|
|
invocation and execution of looper.
|
|
|
|
--
|
|
|
|
The index numbers are generated from a baseline file that is in
|
|
pgms/index.base. You can put tests that you wish in this file.
|
|
All you need to do is take the results/log file from your
|
|
baseline machine, edit out the comment and blank lines, and sort
|
|
the result (vi/ex command: 1,$!sort). The sort in necessary
|
|
because the process of generating the index report uses join (1).
|
|
You can regenerate the reports by running "make report."
|
|
|
|
--
|
|
|
|
========================= Jan 90 =============================
|
|
Tom Yager has joined the effort here at BYTE; he is responsible
|
|
for many refinements in the UNIX benchmarks.
|
|
|
|
The memory access tests have been deleted from the benchmarks.
|
|
The file access tests have been reversed so that the test is run
|
|
for a fixed time. The amount of data transfered (written, read,
|
|
and copied) is the variable. !WARNING! This test can eat up a
|
|
large hunk of disk space.
|
|
|
|
The initial line of all shell scripts has been changed from the
|
|
SCO and XENIX form (:) to the more standard form "#! /bin/sh".
|
|
But different systems handle shell switching differently. Check
|
|
the documentation on your system and find out how you are
|
|
supposed to do it. Or, simpler yet, just run the benchmarks from
|
|
the Bourne shell. (You may need to set SHELL=/bin/sh as well.)
|
|
|
|
The options to Run have not been checked in a while. They may no
|
|
longer function. Next time, I'll get back on them. There needs to
|
|
be another option added (next time) that halts testing between
|
|
each test. !WARNING! Some systems have caches that are not getting flushed
|
|
before the next test or iteration is run. This can cause
|
|
erroneous values.
|
|
|
|
========================= Sept 89 =============================
|
|
The database (db) programs now have a tuneable message queue space.
|
|
queue space. The default set in the Run script is 1024 bytes.
|
|
Other major changes are in the format of the times. We now show
|
|
Arithmetic and Geometric mean and standard deviation for User
|
|
Time, System Time, and Real Time. Generally, in reporting, we
|
|
plan on using the Real Time values with the benchs run with one
|
|
active user (the bench user). Comments and arguments are requested.
|
|
|
|
contact: BIX bensmith or rick_g
|