Document Title: execution_without_SPEC_tools.txt Subject: How to use the SPEC CPU2000 benchmarks while making minimal use of SPEC's tool set. Last updated: 29-Oct-2001 jh (To check for possible updates to this document, please see http://www.spec.org/cpu2000/docs/ ) ------------------------------------------------------------------------ This document is for the user who: - wants to work directly with benchmark source code and compile by hand (rather than through the SPEC supplied tools) - wants to run the benchmarks directly from the command line (rather than via the SPEC supplied tools) - does not care to submit a result to SPEC - or at least doesn't want to yet - and so doesn't want the features that the tools provide to do formal, submittable runs. If the above describes you, here is a suggested path which should lead quickly to your desired state. (1) Please read just one page from www.spec.org/cpu2000/docs/runrules.html namely these two sections: 4.5 Research and Academic usage of CPU2000 4.6 Required Disclosures These sections acknowledge that the suite may be used in ways other than the formal environment enforced by the SPEC tools; but they warn that if you plan to publish your results, you should be able to state HOW your usage of the suite differs from the standard usage. So even if you skip over the tools and the run-rules today, you should plan a time to come back and learn them later. (2) Get through a successful installation, and verify that you can at least build and run one benchmark under the tools. Yes, we are about to teach you how to bypass the tools most of the time -- but the bypass still makes some minimal use of them. So you need a working toolset. If you can't make it through $SPEC/docs/install_guide_unix.html or %SPEC%\docs\install_guide_nt.html please send email to cpu2000support@spec.org and we'll try to help you. (3) Pick a benchmark that will be your starting point. There are 26 benchmarks in the CPU2000 suite; pick one that you'd like to start with. If you're having trouble deciding, please see the descriptions at http://www.spec.org/cpu2000 . If you're still not sure, start with 171.swim (Fortran) or 181.mcf (C). These are two of the shortest benchmarks for lines of code, and therefore relatively easy to understand, and they are also interesting benchmarks for their effects on memory systems. (4) Pick a config file for an environment that resembles your environment. You'll find a variety of config files on the kit (in $SPEC/config/ [Unix-like systems] or %SPEC%\config\ [NT]) or at http://www.spec.org (with the submitted CPU2000 results). Don't worry if the config file you pick doesn't exactly match your environment; you're just looking for a starting point. (5) Obtain a working copy of the sources for your benchmark. Now that you know what benchmark you're starting from, and what config file, you're ready to get your own copy of the sources. For example, let's suppose that you want to work with 171.swim, and your environment is at least partially similar to the environment described in the comments for $SPEC/config/alpha-linux-gem.cfg [Other Unix-like systems should behave similarly to the example in this section; for an NT'ish example, see the next section.] For the Linux example, you would enter the following commands: % sh (or invoke some other Bourne-compatible shell) $ cd xxx (wherever you installed the SPEC CPU2000 tree) $ . ./shrc (that's dot-space-dot-slash-shrc) $ runspec --action=build --tune=base --config=alpha-linux-gem swim IT IS LIKELY THAT THE ABOVE COMMAND WILL APPEAR TO FAIL, unless your environment is very similar to the environment assumed by the config file. IGNORE THE ERROR MESSAGES, because all you really want to do is to create a directory with a working copy of the sources. This has been done, and you can get there using the following commands: $ cd $SPEC/be*/CF*/*swim/run $ grep build list ...This will print a string for each subdirectory, including ...the one where the build was just attempted. If this is ...your first time here, it will be directory 00000001 $ cd 00000001 (6) Observe how the tools attempted to build the benchmark, by using specmake with the dry run ("-n") switch. You can now work in this directory, including making source code changes and trying other build commands, without affecting the original sources. Here is an NT example; the commands on Unix would be highly similar, with the exception that the slashes go the other direction, and you would use "grep" instead of "findstr": C:\cpu2000>shrc C:\cpu2000>echo off Setting environment for using Visual Fortran tools PATH=^ C:\cpu2000\bin;^ C:\Program Files\Microsoft Visual Studio\Common\Msdev98\BIN;^ C:\Program Files\Microsoft Visual Studio\DF98\BIN;^ C:\Program Files\Microsoft Visual Studio\VC98\BIN;^ C:\Program Files\Microsoft Visual Studio\Common\Tools;^ C:\Program Files\Microsoft Visual Studio\Common\Tools\WINNT;^ C:\WINNT\system32;^ C:\WINNT C:\cpu2000>runspec --action=build --config=intel_nt_visual_studio ^ More? --tune=base swim runspec v2.00 - Copyright (C) 1999 Standard Performance Evaluation Corp Loading standard modules.............. Loading runspec modules............. Identifying output formats...asc...config...html...pdf...ps...raw... We will use: 171.swim Compiling Binaries Building 171.swim ref base dec15a default Build Complete C:\cpu2000>cd be*\CF*\*swim\run C:\cpu2000\benchspec\CFP2000\171.swim\run>findstr build list 00000001 dir=C:/cpu2000/benchspec/CFP2000/171.swim/run/00000001 ext=dec15a lock=0 type=build username=Administrator C:\cpu2000\benchspec\CFP2000\171.swim\run>cd *01 C:\cpu2000\benchspec\CFP2000\171.swim\run\00000001>specmake -n f90 -Bd -fast -optimize:5 -traceback swim.f -Feswim (7) Once you have succeeded in building an executable image, place it in the directory where the tools expect to find it, with the extension specified in your config file. For example, if you are working with 171.swim and your config file has the line ext=dec15a you would copy your executable to $SPEC/benchspec/CFP2000/171.swim/exe/swim_base.dec15a [Unix] or %SPEC%\benchspec\CFP2000\171.swim\exe\swim_base.dec15a [NT] Edit your config file and add the line: check_md5=0 (8) Obtain a run directory for the test (minimal) dataset, and observe how the tools run the benchmark by using specinvoke with the dry run ("-n") switch. For example: C:\cpu2000\benchspec\CFP2000\171.swim\run>runspec --action=run ^ More? --config=intel_nt_visual_studio --tune=base --size=test ^ More? --iterations=1 swim runspec v2.00 - Copyright (C) 1999 Standard Performance Evaluation Corp Loading standard modules.............. Loading runspec modules............. Identifying output formats...asc...config...html...pdf...ps...raw... We will use: 171.swim Compiling Binaries Up to date 171.swim test base dec15a default Setting Up Run Directories Setting up 171.swim test base dec15a default: existing Running Benchmarks Running 171.swim test base dec15a default Error: 1x171.swim Run Complete C:\cpu2000\benchspec\CFP2000\171.swim\run>findstr test list 00000002 dir=C:/cpu2000/benchspec/CFP2000/171.swim/run/00000002 ext=dec15a lock=0 mach=default size=test tune=base type=run username=Administrator C:\cpu2000\benchspec\CFP2000\171.swim\run>cd *02 C:\cpu2000\benchspec\CFP2000\171.swim\run\00000002>specinvoke -n ../00000002/swim_base.dec15a < swim.in > swim.out 2> swim.err As you can see above, swim is invoked in a simple, straightforward way. Other benchmarks have more complex invocations. (9) Repeat the previous step to create directories for the other available workloads: --size=train and --size=ref (10) Important: if you are at all interested in saving your work, move the run/nnnnnnnn directories to some safer location. That way, your work areas will not be deleted by the first user who comes along and says "runspec --action=clean". ----------------------------------------------------------------------------- Copyright (C) 1999-2000 Standard Performance Evaluation Corporation All Rights Reserved