SPEC OMP2012 Result File Fields

The date when all the hardware necessary to run the result is generally available. For example, if the CPU is available in Aug-2012, but the memory is not available until Oct-2012, then the hardware availability date is Oct-2012 (unless some other component pushes it out farther).

2.1.8 Software Availability

(Also, "Software Avail")

The date when all the software necessary to run the result is generally available. For example, if the operating system is available in Aug-2012, but the compiler or other libraries are not available until Oct-2012, then the software availability date is Oct-2012 (unless some other component pushes it out farther).

2.1.9 Test date

The date when the test is run. This value is obtained from the system under test, unless the tester explicitly changes it.

2.1.10 Test sponsor

The name of the organization or individual that sponsored the test. Generally, this is the name of the license holder.

2.1.11 Tested by

The name of the organization or individual that ran the test. If there are installations in multiple geographic locations, sometimes that will also be listed in this field.

2.2 Result table

In addition to the graph, the results of the individual benchmark runs are also presented in table form.

2.2.1 Benchmark

The name of the benchmark.

2.2.2 Threads

The number of OpenMP threads (OMP_NUM_THREADS) that were used for this run.

2.2.3 Seconds

This is the amount of time in seconds that the benchmark took to run.

2.2.4 Ratio

This is the ratio of benchmark run time to the run time on the reference platform.

2.2.5 Energy kJoules

This is the amount of energy consumed (in kiloJoules) during the execution of the benchmark. This will only be present if the option power metric is run.

2.2.6 Maximum Power

This is the maximum rate of power consumed (in Watts) during the execution of the benchmark. This will only be present if the option power metric is run.

2.2.7 Average Power

This is the average rate of power consumed (in Watts) during the execution of the benchmark. This will only be present if the option power metric is run.

2.2.8 EnergyRatio

This is the ratio of benchmark average power consumption to the run time on the reference platform. This will only be present if the option power metric is run.

2.3 Notes/Tuning Information

(Also, "Notes/Tuning Information (Continued)")

This section is where the tester provides notes about compiler flags used, system settings, and other items that do not have dedicated fields elsewhere in the result.

Run rules relating to these items can be found in section 4.2 of the OMP2012 Run and Reporting Rules.

2.3.1 Compiler Invocation Notes

(Also, "Compiler Invocation Notes (Continued)")

This section is where the tester provides notes about how the various compilers were invoked, whether any special paths had to be used, etc.

2.3.2 Submit Notes

(Also, "Submit Notes (Continued)")

This section is where the tester provides notes about how the config file submit option was used to assign processes to processors.

2.3.3 Portability Notes

(Also, "Portability Notes (Continued)")

This section is where the tester provides notes about portability options and flags used to build the various benchmarks.

2.3.4 Base Tuning Notes

(Also, "Base Tuning Notes (Continued)")

This section is where the tester provides notes about baseline optimization options and flags used to build the various benchmarks.

2.3.5 Peak Tuning Notes

(Also, "Peak Tuning Notes (Continued)")

This section is where the tester provides notes about peak optimization options and flags used to build the various benchmarks.

2.3.6 Operating System Notes

(Also, "Operating System Notes (Continued)")

This section is where the tester provides notes about changes to the default operating system state and other OS-specific tuning information.

2.3.7 Platform Notes

(Also, "Platform Notes (Continued)")

This section is where the tester provides notes about changes to the default hardware state and other non-OS-specific tuning information.

2.3.8 Component Notes

(Also, "Component Notes (Continued)")

This section is where the tester provides information about various components needed to build a particular system. This section is only used if the system under test is built from parts and not sold as a whole system.

2.3.9 General Notes

(Also, "General Notes (Continued)")

This section is where the tester provides notes about things not covered in the other notes sections.

2.4 Compilation Flags Used

(Also, "Compilation Flags Used (Continued)")

This section is generated automatically by the benchmark tools. It details compilation flags used and provides links (in the HTML and PDF result formats) to descriptions of those flags.

2.4.1 Compiler Invocation

(Also, "Base Compiler Invocation" and "Peak Compiler Invocation")

This section lists the ways that the various compilers are invoked.

2.4.2 Portability Flags

(Also, "Base Portability Flags" and "Peak Portability Flags")

This section lists compilation flags that are used for portability.

2.4.3 Optimization Flags

(Also, "Base Optimization Flags" and "Peak Optimization Flags")

This section lists compilation flags that are used for optimization.

2.4.4 Other Flags

(Also, "Base Other Flags" and "Peak Other Flags")

This section lists compilation flags that are classified as neither portability nor optimization.

2.4.5 Unknown Flags

(Also, "Base Unknown Flags" and "Peak Unknown Flags")

This section of the reports lists compilation flags used that are not described in any flags description file. Results with unknown flags are marked "invalid" and may not be published. This marking may be removed by reformatting the result using a flags file that describes all of the unknown flags.

2.4.6 Forbidden Flags

(Also, "Base Forbidden Flags" and "Peak Forbidden Flags")

This section of the reports lists compilation flags used that are designated as "forbidden". Results using forbidden flags are marked "invalid" and may not be published.

2.5 Errors

This section is automatically inserted by the benchmark tools when there are errors present that prevent the result from being a valid reportable result.

3. Hardware description

Run rules relating to these items can be found in section 4.2.2 of the OMP2012 Run and Reporting Rules.

3.1 CPU Name

A manufacturer-determined processor formal name.

3.2 CPU Characteristics

Technical characteristics to help identify the processor.

3.3 CPU MHz

The clock frequency of the CPU, expressed in megahertz.

3.4 Maximum CPU MHz

The maximum clock frequency of the CPU, expressed in megahertz. This is referred to by some vendors as the Turbo frequency.

3.5 FPU

The type of floating-point unit used in the system.

3.6 CPU(s) enabled

The number of CPUs that were enabled and active during the benchmark run. More information about CPU counting is in the run rules.

3.7 CPU(s) orderable

The number of CPUs that can be ordered in a system of the type being tested.

3.8 Primary Cache

Description (size and organization) of the CPU's primary cache. This cache is also referred to as "L1 cache".

3.9 Secondary Cache

Description (size and organization) of the CPU's secondary cache. This cache is also referred to as "L2 cache".

3.10 L3 Cache

Description (size and organization) of the CPU's tertiary, or "Level 3" cache.

3.11 Other Cache

Description (size and organization) of any other levels of cache memory.

3.12 Memory

Description of the system main memory configuration. End-user options that affect performance, such as arrangement of memory modules, interleaving, latency, etc, are documented here.

3.13 Disk Subsystem

A description of the disk subsystem (size, type, and RAID level if any) of the storage used to hold the benchmark tree during the run.

3.14 Other Hardware

Any additional equipment added to improve performance.

4. Software description

Run rules relating to these items can be found in section 4.2.3 of the OMP2012 Run and Reporting Rules.

4.1 Operating System

The operating system name and version. If there are patches applied that affect performance, they must be disclosed in the notes.

4.2 Auto Parallel

Were multiple threads/cores/chips employed by a parallelizing compiler? Note that only OpenMP parallelism is allowed. Please see the run rules relating to this at section 1.1.4 of the OMP2012 Run and Reporting Rules.

4.3 Compiler

The names and versions of all compilers, preprocessors, and performance libraries used to generate the result.

4.4 File System

The type of the filesystem used to contain the run directories.

4.5 System State

The state (sometimes called "run level") of the system while the benchmarks were being run. Generally, this is "single user", "multi-user", "default", etc.

4.6 Base Pointers

Indicates whether all the benchmarks in base used 32-bit pointers, 64-bit pointers, or a mixture. For example, if the C and C++ benchmarks used 32-bit pointers, and the Fortran benchmarks used 64-bit pointers, then "32/64-bit" would be reported here.

4.7 Peak Pointers

Indicates whether all the benchmarks in peak used 32-bit pointers, 64-bit pointers, or a mixture.

4.8 Other Software

Any performance-relevant non-compiler software used, including third-party libraries, accelerators, etc.

5. Power and Temperature information

Run rules relating to these items can be found in section 4.2.6 of the OMP2012 Run and Reporting Rules.

5.1 Power Supply

This field is for the number and rating of the power supplies used in this system for this run.

5.2 Power Supply Details

This field is for more details about the power supply, like a part number or some other identifier.

5.3 Maximum Power (W)

This is the maximum power (in Watts) that was measured during the entire benchmark suite run.

5.4 Idle Power (W)

This is a 60 second measurement of idle power (in Watts) on the machine that is made after the benchmark has been run and the system was given time 10 seconds to rest.

5.5 Minimum Temperature (C)

This is the minimim temperature measured (in C) that was registered during the entire benchmark suite run.

5.6 Current Ranges Used

This is the current ranges that were used by the power analyzer as reported by PTDaemon.

5.7 Voltage Range Used

This is the voltage range that was used by the power analyzer as reported by PTDaemon.

5.8 Power Analyzer

This is the Power Analyzer name used to connect PTDaemon to the power analyzer. If more than one power analyzer was used, there will be multiple descriptions presented.

5.9 Hardware Vendor

This is the name of the company that provides the power analyzer or temperature meter.

5.10 Model

This is the model of the power analyzer or temperature meter.

5.11 Serial Number

This is the serial number of the power analyzer being used.

5.12 Input Connection

This is a description of the interface used to connect the power analyzer or temperature meter to the PTDaemon host system, e.g. RS-232 (serial port), USC, GPIB, etc.

5.13 Metrology Institute

Name of the national metrology institute, which specifies the calibration standards for power analyzers, appropriate for the Test Location reported in the FDR. Calibration should be done according to the standard of the country where the test was performed or where the power analyzer was manufactured.

Examples from accepted result respors: Country Metrology Institute

USA NIST (National Institute of Standards and Technology)
Germany PTB (Physikalisch Technische Bundesanstalt)
Japan AIST (National Institute of Advanced Science and Technology)
Taiwan (ROC) NML (National Measurement Laboratory)
China CNAS (China National Accreditation Service for Conformity Assessment)

A list of national metrology institutes for many countries is maintained by NIST here http://gsi.nist.gov/global/index.cfm.

5.14 Calibration By

This is name of the organization that performed the power analyzer calibration.

5.15 Calibration Label

This is a number or character string which uniquely identifies this meter calibration event. May appear on the calibration certificate or on a sticker applied to the power analyzer. The format of this number is specified by the metrology institute.

5.16 Calibration Date

The date (DD-MMM-YYYY) the calibration certificate was issued, from the calibration label or the calibration certificate.

5.17 PTDaemon Version

This is the version of the PTDaemon. It is automatically supplied by the tools.

5.18 Setup Description

This is a brief description of how the power analyzer or temperature sensor was used. This could include which power supply was connected to this power analyzer, or how far away this temperature sensor was from the air intake of the system.

5.19 Temperature Sensor

This is the name used to connect the PTDaemon to the temperature sensor. If more that one temperature sensor was used, there will be multiple descriptions presented.

6. Other information

6.1 Median results

For a reportable OMP2012 run, three iterations of each benchmark are run, and the median of the three runs is selected to be part of the overall metric. In output formats that support it, the medians in the result table are underlined in bold.

6.2 Run order

Each iteration now consists of running each benchmark in order. For example, given benchmarks "910.aaa", "920.bbb", and "930.ccc", here's what you might see as the benchmarks were run:

OMPG2012

    Running (#1) 910.aaa ref base oct09a default
    Running (#1) 920.bbb ref base oct09a default
    Running (#1) 930.ccc ref base oct09a default
    Running (#2) 910.aaa ref base oct09a default
    Running (#2) 920.bbb ref base oct09a default
    Running (#2) 930.ccc ref base oct09a default
    Running (#3) 910.aaa ref base oct09a default
    Running (#3) 920.bbb ref base oct09a default
    Running (#3) 930.ccc ref base oct09a default

When you read the results table from a run the results in the results table are listed in the order that they were run, in column-major order. In other words, if you're interested in the base scores as they were produced, start in the upper-lefthand column and read down the first column, then read the middle column, then the right column.

If the benchmarks were run with both base and peak tuning, all base runs were completed before starting peak.

W3C XHTML 1.0

W3C CSS