600.perlbench_s
SPEC CPU®2017 Benchmark Description

Benchmark Name

600.perlbench_s

Benchmark Authors

Larry Wall, et. al.

Benchmark Program General Category

Programming language

Benchmark Description

600.perlbench_s is a cut-down version of Perl v5.22.1, the popular scripting language. SPEC's version of Perl has had most of OS-specific features removed. To understand how SPEC's version differs from the original, please see the comments in spec_config.h.

Input Description

The reference workload for 600.perlbench_s consists of three scripts:

• The primary component of the workload is the Open Source spam checking software SpamAssassin. SpamAssassin is used to score a couple of known corpora of both spam and ham (non-spam), as well as a sampling of mail generated from a set of random components. SpamAssassin has been heavily patched to avoid doing file I/O, and currently does not use the Bayesian filtering.
• Another component is the popular freeware email-to-HTML converter MHonArc. Email messages are generated randomly and converted to HTML. In addition to MHonArc, which was lightly patched to avoid file I/O, this component also uses several standard modules from the CPAN (Comprehensive Perl Archive Network).
• The third script (which also uses the mail generator for convenience) exercises a slightly-modified version of the 'specdiff' script, which is a part of the CPU2017 tool suite.

The training workload is similar, but not identical, to the reference workload from CPU2000. The test workload consists of the non-system- specific parts of the actual Perl 5.20.1 test harness.

Adding workloads is just a matter of adding another script and associated input data to the input directories. The script must have a .pl extension. If it needs input parameters, it will also be necessary to edit object.pm to enable those to be read from a file.

Output Description

In the case of the mail-based benchmarks, a line with salient characteristics (number of header lines, number of body lines, etc) is output for each message generated. During processing, MD5 hashes of the contents of output "files" (in memory) are computed and output. For SpamAssassin, the message's score and the rules that it triggered are also output.

Programming Language

ANSI C

Known Portability Issues

1. SPEC_CPU_NEED_POSIX_IDS: Windows compilers that do not automatically define _MSC_VER may define SPEC_CPU_NEED_POSIX_IDS as a portability flag to enable compilation.

2. Aliasing: There are known aliasing issues. The internal data structures that represent Perl's variables are accessed in such as a way as to violate ANSI aliasing rules. Compilation with optimizations that rely on strict compliance to ANSI C aliasing rules will most likely produce binaries that will not validate. A typical symptom that you might see is:

   Attempt to free unreferenced scalar: SV 0x7fa54a8031c0 at lib/strict.pm line 38.
   perlbench_r_base.may07a-m64(94624,0x7ffff25803c0) malloc: *** error for object 0x7fa54a4138a8: pointer being freed was not
   allocated
   *** set a breakpoint in malloc_error_break to debug
   Abort trap: 6

   Solution: Tell your compiler to be less aggressive in its optimizations, in particular as regards aliasing issues.
   For example, with GCC, you might try:
   -fno-strict-aliasing

   Note that in accordance with the same-for-all rule www.spec.org/cpu2017/Docs/runrules.html#BaseFlags, it is not allowed to set your aliasing flag as a PORTABILITY option. Instead, it must be applied to all of Base. The Example GCC config files as updated for SPEC CPU 2017 v1.1.5 obey this rule.

3. GCC -Ofast and unsafe math: Users of GCC's -Ofast option may encounter problems with the test workload failing to validate. If you go to the run directory and look at file test.out.mis, you may see output such as:

   $ head -2 test.out.mis
   0185:  t/op/sprintf2 ............................... ok
          t/op/sprintf2 ............................... FAILED at test 1459 

   or:

   0155:  t/op/pack ................................... ok
          t/op/pack ................................... FAILED at test 237
         

   Adding -fno-unsafe-math-optimizations and -fno-finite-math-only fixes the problem. Both flags are needed in order to resolve both of the above symptoms.

   Note that in accordance with the same-for-all rule www.spec.org/cpu2017/Docs/runrules.html#BaseFlags, it is not allowed to set the above flags as PORTABILITY options. Instead, they must be applied to all of Base. The Example GCC config files as updated for SPEC CPU 2017 v1.1.5 obey this rule.

Sources and Licensing

600.perlbench_s is based on perl-5.22.1, plus several third-party modules, including:

• HTML-Parser
• IO-stringy
• MailTools
• MHonArc
• SpamAssassin
• TimeDate

Please see details in the document SPEC CPU®2017 Licenses.

To avoid any possible conflict with standard executables, the benchmark is given the name 'perlbench'. Its differences are primarily described in the comments to spec_config.h. In brief: perlbench is a constrained version of perl which tries to remove most operating-system-specific functions, while leaving behind a set of features which are interesting as a CPU benchmark. The SPEC CPU® suites intentionally do not spend significant time in operating system services or IO.

References

• Perl Mongers: http://www.perl.org/
• O'Reilly's Perl Pages: http://www.perl.com/
• The Comprehensive Perl Archive Network: https://www.cpan.org/
• SpamAssassin: http://spamassassin.apache.org/
• MHonArc: https://www.mhonarc.org/

Last updated: $Date: 2020-09-23 10:06:01 -0400 (Wed, 23 Sep 2020) $

Copyright © 2017-2020 Standard Performance Evaluation Corporation (SPEC®)