SPECmail2001 Support FAQ Version 1.01

Errors Running the Benchmark

1. Using Java Version 1.1.8
2. Benchmark Manager Only Prints Version Number
3. Connecting to Remote Clients Failed: InvalidClassException
4. Couldn't Read Config File Error
5. Verification Failed
6. Client does not start up/Benchmark manager claims that client does not exist
7. When running two clients on the same system, only one client does work
8. NoClassDefFoundError in MS Windows

Errors in the Report

1. Sink Receipt Count could not be found in the raw file
2. Quality of Service (QoS) Failures
3. NoClassDefFoundError: java/util/ArrayList

Errors Running the Benchmark

1. Using Java Version 1.1.8

The SPEC Mail Server subcommittee suggests you use Java version 1.2.2 or later. If you must use Java version 1.1.8 to run the load generators and benchmark manager, you should use the command 'jre' instead of 'java'. You will also need to include the classes.zip file in your CLASSPATH. If you see the following error:

Unable to initialize threads: cannot find class java/lang/Thread
Could not create Java VM

the classes.zip file was not entered into your CLASSPATH correctly. If you see the following warning message:

WARNING: specmail2001.jar was not the first file in CLASSPATH

the classes.zip file is probably the first file in your CLASSPATH. The classes.zip file should be the last file listed in your CLASSPATH.

If you are using the batch scripts provided, you will need to modify them to use 'jre' instead of 'java' and include the classes.zip file at the end of the CLASSPATH definition before you attempt to run the scripts.  Note that the SPECmail2001 reporter does not support Java version 1.1.8, it requires Java version 1.2.2.

2. Benchmark Manager Only Prints Version Number

The scripts provided to run the benchmark manager use the '-v' flag which prints out the version number of the code. If you would like to run the benchmark manager using a script, you should modify the provided script to use the appropriate options for your configuration. The first time you run the benchmark, you should use the command:

java specmail -initonly

which will only initialize the mail servers mailboxes.  This initialization takes a significant amount of time, so by default the benchmark assumes the mailboxes have been initialized prior to starting. The benchmark maintains the mailboxes in a steady state during the run, so under normal conditions you will only need to initialize the mailboxes once. You can verify that the mailboxes were initialized correctly using the command:

java specmail -verifyonly

When performing a benchmark run to generate results for publication, you should use the command:

java specmail -compliant

The -compliant flag will cause the benchmark to terminate if an error or warning is encountered during the run. Without the -compliant flag, warnings will be printed to the console but the benchmark will continue to run.

3. Connecting to Remote Clients Failed: InvalidClassException

When launching the benchmark manager and adding hosts, you may see an error containing text similar to:

Error: Name lookup for load generator alt100 exception error
unmarshalling return; nested exception is:
        java.io.InvalidClassException: specmailclient_Stub; Local 
class not compatible: Stream classdesc serialVersionUID=xxx local
class serial VersionUID=yyy
...

This error indicates that you have non-compatible versions of the benchmark code on your clients. All clients (running the benchmark manager, load generators and/or mail sinks) must have the same version of the benchmark software installed and running in order to run the benchmark.

4. Couldn't Read Config File Error

When launching the benchmark manager, you may see an error containing text similar to:

Error: couldn't read config file: exception java.lang.NullPointerException
java.lang.NullPointerException
at org.spec.specmail.Configuration.parseList(Configuration.java, Compiled Code)
...

This error indicates that one of the configuration file parameters (specifically, one of the lists) could not be found. Check your configuration file to make sure that you have not accidentally commented out one of the configuration parameters or misspelled a parameter name.

Note that when updating the benchmark from v1.00 to v1.01, two of the configuration file parameters were changed. If you are using a configuration file from v1.00 in a version of the benchmark v1.01 or greater, you need to change

SINK_SERVER to SINK_SERVERS (with an 'S' on the end)
and
REMOTE_DOMAIN to REMOTE_DOMAINS (with an 'S' on the end)

5. Verification Failed

When running the verification step, you may see the following error in the results:

Failed the verification (total mailbox count == NNN)!

This message may have occurred because you do not have enough mailboxes in your configuration. Our testing indicates that you must have at least 10,000 mailboxes in your configuration to pass the verification step. The benchmark expects the mail boxes to be pre-populated, and the verification step checks to make sure the mail boxes have the correct distribution of messages (both number and size). So it is necessary to have sufficient mailboxes in your configuration to contain the distribution.  If you would like to test with fewer than 10,000 mailboxes, turn verification off using the -noverify parameter.

If you have more than 10,000 mailboxes in your configuration and you received this message, it is possible that your mail store distribution is out of tolerance. The additional text following the 'Failed verification' message should indicate the amount of deviation. In this case we recommend you re-initialize the mailboxes using the '-init' flag.

6. Client does not start up/Benchmark manager claims that client does not exist

There is a known bug the software in which it does not correctly handle client names listed in the configuration file with their full domain name (i.e systemname.domainname.com). To work around this bug, we suggest you define your clients locally on the benchmark manager and list them in the configuration file without the domain names.

7. When running two clients on the same system, only one client does work

There is a known bug in the software that causes this behavior when client names are listed in the configuration file using their full domain name (i.e systemname.domainname.com). To work around this bug, we suggest you define your clients locally on the benchmark manager and list them in the configuration file without the domain names.

8. NoClassDefFoundError in MS Windows

When starting a load generator or the benchmark manager in MS Windows, you may see the following:

Exception in thread "main" java.lang.NoClassDefFoundError: specmailclient

First you should verify that your CLASSPATH is set correctly. You can do this in MS Windows at a command prompt by typing 'set CLASSPATH'. If the CLASSPATH does contain the JAR files correctly, look to see if there are spaces in the path (for example, if you installed it in C:\Documents and Settings or C:\Program Files). We have seen problems in MS Windows where the CLASSPATH definition had spaces in it. To correct the problem try installing the benchmark into another location on the system that does not contain a space in the path.

Errors in the Report

1. Sink Receipt Count could not be found in the raw file

You may see the following error in the report:

Sink Receipt Count could not be found in the raw file.

This means that no messages were received by the mail sinks. Check your SUT to see if the outgoing mail is being queued. If the mail is being queued, there are two possible causes for this:

• The mail sinks are not monitoring the correct port or the mail server is not sending to the correct port on the mail sink. Check
• The mail server does not recognize the remote domain.

If mail is not being queued, then the server is probably sending the mail to a system that is running sendmail or other SMTP processes instead of the mail sink.

2. Quality of Service (QoS) Failures

In the report, you see:

• SMTP Connect Count could not be found in the raw file.
• SMTP Hello Count could not be found in the raw file.
• SMTP Mail From Count could not be found in the raw file.
• SMTP Rcpt To Count could not be found in the raw file.
• SMTP Data Count could not be found in the raw file.
• SMTP Quit Count could not be found in the raw file.
• POP Connect Count could not be found in the raw file.
• POP User ID Count could not be found in the raw file.
• POP Password Count could not be found in the raw file.
• POP Status Count could not be found in the raw file.
• POP Retrieve Count could not be found in the raw file.
• POP Delete Count could not be found in the raw file.
• POP Quit Count could not be found in the raw file.
• Delivery Time Pass Count could not be found in the raw file.
• SMTP Data Count could not be found in the raw file.
• POP Retrieve Count could not be found in the raw file.
• SMTP to Remote Count could not be found in the raw file.

This indicates that the reporter could not find the results of the benchmark run at the 100% load level. When generating the report, the reporter checks the counts above to verify that the SUT passed all the quality of service criteria for the 100% load level.  If you only run initialization or override the LOAD_FACTORS setting and exclude 100%, you will see this type of error.

3. NoClassDefFoundError: java/util/ArrayList

When running the reporter, you may see the following:

java.lang.NoClassDefFoundError: java/util/ArrayList

The SPECmail2001 reporter requires Java version 1.2.2 or later, version 1.1.8 is not supported by the reporter.