SPEC Cloud IaaS 2016 Benchmark - User Guide¶
Contents:
- Getting started
- Set up Your Cloud for the Benchmark
- Before Setting up the Benchmark
- Benchmark Harness: CBTOOL and Benchmark Drivers Setup and Preparation
- Introduction to CBTOOL
- Preparing a QCOW2 Benchmark Harness Image for Ubuntu (CBTOOL and Benchmark Drivers)
- Preparing a QCOW2 Benchmark Harness Image for Centos (CBTOOL and Benchmark Drivers)
- Preparing an EC2 Benchmark Harness Image (CBTOOL and Benchmark Drivers)
- Preparing a Google Compute Engine Benchmark Harness Image (CBTOOL and Benchmark Drivers)
- Preparing a Digital Ocean Benchmark Harness Image (CBTOOL and Benchmark Drivers)
- Configuring CBTOOL For Your OpenStack Cloud
- Configuring CBTOOL For Amazon Elastic Compute Cloud
- Configuring CBTOOL For Google Compute Engine
- Configuring CBTOOL For Digital Ocean
- Multiple Network Interfaces on Benchmark Harness Machine
- Timezone and NTP server
- Adding a New Cloud Adapter (native method)
- Adding a New Cloud Adapter (simplified libcloud method)
- How Provisioning Scripts are Executed
- Cloud Configuration Gathering Scripts Through Cloud APIs
- Running Benchmark with a Simulated Cloud
- Preparing Workload Images
- Timezone
- Tips for Preparing Workload Images
- (x86_64/Ubuntu/Trusty/QCOW2) - Common workload image
- (x86_64/Ubuntu/Trusty/QCOW2) - Cassandra and YCSB
- (x86_64/Ubuntu/Trusty/QCOW2) - KMeans and Hadoop
- (x86_64/CentOS/7.1/QCOW2) - Common workload image
- (x86_64/CentOS/7.1/QCOW2) - Cassandra and YCSB
- (x86_64/CentOS/7.1/QCOW2) - KMeans and Hadoop
- (ppc_64/Ubuntu/Trusty/QCOW2) - Common workload image
- (ppc_64/Ubuntu/Trusty/QCOW2) - Cassandra and YCSB
- (ppc_64/Ubuntu/Trusty/QCOW2) - KMeans and Hadoop
- (AMI_x86_64/Ubuntu/Trusty/QCOW2) - Common workload image
- (AMI_x86_64/Ubuntu/Trusty/QCOW2) - Cassandra and YCSB
- (AMI_x86_64/Ubuntu/Trusty/QCOW2) KMeans and Hadoop
- (gcloud_x86_64/Ubuntu/Trusty/QCOW2) - Common workload image
- (gcloud/x86_64/Ubuntu/Trusty) - Cassandra and YCSB
- (gcloud/x86_64/Ubuntu/Trusty) - KMeans and Hadoop
- Upload Images in Your Cloud, Launch VM, and Launch AI
- Setting up Parameters for Measurements
- Running Baseline Phase For the First Time With Your Cloud
- Running Elasticity + Scalability For the First Time With Your Cloud
- Compliant Run For Result Submission
- Submit Results
- FAQ
- 1. CBTOOL fails to start.
- 2. CBTOOL fails to communicate with its own API.
- 3. Attempting to attach a single VM/Container instance through CBTOOL’s CLI - for testing purposes - results in an error message. How can I debug?
- 4. Attempting to attach a single Application Instance (AI) through CBTOOL’s CLI - for testing purposes - results in an error message. How can I debug?
- 5. Baseline Application Instances are deployed successfully, but fail to produce any application metric samples.
- 6. Baseline phase hangs. What is happening?
- 7. How can I check if CBTOOL scripts are failing?
- 8. Supporting evidence collection is not working for baseline.
- 9. When using CBTOOL with the OpenStack Cloud Adapter, can I instruct it to create each new Application Instance on its own tenant/network/subnet/router?
- 10. What is the difference between maximum_ais and reported_ais in the benchmark’s configuration file osgcloud_rules.yaml ?
- 11. What does daemon_parallelism and attach_parallelism in osgcloud_rules.yaml mean?
- 12. Verify rules indicate an ‘ERROR’ in rules. Shall I proceed with the experiment?
- 13. I am running baseline phase for testing and do not want the instances created during baseline phase to be destroyed.
- 14. How do I check the detailed timeline for deployment of a given Application Instance (AI)?
- 15. The provisioning SLA for a given Application Instance was violated (run code = 5). How do I check which component in the creation of application instance took the most time?
- 16. Which OpenStack releases are supported by SPEC Cloud IaaS 2016?
