SPECweb2009 Release 1.20 E-commerce Workload Design Document

Overview

The E-commerce workload in SPECweb2009 is designed to simulate a Web server that sells computer systems; this includes allowing end users to search, browse, customize, and purchase products.  The workload was developed by analyzing log files of actual E-commerce sites, as well as browsing popular Web stores to gather statistics such as average page size, image sizes and access frequencies (including If-Modified-Since caching from the browser side), and capturing form data a user typically fills out when purchasing products.

Dynamic pages

By observing the requests to typical E-commerce Web stores, it is apparent that three distinct phases are involved in an E-commerce transaction; these are described below.

Phase 1: Browse

The first phase involves a user visiting the home page, possibly selecting a country or region (and thus being directed to the appropriate localized set of pages), selecting the type of customer (home user; small/medium/large business; local/state/federal government; education; healthcare), and being redirected to the appropriate section of products for that customer.  Then the customer must select the type of product they're interested in, and then choosing a specific product model.  Another possibility is using the site's search functionality to find product(s) of interest.

The dynamic pages in this phase must:

• Establish a session cookie to track a user's flow within the system

• Displaying a list of countries/regions on the home page

• Handle search requests, which involves submitting queries to the backend, retrieving the results, and displaying them back to the user

• Display customized lists of product lines, product models, and product details (depending upon the selections the user has made)

The name and purpose of each dynamic page in this phase is below.

• index: Home page; user forms for region selection, searching for products; links for browsing products based upon customer type

• search: Displays search results; user form for new searches

• browse: Displays links for the various product lines based upon the customer type

• browse_productline: Displays a table with information about each product within a product line; each product has a hyperlink to more details

• productdetail: Displays a bulleted list of product details regarding an individual product; contains a hyperlink to customize the product

Phase 2: Customization

The second phase involves configuring a product, which involves user interaction via form submissions.  First, the internal components are selected by the user (i.e. processor speed, amount of memory, hard drive size); the next page consists of further customizations, i.e. choosing the type of warranty, service, and support they want included, and finally any optional accessories (cables, printers, software, etc.).

The dynamic pages in this phase must:

• Display a list of customizations for each stage mentioned above

• Display a price for the item and the customizations the user has selected

This phase contains one page, but it is requested multiple times, as it returns different customizations depending upon which stage the user is in (1, 2, or 3).

• customize: Displays a table with radio buttons that list possible customizations along with prices; contains a form to update the price and selections or continue to the next page (or add to cart)

Phase 3: Purchase

The third and final phase is the actual "E-commerce" phase.  Once the user has configured the product to his/her liking, the user adds the product to the shopping cart.  The cart page allows users to change item quantities, remove items, or save the cart for future retrieval.  When the user clicks "Checkout" from the cart, the session must become secure (this is accomplished via a redirect into HTTPS).  There are multiple pages in this SSL stage: login/registration, entering billing and shipping information, payment details, verifying and submitting the order, and the confirmation page.

The dynamic pages in this phase must handle:

• Cart management (displaying cart contents and line item prices, allowing user modification of quantities)

• Existing user login or new user registration

• Processing shipment and billing details, including validation of the submitted data

• Displaying a confirmation number and shipment date when the order is confirmed

The name and function of each page is described below:

• cart: Displays cart contents; form with input fields for changing cart quantities, and buttons for updating quantities, saving the cart, and checkout

• login: Displays two forms, one for logging in with an existing user account, the other for registering a new user account

• shipping: Displays a form with shipping address and shipping method; performs validation on fields when the form is submitted

• billing: Displays a form with billing details; performs validation on fields when the form is submitted

• confirm: Displays a summary of the cart, shipping and billing information, along with a button to confirm the order; once confirmed, displays a similar page with a ship date and confirmation number

Markov chain

SPECweb2009 is based upon a page-based model; that is, it issues a request to a dynamic page and requests all the images that would normally exist within the page as HTML image tags.  A Markov chain in the harness allows simulation of the relative page request frequencies as seen from the server side.  This is represented in the prime client's SPECweb_Ecommerce.config (see the STATE_n lines).  Below is a diagram of the likelihood of transitioning from one state into another:

Static file set

The static portion of the E-commerce file set is generated by Wafgen.  Each workload has a fixed file set and a file set that scales with the number of simultaneous user sessions requested.

Fixed file set

The fixed file set consists of two types of files: images that an HTML page would reference via <IMG> tags in the HTML (and that a browser would request upon receipt of a page), and "padding".  Padding consists of random text that is inserted at the bottom of a dynamic page to bring the file size up to what was observed with real-world E-commerce Web pages (which have, among other things, JavaScript code and numerous layout tags).  The page image sizes of the fixed file set were determined by analyzing and averaging file sizes observed; the sizes range from very small (less than 100 bytes) to ~22 KB; the former are usually "spacer" images used throughout the site for aligning tables, while the latter tend to be photo-quality JPEG images.  The page images used in the E-commerce workload, along with their size and percentage of being cached by the browser (i.e. receiving an HTTP 304 Not Modified response from the SUT) are listed in the table below.

Scaling file set

Product images are the component of the E-commerce file set that scales as the number of requested simultaneous sessions increases.  Each directory represents a product line an E-commerce store would carry.  The number of directories is determined using the following formula:

directory count = 5 * SIMULTANEOUS_SESSIONS

During a benchmark run, a Zipf distribution is used to access each directory.  A Zipf distribution is a distribution where the probability of selecting the nth item is proportional to 1/n. Zipf distributions are empirically associated with situations where there are many equal-cost alternatives.

Each directory consists of 10 "products", and each product has three images associated with it, which represent different views of a product a customer is interested in.  Within a directory, one of the 10 products is chosen using a random distribution.  Then, three image requests are made for that product, one from each class.  The classes are shown in the table below:

The sizes, frequencies, and directory scaling factor were determined from aggregating server-side Web server logs and observing client-side Web browser caches.

Copyright © 2005-2010 Standard Performance Evaluation Corporation.  All rights reserved.