Performance

Configuration

Common Workload Notes

Notes for Banking Workload

SUT Notes

• 1xSmart Array P600 with 256MB cache
• 2xModular Smart Array 50 Enclosures, 1+1 configuration
• 1 drive RAID 0 for OS Drive, 7 drives RAID 0 for Log, 20 Drives RAID 0 for fileset
• 8 client subnets connected through 3400cl switch, BeSim directly connected to SUT

Operating System Notes

• ulimit -n 1024000, sets number of open files, default 1024
• ulimit -u 1000000, sets max user processes, default 131071
• net.core.optmem_max = 10000000 #default 10240, maximum amount of option memory buffers, default 20480
• net.core.rmem_default = 10000000, default receive socket buffer size, default 135168
• net.core.rmem_max = 10000000, maximum receive socket buffer size, default 131071
• net.core.wmem_default = 10000000, default send socket buffer size, default 135168
• net.core.optmem_max = 10000000, maximum send socket buffer size, default 131071
• net.core.netdev_max_backlog=300000, number of unprocessed input packets before kernel starts dropping them, default 300
• net.ipv4.tcp_max_tw_buckets = 2000000, set TCP time-wait buckets pool size, default 180000
• net.ipv4.tcp_rmem = 30000000 30000000 30000000, maximum TCP read-buffer space allocatable, default 4096 87380 174760
• net.ipv4.tcp_wmem = 30000000 30000000 30000000, maximum TCP write-buffer space allocatable, default 4096 16384 131072
• net.ipv4.tcp_mem=30000000 30000000 30000000, maximum TCP buffer space allocatable, default 392192 392704 393216
• net.ipv4.tcp_timestamps = 0, turns TCP timestamp support off, default on
• net.ipv4.tcp_window_scaling=0, turns TCP window scaling support off, default on
• net.ipv4.conf.all.rp_filter = 1 ,enables source route verification, default 0
• vm.min_free_kbytes = 200000, size of memory kernel will always keep free, default 16382
• net.core.somaxconn=40960, size of listen que for accepting new TCP connections, default 128
• net.core.netdev_max_backlog=300000 ,maximum length of the input queues for the processors, default 300
• fs.file-max = 500000 ,maximum number of open files, default 1677721
• File systems mounted with 'noatime,noacl,notail' to disable access time updates, control lists
• irq_balancer daemon disabled, default enabled
• echo 02 > /proc/irq/74/smp_affinity # bind eth1 irq to processor 2
• echo 04 > /proc/irq/82/smp_affinity # bind eth2 irq to processor 3
• echo 08 > /proc/irq/90/smp_affinity # bind eth7 irq to processor 4
• echo 02 > /proc/irq/98/smp_affinity # bind eth8 irq to processor 2
• echo 01 > /proc/irq/66/smp_affinity # bind eth0 irq to processor 1
• echo 08 > /proc/irq/209/smp_affinity # bind eth3,5 irq to processor 4
• echo 02 > /proc/irq/217/smp_affinity # bind eth4,6 irq to processor 2
• echo 01 > /proc/irq/0/smp_affinity # bind timer irq to processor 1
• echo 02 > /proc/irq/169/smp_affinity # bind cciss0 irq to processor 2
• echo 01 > /proc/irq/201/smp_affinity # bind cciss1 irq to processor 1
• TSO enabled on client-data interfaces, default enabled
• e1000 v6.3.9-NAPI installed, options e1000 RxDescriptors=2048,2048,2048,2048,2048,2048,2048,2048 TxDescriptors=2048,2048,2048,2048,2048,2048,2048,2048 InterruptThrottleRate=4000,4000,4000,4000,4000,4000,4000,4000 FlowControl=0,0,0,0,0,0,0,0

Web Server Software Notes

• Rock Web Server conf/bank.conf
• server_log /var/log/httpd/httpd.log
• validate_static 604800 #static file modification validation, default 0
• validate_httpmod -1 #disables HTTP module modification validation, default 0
• connection_timeout 120 #connection inactivity timeout, default 60
• keepalive_max 100000 #max keepalive requests on a connection, default 100
• cache_memory_size 16384 #cache manager memory, default half system memory
• cache_replacement random #random cache replacement
• header_etag_on 0 #disables etag header in HTTP replies, default 1
• header_server_on 0 #disables server ehader in HTTP replies, default 1
• tcp_send_buf_size 1048576 #sets socket send buffer size
• log_buf_size 1048576 #sets access log buffer size, default 65536
• port_getn 256 #IO evente retrieve from the OS at one time, default 1
• direct_io 1 #uses direct IO when possible
• servlet_session_count 37917 #estimate number of servlet sessions, default 23191
• servlet_session_timeout 900 #servlet session timeout, default 600
• worker 131.100.100.160/443
• worker 131.101.100.160/443
• worker 131.102.100.160/443
• worker 131.103.100.160/443
• worker 131.104.100.160/443
• worker 131.105.100.160/443
• worker 131.106.100.160/443
• worker 131.107.100.160/443
• host server:443
• document_root /web
• index_directory yes
• ssl_key_file /usr/httpd/etc/key-cert/ssl.key #1024bit length key
• ssl_cert_file /usr/httpd/etc/key-cert/ssl.cer #created via Rock Admin Server
• access_log /var/log/httpd/access.log
• error_log /var/log/httpd/error.log
• access_log_format commonlog_binary
• default_mime_type text/html
• ssl_session_max 447931
• ssl_session_timeout 600
• file /bank/bankjsp
• cgi_type gateway
• cgi_listener 127.0.0.1/8080
• /file
• /host

Script Engine Notes

• JAVA_OPT=-Xms4g -Xmx4g -Xcompactexplicitgc -Xss512k -d64 -server
• 1 Tomcat instance used to deploy JSP scripts on port 8080
• Tomcat server.xml parameters
• maxThreads=15000, max number of sessions, default 150
• minSpareThreads=2048, number of threads created when connector starts, default 25
• maxSpareThreads=2048, max number of unused threads allowed, default 75
• acceptCount=10000, max queue length for connection requests, default 100
• connectionTimeout=0, no connection timeout, default 20000
• maxKeepAliveRequests=-1, default unspecified
• path=/bank/bankjp, docbase=/web/bank/app, debug=0, default unspecified
• Tomcat web.xml parameters
• classdebuginfo = false, class file compiled with debug info, default true
• development = false, development mode, default true
• mappedfile = false, debugging info, default true
• keepgenerated = false, keep generated class files between sessions, default true

HTTP Script Notes

• SPEC-provided JSP scripts used without modification

Client Notes

• TCP/IP registry tuning parameters [HKLMSystemCurrentControlSetServicesTcpipParameters]:
• TcpTimedWaitDelay=0x3c, sets TIME_WAIT parameter to 60 seconds (non-RFC 1122), default is 240
• MaxFreeTwTcbs=400000, sets maximum number of TIME_WAIT tcbs to 400,000
• TcpWindowSize=65535, sets TCP send/receive window size, default 8192

BESIM Notes

• Web Server Configuration
• user nobody #configures the user under which httpd runs
• server_log /var/log/httpd/httpd.log #configures the location of the httpd server log file
• validate_static -1 #defines the modification checking interval for static pages. If -1, no checking is done
• validate_httpmod -1 #disables HTTP module modification validation, default 0
• header_etag_on 0 #disables etag header in HTTP replies, default 1
• header_server_on 0 #disables server header in HTTP replies, Default 1
• accept_max 1 #maximum number of client connections httpd can accept at a time, default 50
• log_buf_size 1048576 #sets access log buffer size, default 65536
• tcp_send_buf_size 1048576 #sets socket send buffer size
• keepalive_max 1000000 #max keepalive requests on a connection, default 100
• connection_timeout 86400 #The number of connection reclaims because they time out.
• worker */81
• worker */81
• worker */81
• worker */81
• host besim:81
• document_root /web
• access_log /var/log/httpd/access_besim.log
• error_log /var/log/httpd/error_besim.log
• access_log_format commonlog
• file /
• index_directory yes
• /file
• file /isapi-bin
• cgi_type isapi
• cgi_where internal
• index_directory yes
• /file
• /host
• host besim:9090
• admin_server yes
• file /
• home_page home.cgi
• cgi_type module
• cgi_where internal
• cgi_regex .*cgi
• /file
• /host
• BESIM OS Configuration
• ethtool -K eth0 tso off # TSO has to be off
• ethtool -K eth1 tso off # TSO has to be off
• ifconfig eth0 txqueuelen 40000
• ifconfig eth1 txqueuelen 40000
• ulimit -n 1000000 #sets number of open files, default 1024
• ulimit -u 1000000 #sets number of user processes, default 1024
• echo 2000000 > /proc/sys/net/ipv4/tcp_max_tw_buckets #sets TCP time-wait buckets pool size, default 180000
• echo 30000000 30000000 30000000 > /proc/sys/net/ipv4/tcp_rmem #maximum TCP write-buffer space allocatable, default 4096 16384 131072
• echo 30000000 30000000 30000000 > /proc/sys/net/ipv4/tcp_wmem #maximum TCP write-buffer space allocatable, default 4096 16384 131072
• echo 30000000 30000000 30000000 > /proc/sys/net/ipv4/tcp_mem #maximum TCP buffer space allocatable, default 31744 32256 32768
• echo 10000000 > /proc/sys/net/core/rmem_max #maximum receive socket buffer size, default 65535
• echo 10000000 > /proc/sys/net/core/rmem_default #default receive socket buffer size, default 65535
• echo 10000000 > /proc/sys/net/core/wmem_max #maximum send socket buffer size, default 65535
• echo 10000000 > /proc/sys/net/core/wmem_default #default send socket buffer size, default 65535
• echo 10000000 > /proc/sys/net/core/optmem_max #default 10240
• echo 500000 > /proc/sys/fs/file-max #maximum number of open files, default 1024

Other Notes

• Besim ISAPI compile instructions see HP-20060824.txt

Errors for Banking Workload

Quality of Service Errors

•   No QOS Errors Found

Validation Errors

• 16 Validation Errors for Iteration 1  :   130.168.85.1 - post_chk_order   

• 16 Validation Errors for Iteration 1  :   130.168.85.2 - post_chk_order   

• 8 Validation Errors for Iteration 1  :   130.168.85.3 - post_chk_order   

• 9 Validation Errors for Iteration 1  :   130.168.85.4 - post_chk_order   

• 19 Validation Errors for Iteration 1  :   130.168.85.5 - post_chk_order   

• 15 Validation Errors for Iteration 1  :   130.168.85.6 - post_chk_order   

• 13 Validation Errors for Iteration 1  :   130.168.85.7 - post_chk_order   

• 12 Validation Errors for Iteration 1  :   130.168.85.8 - post_chk_order   

• 21 Validation Errors for Iteration 1  :   130.168.85.9 - post_chk_order   

• 10 Validation Errors for Iteration 1  :   130.168.85.10 - post_chk_order   

• 12 Validation Errors for Iteration 1  :   130.168.85.12 - post_chk_order   

• 9 Validation Errors for Iteration 1  :   130.168.85.13 - post_chk_order   

• 15 Validation Errors for Iteration 1  :   130.168.85.14 - post_chk_order   

• 10 Validation Errors for Iteration 1  :   130.168.85.15 - post_chk_order   

• 11 Validation Errors for Iteration 1  :   130.168.85.16 - post_chk_order   

• 13 Validation Errors for Iteration 1  :   130.168.85.17 - post_chk_order   

• 4 Validation Errors for Iteration 1  :   130.168.85.18 - post_chk_order   

• 15 Validation Errors for Iteration 1  :   130.168.85.19 - post_chk_order   

• 9 Validation Errors for Iteration 1  :   130.168.85.20 - post_chk_order   

• 9 Validation Errors for Iteration 1  :   130.168.85.21 - post_chk_order   

• 16 Validation Errors for Iteration 1  :   130.168.85.22 - post_chk_order   

• 18 Validation Errors for Iteration 1  :   130.168.85.23 - post_chk_order   

• 19 Validation Errors for Iteration 1  :   130.168.85.24 - post_chk_order   

• 12 Validation Errors for Iteration 1  :   130.168.85.25 - post_chk_order   

• 12 Validation Errors for Iteration 2  :   130.168.85.1 - post_chk_order   

• 12 Validation Errors for Iteration 2  :   130.168.85.2 - post_chk_order   

• 17 Validation Errors for Iteration 2  :   130.168.85.3 - post_chk_order   

• 11 Validation Errors for Iteration 2  :   130.168.85.4 - post_chk_order   

• 12 Validation Errors for Iteration 2  :   130.168.85.5 - post_chk_order   

• 9 Validation Errors for Iteration 2  :   130.168.85.6 - post_chk_order   

• 14 Validation Errors for Iteration 2  :   130.168.85.7 - post_chk_order   

• 11 Validation Errors for Iteration 2  :   130.168.85.8 - post_chk_order   

• 13 Validation Errors for Iteration 2  :   130.168.85.9 - post_chk_order   

• 9 Validation Errors for Iteration 2  :   130.168.85.10 - post_chk_order   

• 17 Validation Errors for Iteration 2  :   130.168.85.12 - post_chk_order   

• 10 Validation Errors for Iteration 2  :   130.168.85.13 - post_chk_order   

• 20 Validation Errors for Iteration 2  :   130.168.85.14 - post_chk_order   

• 16 Validation Errors for Iteration 2  :   130.168.85.15 - post_chk_order   

• 13 Validation Errors for Iteration 2  :   130.168.85.16 - post_chk_order   

• 17 Validation Errors for Iteration 2  :   130.168.85.17 - post_chk_order   

• 13 Validation Errors for Iteration 2  :   130.168.85.18 - post_chk_order   

• 9 Validation Errors for Iteration 2  :   130.168.85.19 - post_chk_order   

• 14 Validation Errors for Iteration 2  :   130.168.85.20 - post_chk_order   

• 12 Validation Errors for Iteration 2  :   130.168.85.21 - post_chk_order   

• 16 Validation Errors for Iteration 2  :   130.168.85.22 - post_chk_order   

• 20 Validation Errors for Iteration 2  :   130.168.85.23 - post_chk_order   

• 11 Validation Errors for Iteration 2  :   130.168.85.24 - post_chk_order   

• 9 Validation Errors for Iteration 2  :   130.168.85.25 - post_chk_order   

• 16 Validation Errors for Iteration 3  :   130.168.85.1 - post_chk_order   

• 12 Validation Errors for Iteration 3  :   130.168.85.2 - post_chk_order   

• 17 Validation Errors for Iteration 3  :   130.168.85.3 - post_chk_order   

• 8 Validation Errors for Iteration 3  :   130.168.85.4 - post_chk_order   

• 16 Validation Errors for Iteration 3  :   130.168.85.5 - post_chk_order   

• 3 Validation Errors for Iteration 3  :   130.168.85.6 - post_chk_order   

• 12 Validation Errors for Iteration 3  :   130.168.85.7 - post_chk_order   

• 11 Validation Errors for Iteration 3  :   130.168.85.8 - post_chk_order   

• 11 Validation Errors for Iteration 3  :   130.168.85.9 - post_chk_order   

• 18 Validation Errors for Iteration 3  :   130.168.85.10 - post_chk_order   

• 16 Validation Errors for Iteration 3  :   130.168.85.12 - post_chk_order   

• 12 Validation Errors for Iteration 3  :   130.168.85.13 - post_chk_order   

• 14 Validation Errors for Iteration 3  :   130.168.85.14 - post_chk_order   

• 14 Validation Errors for Iteration 3  :   130.168.85.15 - post_chk_order   

• 17 Validation Errors for Iteration 3  :   130.168.85.16 - post_chk_order   

• 21 Validation Errors for Iteration 3  :   130.168.85.17 - post_chk_order   

• 13 Validation Errors for Iteration 3  :   130.168.85.18 - post_chk_order   

• 17 Validation Errors for Iteration 3  :   130.168.85.19 - post_chk_order   

• 12 Validation Errors for Iteration 3  :   130.168.85.20 - post_chk_order   

• 18 Validation Errors for Iteration 3  :   130.168.85.21 - post_chk_order   

• 13 Validation Errors for Iteration 3  :   130.168.85.22 - post_chk_order   

• 19 Validation Errors for Iteration 3  :   130.168.85.23 - post_chk_order   

• 21 Validation Errors for Iteration 3  :   130.168.85.24 - post_chk_order   

• 12 Validation Errors for Iteration 3  :   130.168.85.25 - post_chk_order   

Notes for Ecommerce Workload

SUT Notes

• 1xSmart Array P600 with 256MB cache
• 2xModular Smart Array 50 Enclosures, 1+1 configuration
• 1 drive RAID 0 for OS Drive, 7 drives RAID 0 for Log, 20 Drives RAID 0 for fileset
• 8 client subnets connected through 3400cl switch, BeSim directly connected to SUT

Operating System Notes

• ulimit -n 1024000, sets number of open files, default 1024
• ulimit -u 1000000, sets max user processes, default 131071
• net.core.optmem_max = 10000000 #default 10240, maximum amount of option memory buffers, default 20480
• net.core.rmem_default = 10000000, default receive socket buffer size, default 135168
• net.core.rmem_max = 10000000, maximum receive socket buffer size, default 131071
• net.core.wmem_default = 10000000, default send socket buffer size, default 135168
• net.core.optmem_max = 10000000, maximum send socket buffer size, default 131071
• net.core.netdev_max_backlog=300000, number of unprocessed input packets before kernel starts dropping them, default 300
• net.ipv4.tcp_max_tw_buckets = 2000000, set TCP time-wait buckets pool size, default 180000
• net.ipv4.tcp_rmem = 30000000 30000000 30000000, maximum TCP read-buffer space allocatable, default 4096 87380 174760
• net.ipv4.tcp_wmem = 30000000 30000000 30000000, maximum TCP write-buffer space allocatable, default 4096 16384 131072
• net.ipv4.tcp_mem=30000000 30000000 30000000, maximum TCP buffer space allocatable, default 392192 392704 393216
• net.ipv4.tcp_timestamps = 0, turns TCP timestamp support off, default on
• net.ipv4.tcp_window_scaling=0, turns TCP window scaling support off, default on
• net.ipv4.conf.all.rp_filter = 1 ,enables source route verification, default 0
• vm.min_free_kbytes = 200000, size of memory kernel will always keep free, default 16382
• net.core.somaxconn=40960, size of listen que for accepting new TCP connections, default 128
• net.core.netdev_max_backlog=300000 ,maximum length of the input queues for the processors, default 300
• fs.file-max = 500000 ,maximum number of open files, default 1677721
• File systems mounted with 'noatime,noacl,notail' to disable access time updates, control lists
• irq_balancer daemon disabled, default enabled
• echo 02 > /proc/irq/74/smp_affinity # bind eth1 irq to processor 2
• echo 04 > /proc/irq/82/smp_affinity # bind eth2 irq to processor 3
• echo 08 > /proc/irq/90/smp_affinity # bind eth7 irq to processor 4
• echo 02 > /proc/irq/98/smp_affinity # bind eth8 irq to processor 2
• echo 01 > /proc/irq/66/smp_affinity # bind eth0 irq to processor 1
• echo 08 > /proc/irq/209/smp_affinity # bind eth3,5 irq to processor 4
• echo 02 > /proc/irq/217/smp_affinity # bind eth4,6 irq to processor 2
• echo 01 > /proc/irq/0/smp_affinity # bind timer irq to processor 1
• echo 02 > /proc/irq/169/smp_affinity # bind cciss0 irq to processor 2
• echo 01 > /proc/irq/201/smp_affinity # bind cciss1 irq to processor 1
• TSO enabled on client-data interfaces, default enabled
• e1000 v6.3.9-NAPI installed, options e1000 RxDescriptors=2048,2048,2048,2048,2048,2048,2048,2048 TxDescriptors=2048,2048,2048,2048,2048,2048,2048,2048 InterruptThrottleRate=4000,4000,4000,4000,4000,4000,4000,4000 FlowControl=0,0,0,0,0,0,0,0

Web Server Software Notes

• Rock Web Server conf/ecommerce.conf
• server_log /var/log/httpd/httpd.log
• validate_static 604800 #static file modification validation, default 0
• validate_httpmod -1 #disables HTTP module modification validation, default 0
• connection_timeout 9200 #connection inactivity timeout, default 60
• keepalive_max 100000 #max keepalive requests on a connection, default 100
• cache_memory_size 24576 #cache manager memory, default half system memory
• cache_replacement lru #lease recent used cache replacement policy, default random
• header_etag_on 0 #disables etag header in HTTP replies, default 1
• header_server_on 0 #disables server ehader in HTTP replies, default 1
• tcp_send_buf_size 1048576 #sets socket send buffer size
• log_buf_size 1048576 #sets access log buffer size, default 65536
• port_getn 256 #IO evente retrieve from the OS at one time, default 1
• direct_io 1 #uses direct IO when possible
• servlet_session_count 57917 #estimate number of servlet sessions, default 23191
• servlet_session_timeout 800 #servlet session timeout, default 600
• fast_read 1 # enable better read when IO wait time is long
• worker 131.100.100.160/443 131.100.100.160/80
• worker 131.101.100.160/443 131.101.100.160/80
• worker 131.102.100.160/443 131.102.100.160/80
• worker 131.103.100.160/443 131.103.100.160/80
• worker 131.104.100.160/443 131.104.100.160/80
• worker 131.105.100.160/443 131.105.100.160/80
• worker 131.106.100.160/443 131.106.100.160/80
• worker 131.107.100.160/443 131.107.100.160/80
• host server:443
• document_root /web
• index_directory yes
• ssl_key_file /usr/httpd/etc/key-cert/ssl.key #1024bit length key
• ssl_cert_file /usr/httpd/etc/key-cert/ssl.cer #created via Rock Admin Server
• access_log /var/log/httpd/access443.log
• error_log /var/log/httpd/error443.log
• access_log_format commonlog_binary
• default_mime_type text/html
• ssl_session_max 447931
• ssl_session_timeout 600
• file /ecommerce/ecomjsp
• cgi_type gateway
• cgi_listener 127.0.0.1/8080
• /file
• /host
• host server:80
• index_directory yes
• default_mime_type text/html
• document_root /web
• access_log /var/log/httpd/access80.log
• access_log_format commonlog_binary
• error_log /var/log/httpd/error80.log
• ssl_session_max 37931
• ssl_session_timeout 0
• file /ecommerce/ecomjsp
• cgi_type gateway
• cgi_listener 127.0.0.1/8080
• /file
• /host

Script Engine Notes

• JAVA_OPT=-Xms4g -Xmx4g -Xcompactexplicitgc -Xss512k -d64 -server
• 1 Tomcat instance used to deploy JSP scripts on port 8080
• Tomcat server.xml parameters
• maxThreads=15000, max number of sessions, default 150
• minSpareThreads=2048, number of threads created when connector starts, default 25
• maxSpareThreads=2048, max number of unused threads allowed, default 75
• acceptCount=10000, max queue length for connection requests, default 100
• connectionTimeout=0, no connection timeout, default 20000
• maxKeepAliveRequests=-1, default unspecified
• path=/bank/bankjp, docbase=/web/bank/app, debug=0, default unspecified
• Tomcat web.xml parameters
• classdebuginfo = false, class file compiled with debug info, default true
• development = false, development mode, default true
• mappedfile = false, debugging info, default true
• keepgenerated = false, keep generated class files between sessions, default true
• LG_BUF_SIZE=102400; default 22528 (ecommerce)

HTTP Script Notes

• SPEC-provided JSP scripts used without modification

Client Notes

• TCP/IP registry tuning parameters [HKLMSystemCurrentControlSetServicesTcpipParameters]:
• TcpTimedWaitDelay=0x3c, sets TIME_WAIT parameter to 60 seconds (non-RFC 1122), default is 240
• MaxFreeTwTcbs=400000, sets maximum number of TIME_WAIT tcbs to 400,000
• TcpWindowSize=65535, sets TCP send/receive window size, default 8192

BESIM Notes

• Web Server Configuration
• user nobody #configures the user under which httpd runs
• server_log /var/log/httpd/httpd.log #configures the location of the httpd server log file
• validate_static -1 #defines the modification checking interval for static pages. If -1, no checking is done
• validate_httpmod -1 #disables HTTP module modification validation, default 0
• header_etag_on 0 #disables etag header in HTTP replies, default 1
• header_server_on 0 #disables server header in HTTP replies, Default 1
• accept_max 1 #maximum number of client connections httpd can accept at a time, default 50
• log_buf_size 1048576 #sets access log buffer size, default 65536
• tcp_send_buf_size 1048576 #sets socket send buffer size
• keepalive_max 1000000 #max keepalive requests on a connection, default 100
• connection_timeout 86400 #The number of connection reclaims because they time out.
• worker */81
• worker */81
• worker */81
• worker */81
• host besim:81
• document_root /web
• access_log /var/log/httpd/access_besim.log
• error_log /var/log/httpd/error_besim.log
• access_log_format commonlog
• file /
• index_directory yes
• /file
• file /isapi-bin
• cgi_type isapi
• cgi_where internal
• index_directory yes
• /file
• /host
• host besim:9090
• admin_server yes
• file /
• home_page home.cgi
• cgi_type module
• cgi_where internal
• cgi_regex .*cgi
• /file
• /host
• BESIM OS Configuration
• ethtool -K eth0 tso off # TSO has to be off
• ethtool -K eth1 tso off # TSO has to be off
• ifconfig eth0 txqueuelen 40000
• ifconfig eth1 txqueuelen 40000
• ulimit -n 1000000 #sets number of open files, default 1024
• ulimit -u 1000000 #sets number of user processes, default 1024
• echo 2000000 > /proc/sys/net/ipv4/tcp_max_tw_buckets #sets TCP time-wait buckets pool size, default 180000
• echo 30000000 30000000 30000000 > /proc/sys/net/ipv4/tcp_rmem #maximum TCP write-buffer space allocatable, default 4096 16384 131072
• echo 30000000 30000000 30000000 > /proc/sys/net/ipv4/tcp_wmem #maximum TCP write-buffer space allocatable, default 4096 16384 131072
• echo 30000000 30000000 30000000 > /proc/sys/net/ipv4/tcp_mem #maximum TCP buffer space allocatable, default 31744 32256 32768
• echo 10000000 > /proc/sys/net/core/rmem_max #maximum receive socket buffer size, default 65535
• echo 10000000 > /proc/sys/net/core/rmem_default #default receive socket buffer size, default 65535
• echo 10000000 > /proc/sys/net/core/wmem_max #maximum send socket buffer size, default 65535
• echo 10000000 > /proc/sys/net/core/wmem_default #default send socket buffer size, default 65535
• echo 10000000 > /proc/sys/net/core/optmem_max #default 10240
• echo 500000 > /proc/sys/fs/file-max #maximum number of open files, default 1024

Other Notes

• Besim ISAPI compile instructions see HP-20060824.txt

Errors for Ecommerce Workload

Quality of Service Errors

•   No QOS Errors Found

Validation Errors

•   No Validation Errors Found

Notes for Support Workload

SUT Notes

• 1xSmart Array P600 with 256MB cache
• 2xModular Smart Array 50 Enclosures, 1+1 configuration
• 1 drive RAID 0 for OS Drive, 7 drives RAID 0 for Log, 20 Drives RAID 0 for fileset
• Support BESIM_POOL_SIZE=512
• 8 client subnets connected through 3400cl switch, BeSim directly connected to SUT

Operating System Notes

• ulimit -n 1024000, sets number of open files, default 1024
• ulimit -u 1000000, sets max user processes, default 131071
• net.core.optmem_max = 10000000 #default 10240, maximum amount of option memory buffers, default 20480
• net.core.rmem_default = 10000000, default receive socket buffer size, default 135168
• net.core.rmem_max = 10000000, maximum receive socket buffer size, default 131071
• net.core.wmem_default = 10000000, default send socket buffer size, default 135168
• net.core.optmem_max = 10000000, maximum send socket buffer size, default 131071
• net.core.netdev_max_backlog=300000, number of unprocessed input packets before kernel starts dropping them, default 300
• net.ipv4.tcp_max_tw_buckets = 2000000, set TCP time-wait buckets pool size, default 180000
• net.ipv4.tcp_rmem = 30000000 30000000 30000000, maximum TCP read-buffer space allocatable, default 4096 87380 174760
• net.ipv4.tcp_wmem = 30000000 30000000 30000000, maximum TCP write-buffer space allocatable, default 4096 16384 131072
• net.ipv4.tcp_mem=30000000 30000000 30000000, maximum TCP buffer space allocatable, default 392192 392704 393216
• net.ipv4.tcp_timestamps = 0, turns TCP timestamp support off, default on
• net.ipv4.tcp_window_scaling=0, turns TCP window scaling support off, default on
• net.ipv4.conf.all.rp_filter = 1 ,enables source route verification, default 0
• vm.min_free_kbytes = 200000, size of memory kernel will always keep free, default 16382
• net.core.somaxconn=40960, size of listen que for accepting new TCP connections, default 128
• net.core.netdev_max_backlog=300000 ,maximum length of the input queues for the processors, default 300
• fs.file-max = 500000 ,maximum number of open files, default 1677721
• File systems mounted with 'noatime,noacl,notail' to disable access time updates, control lists
• irq_balancer daemon disabled, default enabled
• echo 02 > /proc/irq/74/smp_affinity # bind eth1 irq to processor 2
• echo 04 > /proc/irq/82/smp_affinity # bind eth2 irq to processor 3
• echo 08 > /proc/irq/90/smp_affinity # bind eth7 irq to processor 4
• echo 02 > /proc/irq/98/smp_affinity # bind eth8 irq to processor 2
• echo 01 > /proc/irq/66/smp_affinity # bind eth0 irq to processor 1
• echo 08 > /proc/irq/209/smp_affinity # bind eth3,5 irq to processor 4
• echo 02 > /proc/irq/217/smp_affinity # bind eth4,6 irq to processor 2
• echo 01 > /proc/irq/0/smp_affinity # bind timer irq to processor 1
• echo 02 > /proc/irq/169/smp_affinity # bind cciss0 irq to processor 2
• echo 01 > /proc/irq/201/smp_affinity # bind cciss1 irq to processor 1
• TSO enabled on client-data interfaces, default enabled
• e1000 v6.3.9-NAPI installed, options e1000 RxDescriptors=2048,2048,2048,2048,2048,2048,2048,2048 TxDescriptors=2048,2048,2048,2048,2048,2048,2048,2048 InterruptThrottleRate=4000,4000,4000,4000,4000,4000,4000,4000 FlowControl=0,0,0,0,0,0,0,0

Web Server Software Notes

• Rock Web Server conf/support.conf
• server_log /var/log/httpd/httpd.log
• validate_static 604800 #static file modification validation, default 0
• validate_httpmod -1 #disables HTTP module modification validation, default 0
• connection_timeout 9200 #connection inactivity timeout, default 60
• keepalive_max 100000 #max keepalive requests on a connection, default 100
• cache_memory_size 24576 #cache manager memory, default half system memory
• cache_replacement lru #random cache replacement
• header_etag_on 0 #disables etag header in HTTP replies, default 1
• header_server_on 0 #disables server ehader in HTTP replies, default 1
• tcp_send_buf_size 1048576 #sets socket send buffer size
• log_buf_size 1048576 #sets access log buffer size, default 65536
• port_getn 256 #IO evente retrieve from the OS at one time, default 1
• direct_io 1 #uses direct IO when possible
• direct_io_size 1048576 #read buffer size for direckt IO, default 65535
• servlet_session_count 57917 #estimate number of servlet sessions, default 23191
• servlet_session_timeout 800 #servlet session timeout, default 600
• cache_file_max_size 4194303 #largest file allowed in cache manager, default 2097152
• disk_worker_count 68 #number of threads to read data from disk for large files, default 21
• worker 131.100.100.160/80
• worker 131.101.100.160/80
• worker 131.102.100.160/80
• worker 131.103.100.160/80
• worker 131.104.100.160/80
• worker 131.105.100.160/80
• worker 131.106.100.160/80
• worker 131.107.100.160/80
• host server:80
• document_root /web
• index_directory yes
• access_log /var/log/httpd/access80.log
• error_log /var/log/httpd/error80.log
• access_log_format commonlog_binary
• default_mime_type text/html
• file /support/suppjsp
• cgi_type gateway
• cgi_listener 127.0.0.1/8080
• /file
• /host

Script Engine Notes

• JAVA_OPT=-Xms4g -Xmx4g -Xcompactexplicitgc -Xss512k -d64 -server
• 1 Tomcat instance used to deploy JSP scripts on port 8080
• Tomcat server.xml parameters
• maxThreads=15000, max number of sessions, default 150
• minSpareThreads=2048, number of threads created when connector starts, default 25
• maxSpareThreads=2048, max number of unused threads allowed, default 75
• acceptCount=10000, max queue length for connection requests, default 100
• connectionTimeout=0, no connection timeout, default 20000
• maxKeepAliveRequests=-1, default unspecified
• path=/bank/bankjp, docbase=/web/bank/app, debug=0, default unspecified
• Tomcat web.xml parameters
• classdebuginfo = false, class file compiled with debug info, default true
• development = false, development mode, default true
• mappedfile = false, debugging info, default true
• keepgenerated = false, keep generated class files between sessions, default true

HTTP Script Notes

• SPEC-provided JSP scripts used without modification

Client Notes

• TCP/IP registry tuning parameters [HKLMSystemCurrentControlSetServicesTcpipParameters]:
• TcpTimedWaitDelay=0x3c, sets TIME_WAIT parameter to 60 seconds (non-RFC 1122), default is 240
• MaxFreeTwTcbs=400000, sets maximum number of TIME_WAIT tcbs to 400,000
• TcpWindowSize=65535, sets TCP send/receive window size, default 8192

BESIM Notes

• Web Server Configuration
• user nobody #configures the user under which httpd runs
• server_log /var/log/httpd/httpd.log #configures the location of the httpd server log file
• validate_static -1 #defines the modification checking interval for static pages. If -1, no checking is done
• validate_httpmod -1 #disables HTTP module modification validation, default 0
• header_etag_on 0 #disables etag header in HTTP replies, default 1
• header_server_on 0 #disables server header in HTTP replies, Default 1
• accept_max 1 #maximum number of client connections httpd can accept at a time, default 50
• log_buf_size 1048576 #sets access log buffer size, default 65536
• tcp_send_buf_size 1048576 #sets socket send buffer size
• keepalive_max 1000000 #max keepalive requests on a connection, default 100
• connection_timeout 86400 #The number of connection reclaims because they time out.
• worker */81
• worker */81
• worker */81
• worker */81
• host besim:81
• document_root /web
• access_log /var/log/httpd/access_besim.log
• error_log /var/log/httpd/error_besim.log
• access_log_format commonlog
• file /
• index_directory yes
• /file
• file /isapi-bin
• cgi_type isapi
• cgi_where internal
• index_directory yes
• /file
• /host
• host besim:9090
• admin_server yes
• file /
• home_page home.cgi
• cgi_type module
• cgi_where internal
• cgi_regex .*cgi
• /file
• /host
• BESIM OS Configuration
• ethtool -K eth0 tso off # TSO has to be off
• ethtool -K eth1 tso off # TSO has to be off
• ifconfig eth0 txqueuelen 40000
• ifconfig eth1 txqueuelen 40000
• ulimit -n 1000000 #sets number of open files, default 1024
• ulimit -u 1000000 #sets number of user processes, default
• echo 2000000 > /proc/sys/net/ipv4/tcp_max_tw_buckets #sets TCP time-wait buckets pool size, default 180000
• echo 30000000 30000000 30000000 > /proc/sys/net/ipv4/tcp_rmem #maximum TCP write-buffer space allocatable, default 4096 16384 131072
• echo 30000000 30000000 30000000 > /proc/sys/net/ipv4/tcp_wmem #maximum TCP write-buffer space allocatable, default 4096 16384 131072
• echo 30000000 30000000 30000000 > /proc/sys/net/ipv4/tcp_mem #maximum TCP buffer space allocatable, default 31744 32256 32768
• echo 10000000 > /proc/sys/net/core/rmem_max #maximum receive socket buffer size, default 65535
• echo 10000000 > /proc/sys/net/core/rmem_default #default receive socket buffer size, default 65535
• echo 10000000 > /proc/sys/net/core/wmem_max #maximum send socket buffer size, default 65535
• echo 10000000 > /proc/sys/net/core/wmem_default #default send socket buffer size, default 65535
• echo 10000000 > /proc/sys/net/core/optmem_max #default 10240
• echo 500000 > /proc/sys/fs/file-max #maximum number of open files, default 1024

Other Notes

• Besim ISAPI compile instructions see HP-20060824.txt

Errors for Support Workload

Quality of Service Errors

•   No QOS Errors Found

Validation Errors

•   No Validation Errors Found