SPEC(R) MPIM2007 Summary Hewlett Packard Enterprise SGI 8600 (Intel Xeon Gold 6148, 2.40 GHz) Tue Oct 10 18:46:51 2017 MPI2007 License: 1 Test date: Oct-2017 Test sponsor: HPE Hardware availability: Jul-2017 Tested by: HPE Software availability: Nov-2017 Base Base Base Peak Peak Peak Benchmarks Ranks Run Time Ratio Ranks Run Time Ratio -------------- ------ --------- --------- ------ --------- --------- 104.milc 768 7.51 208 S 768 7.51 208 S 104.milc 768 6.91 226 S 768 6.91 226 S 104.milc 768 6.99 224 * 768 6.99 224 * 107.leslie3d 768 21.4 244 S 960 20.1 259 * 107.leslie3d 768 21.1 248 * 960 21.4 244 S 107.leslie3d 768 19.5 267 S 960 19.1 273 S 113.GemsFDTD 768 169 37.2 * 384 160 39.3 S 113.GemsFDTD 768 169 37.2 S 384 161 39.2 * 113.GemsFDTD 768 170 37.2 S 384 161 39.2 S 115.fds4 768 14.6 134 S 704 12.2 160 S 115.fds4 768 13.5 144 S 704 12.7 154 S 115.fds4 768 13.8 142 * 704 12.6 155 * 121.pop2 768 65.0 63.5 S 448 59.6 69.3 S 121.pop2 768 65.2 63.3 S 448 59.8 69.0 S 121.pop2 768 65.0 63.5 * 448 59.7 69.1 * 122.tachyon 768 28.8 97.1 * 1024 25.3 111 * 122.tachyon 768 28.8 97.2 S 1024 25.4 110 S 122.tachyon 768 29.1 96.2 S 1024 25.1 111 S 126.lammps 768 79.6 36.6 S 384 74.5 39.1 * 126.lammps 768 80.7 36.1 S 384 74.2 39.3 S 126.lammps 768 79.6 36.6 * 384 75.3 38.7 S 127.wrf2 768 23.3 335 S 896 22.1 353 S 127.wrf2 768 23.5 331 S 896 22.1 352 * 127.wrf2 768 23.5 331 * 896 22.8 342 S 128.GAPgeofem 768 7.07 292 * 1024 6.47 319 S 128.GAPgeofem 768 7.05 293 S 1024 6.36 325 S 128.GAPgeofem 768 7.14 289 S 1024 6.40 323 * 129.tera_tf 768 19.0 146 S 1024 15.7 176 * 129.tera_tf 768 19.2 144 S 1024 15.6 177 S 129.tera_tf 768 19.1 145 * 1024 16.1 172 S 130.socorro 768 20.3 188 * 640 19.8 193 * 130.socorro 768 20.1 190 S 640 19.8 193 S 130.socorro 768 20.4 187 S 640 19.6 195 S 132.zeusmp2 768 16.9 183 S 512 16.1 193 S 132.zeusmp2 768 16.7 185 S 512 15.4 201 S 132.zeusmp2 768 16.9 184 * 512 15.6 198 * 137.lu 768 15.4 239 S 512 14.3 257 S 137.lu 768 15.2 241 S 512 14.2 260 * 137.lu 768 15.3 240 * 512 14.0 262 S ============================================================================== 104.milc 768 6.99 224 * 768 6.99 224 * 107.leslie3d 768 21.1 248 * 960 20.1 259 * 113.GemsFDTD 768 169 37.2 * 384 161 39.2 * 115.fds4 768 13.8 142 * 704 12.6 155 * 121.pop2 768 65.0 63.5 * 448 59.7 69.1 * 122.tachyon 768 28.8 97.1 * 1024 25.3 111 * 126.lammps 768 79.6 36.6 * 384 74.5 39.1 * 127.wrf2 768 23.5 331 * 896 22.1 352 * 128.GAPgeofem 768 7.07 292 * 1024 6.40 323 * 129.tera_tf 768 19.1 145 * 1024 15.7 176 * 130.socorro 768 20.3 188 * 640 19.8 193 * 132.zeusmp2 768 16.9 184 * 512 15.6 198 * 137.lu 768 15.3 240 * 512 14.2 260 * SPECmpiM_base2007 140 SPECmpiM_peak2007 151 BENCHMARK DETAILS ----------------- Type of System: Homogeneous Total Compute Nodes: 32 Total Chips: 64 Total Cores: 1280 Total Threads: 2560 Total Memory: 6 TB Base Ranks Run: 768 Minimum Peak Ranks: 384 Maximum Peak Ranks: 1024 C Compiler: Intel C Composer XE for Linux, Version 18.0.0.128 Build 20170811 C++ Compiler: Intel C++ Composer XE for Linux, Version 18.0.0.128 Build 20170811 Fortran Compiler: Intel Fortran Composer XE for Linux, Version 18.0.0.128 Build 20170811 Base Pointers: 64-bit Peak Pointers: 64-bit MPI Library: HPE Performance Software - Message Passing Interface 2.17 Other MPI Info: OFED 3.2.2 Pre-processors: None Other Software: None Node Description: HPE XA730i Gen10 Server Node ============================================== HARDWARE -------- Number of nodes: 32 Uses of the node: compute Vendor: Hewlett Packard Enterprise Model: SGI 8600 (Intel Xeon Gold 6148, 2.40 GHz) CPU Name: Intel Xeon Gold 6148 CPU(s) orderable: 1-2 chips Chips enabled: 2 Cores enabled: 40 Cores per chip: 20 Threads per core: 2 CPU Characteristics: Intel Turbo Boost Technology up to 3.70 GHz CPU MHz: 2400 Primary Cache: 32 KB I + 32 KB D on chip per core Secondary Cache: 1 MB I+D on chip per core L3 Cache: 27.5 MB I+D on chip per chip Other Cache: None Memory: 192 GB (12 x 16 GB 2Rx4 PC4-2666V-R) Disk Subsystem: None Other Hardware: None Adapter: Mellanox MT27700 with ConnectX-4 ASIC Number of Adapters: 2 Slot Type: PCIe x16 Gen3 8GT/s Data Rate: InfiniBand 4X EDR Ports Used: 1 Interconnect Type: InfiniBand SOFTWARE -------- Adapter: Mellanox MT27700 with ConnectX-4 ASIC Adapter Driver: OFED-3.4-2.1.8.0 Adapter Firmware: 12.18.1000 Operating System: Red Hat Enterprise Linux Server 7.3 (Maipo), Kernel 3.10.0-514.2.2.el7.x86_64 Local File System: LFS Shared File System: LFS System State: Multi-user, run level 3 Other Software: SGI Management Center Compute Node 3.5.0, Build 716r171.rhel73-1705051353 Node Description: Lustre FS =========================== HARDWARE -------- Number of nodes: 4 Uses of the node: fileserver Vendor: Hewlett Packard Enterprise Model: Rackable C1104-GP2 (Intel Xeon E5-2690 v3, 2.60 GHz) CPU Name: Intel Xeon E5-2690 v3 CPU(s) orderable: 1-2 chips Chips enabled: 2 Cores enabled: 24 Cores per chip: 12 Threads per core: 1 CPU Characteristics: Intel Turbo Boost Technology up to 3.50 GHz Hyper-Threading Technology disabled CPU MHz: 2600 Primary Cache: 32 KB I + 32 KB D on chip per core Secondary Cache: 256 KB I+D on chip per core L3 Cache: 30 MB I+D on chip per chip Other Cache: None Memory: 128 GB (8 x 16 GB 2Rx4 PC4-2133P-R) Disk Subsystem: 684 TB RAID 6 48 x 8+2 2TB 7200 RPM Other Hardware: None Adapter: Mellanox MT27700 with ConnectX-4 ASIC Number of Adapters: 2 Slot Type: PCIe x16 Gen3 Data Rate: InfiniBand 4X EDR Ports Used: 1 Interconnect Type: InfiniBand SOFTWARE -------- Adapter: Mellanox MT27700 with ConnectX-4 ASIC Adapter Driver: OFED-3.3-1.0.0.0 Adapter Firmware: 12.14.2036 Operating System: Red Hat Enterprise Linux Server 7.3 (Maipo), Kernel 3.10.0-514.2.2.el7.x86_64 Local File System: ext3 Shared File System: LFS System State: Multi-user, run level 3 Other Software: None Interconnect Description: InfiniBand (MPI and I/O) ================================================== HARDWARE -------- Vendor: Mellanox Technologies and SGI Model: SGI P0002145 Switch Model: SGI P0002145 Number of Switches: 4 Number of Ports: 36 Data Rate: InfiniBand 4X EDR Firmware: 11.0350.0394 Topology: Enhanced Hypercube Primary Use: MPI and I/O traffic Base Tuning Notes ----------------- src.alt used: 129.tera_tf->add_rank_support src.alt used: 130.socorro->nullify_ptrs Submit Notes ------------ The config file option 'submit' was used. General Notes ------------- Software environment: export MPI_CONNECTIONS_THRESHOLD=0 export MPI_REQUEST_MAX=65536 export MPI_TYPE_MAX=32768 export MPI_IB_RAILS=2 export MPI_IB_IMM_UPGRADE=false export MPI_IB_DCIS=2 export MPI_IB_HYPER_LAZY=false ulimit -s unlimited BIOS settings: AMI BIOS version SAED7177, 07/17/2017 Job Placement: Each MPI job was assigned to a topologically compact set of nodes. Additional notes regarding interconnect: The Infiniband network consists of two independent planes, with half the switches in the system allocated to each plane. I/O traffic is restricted to one plane, while MPI traffic can use both planes. Base Compiler Invocation ------------------------ C benchmarks: icc C++ benchmarks: 126.lammps: icpc Fortran benchmarks: ifort Benchmarks using both Fortran and C: icc ifort Base Portability Flags ---------------------- 121.pop2: -DSPEC_MPI_CASE_FLAG 127.wrf2: -DSPEC_MPI_CASE_FLAG -DSPEC_MPI_LINUX 130.socorro: -assume nostd_intent_in Base Optimization Flags ----------------------- C benchmarks: -O3 -xCORE-AVX512 -no-prec-div -ipo C++ benchmarks: 126.lammps: -O3 -xCORE-AVX512 -no-prec-div -ansi-alias -ipo Fortran benchmarks: -O3 -xCORE-AVX512 -no-prec-div -ipo Benchmarks using both Fortran and C: -O3 -xCORE-AVX512 -no-prec-div -ipo Base Other Flags ---------------- C benchmarks: -lmpi C++ benchmarks: 126.lammps: -lmpi Fortran benchmarks: -lmpi Benchmarks using both Fortran and C: -lmpi Peak Compiler Invocation ------------------------ C benchmarks: icc C++ benchmarks: 126.lammps: icpc Fortran benchmarks: ifort Benchmarks using both Fortran and C (except as noted below): icc ifort 130.socorro: /sw/sdev/intel/parallel_studio_xe_2017_update4/compilers_and_libraries_2017.4.196/linux/bin/intel64/icc /sw/sdev/intel/parallel_studio_xe_2017_update4/compilers_and_libraries_2017.4.196/linux/bin/intel64/ifort Peak Portability Flags ---------------------- Same as Base Portability Flags Peak Optimization Flags ----------------------- C benchmarks: 104.milc: basepeak = yes 122.tachyon: -O3 -xCORE-AVX512 -no-prec-div -ipo C++ benchmarks: 126.lammps: -O3 -xCORE-AVX512 -no-prec-div -ansi-alias -ipo Fortran benchmarks: -O3 -xCORE-AVX512 -no-prec-div -ipo Benchmarks using both Fortran and C: -O3 -xCORE-AVX512 -no-prec-div -ipo Peak Other Flags ---------------- Same as Base Other Flags The flags file that was used to format this result can be browsed at http://www.spec.org/mpi2007/flags/HPE_x86_64_Intel18_flags.html You can also download the XML flags source by saving the following link: http://www.spec.org/mpi2007/flags/HPE_x86_64_Intel18_flags.xml SPEC and SPEC MPI are registered trademarks of the Standard Performance Evaluation Corporation. All other brand and product names appearing in this result are trademarks or registered trademarks of their respective holders. ----------------------------------------------------------------------------- For questions about this result, please contact the tester. For other inquiries, please contact webmaster@spec.org. Copyright 2006-2010 Standard Performance Evaluation Corporation Tested with SPEC MPI2007 v2.0.1. Report generated on Wed Oct 25 17:11:55 2017 by MPI2007 ASCII formatter v1463. Originally published on 25 October 2017.