SPEC CPU2017 software OS and BIOS Settings Descriptions for xFusion Platform systems

Operating System Tuning Parameters

cpupower frequency-set

cpupower utility is a collection of tools for power efficiency of processor. frequency-set sub-command controls settings for processor frequency. "-g [governor]" specifies a policy to select processor frequency. The performance governor statically sets frequency of the processor cores specified by "-c" option to the highest possible for maximum performance.

cpupower idle-set

idle-set sub-command of cpupower utility controls a processor idle state (C-state) of the kernel. "-d [state_no]>" option disables a specific processor idle state. Disabling idle state can reduce the idle-wakeup delay, but it results in substantially higher power consumption. By default, processor idle states of all CPU cores are set.

irqbalance

Disabled through "service irqbalance stop". Depending on the workload involved, the irqbalance service reassigns various IRQ's to system CPUs. Though this service might help in some situations, disabling it can also help environments which need to minimize or eliminate latency to more quickly respond to events.

isolcpus

This kernel option excludes a specified processor from load balancing by the kernel scheduler. This prevents the scheduler from scheduling any user-space threads on this processor.

nohz_full

This kernel option sets adaptive tick mode (NOHZ_FULL) to specified processors. Since the number of interrupts is reduced to ones per second, latency-sensitive applications can take advantage of it.

numa_balancing

This OS setting controls automatic NUMA balancing on memory mapping and process placement. Setting 0 disables this feature. It is enabled by default (1).

sched_latency_ns

This OS setting configures targeted preemption latency for CPU bound tasks. The default value is 24000000 (ns).

sched_migration_cost_ns

Amount of time after the last execution that a task is considered to be "cache hot" in migration decisions. A "hot" task is less likely to be migrated to another CPU, so increasing this variable reduces task migrations. The default value is 500000 (ns).

sched_min_granularity_ns

This OS setting controls the minimal preemption granularity for CPU bound tasks. As the number of runnable tasks increases, CFS(Complete Fair Scheduler), the scheduler of the Linux kernel, decreases the timeslices of tasks. If the number of runnable tasks exceeds sched_latency_ns/sched_min_granularity_ns, the timeslice becomes number_of_running_tasks * sched_min_granularity_ns. The default value is 10000000(ns).

sched_wakeup_granularity_ns

This OS setting controls the wake-up preemption granularity. Increasing this variable reduces wake-up preemption, reducing disturbance of compute bound tasks. Lowering it improves wake-up latency and throughput for latency critical tasks, particularly when a short duty cycle load component must compete with CPU bound components. The default value is 15000000 (ns).

Firmware / BIOS / Microcode Settings

Hardware Prefetcher (Default = Enabled)

This BIOS option allows the enabling/disabling of a processor mechanism to prefetch data into the cache according to a pattern-recognition algorithm In some cases, setting this option to Disabled may improve performance. Users should only disable this option after performing application benchmarking to verify improved performance in their environment.

Turbo Mode (Default = Enabled)

Intel Turbo boost Technology, Enabling this option allows the processor cores to automatically increase its frequency and increasing performance if it is running below power, temperature.

Enable LP [Global] (Default = Enabled)

The Intel Hyper-Threading knob has been renamed Enable LP [Global] to represent the number of logical processors (LP). Enabling this option allows to use processor resources more efficiently, enabling multiple threads to run on each core and increases processor throughput, improving overall performance on threaded software.

ALL LPs: Hyper-Threading is enabled, each physical processor core functions as two logical processor cores.

Single LP: Run a single logical processor per core.

Performance Profile (Default = Custom)

Values for this BIOS setting can be:

Custom: Allows the user to setup all of the BIOS options according to their requirement.

Performance: Maximize the performance of the server.

Efficiency: Maximize the power efficiency of the server.

Load Balance: The system's performance and power consumption will be adjusted automatically according to the loading.

CPU C6 Report (Default = Disabled)

Enable or disable reporting of the CPU C6 State (ACPI C3) to the OS.

Enhanced Halt State (C1E) (Default = Disabled)

When set to Enabled, the processor is allowed to switch to nimimum performance and save power when idle.

Sub NUMA Cluster（SNC）(Default = Disabled)

Sub NUMA Clustering (SNC) is a feature for breaking up the LLC into disjoint clusters based on address range,with each cluster bound to a subset of the memory controllers in the system.It improves average latency to the LLC.

Values for this BIOS option can be:

Disabled: SNC disabled will support 1-cluster and 4-way IMC interleave.

Enable SNC2 (2-clusters): SNC2 Enabled supports 2-clusters SNC and 2-way IMC interleave.

Enable SNC4 (4-clusters): SNC4 Enabled supports 4-clusters SNC and 1-way IMC interleave.

Last Level Cache (LLC) Prefetch (Default = Enabled)

The last level cache (LLC) prefetch is a prefetcher added to the Intel Xeon Scalable processor family as a result of the non-inclusive cache architecture. The LLC prefetcher is an additional prefetch mechanism on top of the existing prefetchers that prefetch data into the core Data Cache Unit (DCU) and Mid-Level Cache (MLC or second-level cache (L2)). Enabling LLC prefetch gives the core prefetcher the ability to prefetch data directly into the LLC without necessarily filling into the L1 and L2 cache. In some cases, setting this option to disabled can improve performance.

Values for this BIOS option can be:

Disabled: Disables the LLC prefetcher. The other core prefetchers are unaffected.

Enabled: Gives the core prefetcher the ability to prefetch data directly to the LLC.

Adaptive Double Device Data Correction (ADDDC) Sparing (Default = Enabled)

Adaptive Double Device Data Correction (ADDDC), which is an enhanced feature to DDDC. This function is used to correct data errors on two memory particles, ADDDC still has single-particle multi-bit error correction capability after the first particle failure occurs and is replaced.

Values for this BIOS option can be:

Enabled: Enable the ADDDC Sparing function.

Disabled: Disable the ADDDC Sparing function.

LLC dead line alloc (Default = Enabled)

LLC dead line allocation. The processor marks the row replaced by the MLC as dead, indicating that the row will not be read again. This function is used to set the allocation policy for the data marked as dead in the LLC.

Values for this BIOS option can be:

Enabled: Allows the LLC to fill dead lines into the LLC if there is free space.

Disabled: The dead lines are dropped and are never filled into the LLC, saving the LLC space.

Stale AtoS (Default = Auto)

The in-memory directory has three states: invalid (I), snoopAll (A), and shared (S). Invalid (I) state means the data is clean and does not exist in any other socket`s cache. The snoopAll (A) state means the data may exist in another socket in exclusive or modified state. Shared (S) state means the data is clean and may be shared across one or more socket`s caches. When doing a read to memory, if the directory line is in the A state we must snoop all the other sockets because another socket may have the line in modified state. If this is the case, the snoop will return the modified data. However, it may be the case that a line is read in A state and all the snoops come back a miss. This can happen if another socket read the line earlier and then silently dropped it from its cache without modifying it.

Values for this BIOS option can be:

Auto: The SnoopAll (A) state is used by default. During uncore post MRC, the state is reconfigured based on the setup knob, number of sockets, and BPS memory.

Enabled: The SnoopAll (A) state is changed to the Shared (S) state.

Disabled: The SnoopAll (A) state is used.