JVM Options:
-d64 -server -XX:AllocatePrefetchDistance=256 -XX:AllocatePrefetchStyle=1
-Xms14g -Xmx14g -Xmn11g -Xss256k -XX:MaxPermSize=256m -XX:+AggressiveOpts
-XX:+UseParallelGC -XX:ParallelGCThreads=24 -verbose:gc -XX:+PrintGCDetails
-XX:+PrintGCTimeStamps -XX:-TraceClassUnloading -XX:+UseParallelOldGC
-XX:ReservedCodeCacheSize=64m -XX:CICompilerCount=8 -XX:+UseCompressedOops
-XX:-UseBiasedLocking -XX:+AlwaysPreTouch -XX:InitialSurvivorRatio=4
-XX:SurvivorRatio=4 -XX:TargetSurvivorRatio=90 -XX:-UseAdaptiveSizePolicy
-XX:LargePageSizeInBytes=2g
-Dweblogic.MuxerClass=weblogic.socket.DevPollSocketMuxer -Dweblogic.SocketReaders=16
-Dweblogic.diagnostics.debug.DebugLogger.DISABLED=true -Doracle.jdbc.defaultRowPrefetch=200
-Dweblogic.management.discover=false -Dweblogic.llr.table.specjds=wl_llr_jent31
-Djavax.xml.parsers.DocumentBuilderFactory=weblogic.xml.jaxp.RegistryDocumentBuilderFactory
-Dweblogic.threadpool.MinPoolSize=30 -Dsun.net.inetaddr.ttl=0 -Dnetworkaddress.cache.ttl=0
-Dcom.sun.xml.ws.spi.db.BindingContextFactory=com.sun.xml.ws.db.glassfish.JAXBRIContextFactory
-Djavax.xml.bind.JAXBContext=com.sun.xml.bind.v2.ContextFactory

Java process started in FX class using /usr/bin/priocntl -e -c FX
Java process started with LD_PRELOAD_64=/usr/lib/sparcv9/libumem.so

JVM Options:
-d64 -server -Xms20g -Xmx20g -Xmn3g -Xss256k -XX:+AggressiveOpts
-XX:+UseParallelGC -XX:ParallelGCThreads=32 -XX:LargePageSizeInBytes=4m
-verbose:gc -XX:+PrintGCDetails -XX:+PrintGCTimeStamps -XX:-TraceClassUnloading
-XX:PermSize=256m -XX:MaxPermSize=512M -XX:+UseCompressedOops
-Dsun.net.inetaddr.ttl=0 -Dnetworkaddress.cache.ttl=0 -Dhttp.keepAliveCache.lifeTime=600000
-Dweblogic.management.discover=false -Dweblogic.SocketReaders=1
-Dweblogic.diagnostics.debug.DebugLogger.DISABLED=true

Java process started in FX class using /usr/bin/priocntl -e -c FX
Java process started with LD_PRELOAD_64=/usr/lib/amd64/libumem.so

Database Options:

db_name                         = spec
control_files                   = +DATA/cntrlspec
cpu_count                       = 128
aq_tm_processes                 = 0
compatible                      = 12.1.0.1.0.0
db_block_size                   = 2048
db_block_checksum               = FALSE
db_block_checking               = FALSE
db_cache_size                   = 14G
db_cache_advice                 = off
db_4k_cache_size                = 10G
db_8k_cache_size                = 4G
db_keep_cache_size              = 10G
db_recycle_cache_size           = 1G
db_file_multiblock_read_count   = 128
db_files                        = 256
db_writer_processes             = 32
deferred_segment_creation       = false
ddl_lock_timeout                = 30
disk_asynch_io                  = TRUE
dml_locks                       = 1000
filesystemio_options            = setall
log_buffer                      = 100663296
log_checkpoint_interval         = 0
log_checkpoints_to_alert        = TRUE
open_cursors                    = 2400
parallel_max_servers            = 100
processes                       = 8000
query_rewrite_enabled           = FALSE
replication_dependency_tracking = FALSE
sessions                        = 12000
shared_pool_size                = 16G
statistics_level                = BASIC
timed_statistics                = false
trace_enabled                   = false
transactions                    = 12000
transactions_per_rollback_segment = 1
undo_management                 = AUTO
undo_retention                  = 600
undo_tablespace                 = undo_ts

Grid Infrastructure was installed to use Oracle ASM for DATA and LOG disk groups.
Oracle database processes run in 5 processors sets within a
logical domain configured on 3 cpu chips (47 cores)
using Oracle VM for SPARC. One core from the first
cpu chip was reserved for the primary domain.
Each processor set was configured with 8 cores (64 strands).
Each process run in the FX class using /usr/bin/priocntl -s -c FX -i pid PID.
Oracle Log Writer run in a single core by executing
in FX class at priority 60 using the command
/usr/bin/priocntl -s -c FX -p 60 -m 60 -i PID.
The remaining cores were used for HBA and Network interrupts.
The Solaris 11.1 logical domain configuration is in the FDA.

JVM Options
-d64 -server -Xms6400M -Xmx6400M -Xss256k -Dsun.net.inetaddr.ttl=0
-Dnetworkaddress.cache.ttl=0 -Djava.net.preferIPv4Stack=true -verbosegc
-Dhttp.keepAliveCache.lifeTime=600000 -XX:+PrintFlagsFinal
-Dcom.sun.xml.ws.spi.db.BindingContextFactory=com.sun.xml.ws.db.glassfish.JAXBRIContextFactory
-Djavax.xml.bind.JAXBContext=com.sun.xml.bind.v2.ContextFactory

Tuning in Primary Domain:
Network Tuning:
ndd -set /dev/udp udp_recv_hiwat 2097152
ndd -set /dev/udp udp_xmit_hiwat 2097152
1x 10 Gbit NIC port was used for DNS requests.
Stop intrd.
Solaris 11.1 Support Repository Update 10.5
Configuration Notes:
Application Servers were configured on 5 processors (80 cores) running 10 Solaris zones.
DB server was configured using 47 cores from 3 processors
One core from one processor ran the primary domain.

Tuning in Appserver Domain:
/etc/system:
set autoup = 345600
Network Tuning:
ndd -set /dev/tcp tcp_conn_req_max_q 40000
ndd -set /dev/tcp tcp_conn_req_max_q0 40000
ndd -set /dev/tcp tcp_xmit_hiwat 524288
ndd -set /dev/tcp tcp_recv_hiwat 524288
ndd -set /dev/tcp tcp_smallest_anon_port 4096
ndd -set /dev/tcp tcp_naglim_def 1
ndd -set /dev/tcp tcp_time_wait_interval 10000
ndd -set /dev/tcp tcp_fin_wait_2_flush_interval 10000
Tuning in /kernel/drv/ixgbe.conf:
ixgbe[4/5/6/7/12]: intr_throttling=3000
ixgbe[0/1/10/11/13]: intr_throttling=1000
Configuration Notes:
10 application server instances were hosted in 10 separate Solaris Zones.
On each chip, 2 zones were bound to a single processor set where
each processor set consisted of 116 HW strands.
The default processor set consisting of 60 HW strands from all 5 chips
was used for network processing.
The Solaris Zone and processor set configuration is in the FDA.
Each Solaris Zone on the system was configured with 2 subnets.
5x 10 Gbit NIC ports were used by 10 zones for driver requests.
5x 10 Gbit NIC ports were used by 10 zones for database access.
Interrupts from the Network controllers were configured across the
processors in the appserver zones using the /usr/sbin/pcitool utility.
Stop intrd.
Solaris 11.1 Support Repository Update 10.5

Tuning in Database Domain:
/etc/system:
set autoup = 345600
Network Tuning:
ndd -set /dev/tcp tcp_conn_req_max_q 40000
ndd -set /dev/tcp tcp_conn_req_max_q0 40000
ndd -set /dev/tcp tcp_xmit_hiwat 524288
ndd -set /dev/tcp tcp_recv_hiwat 524288
ndd -set /dev/tcp tcp_smallest_anon_port 4096
ndd -set /dev/tcp tcp_naglim_def 1
ndd -set /dev/tcp tcp_time_wait_interval 10000
ndd -set /dev/tcp tcp_fin_wait_2_flush_interval 10000
Configuration Notes:
Oracle database runs in a logical domain configured with 3 cpu chips (47 cores).
External Disk configuration:
6x Sun Server X3-2L each configured with 4x F40 Flash PCI Cards
and COMSTAR software used for database files.
2x 2540 M2 Storage Arrays each configured with 12x 600GB disks
and 2x RAID controllers
The COMSTAR configuration for the database files is in the FDA
5x 10 Gbit NIC ports were used for database access.
Interrupts from the HBAs and Network controllers were configured across
the processors using the /usr/sbin/pcitool utility.
Stop intrd.
Solaris 11.1 Support Repository Update 10.5

Tuning in /etc/system:
set ddi_msix_alloc_limit=8
set ip:ip_soft_rings_cnt=16
Network Tuning:
ndd -set /dev/tcp tcp_conn_req_max_q 40000
ndd -set /dev/tcp tcp_conn_req_max_q0 40000
ndd -set /dev/tcp tcp_xmit_hiwat 524288
ndd -set /dev/tcp tcp_recv_hiwat 524288
ndd -set /dev/tcp tcp_smallest_anon_port 4096
ndd -set /dev/tcp tcp_naglim_def 1
ndd -set /dev/tcp tcp_time_wait_interval 10000
Configuration Notes:
16 Driver instances ran on 16 systems.
10 Emulator instances ran on 5 systems.