EG: Embedded Group
SPEC EG, formerly EEMBC, develops industry-standard benchmarks for the hardware
and software used in autonomous driving, mobile imaging, the Internet of Things,
mobile devices, and many other applications. Benchmark suites are developed by
working groups of our members who share an interest in developing clearly defined
standards for measuring the performance and energy efficiency of embedded processor
implementations, from IoT edge nodes to next-generation advanced driver-assistance systems.
The primary audience for the scores yielded by SPEC EG benchmarks are companies
creating systems that rely on embedded microcontrollers and microprocessors:
everything from smartphones to solar panels. Although such manufacturers typically
have their own proprietary benchmarks, SPEC EG provides an attractive alternative
to all the complexities of purchasing hardware, setting it up, and then running
all the measurements on a candidate group of processors. The SPEC EG benchmark
suites allow manufacturers to avoid this cumbersome process by providing a uniform,
rigorously defined standard of measurement that makes it easier for them to compare
alternative solutions. This benefits not only the large manufacturers but everyone
in the ecosystem of developers, integrators, and smaller enterprises that may or may
not have access to proprietary benchmarks of their own or someone else.
SPEC EG was originally founded in 1997 as the independent EEMBC consortium which was
later acquired by SPEC in 2023. In the more than 25 years since EEMBC's founding,
it has accumulated a database of benchmark scores that everyone in the EEMBC community
can use in product development, R&D, or as a baseline for comparison with their
own benchmark test results. Published scores, when available, can be accessed from
each of the main benchmark pages.
Joining SPEC EG
We welcome your interest in joining SPEC EG!
SPEC EG Benchmark Suites
- ULPMark®
-
In 2014, the ULP team introduced the ULPMark-CoreProfile (or -CP for short).
The benchmark runs an active workload for a period of time, then goes to sleep.
The energy measurement during the duty cycle reflects a real-life test of embedded
low power beyond a simple sleep number.
The ULPMark-PeripheralProfile (or -PP for short) launched in 2016 examines
the energy cost of four peripherals: real-time clock, pulse-width modulation,
analog-to-digital conversion, and SPI communication.
ULPMark-CoreMark (or ULPMark-CM for short) launched in 2019 measures the energy
of CoreMark in a consistent environment. It is EEMBC's first active-power benchmark.
- IoTMark®
-
IoTMark builds on ULPMark by adding a sensor emulation module (the IO Manager)
and a radio gateway emulator (the Radio Manager). The execution profile incorporates
the types of behavior an IoT edge node would perform. The first benchmark in this
series, IoTMark-BLE, uses a Bluetooth Low Energy (BLE) radio as the gateway and
an I2C device as the sensor. The next in the suite is IoTMark-Wi-Fi, which explores
link- and application-layer energy efficiency for 802.11 devices.
- SecureMark®
-
Security comes at a cost, both in programming complexity and energy. As IoT
secure becomes even more important, designers must assess the energy cost of the
design. SecureMark provides security-specific profiles to assist in this analysis.
SecureMark-TLS provides a method to account for the cost of a TLS handshake using
elliptic curve key exchange and signing, as well as AES128 CCM & ECB for a
ciphers and SHA256 for hashing.
Heterogeneous Compute
EEMBC's original benchmarks focused on single core 8- and 16-bit embedded processors.
These benchmarks aren't suitable for todays mixed-core heterogeneous platforms deployed
for high-performance computation. The benchmarks in this category differ from single-core
or symmetric multi-core performance benchmarks EEMBC has developed in the past, in that
they provide more sophisticated frameworks for asymmetric compute.
- ADASMark™
-
ADASMark focuses benchmarking a typical vision pipeline that may be used in advanced
driver-assistance systems (ADAS) platforms. Built on OpenCL, the pipeline may be
distributed among CPUs, GPUs and DSPs.
- MLMark™
-
Machine Learning on the edge has become a reality. This benchmark strives to categories
and analyze several broad classes of popular ML neural networks on embedded edge-compute
platforms.
Single-core Processor Performance
The original EEMBC benchmarks, these suites are still in popular use today for compiler testing,
regressions in new embedded core RTL, and academic analysis.
- AudioMark™
-
A modern audio pipeline with multiple datatypes, larger code size, DSP, and machine learning.
- CoreMark®
-
A simple, yet sophisticated test of the functionality of a processor core; it produces
a single-number score allowing users to make quick comparisons between processors.
- AutoBench™
-
The algorithms in the benchmark analyze processor performance in automotive, industrial,
and general-purpos applications.
- DENBench™ (Digital Entertainment)
-
This benchmark addresses digital entertainment products such as smartphones, MP3 players,
digital cameras, TV set-top boxes, and in-car entertainment systems.
- Networking 2.0
-
Routers and switches can benefit from this benchmark's analysis of performance associated
with moving packets in networking applications.
- OABench™ (Office Automation)
-
This benchmark approximates office automation tasks performed by processors in printers,
plotters, and other office automation systems that handle text and image processing tasks.
- TeleBench™
-
The telecommunications suite approximates performance of processors in modem, xDSL, and
related fixed-telecom applications.
Symmetric Multi-core Processor Performance
Our multicore suites are built on the multi-instance test harness (MITH) which exploits the
POSIX pthreads interace for testing both context- and worker-level parallelism.
- CoreMark®-Pro
-
CoreMark-Pro builds on the original CoreMark benchmark by adding context-level parallelism
and 7 new workloads which cover integer and floating-point performance.
- AutoBench™-2.0
-
The original AutoBench benchmark is back in parallel form, with more workloads concentrating
on aggregated tasks (combinations of AutoBench-1.1 kernels) that reflect the increase in
compute demand of automotive electronic control units (ECU).
- FPMark™
-
FPMark focuses entirely on single- and double-precision floating-point workloads. From
Gauss-Jordan Elimination to Black Schols computation, FPMark covers a broad range of
intense floating point analysis.
- MultiBench™
-
To analyze worker-level parallelism in addition to context-level, MultiBench contains
integer workloads that scale in both directions.
Phone and Tablet
- BrowsingBench™
-
Measure browser performance with BrowsingBench, a collection of webpages loaded using a
local Nginx server over a wired LAN connection.