SPEC 2021 Year in Review

By David Reiner, President

As all of us at SPEC continue into a very busy 2022, I’d like to reflect on what we accomplished during 2021, which, despite the continued headwinds from the pandemic, was an exciting and productive year at SPEC.

New committees and groups

2021 saw the formation of two new committees at SPEC. The Open Systems Group (OSG) Machine Learning (ML) Working Group became a full Machine Learning Committee. Its charter includes developing, releasing and maintaining new ML benchmarks, as well as collaborating with other SPEC committees as they incorporate AI and ML into their benchmarks.

The International Standards Group (ISG), a new 5th group of SPEC, was established in 2020 and is now fully up and running and meeting regularly. The group launched its second committee, the ISG Client Efficiency Committee, which is focused on standardized measurement and reporting of power efficiency across a broad diversity of client PC form factors, such as desktops and laptops. The goal of this committee is to create a client computer energy efficiency benchmark for use by worldwide governments in regulations and programs.

New benchmarks

I’m also very excited to report that in 2021, we released three very important new benchmarks and updated four others. A huge thank you to all the volunteers for the tremendous amount of time and effort they put into this work!

The SPECapc™ for Solidworks™ 2021 benchmark, which measures the latest version of Dassault Systèmes’ SOLIDWORKS 2021 CAD/CAM application, includes 10 models and 50 tests exercising a full range of graphics and CPU functionality.

The SPEChpc™ 2021 Benchmark Suites, which measure real-world performance for state-of-the-art High Performance Computing (HPC) systems, provide researchers, developers, hardware and software vendors, compiler vendors, data center operators and end users with fair comparisons of the performance of different HPC systems.

The SPECvirt® Datacenter 2021 benchmark, a new multi-host benchmark, measures the performance of a scaled-out datacenter and provides a better way to measure a virtualized platform’s ability to model a dynamic datacenter virtual environment.

The four updated benchmarks were:

SPEC CPU® 2017 v1.1.7
SPECviewperf® 2020 v3.0
SPEC® SERT® 2.0.4
SPECworkstation® 3.1.

New members

Thanks to the unwavering dedication of our many volunteers, SPEC continues to be one of the most highly respected, vendor-neutral, global. industry-standard benchmarking organizations. We are delighted to have welcomed eight new distinguished members to our organization in 2021. VeriSilicon joined the Graphics and Workstation Group (GWPG). Interuniversity Microelectronics Centre (IMEC), Next Silicon Inc., Siemens Industry Software, and Texas Advanced Computing Center – UT Austin joined the High Performance Group (HPG). And Ampere Computing joined the Open Systems Group (OSG).

New members bring a wealth of knowledge, experience, and diversity to SPEC. We welcome all interested organizations to join SPEC and help us develop the next generation of real-world relevant performance and energy efficiency benchmarks. Quality benchmarks have a critical role in measuring and incentivizing computational improvements, so the world’s toughest problems can be solved more quickly while conserving our planet’s precious resources.

2022 and beyond

Speaking of the coming year, SPEC’s future is bright, and opportunities abound. Looking forward, SPEC is pursing further integration of ML workloads across our benchmarks where relevant, as well as, building upon our SPEC Cloud® IaaS 2018 benchmark by looking for ways to ease the use and explicitly design our benchmarks for use in the Cloud. At the SPEC executive level, we plan to continue the work of simplifying, documenting, and improving our internal processes for additional transparency and consistency and to enable SPEC members to stay focused on benchmark development.

As an organization driven and resourced by volunteers, I encourage members as well as industry stakeholders to bring forward proposals for enhancements to our existing benchmarks, as well as areas of compute that have not yet experienced the benefits of a quality industry-standard benchmark. Both new committees started in 2021 were initiated by a member proposal and resource offer.

As our hearts and prayers remain focused on so many who have been tragically affected by the pandemic, and now war, it is heartening to see SPEC thriving and growing, and I ask for your continued contributions and support to help us take the next step in building better benchmarks. It won’t happen without you.