Video Area Home >> Video-Codec Comparisons >> HEVC/AV1 Video Codecs Comparison 2020

MSU Video Codecs Comparison 2020 Part 1: FullHD, objective

Fifteen Annual Video-Codecs Comparison by MSU

Video group head: Dr. Dmitriy Vatolin
Project head: Dr. Dmitriy Kulikov
Measurements, analysis: Dr. Mikhail Erofeev,
Anastasia Antsiferova,
Egor Sklyarov,
Alexander Yakovenko,
Nickolay Safonov

Navigation

Summary

TL;DR Download the report

This report presents the results of video codecs comparison using objective quality measurement. Main points of this test:

  • 50 FullHD videos
  • 20 video encoders of different standards (H.265/HEVC, AV1, H.264/AVC, and others)
  • 2 encoding use cases
    • Offline encoding (1 fps)
    • Online encoding (30 fps)
  • Metrics: SSIM, PSNR, VMAF (+ additional color-planes)

Offline (1 fps) Online (30 fps)
Best quality (SSIM) 1st: Aurora AV1 Encoder (AV1), BVC2.0
2nd: QAV1 (AV1)
3rd: Tencent V265 (HEVC), aom (AV1) 		1st: Tencent V265
2nd: XCCZM265
3rd: S265
Best quality (VMAF) 1st: BVC2.0
2nd: Aurora AV1 Encoder (AV1)
3rd: Tencent V265 (HEVC) 		1st: Tencent V265
2nd: BD265, XCCZM265
3rd: S265
Best quality (PSNR avg.log.) 1st: S265 (HEVC), BVC2.0
2nd: Tencent V265 (HEVC), S265 (HEVC)
3rd: Aurora AV1 Encoder (AV1) 		1st: S265, Tencent V265
2nd: XCCZM265
3rd: BD265
Best quality (PSNR avg.MSE) 1st: BVC2.0
2nd: Aurora AV1 Encoder (AV1)
3rd: S265 (HEVC), Tencent V265 (HEVC),
QAV1 (AV1) 		1st: Tencent V265, S265
2nd: XCCZM265
3rd: S265, BD265
Best speed-quality trade-off (SSIM)
The most frequent Pareto-optimal encoders for all videos 		SIF Codec, Aurora AV1 Encoder, x264 Tencent V265

The biggest number of codecs took part in comparison of Offline encoding (1 fps). The winners vary for different objective quality metrics, the leaders for bith SSIM and VMAF metrics are: Aurora AV1 Encoder, BVC2.0 and Tencent V265. The participants were rated using BSQ-rate (enhanced BD-rate) scores [1].

[1] A. Zvezdakova, D. Kulikov, S. Zvezdakov, D. Vatolin, "BSQ-rate: a new approach for video-codec performance comparison and drawbacks of current solutions," 2020.

Download & Buy Report

Free version Enterprise version
Use cases Offline (1 fps) (partially) Offline (1 fps), Online (1 fps)
Per-sequence-results 1 of 50 sequences (only Offline use case) All 50 sequences for all use cases (in interactive charts)
Metric: YUV-SSIM YES YES
Videos YES
Only description 		YES
Description and sources
Codec info YES
Developer, version 		YES
Encoding presets, developer, version
Other objective metrics (in addition to YUV-SSIM) None
(only for overall scores) 		YES
Y-VMAF(0.6.3), Y-SSIM, U-SSIM, V-SSIM, YUV-PSNR, Y-PSNR, U-PSNR, V-PSNR
Per-frame metrics results None YES
PDF report 48 pages 70 pages
HTML report 29 interactive charts 15000+ interactive charts
Price Free $950
Download PDF & HTML (17 MB)

PDF & HTML reports + videos description (ZIP, 17 MB) 		Buy

You will receive enterprise versions of all 2020 reports (FullHD, Subjective, 4K)

Participated codecs

Codec name Use cases Standard Version
1 aom
AOMedia 		Offline (1 FPS) AV1 2.0.0-287-g2aa13c436, Windows
2 Aurora AV1 Encoder
Visionular 		Offline (1 FPS) AV1 2.0, Windows
3 BD265
Baidu Inc. 		Offline (1 FPS),
Online (30 FPS) 		H.265/HEVC 2.0, Windows
4 BVC2.0
Bytedance Inc. 		Offline (1 FPS) Other V0, Windows
5 donkey
ChangKuoLao 		Offline (1 FPS) AV1 0.4.2c-a60655ae, Windows
6 QAV1
iQIYI Inc. 		Offline (1 FPS) AV1 v1.1, Linux
7 rav1e
The rav1e contributors 		Offline (1 FPS) AV1 0.3.0 (p20200515), Windows
8 Reference x265
MulticoreWare, Inc. 		Offline (1 FPS),
Online (30 FPS) 		H.265/HEVC 3.3+21-6bb2d88029c2, Windows
9 S265 (bitrate & CRF modes)
Alibaba Group 		Offline (1 FPS),
Online (30 FPS) 		H.265/HEVC v5.0.2, Windows
10 SIF Codec
SIF Codec LLC 		Offline (1 FPS),
Online (30 FPS) 		SIF 1.91, Windows
11 SVT-AV1
Open Visual Cloud 		Offline (1 FPS) AV1 v0.8.3, Windows
12 SVT-HEVC
Open Visual Cloud 		Offline (1 FPS),
Online (30 FPS) 		H.265/HEVC v1.4.3, Windows
13 SVT-VP9
Open Visual Cloud 		Offline (1 FPS),
Online (30 FPS) 		VP9 v0.2.0, Windows
14 sz265
RayShaper 		Offline (1 FPS),
Online (30 FPS) 		H.265/HEVC v1.0.0, Linux
15 Tencent V265
Tencent 		Offline (1 FPS),
Online (30 FPS) 		H.265/HEVC v1.4.5, Windows
16 VP9
The WebM Project 		Offline (1 FPS) VP9 v1.8.2, Windows
17 x264
x264 project 		Offline (1 FPS),
Online (30 FPS) 		H.264/AVC 0.160.3000 33f9e14, Windows
18 x265
MulticoreWare, Inc. 		Offline (1 FPS),
Online (30 FPS) 		H.265/HEVC 3.3+33-3116be008af1, Windows
19 XCCZM265
XCCZM Codec Team 		Offline (1 FPS),
Online (30 FPS) 		H.265/HEVC v3.2_0607, Windows
20 xin26x
A Father (xin26x) 		Offline (1 FPS),
Online (30 FPS) 		H.265/HEVC 1.1, Windows

Comparison Rules

HEVC codec testing objectives

The main goal of this report is the presentation of a comparative evaluation of the quality of new HEVC codecs and codecs of other standards using objective measures of assessment. The comparison was done using settings provided by the developers of each codec. Nevertheless, we required all presets to satisfy minimum speed requirement on the particular use case. The main task of the comparison is to analyze different encoders for the task of transcoding video – e.g., compressing video for personal use.

Test Hardware Characteristics

  • CPU: Intel Socket 1151 Core i7 8700K (Coffee Lake) (3.7Ghz, 6C12T, TDP 95W)
  • Mainboard: ASRock Z370M Pro4
  • RAM: Crucial CT16G4DFD824A 2x16GB (totally 32 GB) DIMM DDR4 2400MHz CL15
  • OS: Windows 10 x64, Linux

For this platform we considered three key use cases with different speed requirements:

  • Online – 1080p@30fps
  • Offline – 1080p@1fps

See more on Call-for-codecs 2020 page

Videos

Videos for testing set were chosen from MSU video collection via a voting among comparison participants, organizers and an independend expert.

Number of videos in MSU video collection
Year # FullHD videos # FullHD samples # 4K videos # 4K samples Total # of videos Total # of samples
2016 3 7 882 2902 885 2909
2017 1996 4638 1544 4561 3540 9299
2018 4342 10330 1946 5503 6288 15833
2020 4945 12402 2091 6016 7036 18418

Bitrate distribution of videos in MSU video collection Videos bitrate distribution

Final video set consists of 50 sequences including new videos from Vimeo and media.xiph.org derf's collection.


Video sequences selection

Descriptions of all test videos are presented in a separate PDF provided with the report.

Codec Analysis and Tuning for Codec Developers and Codec Users

Computer Graphics and Multimedia Laboratory of Moscow State University:

  • 17+ years working in the area of video codec analysis and tuning using objective quality metrics and subjective comparisons.
  • 30+ reports of video codec comparisons and analysis (H.265, H.264, AV1, VP9, MPEG-4, MPEG-2, decoders' error recovery).
  • Methods and algorithms for codec comparison and analysis development, separate codec's features and codec's options analysis.

We could perform next tasks for codec developers and codec users.

Strong and Weak Points of Your Codec

  • Deep encoder parts analysis (ME, RC on GOP, mode decision, etc).
  • Weak and strong points for your encoder and complete information about encoding quality on different content types.
  • Encoding Quality improvement by the pre and post filtering (including technologies licensing).

Independent Codec Estimation Comparing to Other Codecs for Different Use-cases

  • Comparative analysis of your encoder and other encoders.
  • We have direct contact with many codec developers.
  • You will know place of your encoder between other newest well-known encoders (compare encoding quality, speed, bitrate handling, etc.).

Encoder Features Implementation Optimality Analysis

We perform encoder features effectiveness (speed/quality trade-off) analysis that could lead up to 30% increase in the speed/quality characteristics of your codec. We can help you to tune your codec and find best encoding parameters.

Thanks

Special thanks to the following contributors of our previous comparisons
Google Intel AMD NVidia
ATI Adobe ISPhone dicas
KDDI R&D labs Dolby Tata Elxsi Octasic
Qualcomm Voceweb Elgato Telecast
Huawei MainConcept Vitec Tencent

Leave a feedback

Contact Information


compression.ru 		in cooperation
with 		Lomonosov MSU
Graphics & Media Lab
(Video Group)
Dubna State University
Institute for Information
Transmission Problems RAS

Subscribe to report updates

Other Materials

Video resources:

3D and stereo video
Projects on 3D and stereo video processing and analysis
MSU Video Quality Measurement tools
Programs with different objective and subjective video quality metrics implementation 		Codecs comparisons
Objective and subjective quality evaluation
tests for video and image codecs
Ext. link: x264 parameters efficiency comparison
Public MSU video filters
Here are available VirtualDub and AviSynth filters. For a given type of digital video filtration we typically develop a family of different algorithms and implementations. Generally there are also versions optimized for PC and hardware implementations (ASIC/FPGA/DSP). These optimized versions can be licensed to companies. Please contact us for details via video(at)graphics.cs.msu_ru. 		Filters for companies
We are working with Intel, Samsung, RealNetworks and other companies on adapting our filters other video processing algorithms for specific video streams, applications and hardware like TV-sets, graphics cards, etc. Some of such projects are non-exclusive. Also we have internal researches. Please let us know via video(at)graphics.cs.msu_ru if you are interested in acquiring a license for such filters or making a custom R&D project on video processing, compression, computer vision.
Video codecs projects
Different research and development
projects on video codecs 		Other
Other information
Last updated: 12-May-2022

Server size: 8069 files, 1215Mb (Server statistics)

Project updated by
Server Team and MSU Video Group

Project sponsored by YUVsoft Corp.

Project supported by MSU Graphics & Media Lab