research-article

Open access

Understanding quality of experience of heuristic-based HTTP adaptive bitrate algorithms

Authors:

Abdelhak Bentaleb,

Christian Timmerer,

Roger Zimmermann,

Hermann HellwagnerAuthors Info & Claims

NOSSDAV '21: Proceedings of the 31st ACM Workshop on Network and Operating Systems Support for Digital Audio and Video

Pages 82 - 89

https://doi.org/10.1145/3458306.3458875

Published: 02 July 2021 Publication History

Abstract

Adaptive bitrate (ABR) algorithms play a crucial role in delivering the highest possible viewer's Quality of Experience (QoE) in HTTP Adaptive Streaming (HAS). Online video streaming service providers use HAS - the dominant video streaming technique on the Internet - to deliver the best QoE for their users. A viewer's delight relies heavily on how the ABR of a media player can adapt the stream's quality to the current network conditions. QoE for video streaming sessions has been assessed in many research projects to give better insight into the significant quality metrics such as startup delay and stall events. The ITU Telecommunication Standardization Sector (ITU-T) P.1203 quality evaluation model allows to algorithmically predict a subjective Mean Opinion Score (MOS) by considering various quality metrics. Subjective evaluation is the best assessment method for examining the end-user opinion over a video streaming session's experienced quality. We have conducted subjective evaluations with crowdsourced participants and evaluated the MOS of the sessions using the ITU-T P.1203 quality model. This paper's main contribution is to investigate the correspondence of subjective and objective evaluations for well-known heuristic-based ABRs.

References

[1]

Mohammed Alreshoodi and John Woods. 2013. Survey on QoE\QoS correlation models for multimedia services. arXiv preprint arXiv:1306.0221 (2013).

[2]

Apple. 2016. HTTP Live Streaming. https://developer.apple.com/streaming/.

[3]

Sabina Baraković and Lea Skorin-Kapov. 2013. Survey and challenges of QoE management issues in wireless networks. Journal of Computer Networks and Communications 2013 (2013).

[4]

Erica Beavers. 2014. How to Encode Multi-Bitrate Videos in MPEG-DASH for MSE Based Media Players. Retrieved Nov 21, 2020 from https://blog.streamroot.io/encode-multi-bitrate-videos-mpeg-dash-mse-based-media-players/

[5]

Abdelhak Bentaleb, Ali C Begen, Saad Harous, and Roger Zimmermann. 2018. Want to play DASH? A game theoretic approach for adaptive streaming over HTTP. In Proceedings of the 9th ACM Multimedia Systems Conference. 13--26.

Digital Library

[6]

Abdelhak Bentaleb, Ali C Begen, Saad Harous, and Roger Zimmermann. 2019. Game of Streaming Players: Is Consensus Viable or an Illusion? ACM Transactions on Multimedia Computing, Communications, and Applications 15, 2s (2019), 1--30.

Digital Library

[7]

Abdelhak Bentaleb, Ali C Begen, and Roger Zimmermann. 2016. SDNDASH: Improving QoE of HTTP adaptive streaming using software defined networking. In Proceedings of the 24th ACM International Conference on Multimedia. 1296--1305.

Digital Library

[8]

Abdelhak Bentaleb, Bayan Taani, Ali C Begen, Christian Timmerer, and Roger Zimmermann. 2018. A survey on bitrate adaptation schemes for streaming media over HTTP. IEEE Communications Surveys & Tutorials 21, 1 (2018), 562--585.

[9]

Bitmovin. 2020. 2020 Video Developer Report. https://go.bitmovin.com/video-developer-report-2020.

[10]

Kjell Brunnström, Sergio Ariel Beker, Katrien De Moor, Ann Dooms, Sebastian Egger, Marie-Neige Garcia, Tobias Hossfeld, Satu Jumisko-Pyykkö, Christian Keimel, Mohamed-Chaker Larabi, et al. 2013. Qualinet white paper on definitions of quality of experience. (2013).

[11]

DASH-IF. 2020. dash.js. [Online] Available: https://reference.dashif.org/dash.js/.

[12]

Luca De Cicco, Vito Caldaralo, Vittorio Palmisano, and Saverio Mascolo. 2013. Elastic: a client-side controller for dynamic adaptive streaming over HTTP (DASH). In 2013 20th International Packet Video Workshop. IEEE, 1--8.

[13]

Zhengfang Duanmu, Wentao Liu, Zhuoran Li, Diqi Chen, Zhou Wang, Yizhou Wang, and Wen Gao. 2020. Assessing the Quality-of-Experience of Adaptive Bitrate Video Streaming. arXiv preprint arXiv:2008.08804 (2020).

[14]

Zhengfang Duanmu, Abdul Rehman, and Zhou Wang. 2018. A Quality-of-Experience Database for Adaptive Video Streaming. IEEE Transactions on Broadcasting 64, 2 (June 2018), 474--487.

[15]

Ericsson. 2020. Ericsson Mobility Report. https://www.ericsson.com/4adc87/assets/local/mobility-report/documents/2020/november-2020-ericsson-mobility-report.pdf.

[16]

Deepti Ghadiyaram, Janice Pan, and Alan C Bovik. 2017. A subjective and objective study of stalling events in mobile streaming videos. IEEE Transactions on Circuits and Systems for Video Technology 29, 1 (2017), 183--197.

Digital Library

[17]

Google. 2020. Shaka Player. [Online] Available: https://github.com/google/shaka-player.

[18]

Tobias Hossfeld, Christian Keimel, Matthias Hirth, Bruno Gardlo, Julian Habigt, Klaus Diepold, and Phuoc Tran-Gia. 2013. Best practices for QoE crowdtesting: QoE assessment with crowdsourcing. IEEE Transactions on Multimedia 16, 2 (2013), 541--558.

Digital Library

[19]

Tianchi Huang, Chao Zhou, Rui-Xiao Zhang, Chenglei Wu, Xin Yao, and Lifeng Sun. 2020. Stick: A Harmonious Fusion of Buffer-based and Learning-based Approach for Adaptive Streaming. In IEEE INFOCOM 2020-IEEE Conference on Computer Communications. IEEE, 1967--1976.

[20]

Te-Yuan Huang, Ramesh Johari, Nick McKeown, Matthew Trunnell, and Mark Watson. 2014. A buffer-based approach to rate adaptation: Evidence from a large video streaming service. In Proceedings of the 2014 ACM Conference on SIGCOMM.

Digital Library

[21]

Apple Inc. 2016. Best practices for creating and deploying HTTP live streaming media for apple devices. Retrieved Nov 21, 2020 from https://developer.apple.com/library/content/technotes/tn2224/_index.html

[22]

Netflix Inc. 2015. Per-Title Encode Optimization. Retrieved Nov 21, 2020 from http://techblog.netflix.com/2015/12/per-title-encode-optimization.html

[23]

ISO/IEC. 2014. 23009-1:2014 - Dynamic adaptive streaming over HTTP (DASH) - Part 1: Media presentation description and segment formats. https://www.iso.org/standard/65274.html.

[24]

P ITU-T. 2020. Methods, metrics and procedures for statistical evaluation, qualification and comparison of objective quality prediction models (P.1401). http://handle.itu.int/11.1002/1000/14159. ITU-T Recommendation (2020).

[25]

P ITU-T Recommendation. 1999. Subjective video quality assessment methods for multimedia applications. International telecommunication union (1999).

[26]

Junchen Jiang, Vyas Sekar, and Hui Zhang. 2012. Improving fairness, efficiency, and stability in http-based adaptive video streaming with festive. In Proceedings of the 8th International Conference on Emerging networking experiments and technologies. 97--108.

Digital Library

[27]

Stefan Lederer, Christopher Müller, and Christian Timmerer. 2012. Dynamic adaptive streaming over HTTP dataset. In Proceedings of the 3rd multimedia systems Conference. 89--94.

Digital Library

[28]

Zhi Li, Xiaoqing Zhu, Joshua Gahm, Rong Pan, Hao Hu, Ali C Begen, and David Oran. 2014. Probe and adapt: Rate adaptation for HTTP video streaming at scale. IEEE Journal on Selected Areas in Communications 32, 4 (2014), 719--733.

[29]

Tarun Mangla, Nawanol Theera-Ampornpunt, Mostafa Ammar, Ellen Zegura, and Saurabh Bagchi. 2016. Video through a crystal ball: Effect of bandwidth prediction quality on adaptive streaming in mobile environments. In Proceedings of the 8th International Workshop on Mobile Video. 1--6.

Digital Library

[30]

Hongzi Mao, Ravi Netravali, and Mohammad Alizadeh. 2017. Neural adaptive video streaming with pensieve. In Proceedings of the Conference of the ACM Special Interest Group on Data Communication. 197--210.

Digital Library

[31]

Karan Mitra, Arkady Zaslavsky, and Christer Åhlund. 2014. QoE modelling, measurement and prediction: A review. arXiv preprint arXiv:1410.6952 (2014).

[32]

Alexander Raake, Marie-Neige Garcia, Werner Robitza, Peter List, Steve Göring, and Bernhard Feiten. 2017. A bitstream-based, scalable video-quality model for HTTP adaptive streaming: ITU-T P.1203.1. In Ninth International Conference on Quality of Multimedia Experience (QoMEX). IEEE, Erfurt. http://ieeexplore.ieee.org/document/7965631/

[33]

Werner Robitza, Steve Göring, Alexander Raake, David Lindegren, Gunnar Heikkilä, Jörgen Gustafsson, Peter List, Bernhard Feiten, Ulf Wüstenhagen, MarieNeige Garcia, Kazuhisa Yamagishi, and Simon Broom. 2018. HTTP Adaptive Streaming QoE Estimation with ITU-T Rec. P.1203 - Open Databases and Software. In 9th ACM Multimedia Systems Conference. Amsterdam.

Digital Library

[34]

Kalpana Seshadrinathan, Rajiv Soundararajan, Alan Conrad Bovik, and Lawrence K Cormack. 2010. Study of subjective and objective quality assessment of video. IEEE Transactions on Image Processing 19, 6 (2010), 1427--1441.

Digital Library

[35]

Michael Seufert, Sebastian Egger, Martin Slanina, Thomas Zinner, Tobias Hoßfeld, and Phuoc Tran-Gia. 2014. A survey on quality of experience of HTTP adaptive streaming. IEEE Communications Surveys & Tutorials 17, 1 (2014), 469--492.

Digital Library

[36]

Kevin Spiteri, Rahul Urgaonkar, and Ramesh K Sitaraman. 2020. BOLA: Near-optimal bitrate adaptation for online videos. IEEE/ACM Transactions on Networking 28, 4 (2020), 1698--1711.

Digital Library

[37]

Yi Sun, Xiaoqi Yin, Junchen Jiang, Vyas Sekar, Fuyuan Lin, Nanshu Wang, Tao Liu, and Bruno Sinopoli. 2016. CS2P: Improving video bitrate selection and adaptation with data-driven throughput prediction. In Proceedings of the 2016 ACM SIGCOMM Conference. 272--285.

Digital Library

[38]

Babak Taraghi, Anatoliy Zabrovskiy, Christian Timmerer, and Hermann Hellwagner. 2020. CAdViSE: cloud-based adaptive video streaming evaluation framework for the automated testing of media players. In Proceedings of the 11th ACM Multi-media Systems Conference. 349--352.

Digital Library

[39]

Christian Timmerer, Matteo Maiero, and Benjamin Rainer. 2016. Which adaptation logic? An objective and subjective performance evaluation of HTTP-based adaptive media streaming systems. arXiv preprint arXiv:1606.00341 (2016).

[40]

Praveen Kumar Yadav, Arash Shafiei, and Wei Tsang Ooi. 2017. Quetra: A queuing theory approach to DASH rate adaptation. In Proceedings of the 25th ACM International Conference on Multimedia. 1130--1138.

Digital Library

[41]

Francis Y Yan, Hudson Ayers, Chenzhi Zhu, Sadjad Fouladi, James Hong, Keyi Zhang, Philip Levis, and Keith Winstein. 2020. Learning in situ: a randomized experiment in video streaming. In 17th {USENIX} Symposium on Networked Systems Design and Implementation ({NSDI} 20). 495--511.

[42]

Xiaoqi Yin, Abhishek Jindal, Vyas Sekar, and Bruno Sinopoli. 2015. A control-theoretic approach for dynamic adaptive video streaming over HTTP. In Proceedings of the 2015 ACM Conference on SIGCOMM. 325--338.

Digital Library

[43]

Alex Zambelli. 2009. Smooth Streaming Technical Overview. Retrieved Nov 21, 2020 from https://docs.microsoft.com/en-us/iis/media/on-demand-smooth-streaming/smooth-streaming-technical-overview

Cited By

Turkkan BDai TRaman AKosar TChen CBulut MZola JSow D(2024)GreenABR+: Generalized Energy-Aware Adaptive Bitrate StreamingACM Transactions on Multimedia Computing, Communications, and Applications10.1145/364989820:9(1-24)Online publication date: 5-Mar-2024
https://dl.acm.org/doi/10.1145/3649898
Hodžić KCosovic MMrdovic SQuinlan JRaca D(2024)DashReStreamer: Framework for Creation of Impaired Video Clips under Realistic Network ConditionsACM Transactions on Multimedia Computing, Communications, and Applications10.1145/3640016Online publication date: 9-Jan-2024
https://dl.acm.org/doi/10.1145/3640016
Jing WLiu CCai HWen XLu ZWang ZZhang H(2024)MEC-Based Super-Resolution Enhanced Adaptive Video Streaming Optimization for Mobile Networks With Satellite BackhaulIEEE Transactions on Network and Service Management10.1109/TNSM.2024.337769321:3(2977-2991)Online publication date: Jun-2024
https://doi.org/10.1109/TNSM.2024.3377693
Show More Cited By

Index Terms

Understanding quality of experience of heuristic-based HTTP adaptive bitrate algorithms
1. Information systems
  1. Information systems applications
    1. Multimedia information systems
      1. Multimedia streaming

Recommendations

End-to-end Quality of Experience Evaluation for HTTP Adaptive Streaming
MM '21: Proceedings of the 29th ACM International Conference on Multimedia

Exponential growth in multimedia streaming traffic over the Internet motivates the research and further investigation of the user's perceived quality of such services. Enhancement of experienced quality by the users becomes more substantial when service ...
Quality of Experience of adaptive video streaming

The usage of HTTP adaptive streaming (HAS) has become widely spread in multimedia services. Because it allows the service providers to improve the network resource utilization and user's Quality of Experience (QoE). Using this technology, the video ...
H2BR: an HTTP/2-based retransmission technique to improve the QoE of adaptive video streaming
PV '20: Proceedings of the 25th ACM Workshop on Packet Video

HTTP-based Adaptive Streaming (HAS) plays a key role in over-the-top video streaming. It contributes towards reducing the rebuffering duration of video playout by adapting the video quality to the current network conditions. However, it incurs ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

NOSSDAV '21: Proceedings of the 31st ACM Workshop on Network and Operating Systems Support for Digital Audio and Video

July 2021

128 pages

ISBN:9781450384353

DOI:10.1145/3458306

Program Chairs:
Andra Lutu
Telefonica, Spain
,
Gwendal Simon
Huawei, France
,
Mylene C. Q. Farias
University of Brasilia, Brazil

Copyright © 2021 Owner/Author.

Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

Sponsors

SIGMM: ACM Special Interest Group on Multimedia

In-Cooperation

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 02 July 2021

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Singapore Ministry of Education Academic Research
Christian Doppler Laboratory ATHENA

Conference

MMSys '21

Sponsor:

SIGMM

MMSys '21: 12th ACM Multimedia Systems Conference

September 28 - October 1, 2021

Istanbul, Turkey

Acceptance Rates

NOSSDAV '21 Paper Acceptance Rate 15 of 52 submissions, 29%;

Overall Acceptance Rate 118 of 363 submissions, 33%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

8
Total Citations
View Citations
713
Total Downloads

Downloads (Last 12 months)236
Downloads (Last 6 weeks)28

Reflects downloads up to 14 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

Turkkan BDai TRaman AKosar TChen CBulut MZola JSow D(2024)GreenABR+: Generalized Energy-Aware Adaptive Bitrate StreamingACM Transactions on Multimedia Computing, Communications, and Applications10.1145/364989820:9(1-24)Online publication date: 5-Mar-2024
https://dl.acm.org/doi/10.1145/3649898
Hodžić KCosovic MMrdovic SQuinlan JRaca D(2024)DashReStreamer: Framework for Creation of Impaired Video Clips under Realistic Network ConditionsACM Transactions on Multimedia Computing, Communications, and Applications10.1145/3640016Online publication date: 9-Jan-2024
https://dl.acm.org/doi/10.1145/3640016
Jing WLiu CCai HWen XLu ZWang ZZhang H(2024)MEC-Based Super-Resolution Enhanced Adaptive Video Streaming Optimization for Mobile Networks With Satellite BackhaulIEEE Transactions on Network and Service Management10.1109/TNSM.2024.337769321:3(2977-2991)Online publication date: Jun-2024
https://doi.org/10.1109/TNSM.2024.3377693
Margetis GTsagkatakis GStamou SStephanidis C(2023)Integrating Visual and Network Data with Deep Learning for Streaming Video Quality AssessmentSensors10.3390/s2308399823:8(3998)Online publication date: 14-Apr-2023
https://doi.org/10.3390/s23083998
Tashtarian FBentaleb AAmirpour HTaraghi BTimmerer CHellwagner HZimmermann R(2023)LALISA: Adaptive Bitrate Ladder Optimization in HTTP-based Adaptive Live StreamingNOMS 2023-2023 IEEE/IFIP Network Operations and Management Symposium10.1109/NOMS56928.2023.10154347(1-9)Online publication date: 8-May-2023
https://doi.org/10.1109/NOMS56928.2023.10154347
Taraghi BHellwagner HTimmerer C(2023)LLL-CAdViSE: Live Low-Latency Cloud-Based Adaptive Video Streaming Evaluation FrameworkIEEE Access10.1109/ACCESS.2023.325709911(25723-25734)Online publication date: 2023
https://doi.org/10.1109/ACCESS.2023.3257099
Taraghi BHaack STimmerer C(2022)Towards Better Quality of Experience in HTTP Adaptive Streaming2022 16th International Conference on Signal-Image Technology & Internet-Based Systems (SITIS)10.1109/SITIS57111.2022.00096(608-615)Online publication date: Oct-2022
https://doi.org/10.1109/SITIS57111.2022.00096
Xiong GDai Z(2022)Proactive Edge Computing for Video Streaming: A Mutual Conversion Model for Varying Requirements on Representations2022 IEEE 8th International Conference on Computer and Communications (ICCC)10.1109/ICCC56324.2022.10065666(1893-1898)Online publication date: 9-Dec-2022
https://doi.org/10.1109/ICCC56324.2022.10065666
Taraghi BNguyen MAmirpour HTimmerer C(2021)Intense: In-Depth Studies on Stall Events and Quality Switches and Their Impact on the Quality of Experience in HTTP Adaptive StreamingIEEE Access10.1109/ACCESS.2021.31076199(118087-118098)Online publication date: 2021
https://doi.org/10.1109/ACCESS.2021.3107619

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents