skip to main content
10.1145/3308558.3313606acmotherconferencesArticle/Chapter ViewAbstractPublication PagesthewebconfConference Proceedingsconference-collections
research-article

Web Experience in Mobile Networks: Lessons from Two Million Page Visits

Published: 13 May 2019 Publication History

Abstract

Measuring and characterizing web page performance is a challenging task. When it comes to the mobile world, the highly varying technology characteristics coupled with the opaque network configuration make it even more difficult. Aiming at reproducibility, we present a large scale empirical study of web page performance collected in eleven commercial mobile networks spanning four countries. By digging into measurement from nearly two million web browsing sessions, we shed light on the impact of different web protocols, browsers, and mobile technologies on the web performance. We find that the impact of mobile broadband access is sizeable. For example, the median page load time using mobile broadband increases by a third compared to wired access. Mobility clearly stresses the system, with handover causing the most evident performance penalties. Contrariwise, our measurements show that the adoption of HTTP/2 and QUIC has practically negligible impact. To understand the intertwining of all parameters, we adopt state-of-the-art statistical methods to identify the significance of different factors on the web performance. Our analysis confirms the importance of access technology and mobility context as well as webpage composition and browser. Our work highlights the importance of large-scale measurements. Even with our controlled setup, the complexity of the mobile web ecosystem is challenging to untangle. For this, we are releasing the dataset as open data for validation and further research.

References

[1]
{n. d.}. A Primer for Web Performance Timing APIs. https://w3c.github.io/perf-timing-primer/. {Online; accessed 20-Feb-2019}.
[2]
{n. d.}. Above-the-fold time (AFT): Useful, but not yet a substitute for user-centric analysis. https://goo.gl/gWmj3r. {Online; accessed 20-Feb-2019}.
[3]
{n. d.}. Alexa: The top 500 sites on the web. https://www.alexa.com/topsites. {Online; accessed 20-Feb-2019}.
[4]
{n. d.}. Different SpeedIndex Measurements. https://bit.ly/2I87znK. {Online; accessed 20-Feb-2019}.
[5]
{n. d.}. MONROE-Browsertime. https://bit.ly/2SejKPo. {Online; accessed 20-Feb-2019}.
[6]
{n. d.}. A simple, client-side approach to measuring Speed Index. https://deanhume.com/a-simple-client-side-approach-to-measuring-speed-index/. {Online; accessed 20-Feb-2019}.
[7]
{n. d.}. sitespeedio/browsertime: Your browser, your page, your scripts!https://github.com/sitespeedio/browsertime. {Online; accessed 20-Feb-2019}.
[8]
{n. d.}. Speed Index - how it works and what it means. https://www.nccgroup.trust/uk/about-us/newsroom-and-events/blogs/2015/june/speed-index--how-it-works-and-what-it-means/. {Online; accessed 07-Aug-2018}.
[9]
{n. d.}. SpeedIndex measurements from the field. https://github.com/WPO-Foundation/RUM-SpeedIndex. {Online; accessed 20-Feb-2019}.
[10]
{n. d.}. Webpage Test. https://www.webpagetest.org. {Online; accessed 20-Feb-2019}.
[11]
{n. d.}. WebPagetest documentation. https://sites.google.com/a/webpagetest.org/docs/using-webpagetest/metrics/speed-index. {Online; accessed 20-Feb-2019}.
[12]
{n. d.}. Xvfb manul page. https://goo.gl/EPBtpt. {Online; accessed 20-Feb-2019}.
[13]
2014. Can SPDY really make the web faster?. In IFIP Networking Conference. 1-9.
[14]
Özgü Alay, Andra Lutu, Miguel Peon Quiros, Vincenzo Mancuso, thomas hirsch, Kristian Evensen, Audun Fosselie Hansen, Stefan Alfredsson, jonas karlsson, Anna Brunström, ali safari Khatouni, M. Mellia, and Marco Ajmone Marsan. 2017. Experience: An Open Platform for Experimentation with Commercial Mobile Broadband Networks. Proc. ACM MobiCom '17(2017), 70-78.
[15]
Athula Balachandran, Vaneet Aggarwal, Emir Halepovic, Jeffrey Pang, Srinivasan Seshan, Shobha Venkataraman, and He Yan. 2014. Modeling Web Quality-of-experience on Cellular Networks. In Proceedings of the 20th Annual International Conference on Mobile Computing and Networking(MobiCom '14). ACM, New York, NY, USA, 213-224.
[16]
Dziugas Baltrunas, Ahmed Elmokashfi, and Amund Kvalbein. 2014. Measuring the Reliability of Mobile Broadband Networks. In Proceedings of the 2014 Conference on Internet Measurement Conference(IMC '14). ACM, New York, NY, USA, 45-58.
[17]
Dziugas Baltrunas, Ahmed Elmokashfi, Amund Kvalbein, and Özgü Alay. 2016. Investigating packet loss in mobile broadband networks under mobility. In 2016 IFIP Networking Conference (IFIP Networking) and Workshops. 225-233.
[18]
M. Belshe, R. Peon, and M. Thomson. 2015. Hypertext Transfer Protocol Version 2 (HTTP/2). RFC 7540 (Proposed Standard).
[19]
Prasenjeet Biswal and Omprakash Gnawali. 2016. Does QUIC Make the Web Faster?. In 2016 IEEE Global Communications Conference (GLOBECOM). 1-6.
[20]
Enrico Bocchi, Luca De Cicco, Marco Mellia, and Dario Rossi. 2017. The Web, the Users, and the MOS: Influence of HTTP/2 on User Experience. In Passive and Active Measurement - 18th International Conference, PAM. 47-59.
[21]
Enrico Bocchi, Luca De Cicco, and Dario Rossi. 2016. Measuring the Quality of Experience of Web users. ACM SIGCOMM Computer Communication Review46, 4 (2016), 8-13.
[22]
Enrico Bocchi, Ali Safari Khatouni, Stefano Traverso, Alessandro Finamore, Maurizio Munafo, Marco Mellia, and Dario Rossi. 2016. Statistical network monitoring: Methodology and application to carrier-grade NAT. Computer Networks107(2016), 20 - 35. Machine learning, data mining and Big Data frameworks for network monitoring and troubleshooting.
[23]
Gaetano Carlucci, Luca De Cicco, and Saverio Mascolo. 2015. HTTP over UDP: An Experimental Investigation of QUIC. In Proceedings of the 30th Annual ACM Symposium on Applied Computing(SAC '15). ACM, New York, NY, USA, 609-614.
[24]
Mallesham Dasari, Santiago Vargas, Arani Bhattacharya, Aruna Balasubramanian, Samir R. Das, and Michael Ferdman. 2018. Impact of Device Performance on Mobile Internet QoE. In Proceedings of the Internet Measurement Conference 2018(IMC '18). ACM, New York, NY, USA, 1-7.
[25]
H. de Saxce´, I. Oprescu, and Y. Chen. 2015. Is HTTP/2 really faster than HTTP/1.1?. In 2015 IEEE Conference on Computer Communications Workshops (INFOCOM WKSHPS). 293-299.
[26]
Jeffrey Erman, Vijay Gopalakrishnan, Rittwik Jana, and Kadangode K Ramakrishnan. 2015. Towards a SPDY'er mobile web?IEEE/ACM Transactions on Networking23, 6 (Dec 2015), 2010-2023.
[27]
R. Fielding 1999. Hypertext Transfer Protocol - HTTP/1.1. RFC 2616 (Draft Standard).
[28]
Qingzhu Gao, Prasenjit Dey, and Parvez Ahammad. 2017. Perceived Performance of Top Retail Webpages In the Wild: Insights from Large-scale Crowdsourcing of Above-the-Fold QoE. In Proceedings of the Workshop on QoE-based Analysis and Management of Data Communication Networks(Internet QoE '17). ACM, New York, NY, USA, 13-18.
[29]
Utkarsh Goel, Moritz Steiner, Mike P Wittie, Martin Flack, and Stephen Ludin. {n. d.}. Measuring What is Not Ours: A Tale of 3rd Party Performance. In ACM Passive and Active Measurements Conference (PAM). Springer, 142-155.
[30]
Utkarsh Goel, Moritz Steiner, Mike P. Wittie, Martin Flack, and Stephen Ludin. 2016. HTTP/2 Performance in Cellular Networks: Poster. In Proceedings of the 22nd Annual International Conference on Mobile Computing and Networking(MobiCom '16). ACM, New York, NY, USA, 433-434.
[31]
Emir Halepovic, Jeffrey Pang, and Oliver Spatscheck. 2012. Can You GET Me Now?: Estimating the Time-to-first-byte of HTTP Transactions with Passive Measurements. In Proceedings of the 2012 Internet Measurement Conference(IMC '12). ACM, New York, NY, USA, 115-122.
[32]
Junxian Huang, Feng Qian, Yihua Guo, Yuanyuan Zhou, Qiang Xu, Z. Morley Mao, Subhabrata Sen, and Oliver Spatscheck. 2013. An In-depth Study of LTE: Effect of Network Protocol and Application Behavior on Performance. In Proceedings of the ACM SIGCOMM 2013 Conference on SIGCOMM(SIGCOMM '13). ACM, New York, NY, USA, 363-374.
[33]
Google Inc.{n. d.}. SPDY: An experimental protocol for a faster web. https://www.chromium.org/spdy/spdy-whitepaper. {Online; accessed 20-Feb-2019}.
[34]
Arash Molavi Kakhki, Samuel Jero, David Choffnes, Cristina Nita-Rotaru, and Alan Mislove. 2017. Taking a Long Look at QUIC: An Approach for Rigorous Evaluation of Rapidly Evolving Transport Protocols. In Proceedings of the 2017 Internet Measurement Conference(IMC '17). ACM, New York, NY, USA, 290-303.
[35]
Conor Kelton, Jihoon Ryoo, Aruna Balasubramanian, and Samir R Das. 2017. Improving User Perceived Page Load Times Using Gaze. In 14th USENIX Symposium on Networked Systems Design and Implementation (NSDI 17). USENIX Association, Boston, MA, 545-559. https://www.usenix.org/conference/nsdi17/technical-sessions/presentation/kelton
[36]
Ali Safari Khatouni, Marco Mellia, Marco Ajmone Marsan, Stefan Alfredsson, Jonas Karlsson, Anna Brunstrom, Ozgu Alay, Andra Lutu, Cise Midoglu, and Vincenzo Mancuso. 2017. Speedtest-like Measurements in 3G/4G Networks: The MONROE Experience. In Teletraffic Congress (ITC 29), 2017 29th International, Vol. 1. IEEE, 169-177.
[37]
H. Kim, J. Lee, I. Park, H. Kim, D. Yi, and T. Hur. 2015. The upcoming new standard HTTP/2 and its impact on multi-domain websites. In 2015 17th Asia-Pacific Network Operations and Management Symposium (APNOMS). 530-533.
[38]
Zahir Koradia, Goutham Mannava, Aravindh Raman, Gaurav Aggarwal, Vinay Ribeiro, Aaditeshwar Seth, Sebastian Ardon, Anirban Mahanti, and Sipat Triukose. 2013. First Impressions on the State of Cellular Data Connectivity in India. In Proceedings of the 4th Annual Symposium on Computing for Development(ACM DEV-4 '13). ACM, New York, NY, USA, Article 3, 10 pages.
[39]
Adam Langley, Alistair Riddoch, Alyssa Wilk, Antonio Vicente, Charles Krasic, Dan Zhang, Fan Yang, Fedor Kouranov, Ian Swett, Janardhan Iyengar, Jeff Bailey, Jeremy Dorfman, Jim Roskind, Joanna Kulik, Patrik Westin, Raman Tenneti, Robbie Shade, Ryan Hamilton, Victor Vasiliev, Wan-Teh Chang, and Zhongyi Shi. 2017. The QUIC Transport Protocol: Design and Internet-Scale Deployment. In Proceedings of the Conference of the ACM Special Interest Group on Data Communication(SIGCOMM '17). ACM, New York, NY, USA, 183-196.
[40]
Y. Liu, Y. Ma, X. Liu, and G. Huang. 2016. Can HTTP/2 Really Help Web Performance on Smartphones?. In 2016 IEEE International Conference on Services Computing (SCC). 219-226.
[41]
Patrick Meenan. 2013. How Fast is Your Web Site?Queue11, 2, Article 60 (March 2013), 11 pages.
[42]
Pe´ter Megyesi, Zsolt Krämer, and Sándor Molnár. 2016. How quick is QUIC?. In 2016 IEEE International Conference on Communications (ICC). 1-6.
[43]
Javad Nejati and Aruna Balasubramanian. 2016. An In-depth Study of Mobile Browser Performance. In Proceedings of the 25th International Conference on World Wide Web(WWW '16). International World Wide Web Conferences Steering Committee, Republic and Canton of Geneva, Switzerland, 1305-1315.
[44]
Ashkan Nikravesh, Hongyi Yao, Shichang Xu, David Choffnes, and Z Morley Mao. 2015. Mobilyzer: An open platform for controllable mobile network measurements. In Proceedings of the 13th Annual International Conference on Mobile Systems, Applications, and Services. ACM, 389-404.
[45]
N. Oda and S. Yamaguchi. 2018. HTTP/2 performance evaluation with latency and packet losses. In 2018 15th IEEE Annual Consumer Communications Networking Conference (CCNC). 1-2.
[46]
Mohammad Rajiullah. 2015. Towards a Low Latency Internet: Understanding and Solutions. Ph.D. Dissertation. Karlstad University, Department of Mathematics and Computer Science.
[47]
Mohammad Rajiullah, AC Mohideen, Felix Weinrank, Raffaello Secchi, Gorry Fairhurst, and Anna Brunstrom. 2017. Understanding multistreaming for web traffic: An experimental study. In 2017 IFIP Networking Conference (IFIP Networking) and Workshops. 1-6.
[48]
E. Rescorla. 2000. HTTP Over TLS. RFC 2818 (Informational). http://www.ietf.org/rfc/rfc2818.txtUpdated by RFCs 5785, 7230.
[49]
John P. Rula, Vishnu Navda, Fabián E. Bustamante, Ranjita Bhagwan, and Saikat Guha. 2014. No ”One-size Fits All”: Towards a Principled Approach for Incentives in Mobile Crowdsourcing. In Proceedings of the 15th Workshop on Mobile Computing Systems and Applications(HotMobile '14). ACM, New York, NY, USA, Article 3, 5 pages.
[50]
Sayandeep Sen, Jongwon Yoon, Joshua Hare, Justin Ormont, and Suman Banerjee. 2011. Can They Hear Me Now?: A Case for a Client-assisted Approach to Monitoring Wide-area Wireless Networks. In Proceedings of the 2011 ACM SIGCOMM Conference on Internet Measurement Conference(IMC '11). ACM, New York, NY, USA, 99-116.
[51]
Muhammad Zubair Shafiq, Lusheng Ji, Alex X. Liu, Jeffrey Pang, Shobha Venkataraman, and Jia Wang. 2013. A First Look at Cellular Network Performance During Crowded Events. In Proceedings of the ACM SIGMETRICS/International Conference on Measurement and Modeling of Computer Systems(SIGMETRICS '13). ACM, New York, NY, USA, 17-28.
[52]
M. Zubair Shafiq, Lusheng Ji, Alex X. Liu, and Jia Wang. 2011. Characterizing and Modeling Internet Traffic Dynamics of Cellular Devices. In Proceedings of the ACM SIGMETRICS Joint International Conference on Measurement and Modeling of Computer Systems(SIGMETRICS '11). ACM, New York, NY, USA, 305-316.
[53]
Sierra-Wireless. {n. d.}. MC7455 miniPCI express (USB 3.0) modem: https://www.sierrawireless.com/products-and-solutions/embedded-solutions/products/mc7455/. ({n. d.}).
[54]
Joel Sommers and Paul Barford. 2012. Cell vs. WiFi: On the Performance of Metro Area Mobile Connections. In Proceedings of the 2012 Internet Measurement Conference(IMC '12). ACM, New York, NY, USA, 301-314.
[55]
Steve Souders. {n. d.}. Moving beyond window.onload(). https://goo.gl/rAeFNR. {Online; accessed 20-Feb-2019}.
[56]
Daniel Stenberg. 2014. HTTP2 Explained. SIGCOMM Computer Communication Review44, 3 (July 2014), 120-128.
[57]
Matteo Varvello, Kyle Schomp, David Naylor, Jeremy Blackburn, Alessandro Finamore, and Kostantina Papagiannaki. 2015. To HTTP/2, or not to HTTP/2, that is the question. arXiv preprint arXiv:1507.06562(2015).
[58]
Matteo Varvello, Kyle Schomp, David Naylor, Jeremy Blackburn, Alessandro Finamore, and Konstantina Papagiannaki. 2016. Is the Web HTTP/2 Yet?. In Passive and Active Measurement - 17th International Conference, PAM. 218-232.
[59]
J Vihervaara and T Alapaholuoma. 2018. The impact of HTTP/2 on the service efficiency of e-commerce websites. In 2018 41st International Convention on Information and Communication Technology, Electronics and Microelectronics (MIPRO). IEEE.
[60]
Xiao Sophia Wang, Aruna Balasubramanian, Arvind Krishnamurthy, and David Wetherall. 2014. How Speedy is SPDY?. In 11th USENIX Symposium on Networked Systems Design and Implementation (NSDI 14). USENIX Association, Seattle, WA, 387-399. https://www.usenix.org/conference/nsdi14/technical-sessions/wang
[61]
Greg White, Jean-François Mule´, and Dan Rice. 2012. Analysis of Google SPDY and TCP initcwnd. Cable Television Laboratories, Inc(2012).
[62]
Maarten Wijnants, Robin Marx, Peter Quax, and Wim Lamotte. 2018. HTTP/2 Prioritization and Its Impact on Web Performance. In Proceedings of the 2018 World Wide Web Conference(WWW '18). International World Wide Web Conferences Steering Committee, Republic and Canton of Geneva, Switzerland, 1755-1764.
[63]
Kyriakos Zarifis, Mark Holland, Manish Jain, Ethan Katz-Bassett, and Ramesh Govindan. 2016. Modeling HTTP/2 Speed from HTTP/1 Traces. Springer International Publishing, Cham, 233-247.
[64]
Torsten Zimmermann, Jan Rüth, Benedikt Wolters, and Oliver Hohlfeld. 2017. How HTTP/2 Pushes the Web: An Empirical Study of HTTP/2 Server Push. In 2017 IFIP Networking Conference (IFIP Networking) and Workshops.

Cited By

View all
  • (2024)Bottleneck Identification in Cloudified Mobile Networks based on Distributed TelemetryIEEE Transactions on Mobile Computing10.1109/TMC.2023.3312051(1-18)Online publication date: 2024
  • (2024)Dissecting the Applicability of HTTP/3 in Content Delivery Networks2024 IEEE 44th International Conference on Distributed Computing Systems (ICDCS)10.1109/ICDCS60910.2024.00091(936-946)Online publication date: 23-Jul-2024
  • (2024)Monitoring Web QoE in Satellite Networks from Passive Measurements2024 IEEE 21st Consumer Communications & Networking Conference (CCNC)10.1109/CCNC51664.2024.10454720(669-676)Online publication date: 6-Jan-2024
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences
WWW '19: The World Wide Web Conference
May 2019
3620 pages
ISBN:9781450366748
DOI:10.1145/3308558
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

In-Cooperation

  • IW3C2: International World Wide Web Conference Committee

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 13 May 2019

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. HTTP/2
  2. MONROE
  3. Measurements
  4. Mobile Broadband
  5. QUIC
  6. TCP
  7. WEB Browsing Experience

Qualifiers

  • Research-article
  • Research
  • Refereed limited

Conference

WWW '19
WWW '19: The Web Conference
May 13 - 17, 2019
CA, San Francisco, USA

Acceptance Rates

Overall Acceptance Rate 1,899 of 8,196 submissions, 23%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)38
  • Downloads (Last 6 weeks)2
Reflects downloads up to 14 Sep 2024

Other Metrics

Citations

Cited By

View all
  • (2024)Bottleneck Identification in Cloudified Mobile Networks based on Distributed TelemetryIEEE Transactions on Mobile Computing10.1109/TMC.2023.3312051(1-18)Online publication date: 2024
  • (2024)Dissecting the Applicability of HTTP/3 in Content Delivery Networks2024 IEEE 44th International Conference on Distributed Computing Systems (ICDCS)10.1109/ICDCS60910.2024.00091(936-946)Online publication date: 23-Jul-2024
  • (2024)Monitoring Web QoE in Satellite Networks from Passive Measurements2024 IEEE 21st Consumer Communications & Networking Conference (CCNC)10.1109/CCNC51664.2024.10454720(669-676)Online publication date: 6-Jan-2024
  • (2023)Website-Aware Protocol Confusion Network for Emergent HTTP/3 Website FingerprintingIEEE Transactions on Information Forensics and Security10.1109/TIFS.2023.326617318(2427-2439)Online publication date: 2023
  • (2023)Boosting TCP & QUIC Performance in mmWave, Terahertz, and Lightwave Wireless Networks: A SurveyIEEE Communications Surveys & Tutorials10.1109/COMST.2023.330182025:4(2862-2891)Online publication date: Dec-2024
  • (2023)Do you agree? Contrasting Google’s Core Web Vitals and the impact of cookie consent banners with actual web QoEQuality and User Experience10.1007/s41233-023-00058-38:1Online publication date: 30-Jun-2023
  • (2022)Causal impact of Android go on mobile web performanceProceedings of the 22nd ACM Internet Measurement Conference10.1145/3517745.3561456(113-129)Online publication date: 25-Oct-2022
  • (2022)FlexHTTPProceedings of the 2nd European Workshop on Machine Learning and Systems10.1145/3517207.3526972(29-36)Online publication date: 5-Apr-2022
  • (2022)Global mobile network aggregatorsProceedings of the 20th Annual International Conference on Mobile Systems, Applications and Services10.1145/3498361.3538942(183-195)Online publication date: 27-Jun-2022
  • (2022)Stateful Versus Stateless Selection of Edge or Cloud Servers Under Latency Constraints2022 IEEE 23rd International Symposium on a World of Wireless, Mobile and Multimedia Networks (WoWMoM)10.1109/WoWMoM54355.2022.00022(110-119)Online publication date: Jun-2022
  • Show More Cited By

View Options

Get Access

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

HTML Format

View this article in HTML Format.

HTML Format

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media