research-article

Investigating the Correlation between Performance Scores and Energy Consumption of Mobile Web Apps

Authors:

Kwame Chan-Jong-Chu,

Miguel Morales Exposito,

Sanjay Sheombar,

Christian Valladares,

Olivier Philippot,

Eoin Martino Grua,

Ivano MalavoltaAuthors Info & Claims

EASE '20: Proceedings of the 24th International Conference on Evaluation and Assessment in Software Engineering

Pages 190 - 199

https://doi.org/10.1145/3383219.3383239

Published: 17 April 2020 Publication History

Abstract

Context. Developers have access to tools like Google Lighthouse to assess the performance of web apps and to guide the adoption of development best practices. However, when it comes to energy consumption of mobile web apps, these tools seem to be lacking. Goal. This study investigates on the correlation between the performance scores produced by Lighthouse and the energy consumption of mobile web apps.

Method. We design and conduct an empirical experiment where 21 real mobile web apps are (i) analyzed via the Lighthouse performance analysis tool and (ii) measured on an Android device running a software-based energy profiler. Then, we statistically assess how energy consumption correlates with the obtained performance scores and carry out an effect size estimation.

Results. We discover a statistically significant negative correlation between performance scores and the energy consumption of mobile web apps (with medium to large effect sizes), implying that an increase of the performance score tend to lead to a decrease of energy consumption.

Conclusions. We recommend developers to strive to improve the performance level of their mobile web apps, as this can also have a positive impact on their energy consumption on Android devices.

References

[1]

Statscounter, "Desktop vs mobile vs tablet market share worldwide," GlobalStats, Tech. Rep., August 2018. [Online]. Available: http://gs.statcounter.com/platform-market-share/desktop-mobile-tablet

[2]

I. Malavolta, "Beyond native apps: web technologies to the rescue!(keynote)," in Proceedings of the 1st International Workshop on Mobile Development. ACM, 2016, pp. 1--2.

[3]

"Native, Web or Hybrid Mobile-app Development," White paper, IBM Corporation, April 2012, document Number: WSW14182USEN.

[4]

M. E. Joorabchi, A. Mesbah, and P. Kruchten, "Real challenges in mobile app development," in International Symposium on Empirical Software Engineering and Measurement. IEEE, 2013, pp. 15--24.

[5]

V. A. J. C. Sam Meder. (2017, March) Driving user growth with performance improvements. [Online]. Available: https://medium.com/@Pinterest_Engineering/driving-user-growth-with-performance-improvements-cfc50dafadd7

[6]

G. Pinto and F. Castor, "Energy efficiency: A new concern for application software developers," ACM, Tech. Rep., 2018. [Online]. Available: http://gustavopinto.org/lost+found/cacm2017.pdf

[7]

M. Nagappan and E. Shihab, "Future trends in software engineering research for mobile apps." in FOSE@ SANER, 2016, pp. 21--32.

[8]

N. Thiagarajan, G. Aggarwal, A. Nicoara, D. Boneh, and J. P. Singh, "Who killed my battery?: analyzing mobile browser energy consumption," in Proceedings of the 21st international conference on World Wide Web. ACM, 2012, pp. 41--50.

[9]

M. K. A. O. K. B. J. Miller. (2018, August) Measure performance with the rail model. [Online]. Available: https://developers.google.com/web/fundamentals/performance/rail

[10]

Google. (2019, February) Lighthouse scoring documentation. [Online]. Available: https://github.com/GoogleChrome/lighthouse/blob/master/docs/scoring.md

[11]

S. Mahajan, N. Abolhassani, P. McMinn, and W. G. Halfond, "Automated repair of mobile friendly problems in web pages," in Proceedings of the 40th International Conference on Software Engineering. ACM, 2018, pp. 140--150.

[12]

F. S. Ocariza Jr, K. Pattabiraman, and B. Zorn, "Javascript errors in the wild: An empirical study," in 22nd International Symposium on Software Reliability Engineering (ISSRE). IEEE, 2011, pp. 100--109.

[13]

(2018, october) Lighthouse batch reporter. [Online]. Available: https://www.npmjs.com/package/lighthouse-batch

[14]

C. Wohlin, P. Runeson, M. Höst, M. C. Ohlsson, B. Regnell, and A. Wesslén, Experimentation in software engineering. Springer Science & Business Media, 2012.

[15]

Google. (2018) Lighthouse v3 scoring guide. [Online]. Available: https://developers.google.com/web/tools/lighthouse/v3/scoring#perf-scoring

[16]

S. S. Shapiro and M. B. Wilk, "An analysis of variance test for normality (complete samples)," Biometrika, vol. 52, no. 3/4, pp. 591--611, 1965.

[17]

S. Vegas, "Analyzing software engineering experiments: everything you always wanted to know but were afraid to ask," in Proceedings of the 39th International Conference on Software Engineering Companion. IEEE Press, 2017, pp. 513--514.

[18]

W. H. Kruskal and W. A. Wallis, "Use of ranks in one-criterion variance analysis," Journal of the American statistical Association, vol. 47, no. 260, pp. 583--621, 1952.

[19]

O. J. Dunn, "Multiple comparisons among means," Journal of the American statistical association, vol. 56, no. 293, pp. 52--64, 1961.

[20]

C. Bonferroni, "Teoria statistica delle classi e calcolo delle probabilita," Pubblicazioni del R Istituto Superiore di Scienze Economiche e Commericiali di Firenze, vol. 8, pp. 3--62, 1936.

[21]

N. Cliff, "Dominance statistics: Ordinal analyses to answer ordinal questions." Psychological bulletin, vol. 114, no. 3, p. 494, 1993.

[22]

R. J. Grissom and J. J. Kim, Effect sizes for research: A broad practical approach. Lawrence Erlbaum Associates Publishers, 2005.

[23]

Spearman Rank Correlation Coefficient. New York, NY: Springer New York, 2008, pp. 502--505. [Online]. Available: https://doi.org/10.1007/978-0-387-32833-1_379

[24]

I. Malavolta, G. Procaccianti, P. Noorland, and P. Vukmirović, "Assessing the impact of service workers on the energy efficiency of progressive web apps," in Proceedings of the 4th International Conference on Mobile Software Engineering and Systems. IEEE Press, 2017, pp. 35--45.

[25]

J. Nejati and A. Balasubramanian, "An in-depth study of mobile browser performance," in Proceedings of the 25th International Conference on World Wide Web. International World Wide Web Conferences Steering Committee, 2016, pp. 1305--1315.

[26]

J. Vesuna, C. Scott, M. Buettner, M. Piatek, A. Krishnamurthy, and S. Shenker, "Caching Doesn't Improve Mobile Web Performance (Much)," in 2016 USENIX Annual Technical Conference (USENIX ATC 16). USENIX Association, 2016.

[27]

L. Cruz and R. Abreu, "Performance-based guidelines for energy efficient mobile applications," in Mobile Software Engineering and Systems (MOBILESoft), 2017 IEEE/ACM 4th International Conference on. IEEE, 2017, pp. 46--57.

[28]

F. Palomba, D. Di Nucci, A. Panichella, A. Zaidman, and A. De Lucia, "On the impact of code smells on the energy consumption of mobile applications," Information and Software Technology, vol. 105, pp. 43--55, 2019.

[29]

S. R. Choudhary, A. Gorla, and A. Orso, "Automated test input generation for android: Are we there yet?(e)," in 2015 30th IEEE/ACM International Conference on Automated Software Engineering (ASE). IEEE, 2015, pp. 429--440.

[30]

F. Palomba, D. Di Nucci, A. Panichella, A. Zaidman, and A. De Lucia, "Lightweight detection of android-specific code smells: The adoctor project," in 24th International Conference on Software Analysis, Evolution and Reengineering (SANER). IEEE, 2017, pp. 487--491.

[31]

D. Di Nucci, F. Palomba, A. Prota, A. Panichella, A. Zaidman, and A. De Lucia, "Petra: a software-based tool for estimating the energy profile of android applications," in Proceedings of the 39th International Conference on Software Engineering Companion. IEEE, 2017, pp. 3--6.

[32]

M. Gottschalk, J. Jelschen, and A. Winter, "Saving energy on mobile devices by refactoring," in EnviroInfo, 2014, pp. 437--444.

[33]

A. Rodriguez, M. Longo, and A. Zunino, "Using bad smell-driven code refactorings in mobile applications to reduce battery usage," Simposio Argentino de, 2015.

Cited By

Llovido JBrogada MMaceda LAbisado M(2024)AppTest: Assessing the Usability and Performance Efficiency of BOSESKO for Digital Participation2024 26th International Conference on Advanced Communications Technology (ICACT)10.23919/ICACT60172.2024.10471927(241-247)Online publication date: 4-Feb-2024
https://doi.org/10.23919/ICACT60172.2024.10471927
Canay ÖKocabıçak Ü(2024)CAWALInformation Processing and Management: an International Journal10.1016/j.ipm.2023.10361761:3Online publication date: 2-Jul-2024
https://dl.acm.org/doi/10.1016/j.ipm.2023.103617
Malavolta INirghin KScoccia GRomano SLombardi SScanniello GLago P(2023)JavaScript Dead Code Identification, Elimination, and Empirical AssessmentIEEE Transactions on Software Engineering10.1109/TSE.2023.326784849:7(3692-3714)Online publication date: Jul-2023
https://doi.org/10.1109/TSE.2023.3267848
Show More Cited By

Index Terms

Investigating the Correlation between Performance Scores and Energy Consumption of Mobile Web Apps
1. Information systems
  1. World Wide Web
    1. Web applications
2. Software and its engineering
  1. Software organization and properties
    1. Extra-functional properties
      1. Software performance

Recommendations

Caveats in Eliciting Mobile App Requirements
EASE '20: Proceedings of the 24th International Conference on Evaluation and Assessment in Software Engineering

Factors such as app stores or platform choices heavily affect functional and non-functional mobile app requirements. We surveyed 45 companies and interviewed ten experts to explore how factors that impact mobile app requirements are understood by ...
A comparative study on the energy consumption of Progressive Web Apps
Abstract
Progressive Web Apps (PWAs) are a promising approach for developing mobile apps, especially when developing apps for multiple mobile systems. As mobile devices are limited with respect to battery capacity, developers should keep the ...
Highlights
- Thorough analysis of the energy consumption of sample apps and real-world apps.
Building a Secure Cross Platform Enterprise Service Mobile Apps Using HTML5
NBIS '15: Proceedings of the 2015 18th International Conference on Network-Based Information Systems

With countless smartphones and mobile devices being widely used, users can have community interaction, recreation, and can even make a purchase through their mobile devices. Therefore, the mobile network service is becoming increasing irreplaceable and ...

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

EASE '20: Proceedings of the 24th International Conference on Evaluation and Assessment in Software Engineering

April 2020

544 pages

ISBN:9781450377317

DOI:10.1145/3383219

General Chairs:
Jingyue Li
Norwegian University of Science and Technology, Norway
,
Letizia Jaccheri
Norwegian University of Science and Technology, Norway
,
Program Chairs:
Torgeir Dingsøyr
SINTEF Digital and Norwegian University of Science and Technology, Norway
,
Ruzanna Chitchyan
University of Bristol, UK

Copyright © 2020 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

In-Cooperation

NTNU: Norwegian University of Science and Technology

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 17 April 2020

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Conference

EASE '20

EASE '20: Evaluation and Assessment in Software Engineering

April 15 - 17, 2020

Trondheim, Norway

Acceptance Rates

Overall Acceptance Rate 71 of 232 submissions, 31%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

13
Total Citations
View Citations
407
Total Downloads

Downloads (Last 12 months)56
Downloads (Last 6 weeks)5

Reflects downloads up to 15 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

Llovido JBrogada MMaceda LAbisado M(2024)AppTest: Assessing the Usability and Performance Efficiency of BOSESKO for Digital Participation2024 26th International Conference on Advanced Communications Technology (ICACT)10.23919/ICACT60172.2024.10471927(241-247)Online publication date: 4-Feb-2024
https://doi.org/10.23919/ICACT60172.2024.10471927
Canay ÖKocabıçak Ü(2024)CAWALInformation Processing and Management: an International Journal10.1016/j.ipm.2023.10361761:3Online publication date: 2-Jul-2024
https://dl.acm.org/doi/10.1016/j.ipm.2023.103617
Malavolta INirghin KScoccia GRomano SLombardi SScanniello GLago P(2023)JavaScript Dead Code Identification, Elimination, and Empirical AssessmentIEEE Transactions on Software Engineering10.1109/TSE.2023.326784849:7(3692-3714)Online publication date: Jul-2023
https://doi.org/10.1109/TSE.2023.3267848
Dornauer BFelderer M(2023)Energy-Saving Strategies for Mobile Web Apps and their Measurement: Results from a Decade of Research2023 IEEE/ACM 10th International Conference on Mobile Software Engineering and Systems (MOBILESoft)10.1109/MOBILSoft59058.2023.00017(75-86)Online publication date: May-2023
https://doi.org/10.1109/MOBILSoft59058.2023.00017
Horn RLahnaoui AReinoso EPeng SIsakov VIslam TMalavolta I(2023)Native vs Web Apps: Comparing the Energy Consumption and Performance of Android Apps and their Web Counterparts2023 IEEE/ACM 10th International Conference on Mobile Software Engineering and Systems (MOBILESoft)10.1109/MOBILSoft59058.2023.00013(44-54)Online publication date: May-2023
https://doi.org/10.1109/MOBILSoft59058.2023.00013
Ngaogate W(2023)Handling Various Conditions in a Web Service Client's Method by Using the Visitor Design Pattern2023 27th International Computer Science and Engineering Conference (ICSEC)10.1109/ICSEC59635.2023.10329704(341-347)Online publication date: 14-Sep-2023
https://doi.org/10.1109/ICSEC59635.2023.10329704
McGill TBamgboye OLiu XKalutharage C(2023)Towards Improving Accessibility of Web Auditing with Google Lighthouse2023 IEEE 47th Annual Computers, Software, and Applications Conference (COMPSAC)10.1109/COMPSAC57700.2023.00246(1594-1599)Online publication date: Jun-2023
https://doi.org/10.1109/COMPSAC57700.2023.00246
Wu ZChen XLee S(2023)A systematic literature review on Android-specific smellsJournal of Systems and Software10.1016/j.jss.2023.111677201:COnline publication date: 1-Jul-2023
https://dl.acm.org/doi/10.1016/j.jss.2023.111677
Şanlıalp İÖztürk MYiğit T(2022)Energy Efficiency Analysis of Code Refactoring Techniques for Green and Sustainable Software in Portable DevicesElectronics10.3390/electronics1103044211:3(442)Online publication date: 1-Feb-2022
https://doi.org/10.3390/electronics11030442
Sehgal RMehrotra DNagpal RSharma R(2022)Green softwareJournal of King Saud University - Computer and Information Sciences10.1016/j.jksuci.2020.10.02234:7(4635-4643)Online publication date: 1-Jul-2022
https://dl.acm.org/doi/10.1016/j.jksuci.2020.10.022
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents