research-article

KeenTune: Automated Tuning Tool for Cloud Application Performance Testing and Optimization

Authors:

Heyuan ShiAuthors Info & Claims

ISSTA 2023: Proceedings of the 32nd ACM SIGSOFT International Symposium on Software Testing and Analysis

Pages 1487 - 1490

https://doi.org/10.1145/3597926.3604920

Published: 13 July 2023 Publication History

Abstract

The performance testing and optimization of cloud applications is challenging, because manual tuning of cloud computing stacks is tedious and automated tuning tools are rare used for cloud services. To address this issue, we introduce KeenTune, an automated tuning tool designed to optimize application performance and facilitate performance testing. KeenTune is a lightweight and flexible tool that can be deployed with to-be-tuned applications with negligible impact on their performance. Specifically, KeenTune uses a surrogate model that can be implemented with machine learning models to filter out less relevant parameters for efficient tuning. Our empirical evaluation shows that KeenTune significantly enhances the throughput performance of Nginx web servers, resulting in performance improvements of up to 90.43% and 117.23% in certain cases. This study highlights the benefits of using KeenTune for achieving efficient and effective performance testing of cloud applications. The video and source code for KeenTune are provided as supplementary materials.

References

[1]

Dana Van Aken, Andrew Pavlo, Geoffrey J. Gordon, and Bohan Zhang. 2017. Automatic Database Management System Tuning Through Large-scale Machine Learning. In SIGMOD Conference. ACM, 1009–1024.

[2]

Alexey Kopytov. 2017. sysbench. https://github.com/akopytov/sysbench

[3]

Alibaba. 2019. Alibaba Cloud Linux. https://github.com/alibaba/cloud-kernel

[4]

Alibaba Group. 2015. ECS. https://www.alibabacloud.com/product/ecs

[5]

James Bergstra, Rémi Bardenet, Yoshua Bengio, and Balázs Kégl. 2011. Algorithms for Hyper-Parameter Optimization. In NeurIPS. 2546–2554.

[6]

Songyun Duan, Vamsidhar Thummala, and Shivnath Babu. 2009. Tuning Database Configuration Parameters with iTuned. Proc. VLDB Endow., 2, 1 (2009), 1246–1257.

Digital Library

[7]

Li. G., X. Zhou, X. Li, and B. Gao. 2019. QTune: A Query-Aware Database Tuning System with Deep Reinforcement Learning. Proc. VLDB Endow., 12 (2019), 2118–2130.

Digital Library

[8]

Hewlett Packard. 2005. netperf. https://github.com/HewlettPackard/netperf

[9]

Scott M. Lundberg and Su-In Lee. 2017. A Unified Approach to Interpreting Model Predictions. In NeurIPS’17, Long Beach, CA, USA, December 4-9, 2017. 4765–4774.

[10]

NGINX, Inc. 2020. nginx. https://nginx.org/en/

[11]

Huawei OpenEuler. 2019. A-Tune. https://gitee.com/openeuler/A-Tune

[12]

M. Sazanovich, A. Nikolskaya, Y. Belousov, and A. Shpilman. 2020. Solving Black-Box Optimization Challenge via Learning Search Space Partition for Local Bayesian Optimization. In NeurIPS (Competition and Demos). 133, 77–85.

[13]

Jasper Snoek, Hugo Larochelle, and Ryan P. Adams. 2012. Practical Bayesian Optimization of Machine Learning Algorithms. In NeurIPS. 2960–2968.

Digital Library

[14]

Jasper Snoek, Oren Rippel, Kevin Swersky, Ryan Kiros, Nadathur Satish, Narayanan Sundaram, Md. Mostofa Ali Patwary, Prabhat, and Ryan P. Adams. 2015. Scalable Bayesian Optimization Using Deep Neural Networks. In ICML (JMLR Workshop and Conference Proceedings, Vol. 37). JMLR.org, 2171–2180.

[15]

Kevin Swersky, Jasper Snoek, and Ryan Prescott Adams. 2013. Multi-Task Bayesian Optimization. In NeurIPS. 2004–2012.

[16]

R. Wang, Q. Wang, Y. Hu, H. Shi, Y. Shen, Y. Zhan, Y. Fu, Z. Liu, X. Shi, and Y. Jiang. 2022. Industry practice of configuration auto-tuning for cloud applications and services. In ESEC/SIGSOFT FSE. ACM, 1555–1565.

[17]

Will Glozer. 2013. wrk. https://github.com/wg/wrk

[18]

Ji Zhang, Yu Liu, Ke Zhou, Guoliang Li, Zhili Xiao, Bin Cheng, Jiashu Xing, Yangtao Wang, Tianheng Cheng, Li Liu, Minwei Ran, and Zekang Li. 2019. An End-to-End Automatic Cloud Database Tuning System Using Deep Reinforcement Learning. In SIGMOD Conference. ACM, 415–432.

[19]

Xinyi Zhang, Hong Wu, Zhuo Chang, Shuowei Jin, Jian Tan, Feifei Li, Tieying Zhang, and Bin Cui. 2021. ResTune: Resource Oriented Tuning Boosted by Meta-Learning for Cloud Databases. In SIGMOD Conference. ACM, 2102–2114.

Digital Library

Cited By

Shen YLiu JXu YSun HWang MGuan NShi HJiang YChristakis MPradel M(2024)Enhancing ROS System Fuzzing through Callback TracingProceedings of the 33rd ACM SIGSOFT International Symposium on Software Testing and Analysis10.1145/3650212.3652111(76-87)Online publication date: 11-Sep-2024
https://dl.acm.org/doi/10.1145/3650212.3652111
Huang HWang HRao JWu SFan HYu CJin HSuo KPan L(2024)vKernel: Enhancing Container Isolation via Private Code and DataIEEE Transactions on Computers10.1109/TC.2024.338398873:7(1711-1723)Online publication date: Jul-2024
https://doi.org/10.1109/TC.2024.3383988

Index Terms

KeenTune: Automated Tuning Tool for Cloud Application Performance Testing and Optimization
1. Software and its engineering
  1. Software notations and tools
    1. Software maintenance tools
  2. Software organization and properties
    1. Extra-functional properties
      1. Software performance

Recommendations

Automated Model-Based Performance Testing for PaaS Cloud Services
COMPSACW '14: Proceedings of the 2014 IEEE 38th International Computer Software and Applications Conference Workshops

Recently, cloud computing has become popular for its unique advantages. Many applications have been migrated to cloud as web services. However, evaluating the performance of cloud services is non-trivial. Performance testing is one of the dominant ...
Automated GUI performance testing

A significant body of prior work has devised approaches for automating the functional testing of interactive applications. However, little work exists for automatically testing their performance. Performance testing imposes additional requirements upon ...
Automated WAIT for Cloud-Based Application Testing
ICSTW '14: Proceedings of the 2014 IEEE International Conference on Software Testing, Verification, and Validation Workshops

Cloud computing is causing a paradigm shift in the provision and use of software. This has changed the way of obtaining, managing and delivering computing services and solutions. Similarly, it has brought new challenges to software testing. A particular ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

ISSTA 2023: Proceedings of the 32nd ACM SIGSOFT International Symposium on Software Testing and Analysis

July 2023

1554 pages

ISBN:9798400702211

DOI:10.1145/3597926

General Chair:
René Just
University of Washington, USA
,
Program Chair:
Gordon Fraser
University of Passau, Germany

Copyright © 2023 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 13 July 2023

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

ISSTA '23

Sponsor:

SIGSOFT

ISSTA '23: 32nd ACM SIGSOFT International Symposium on Software Testing and Analysis

July 17 - 21, 2023

WA, Seattle, USA

Acceptance Rates

Overall Acceptance Rate 58 of 213 submissions, 27%

Upcoming Conference

ISSTA '25

Sponsor:
sigsoft

34th ACM SIGSOFT International Symposium on Software Testing and Analysis

June 25 - 28, 2025

Trondheim , Norway

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

1
Total Citations
View Citations
259
Total Downloads

Downloads (Last 12 months)211
Downloads (Last 6 weeks)33

Reflects downloads up to 21 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

Shen YLiu JXu YSun HWang MGuan NShi HJiang YChristakis MPradel M(2024)Enhancing ROS System Fuzzing through Callback TracingProceedings of the 33rd ACM SIGSOFT International Symposium on Software Testing and Analysis10.1145/3650212.3652111(76-87)Online publication date: 11-Sep-2024
https://dl.acm.org/doi/10.1145/3650212.3652111
Huang HWang HRao JWu SFan HYu CJin HSuo KPan L(2024)vKernel: Enhancing Container Isolation via Private Code and DataIEEE Transactions on Computers10.1109/TC.2024.338398873:7(1711-1723)Online publication date: Jul-2024
https://doi.org/10.1109/TC.2024.3383988

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents