research-article

Industry practice of configuration auto-tuning for cloud applications and services

Authors:

Yu JiangAuthors Info & Claims

ESEC/FSE 2022: Proceedings of the 30th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering

Pages 1555 - 1565

https://doi.org/10.1145/3540250.3558962

Published: 09 November 2022 Publication History

Abstract

Auto-tuning attracts increasing attention in industry practice to optimize the performance of a system with many configurable parameters. It is particularly useful for cloud applications and services since they have complex system hierarchies and intricate knob correlations. However, existing tools and algorithms rarely consider practical problems such as workload pressure control, the support for distributed deployment, and expensive time costs, etc., which are utterly important for enterprise cloud applications and services. In this work, we significantly extend an open source tuning tool – KeenTune to optimize several typical enterprise cloud applications and services. Our practice is in collaboration with enterprise users and tuning tool developers to address the aforementioned problems. Specifically, we highlight five key challenges from our experiences and provide a set of solutions accordingly. Through applying the improved tuning tool to different application scenarios, we achieve 2%-14% improvements for the performance of MySQL, OceanBase, nginx, ingress-nginx, and 5%-70% improvements for the performance of ACK cloud container service.

References

[1]

Dana Van Aken, Andrew Pavlo, Geoffrey J. Gordon, and Bohan Zhang. 2017. Automatic Database Management System Tuning Through Large-scale Machine Learning. In Proceedings of the 2017 ACM International Conference on Management of Data, SIGMOD Conference 2017, Chicago, IL, USA, May 14-19, 2017, Semih Salihoglu, Wenchao Zhou, Rada Chirkova, Jun Yang, and Dan Suciu (Eds.). ACM, 1009–1024. https://doi.org/10.1145/3035918.3064029

Digital Library

[2]

Alexey Kopytov. 2017. sysbench. https://github.com/akopytov/sysbench

[3]

Alibaba. 2019. Alibaba Cloud Linux. https://github.com/alibaba/cloud-kernel

[4]

Alibaba Group. 2015. ECS. https://www.alibabacloud.com/product/ecs

[5]

Alibaba Group. 2018. ACK. https://www.alibabacloud.com/product/kubernetes

[6]

Omid Alipourfard, Hongqiang Harry Liu, Jianshu Chen, Shivaram Venkataraman, Minlan Yu, and Ming Zhang. 2017. CherryPick: Adaptively Unearthing the Best Cloud Configurations for Big Data Analytics. In NSDI. USENIX Association, 469–482.

Digital Library

[7]

James Bergstra, Rémi Bardenet, Yoshua Bengio, and Balázs Kégl. 2011. Algorithms for Hyper-Parameter Optimization. In NIPS. 2546–2554.

[8]

James Bergstra, Daniel Yamins, and David D. Cox. 2013. Making a Science of Model Search: Hyperparameter Optimization in Hundreds of Dimensions for Vision Architectures. In ICML (1) (JMLR Workshop and Conference Proceedings, Vol. 28). JMLR.org, 115–123.

[9]

Canonical. 2004. The ubuntu Project. https://ubuntu.com

[10]

Centos. 2019. The CentOS Project. https://www.centos.org

[11]

Tianqi Chen and Carlos Guestrin. 2016. XGBoost: A Scalable Tree Boosting System. In KDD. ACM, 785–794.

[12]

Debian. 1997. The Debian Project. https://www.debian.org

[13]

Songyun Duan, Vamsidhar Thummala, and Shivnath Babu. 2009. Tuning Database Configuration Parameters with iTuned. Proc. VLDB Endow., 2, 1 (2009), 1246–1257. https://doi.org/10.14778/1687627.1687767

Digital Library

[14]

Hewlett Packard. 2005. netperf. https://github.com/HewlettPackard/netperf

[15]

Huawei. 2021. A-Tune. https://gitee.com/openeuler/A-Tune

[16]

Ilija Ilievski, Taimoor Akhtar, Jiashi Feng, and Christine Annette Shoemaker. 2017. Efficient Hyperparameter Optimization for Deep Learning Algorithms Using Deterministic RBF Surrogates. In AAAI. AAAI Press, 822–829.

[17]

KeenTune. 2021. KeenTune. https://openanolis.cn/sig/keentune

[18]

Kubernetes. 2016. ingress-nginx. https://github.com/kubernetes/ingress-nginx

[19]

Guoliang Li, Xuanhe Zhou, Shifu Li, and Bo Gao. 2019. QTune: A Query-Aware Database Tuning System with Deep Reinforcement Learning. Proc. VLDB Endow., 12, 12 (2019), 2118–2130. https://doi.org/10.14778/3352063.3352129

Digital Library

[20]

Lisha Li, Kevin G. Jamieson, Giulia DeSalvo, Afshin Rostamizadeh, and Ameet Talwalkar. 2017. Hyperband: Bandit-Based Configuration Evaluation for Hyperparameter Optimization. In ICLR (Poster). OpenReview.net.

[21]

Linux. 2020. sysctl. https://man7.org/linux/man-pages/man8/sysctl.8.html

[22]

Scott M. Lundberg and Su-In Lee. 2017. A Unified Approach to Interpreting Model Predictions. In NIPS. 4765–4774.

Digital Library

[23]

Microsoft. 2021. Neural Network Intelligence. https://github.com/microsoft/nni

[24]

NGINX, Inc. 2020. nginx. https://nginx.org/en/

[25]

OceanBase. 2010. oceanbase. https://github.com/oceanbase/oceanbase

[26]

OpenAnolis. 2020. Anolis OS. https://openanolis.cn/anolisos

[27]

OpenAnolis. 2020. OpenAnolis community. https://openanolis.cn

[28]

Oracle Corporation. 2010. MySQL 8.0 Reference Manual. https://www.mysql.com

[29]

Tirthak Patel and Devesh Tiwari. 2020. CLITE: Efficient and QoS-Aware Co-Location of Multiple Latency-Critical Jobs for Warehouse Scale Computers. In HPCA. IEEE, 193–206.

[30]

F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and E. Duchesnay. 2011. Scikit-learn: Machine Learning in Python. Journal of Machine Learning Research, 12 (2011), 2825–2830.

Digital Library

[31]

Alex Renda, Yishen Chen, Charith Mendis, and Michael Carbin. 2020. DiffTune: Optimizing CPU Simulator Parameters with Learned Differentiable Surrogates. In MICRO. IEEE, 442–455.

[32]

Marco Túlio Ribeiro, Sameer Singh, and Carlos Guestrin. 2016. "Why Should I Trust You?": Explaining the Predictions of Any Classifier. 1135–1144.

[33]

Jasper Snoek, Hugo Larochelle, and Ryan P. Adams. 2012. Practical Bayesian Optimization of Machine Learning Algorithms. In NIPS. 2960–2968.

Digital Library

[34]

Standard Performance Evaluation Corporation. 2015. SPECjbb2015. https://www.spec.org/jbb2015

[35]

Stressapptest. 2017. stressapp. https://github.com/stressapptest/stressapptest

[36]

Kevin Swersky, Jasper Snoek, and Ryan Prescott Adams. 2013. Multi-Task Bayesian Optimization. In NIPS. 2004–2012.

[37]

TPC. 1992. TPC-C. https://www.tpc.org/tpcc

[38]

TPC. 2007. TPC-H. https://www.tpc.org/tpch

[39]

Will Glozer. 2013. wrk. https://github.com/wg/wrk

[40]

Ji Zhang, Yu Liu, Ke Zhou, Guoliang Li, Zhili Xiao, Bin Cheng, Jiashu Xing, Yangtao Wang, Tianheng Cheng, Li Liu, Minwei Ran, and Zekang Li. 2019. An End-to-End Automatic Cloud Database Tuning System Using Deep Reinforcement Learning. In Proceedings of the 2019 International Conference on Management of Data, SIGMOD Conference 2019, Amsterdam, The Netherlands, June 30 - July 5, 2019, Peter A. Boncz, Stefan Manegold, Anastasia Ailamaki, Amol Deshpande, and Tim Kraska (Eds.). ACM, 415–432. https://doi.org/10.1145/3299869.3300085

Digital Library

[41]

Xinyi Zhang, Hong Wu, Zhuo Chang, Shuowei Jin, Jian Tan, Feifei Li, Tieying Zhang, and Bin Cui. 2021. ResTune: Resource Oriented Tuning Boosted by Meta-Learning for Cloud Databases. In SIGMOD Conference. ACM, 2102–2114.

Digital Library

Cited By

Wang QWang RHu YShi XLiu ZMa TSong HShi HJust RFraser G(2023)KeenTune: Automated Tuning Tool for Cloud Application Performance Testing and OptimizationProceedings of the 32nd ACM SIGSOFT International Symposium on Software Testing and Analysis10.1145/3597926.3604920(1487-1490)Online publication date: 13-Jul-2023
https://doi.org/10.1145/3597926.3604920
Chen WLi LYu Z(2023)INTOP: Improving Nginx Performance by Tuning OS Parameters2023 International Conference on High Performance Big Data and Intelligent Systems (HDIS)10.1109/HDIS60872.2023.10499598(16-23)Online publication date: 6-Dec-2023
https://doi.org/10.1109/HDIS60872.2023.10499598

Index Terms

Industry practice of configuration auto-tuning for cloud applications and services

Index terms have been assigned to the content through auto-classification.

Recommendations

KeenTune: Automated Tuning Tool for Cloud Application Performance Testing and Optimization
ISSTA 2023: Proceedings of the 32nd ACM SIGSOFT International Symposium on Software Testing and Analysis

The performance testing and optimization of cloud applications is challenging, because manual tuning of cloud computing stacks is tedious and automated tuning tools are rare used for cloud services. To address this issue, we introduce KeenTune, an ...
A model driven method to deploy auto-scaling configuration for cloud services
RELENG 2016: Proceedings of the 4th International Workshop on Release Engineering

Vendor lock-in is the issues in auto-scaling configuration; scaling configuration of a service cannot automatically transfer when the service is migrated from one cloud to another cloud. To facilitate fast service deployment, there is a need to ...
A model-driven approach to describe and predict the performance of composite services
WOSP '07: Proceedings of the 6th international workshop on Software and performance

Distributed applications are rapidly converging towards the adoption of a computing paradigm based on service-oriented architectures (SOA), according to which an application results from the composition of a set of services in execution on networked ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

ESEC/FSE 2022: Proceedings of the 30th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering

November 2022

1822 pages

ISBN:9781450394130

DOI:10.1145/3540250

General Chair:
Abhik Roychoudhury
National University of Singapore, Singapore
,
Program Chairs:
Cristian Cadar
Imperial College London, UK
,
Miryung Kim
University of California at Los Angeles, USA

Copyright © 2022 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 09 November 2022

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

ESEC/FSE '22

Sponsor:

ESEC/FSE '22: 30th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering

November 14 - 18, 2022

Singapore, Singapore

Acceptance Rates

Overall Acceptance Rate 112 of 543 submissions, 21%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

2
Total Citations
View Citations
325
Total Downloads

Downloads (Last 12 months)131
Downloads (Last 6 weeks)22

Reflects downloads up to 21 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

Wang QWang RHu YShi XLiu ZMa TSong HShi HJust RFraser G(2023)KeenTune: Automated Tuning Tool for Cloud Application Performance Testing and OptimizationProceedings of the 32nd ACM SIGSOFT International Symposium on Software Testing and Analysis10.1145/3597926.3604920(1487-1490)Online publication date: 13-Jul-2023
https://doi.org/10.1145/3597926.3604920
Chen WLi LYu Z(2023)INTOP: Improving Nginx Performance by Tuning OS Parameters2023 International Conference on High Performance Big Data and Intelligent Systems (HDIS)10.1109/HDIS60872.2023.10499598(16-23)Online publication date: 6-Dec-2023
https://doi.org/10.1109/HDIS60872.2023.10499598

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents