research-article

Black-box performance models: prediction based on observation

Authors:

Wolfgang TheilmannAuthors Info & Claims

QUASOSS '09: Proceedings of the 1st international workshop on Quality of service-oriented software systems

Pages 19 - 24

https://doi.org/10.1145/1596473.1596479

Published: 25 August 2009 Publication History

Abstract

Software performance engineering enables software architects to find potential performance problems, such as bottlenecks and long delays, prior to implementation and testing. Such early feedback on the system's performance is essential to develop and maintain efficient and scalable applications. However, the unavailability of data necessary to design performance models often hinders its application in practice. During system maintenance, the existing system has to be included into the performance model. For large, heterogeneous, and complex systems that have grown over time, modelling becomes infeasible due to the sheer size and complexity of the systems. Re-engineering approaches also fail due to the large and heterogeneous technology stack. Especially for such systems, performance prediction is essential. In this position statement, we propose goal-oriented abstractions of large parts of a software system based on systematic measurements. The measurements provide the information necessary to determine Black-box Performance Models that directly capture the influence of a system's usage and workload on performance (response time, throughput, and resource utilisation). We outline the research challenges that need to be addressed in order to apply Black-box Performance Models.

References

[1]

S. Balsamo, A. Di Marco, P. Inverardi, and M. Simeoni. Model-Based Performance Prediction in Software Development: A Survey. IEEE Transactions on Software Engineering, 30(5):295--310, May 2004.

Digital Library

[2]

S. Becker, H. Koziolek, and R. Reussner. The Palladio component model for model-driven performance prediction. Journal of Systems and Software, 82:3--22, 2009.

Digital Library

[3]

M. Courtois and C. M. Woodside. Using regression splines for software performance analysis. In Proc 2nd Int. Workshop on Software and Performance (WOSP2000), pages 105--114, Ottawa, Canada, September 2000. ACM.

Digital Library

[4]

P. J. Courtois. Decomposability, instabilities, and saturation in multiprogramming systems. Commun. ACM, 18(7):371--377, 1975.

Digital Library

[5]

P. J. Courtois. On time and space decomposition of complex structures. Commun. ACM, 28(6):590--603, 1985.

Digital Library

[6]

G. Franks. Performance Analysis of Distributed Server Systems. PhD thesis, Department of Systems and Computer Engineering, Carleton University, Ottawa, Ontario, Canada, December 1999.

Digital Library

[7]

J. Happe, S. Becker, C. Rathfelder, H. Friedrich, and R. H. Reussner. Parametric Performance Completions for Model-Driven Performance Prediction. Performance Evaluation, 2009. Accepted for publication in 2009.

Digital Library

[8]

R. Jain. The Art of Computer Systems Performance Analysis : Techniques for Experimental Design, Measurement, Simulation, and Modeling. Wiley, 1991.

[9]

K. Krogmann. Reengineering of Software Component Models to Enable Architectural Quality of Service Predictions. In R. H. Reussner, C. Szyperski, and W. Weck, editors, Proceedings of WCOP 2007, volume 2007-13 of Interne Berichte, pages 23--29, 2007. Universität Karlsruhe (TH).

[10]

M. Kuperberg, K. Krogmann, and R. Reussner. Performance Prediction for Black-Box Components using Reengineered Parametric Behaviour Models. In Proceedings of CBSE 2008, volume 5282 of LNCS, pages 48--63. Springer-Verlag, 2008.

Digital Library

[11]

E. Lazowska, J. Zahorjan, G. S. Graham, and K. C. Sevcik. Quantitative System Performance - Computer System Analysis Using Queueing Network Models. Prentice-Hall, 1984.

Digital Library

[12]

D. A. Menascé, V. A. F. Almeida, and L. W. Dowdy. Performance by Design. Prentice Hall, 2004.

[13]

K. Sachs, S. Kounev, J. Bacon, and A. Buchmann. Performance evaluation of message-oriented middleware using the SPECjms2007 benchmark. Performance Evaluation, to appear, 2009.

Digital Library

[14]

M. Tanelli, D. Ardagna, and M. Lovera. LPV model identification in virtualized service center environments. In Proceedings of the 15th IFAC Symposium on System Identification (SYSID 2009), 2009. To Appear.

[15]

L. G. Williams and C. U. Smith. Making the Business Case for Software Performance Engineering. In Proceedings of the 29th International Computer Measurement Group Conference, pages 349--358. Computer Measurement Group, 2003.

[16]

C. M. Woodside and M. Litoiu. Performance model estimation and tracking using optimal filters. IEEE Transactions on Software Engineering, 34(3):391--406, 2008.

Digital Library

[17]

M. Woodside, V. Vetland, M. Courtois, and S. Bayarov. Resource Function Capture for Performance Aspects of Software Components and Sub-Systems In Performance Engineering: State of the Art and Current Trends, volume 2047 of LNCS, pages 239--256. Springer-Verlag, 2001.

Digital Library

Cited By

Zheng SZhang L(2010)Queueing models based performance evaluation approach for Video On Demand back office system2010 IEEE International Conference on Software Engineering and Service Sciences10.1109/ICSESS.2010.5552438(303-307)Online publication date: Jul-2010
https://doi.org/10.1109/ICSESS.2010.5552438
Happe JWestermann DSachs KKapová L(2010)Statistical inference of software performance models for parametric performance completionsProceedings of the 6th international conference on Quality of Software Architectures: research into Practice - Reality and Gaps10.1007/978-3-642-13821-8_4(20-35)Online publication date: 23-Jun-2010
https://dl.acm.org/doi/10.1007/978-3-642-13821-8_4

Index Terms

Black-box performance models: prediction based on observation

Recommendations

Using black-box performance models to detect performance regressions under varying workloads: an empirical study
Abstract
Performance regressions of large-scale software systems often lead to both financial and reputational losses. In order to detect performance regressions, performance tests are typically conducted in an in-house (non-production) environment using ...
TAU Performance System
IWOCL '22: Proceedings of the 10th International Workshop on OpenCL

The TAU Performance System 1 is a versatile performance evaluation tool that supports OpenCL, DPC++/SYCL, OpenMP, and other GPU runtimes. It features a performance profiling and tracing module that is widely portable and can access hardware performance ...
A Reference Architecture for Online Performance Model Extraction in Virtualized Environments
ICPE '16 Companion: Companion Publication for ACM/SPEC on International Conference on Performance Engineering

Performance models can support decisions throughout the life-cycle of a software system. However, the manual construction of such performance models is a complex and time-consuming task requiring deep system knowledge. Therefore, automatic approaches ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

QUASOSS '09: Proceedings of the 1st international workshop on Quality of service-oriented software systems

August 2009

60 pages

ISBN:9781605587097

DOI:10.1145/1596473

Program Chairs:
Heiko Koziolek
ABB Corporate Research, Germany
,
Paul Pettersson
Mälardalen University, Sweden
,
Petr Hnetynka
Charles University, Czech Republic

Copyright © 2009 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 25 August 2009

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

ESEC/FSE09

Sponsor:

ESEC/FSE09: Joint 12th European Software Engineering Conference

August 25, 2009

Amsterdam, The Netherlands

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

2
Total Citations
View Citations
312
Total Downloads

Downloads (Last 12 months)4
Downloads (Last 6 weeks)0

Reflects downloads up to 21 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

Zheng SZhang L(2010)Queueing models based performance evaluation approach for Video On Demand back office system2010 IEEE International Conference on Software Engineering and Service Sciences10.1109/ICSESS.2010.5552438(303-307)Online publication date: Jul-2010
https://doi.org/10.1109/ICSESS.2010.5552438
Happe JWestermann DSachs KKapová L(2010)Statistical inference of software performance models for parametric performance completionsProceedings of the 6th international conference on Quality of Software Architectures: research into Practice - Reality and Gaps10.1007/978-3-642-13821-8_4(20-35)Online publication date: 23-Jun-2010
https://dl.acm.org/doi/10.1007/978-3-642-13821-8_4

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents