Article

Parameter space exploration with Gaussian process trees

Authors:

Robert B. Gramacy,

Herbert K. H. Lee,

William G. MacreadyAuthors Info & Claims

ICML '04: Proceedings of the twenty-first international conference on Machine learning

Page 45

https://doi.org/10.1145/1015330.1015367

Published: 04 July 2004 Publication History

Abstract

Computer experiments often require dense sweeps over input parameters to obtain a qualitative understanding of their response. Such sweeps can be prohibitively expensive, and are unnecessary in regions where the response is easy predicted; well-chosen designs could allow a mapping of the response with far fewer simulation runs. Thus, there is a need for computationally inexpensive surrogate models and an accompanying method for selecting small designs. We explore a general methodology for addressing this need that uses non-stationary Gaussian processes. Binary trees partition the input space to facilitate non-stationarity and a Bayesian interpretation provides an explicit measure of predictive uncertainty that can be used to guide sampling. Our methods are illustrated on several examples, including a motivating example involving computational fluid dynamics simulation of a NASA reentry vehicle.

References

[1]

Box, G. E. P., Hunter, W. G., & Hunter, J. S. (1978). Statistics for experimenters. New York: Wiley.

[2]

Breiman, L., Friedman, J. H., Olshen, R., & Stone, C. (1984). Classification and regression trees. Belmont, CA: Wadsworth.

[3]

Chipman, H., George, E., & McCulloch, R. (1998). Bayesian CART model search (with discussion). Journal of the American Statistical Association, 93, 935--960.

[4]

Cohn, D. (1994). Neural network exploration using optimal experimental design. Advances in Neural Information Processing Systems (pp. 679--686). Morgan Kaufmann Publishers.

[5]

Denison, D., Mallick, B., & Smith, A. (1998). A Bayesian CART algorithm. Biometrika, 85, 363--377.

[6]

Gelman, A., Carlin, J. B., Stern, H. S., & Rubin, D. B. (1995). Bayesian data analysis. London: Chapman and Hall.

[7]

Higdon, D. (2002). Space and space-time modeling using process convolutions. Quantitative Methods for Current Environmental Issues (pp. 37--56). London: Springer-Verlag.

[8]

Hjort, N. L., & Omre, H. (1994). Topics in spatial statistics. Scandinavian Journal of Statistics, 21, 289--357.

[9]

Kim, H.-M., Mallick, B. K., & Holmes, C. C. (2002). Analyzing non-stationary spatial data using piecewise Gaussian processes (Technical Report). Texas A&M University -- Corpus Christi.

[10]

Mackay, D. (1992). Information-based objective functions for active data selection. Neural Computation, 4, 589--603.

Digital Library

[11]

Matheron, G. (1963). Principles of geostatistics. Economic Geology, 58, 1246--1266.

[12]

McKay, M. D., Conover, W. J., & Beckman, R. J. (1979). A comparison of three methods for selecting values of input variables in the analysis of output from a computer code. Technometrics, 21, 239--245.

[13]

O'Hagan, A., Kennedy, M. C., & Oakley, J. E. (1999). Uncertainty analysis and other inference tools for complex computer codes. Bayesian Statistics 6 (pp. 503--524). Oxford University Press.

[14]

Rasmussen, C. E., & Ghahramani, Z. (2002). Infinite mixtures of Gaussian process experts. Advances in Neural Information Processing Systems. MIT Press.

[15]

Richardson, S., & Green, P. J. (1997). On Bayesian analysis of mixtures with an unknown number of components. Journal of the Royal Statistical Society, Series B, Methodological, 59, 731--758.

[16]

Rogers, S. E., Aftosmis, M. J., Pandya, S. A., N. M. Chaderjian, E. T. T., & Ahmad, J. U. (2003). Automated cfd parameter studies on distributed parallel computers. 16th AIAA Computational Fluid Dynamics Conference. AIAA Paper 2003--4229.

[17]

Sacks, J., Welch, W. J., Mitchell, T. J., & Wynn, H. P. (1989). Design and analysis of computer experiments. Statistical Science, 4, 409--435.

[18]

Seo, S., Wallat, M., Graepel, T., & Obermayer, K. (2000). Gaussian process regression: Active data selection and test point rejection. Proceedings of the International Joint Conference on Neural Networks IJCNN 2000 (pp. 241--246). IEEE.

Digital Library

[19]

Tresp, V. (2001). Mixtures of gaussian processes. Advances in Neural Information Processing Systems 13 (pp. 654--660). MIT Press.

Cited By

Jiang MKhandait TPedrielli G(2023)CGPT: A Conditional Gaussian Process Tree for Grey-Box Bayesian Optimization2023 Winter Simulation Conference (WSC)10.1109/WSC60868.2023.10408654(564-575)Online publication date: 10-Dec-2023
https://doi.org/10.1109/WSC60868.2023.10408654
Ghosh MWu LHao QZhou Q(2022)A random forest with multi-fidelity Gaussian process leaves for modeling multi-fidelity data with heterogeneityComputers and Industrial Engineering10.1016/j.cie.2022.108746174:COnline publication date: 1-Dec-2022
https://dl.acm.org/doi/10.1016/j.cie.2022.108746
Nguyen TBouzerdoum APhung S(2018)Stochastic variational hierarchical mixture of sparse Gaussian processes for regressionMachine Language10.1007/s10994-018-5721-5107:12(1947-1986)Online publication date: 1-Dec-2018
https://dl.acm.org/doi/10.1007/s10994-018-5721-5
Show More Cited By

Parameter space exploration with Gaussian process trees
1. Computing methodologies

Recommendations

Fast Gaussian process methods for point process intensity estimation
ICML '08: Proceedings of the 25th international conference on Machine learning

Point processes are difficult to analyze because they provide only a sparse and noisy observation of the intensity function driving the process. Gaussian Processes offer an attractive framework within which to infer underlying intensity functions. The ...
In-Space Propulsion Technology products for NASA's future science and exploration missions
AERO '11: Proceedings of the 2011 IEEE Aerospace Conference

Since 2001, the In-Space Propulsion Technology (ISPT) project has been developing and delivering in-space propulsion technologies that will enable or enhance NASA robotic science missions. These in-space propulsion technologies are applicable, and ...
Multi-layer designs and composite gaussian process models with engineering applications

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

ICML '04: Proceedings of the twenty-first international conference on Machine learning

July 2004

934 pages

ISBN:1581138385

DOI:10.1145/1015330

Conference Chair:
Carla Brodley
Purdue University/Tufts University

Copyright © 2004 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 04 July 2004

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Qualifiers

Article

Acceptance Rates

Overall Acceptance Rate 140 of 548 submissions, 26%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

24
Total Citations
View Citations
665
Total Downloads

Downloads (Last 12 months)20
Downloads (Last 6 weeks)3

Reflects downloads up to 15 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

Jiang MKhandait TPedrielli G(2023)CGPT: A Conditional Gaussian Process Tree for Grey-Box Bayesian Optimization2023 Winter Simulation Conference (WSC)10.1109/WSC60868.2023.10408654(564-575)Online publication date: 10-Dec-2023
https://doi.org/10.1109/WSC60868.2023.10408654
Ghosh MWu LHao QZhou Q(2022)A random forest with multi-fidelity Gaussian process leaves for modeling multi-fidelity data with heterogeneityComputers and Industrial Engineering10.1016/j.cie.2022.108746174:COnline publication date: 1-Dec-2022
https://dl.acm.org/doi/10.1016/j.cie.2022.108746
Nguyen TBouzerdoum APhung S(2018)Stochastic variational hierarchical mixture of sparse Gaussian processes for regressionMachine Language10.1007/s10994-018-5721-5107:12(1947-1986)Online publication date: 1-Dec-2018
https://dl.acm.org/doi/10.1007/s10994-018-5721-5
Chérel GCottineau CReuillon RPumain DReuillon R(2017)Using Models to Explore Possible Futures (Contingency and Complexity)Urban Dynamics and Simulation Models10.1007/978-3-319-46497-8_5(81-95)Online publication date: 18-Jan-2017
https://doi.org/10.1007/978-3-319-46497-8_5
Shahriari BSwersky KWang ZAdams Rde Freitas N(2016)Taking the Human Out of the Loop: A Review of Bayesian OptimizationProceedings of the IEEE10.1109/JPROC.2015.2494218104:1(148-175)Online publication date: Jan-2016
https://doi.org/10.1109/JPROC.2015.2494218
Nguyen TBouzerdoum APhung S(2016)Variational inference for infinite mixtures of sparse Gaussian processes through KL-correction2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)10.1109/ICASSP.2016.7472143(2579-2583)Online publication date: Mar-2016
https://doi.org/10.1109/ICASSP.2016.7472143
Chérel GCottineau CReuillon R(2015)Beyond Corroboration: Strengthening Model Validation by Looking for Unexpected PatternsPLOS ONE10.1371/journal.pone.013821210:9(e0138212)Online publication date: 14-Sep-2015
https://doi.org/10.1371/journal.pone.0138212
Gambi AHummer WDustdar SPaige RHarman MWilliams J(2013)Testing elastic systems with surrogate modelsProceedings of the 1st International Workshop on Combining Modelling and Search-Based Software Engineering10.5555/2662572.2662577(8-11)Online publication date: 19-May-2013
https://dl.acm.org/doi/10.5555/2662572.2662577
Hamze FWang Zde Freitas N(2013)Self-Avoiding Random Dynamics on Integer Complex SystemsACM Transactions on Modeling and Computer Simulation10.1145/2414416.241479023:1(1-25)Online publication date: 1-Jan-2013
https://dl.acm.org/doi/10.1145/2414416.2414790
Gambi AHummer WDustdar S(2013)Testing elastic systems with surrogate models2013 1st International Workshop on Combining Modelling and Search-Based Software Engineering (CMSBSE)10.1109/CMSBSE.2013.6604429(8-11)Online publication date: May-2013
https://doi.org/10.1109/CMSBSE.2013.6604429
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents