Computation
See recent articles
Showing new listings for Friday, 18 October 2024
- [1] arXiv:2410.13261 (cross-list from stat.ME) [pdf, html, other]
-
Title: Novel Bayesian algorithms for ARFIMA long-memory processes: a comparison between MCMC and ABC approachesSubjects: Methodology (stat.ME); Computation (stat.CO)
This paper presents a comparative study of two Bayesian approaches - Markov Chain Monte Carlo (MCMC) and Approximate Bayesian Computation (ABC) - for estimating the parameters of autoregressive fractionally-integrated moving average (ARFIMA) models, which are widely used to capture long-memory in time series data. We propose a novel MCMC algorithm that filters the time series into distinct long-memory and ARMA components, and benchmarked it against standard approaches. Additionally, a new ABC method is proposed, using three different summary statistics used for posterior estimation. The methods are implemented and evaluated through an extensive simulation study, as well as applied to a real-world financial dataset, specifically the quarterly U.S. Gross National Product (GNP) series. The results demonstrate the effectiveness of the Bayesian methods in estimating long-memory and short-memory parameters, with the filtered MCMC showing superior performance in various metrics. This study enhances our understanding of Bayesian techniques in ARFIMA modeling, providing insights into their advantages and limitations when applied to complex time series data.
Cross submissions (showing 1 of 1 entries)
- [2] arXiv:2408.06894 (replaced) [pdf, html, other]
-
Title: Exploring the generalizability of the optimal 0.234 acceptance rate in random-walk Metropolis and parallel tempering algorithmsComments: Submitted to Communications in Statistics - Simulation and ComputationSubjects: Computation (stat.CO)
For random-walk Metropolis (RWM) and parallel tempering (PT) algorithms, an asymptotic acceptance rate of around 0.234 is known to be optimal in the high-dimensional limit. However, its practical relevance is uncertain due to restrictive derivation conditions. We synthesise previous theoretical advances in extending the 0.234 acceptance rate to more general settings, and demonstrate its applicability with a comprehensive empirical simulation study on examples examining how acceptance rates affect Expected Squared Jumping Distance (ESJD). Our experiments show the optimality of the 0.234 acceptance rate for RWM is surprisingly robust even in lower dimensions across various proposal and multimodal target distributions which may or may not have an i.i.d. product density. Parallel tempering experiments also show that the idealized 0.234 spacing of inverse temperatures may be approximately optimal for low dimensions and non i.i.d. product target densities, and that constructing an inverse temperature ladder with spacings given by a swap acceptance of 0.234 is a viable strategy. However, we observe the applicability of the 0.234 acceptance rate heuristic diminishes for both RWM and PT algorithms below a certain dimension which differs based on the target density, and that inhomogeneously scaled components in the target density further reduces its applicability in lower dimensions.
- [3] arXiv:2303.14801 (replaced) [pdf, html, other]
-
Title: FAStEN: An Efficient Adaptive Method for Feature Selection and Estimation in High-Dimensional Functional RegressionsJournal-ref: Journal of Computational and Graphical Statistics 2024Subjects: Methodology (stat.ME); Computation (stat.CO); Machine Learning (stat.ML)
Functional regression analysis is an established tool for many contemporary scientific applications. Regression problems involving large and complex data sets are ubiquitous, and feature selection is crucial for avoiding overfitting and achieving accurate predictions. We propose a new, flexible and ultra-efficient approach to perform feature selection in a sparse high dimensional function-on-function regression problem, and we show how to extend it to the scalar-on-function framework. Our method, called FAStEN, combines functional data, optimization, and machine learning techniques to perform feature selection and parameter estimation simultaneously. We exploit the properties of Functional Principal Components and the sparsity inherent to the Dual Augmented Lagrangian problem to significantly reduce computational cost, and we introduce an adaptive scheme to improve selection accuracy. In addition, we derive asymptotic oracle properties, which guarantee estimation and selection consistency for the proposed FAStEN estimator. Through an extensive simulation study, we benchmark our approach to the best existing competitors and demonstrate a massive gain in terms of CPU time and selection performance, without sacrificing the quality of the coefficients' estimation. The theoretical derivations and the simulation study provide a strong motivation for our approach. Finally, we present an application to brain fMRI data from the AOMIC PIOP1 study. Complete FAStEN code is provided at this https URL.
- [4] arXiv:2405.11780 (replaced) [pdf, html, other]
-
Title: General bounds on the quality of Bayesian coresetsComments: 23 pages, 3 figures. Appearing in NeurIPS 2024Subjects: Machine Learning (stat.ML); Machine Learning (cs.LG); Statistics Theory (math.ST); Computation (stat.CO)
Bayesian coresets speed up posterior inference in the large-scale data regime by approximating the full-data log-likelihood function with a surrogate log-likelihood based on a small, weighted subset of the data. But while Bayesian coresets and methods for construction are applicable in a wide range of models, existing theoretical analysis of the posterior inferential error incurred by coreset approximations only apply in restrictive settings -- i.e., exponential family models, or models with strong log-concavity and smoothness assumptions. This work presents general upper and lower bounds on the Kullback-Leibler (KL) divergence of coreset approximations that reflect the full range of applicability of Bayesian coresets. The lower bounds require only mild model assumptions typical of Bayesian asymptotic analyses, while the upper bounds require the log-likelihood functions to satisfy a generalized subexponentiality criterion that is weaker than conditions used in earlier work. The lower bounds are applied to obtain fundamental limitations on the quality of coreset approximations, and to provide a theoretical explanation for the previously-observed poor empirical performance of importance sampling-based construction methods. The upper bounds are used to analyze the performance of recent subsample-optimize methods. The flexibility of the theory is demonstrated in validation experiments involving multimodal, unidentifiable, heavy-tailed Bayesian posterior distributions.
- [5] arXiv:2409.08928 (replaced) [pdf, other]
-
Title: Self-Organized State-Space Models with Artificial DynamicsComments: 102 pages (28 pages for the paper, 6 for the appendix and 68 for the supplementary material), 4 figuresSubjects: Statistics Theory (math.ST); Computation (stat.CO); Methodology (stat.ME)
We consider a state-space model (SSM) parametrized by some parameter $\theta$ and aim at performing joint parameter and state inference. A popular idea to carry out this task is to replace $\theta$ by a Markov chain $(\theta_t)_{t\geq 0}$ and then to apply a filtering algorithm to the extended, or self-organized SSM (SO-SSM). However, the practical implementation of this idea in a theoretically justified way has remained an open problem. In this paper we fill this gap by introducing constructions of $(\theta_t)_{t\geq 0}$ that ensure the validity of the SO-SSM for joint parameter and state inference. Notably, we show that such SO-SSMs can be defined even if $\|\mathrm{Var}(\theta_{t}|\theta_{t-1})\|\rightarrow 0$ slowly as $t\rightarrow\infty$. This result is important since these models can be efficiently approximated using a particle filter. While SO-SSMs have been introduced for online inference, the development of iterated filtering (IF) has shown that they can also serve for computing the maximum likelihood estimator of an SSM. We also derive constructions of $(\theta_t)_{t\geq 0}$ and theoretical guarantees tailored to these specific applications of SO-SSMs and introduce new IF algorithms. From a practical point of view, the algorithms we develop are simple to implement and only require minimal tuning to perform well.
- [6] arXiv:2410.01024 (replaced) [pdf, html, other]
-
Title: GPTreeO: An R package for continual regression with dividing local Gaussian processesComments: Updated the bibliography, and is now equivalent to the journal submissionSubjects: Machine Learning (cs.LG); Computation (stat.CO)
We introduce GPTreeO, a flexible R package for scalable Gaussian process (GP) regression, particularly tailored to continual learning problems. GPTreeO builds upon the Dividing Local Gaussian Processes (DLGP) algorithm, in which a binary tree of local GP regressors is dynamically constructed using a continual stream of input data. In GPTreeO we extend the original DLGP algorithm by allowing continual optimisation of the GP hyperparameters, incorporating uncertainty calibration, and introducing new strategies for how the local partitions are created. Moreover, the modular code structure allows users to interface their favourite GP library to perform the local GP regression in GPTreeO. The flexibility of GPTreeO gives the user fine-grained control of the balance between computational speed, accuracy, stability and smoothness. We conduct a sensitivity analysis to show how GPTreeO's configurable features impact the regression performance in a continual learning setting.
- [7] arXiv:2410.11115 (replaced) [pdf, html, other]
-
Title: Randomized Iterative Solver as Iterative Refinement: A Simple Fix Towards Backward StabilitySubjects: Numerical Analysis (math.NA); Computation (stat.CO)
Iterative sketching and sketch-and-precondition are well-established randomized algorithms for solving large-scale, over-determined linear least-squares problems. In this paper, we introduce a new perspective that interprets Iterative Sketching and Sketching-and-Precondition as forms of Iterative Refinement. We also examine the numerical stability of two distinct refinement strategies, iterative refinement and recursive refinement, which progressively improve the accuracy of a sketched linear solver. Building on this insight, we propose a novel algorithm, Sketched Iterative and Recursive Refinement (SIRR), which combines both refinement methods. SIRR demonstrates a \emph{four order of magnitude improvement} in backward error compared to iterative sketching, achieved simply by reorganizing the computational order, ensuring that the computed solution exactly solves a modified least-squares system where the coefficient matrix deviates only slightly from the original matrix. To the best of our knowledge, \emph{SIRR is the first asymptotically fast, single-stage randomized least-squares solver that achieves both forward and backward stability}.