Design-time learning-rate selection for generalised Bayesian experimental design

Problem statement

Generalised (Gibbs) Bayesian inference replaces the likelihood in the Bayesian update with a loss function scaled by a learning rate $\omega > 0$: the Gibbs posterior is $\pi(\boldsymbol\theta \mid \boldsymbol y) \propto \exp(-\omega\,\ell_{\boldsymbol\theta}(\boldsymbol y))\,\pi(\boldsymbol\theta)$. The learning rate decides how far each observation moves the posterior away from the prior, and under model misspecification it is the dial that trades statistical efficiency against robustness. Every deployment of generalised Bayes must choose it. In the static setting this choice has a literature, because data exists to calibrate against. Sequential experimental design inverts the situation: the experimenter must fix $\omega$, or a rule that produces it, before the first observation arrives, and the choice then acts twice. It defines the Gibbs posterior used for parameter inference, and it enters the acquisition function (the generalised expected information gain) that selects every subsequent design. A wrong $\omega$ therefore corrupts both what is learned and what is measured next, and the two errors compound over the experimental horizon. The state-of-the-art framework for robust sequential design via Gibbs inference, GBOED (Barlas, Sloman and Kaski, 2025, arXiv:2511.07671), demonstrates empirically that performance depends strongly on $\omega$, and selects it per misspecification scenario: for example $\omega = 0.8$ for well-specified pharmacokinetics runs and $\omega = 0.1$ for outlier-contaminated ones. Which scenario applies is exactly what an experimenter does not know at design time. The authors state plainly that existing selection methods are unsuitable for the design setting because they require data that the setting is about to collect. The unsolved problem: produce a principled rule that selects $\omega$, either as a fixed value or as a pre-registered schedule $\omega(i)$ over experiments $i = 1, \dots, T$, using only what exists before data collection, namely the prior, the statistical model, the loss function, and the design space, at most updated with the observations acquired inside the same experiment, and show that designs and inferences made under that rule retain the robustness that scenario-tuned $\omega$ delivers. Why a resolution is a big win. First, it converts every robust-design result achieved with scenario-tuned $\omega$ into a deployable procedure, unblocking use in the settings sequential design exists for: expensive, small-sample experiments in systems biology, pharmacology, psychology, and medical imaging, where each observation costs money or patient exposure and outliers are routine. Second, it settles the attribution question that GBOED's own ablations raise, namely whether acquisition design or learning-rate selection is the binding constraint in robust sequential design; today the two are confounded because $\omega$ is tuned by hand. Third, sequential design is the hardest instance of the general selection problem, since no data exists at time zero; a rule that works there also hands the static generalised-Bayes community a default that does not require a dataset first.

Current state

Learning-rate selection in generalised Bayesian inference has an established literature, and all of it presupposes data in hand. Wu and Martin (2023) compare the main proposals, including SafeBayes-type methods descending from Grünwald and van Ommen (2017) and bootstrap-based calibration that matches frequentist coverage; every compared method needs a dataset generated by the true data-generating process before $\omega$ can be computed. Knoblauch, Jewson and Damoulas (2022) supply the optimisation-centric foundation of generalised variational inference but treat $\omega$ as a modelling choice, not a quantity with a selection procedure. In the experimental design setting, Overstall, Holloway-Brown and McGree (2023) first brought Gibbs inference to design, but their framework requires a trusted "designer distribution" assumed close to the true data-generating process, which relocates rather than solves the calibration problem. Barlas, Sloman and Kaski (2025, arXiv:2511.07671) is the current state of the art: it derives a Gibbs expected information gain, proves it computable by self-normalised importance sampling, and shows robustness gains under outlier contamination and misspecified error distributions across three benchmark problems. But its learning rates are fixed by hand, differently per problem and per scenario (Appendix E of that paper), its Appendix D analyses the sensitivity without producing a selection rule, and its discussion names the absence of a design-suitable selection method as an open limitation. In the conjugate Gaussian-process line, Altamirano, Briol and Knoblauch (2024) and Laplante, Altamirano, Duncan, Knoblauch and Briol (2025) fix $\omega$ from the assumed Gaussian observation variance. That rule does not transfer outside Gaussian models, and, as Barlas et al. note, it yields $\omega > 1$ whenever the assumed variance exceeds $2$, placing more weight on the data exactly when misspecification argues for less. The problem therefore stays open on both fronts: no published method selects $\omega$ for sequential design without data the setting does not yet have, and no published method avoids conditioning on knowledge of which misspecification scenario applies.

Resolution criteria

The challenge is solved by a published method (paper or preprint with released code) that satisfies all three criteria. 1. **Design-time input only.** The method outputs $\omega$, or a schedule $\omega(i)$ for $i = 1, \dots, T$, from the prior $\pi(\boldsymbol\theta)$, the statistical model $p(\boldsymbol y \mid \boldsymbol\theta, \boldsymbol\xi)$, the loss $\ell_{\boldsymbol\theta}(\boldsymbol\xi, \boldsymbol y)$, and the design space $\mathcal{X}$, optionally updated using only observations acquired inside the same experiment. It uses no draws from the true data-generating process, no held-out data, and no input that encodes whether or how the model is misspecified. 2. **One rule across scenarios on a shared benchmark.** On at least two sequential design problems, including at least one of the three GBOED benchmarks (Bayesian linear regression, pharmacokinetics, location finding; arXiv:2511.07671, Appendix E), the method runs with the identical rule under well-specified, outlier-contaminated, and misspecified-error-distribution data-generating processes, reporting MMD and NLL with standard errors over at least 90 replications. Solved means: under each misspecified scenario the method beats standard BOED on MMD and NLL by more than one standard error, and under the well-specified scenario it stays within one standard error of standard BOED. That is robustness without a scenario oracle and without surrendering the well-specified case. 3. **A stated guarantee.** A theorem connects the selected $\omega$ to a named criterion (posterior consistency, calibration or coverage, regret, or a robustness bound over a stated misspecification class) under stated assumptions. An empirical rule with no statement of when it works does not close the challenge. A paper meeting criteria 1 and 2 but not 3 keeps the challenge open and is the natural linked-paper response to attach here.

Citations

1. Barlas, Y. Z., Sloman, S. J., and Kaski, S. (2025). Robust Experimental Design via Generalised Bayesian Inference. arXiv:2511.07671.
2. Wu, P.-S. and Martin, R. (2023). A Comparison of Learning Rate Selection Methods in Generalized Bayesian Inference. Bayesian Analysis, 18(1):105-132.
3. Bissiri, P. G., Holmes, C. C., and Walker, S. G. (2016). A general framework for updating belief distributions. Journal of the Royal Statistical Society Series B: Statistical Methodology, 78(5):1103-1130.
4. Grunwald, P. and van Ommen, T. (2017). Inconsistency of Bayesian Inference for Misspecified Linear Models, and a Proposal for Repairing It. Bayesian Analysis, 12(4):1069-1103.
5. Overstall, A. M., Holloway-Brown, J., and McGree, J. M. (2023). Gibbs optimal design of experiments. arXiv:2310.17440.
6. Altamirano, M., Briol, F.-X., and Knoblauch, J. (2024). Robust and conjugate Gaussian process regression. International Conference on Machine Learning, PMLR. arXiv:2311.00463.
7. Laplante, W., Altamirano, M., Duncan, A., Knoblauch, J., and Briol, F.-X. (2025). Robust and Conjugate Spatio-Temporal Gaussian Processes. arXiv:2502.02450.
8. Knoblauch, J., Jewson, J., and Damoulas, T. (2022). An Optimization-centric View on Bayes' Rule: Reviewing and Generalizing Variational Inference. Journal of Machine Learning Research, 23(132):1-109.

Linked papers

• Robust Experimental Design via Generalised Bayesian Inferencerelated1 Sept 2026