Workshop on "Optimization and sampling for inverse problems" -

Program

10:30 - 11:15: Antonin CHAMBOLLE (CEREMADE, Paris) - A study of minibatch rectified flow matching -
Abstract
Rectified flow matching is often presented as a method for computing optimal transportation between two distributions. We show that this is not always clear, but likely for the “minibatch” version.

This is joint work with Johannes Hertrich and Julie Delon (ENS, PSL).
11:15 - 12:00: Mame Diarra FALL (LITIS, Rouen) - Bayesian computational imaging with RED priors and geometry-aware sampling -
Abstract
Inverse problems are ubiquitous in signal and image processing. As inverse problems are known to be ill-posed or, at least, ill-conditioned, they require regularization through the introduction of additional constraints to mitigate the lack of information provided by the observations. A common difficulty lies in selecting an appropriate regularizer, which has a decisive influence on the quality of the reconstruction. Another challenge concerns the level of confidence we may have in the reconstructed signal or image. These two tasks - regularization and uncertainty quantification - can be addressed simultaneously within the Bayesian framework. This approach makes it possible to incorporate additional information by specifying a marginal distribution for the image, known as the prior distribution. The traditional approach consists in defining the prior analytically, as a hand-crafted explicit function chosen to promote specific desired properties of the recovered image. Following the surge in deep learning, data-driven regularization using priors specified by neural networks has become widespread in image inverse problems. Popular approaches within this framework include Plug-and-Play (PnP) [1] and Regularization by Denoising (RED) [2].
In the first part of the talk, I will present the probabilistic approach to the RED framework we have introduced in [3], which defines a new probability distribution based on a RED potential that can be used as the prior distribution in a Bayesian inversion task. We also introduce a dedicated Markov chain Monte Carlo (MCMC) sampling algorithm that leverages optimisation-based splitting techniques to efficiently draw samples from the posterior distribution. In addition, we provide a theoretical analysis guaranteeing convergence to the target distribution and quantifying the convergence rate. The effectiveness of the proposed approach is illustrated on various linear inverse restoration tasks such as image deblurring, inpainting, and super-resolution. The second part of the talk will be devoted to an extesion of this approach we propose in [4], for solving Poisson inverse problems. We also develop a Monte Carlo sampling algorithm that incorporates the underlying non-Euclidean geometry of the problem while leveraging splitting and data augmentation techniques. The proposed approach has been evaluated on different tasks such as denoising, deblurring, and positron emission tomography (PET) reconstruction. The presentation is based on joint works with E.C. Faye (Univ. Orléans), N. Dobigeon (Univ. Toulouse) and É. Barat (CEA).

References

[1] S. V. Venkatakrishnan et al. « Plug-and-Play priors for model based reconstruction ». In IEEE Global Conf. on Signal and Information Processing, pp 945-948, 2013.
[2] Y. Romano, M. Elad and P. Milanfar, « The little engine that could: Regularization by denoising (RED), » SIAM Journal on Imaging Sciences, 10(4):1804–1844, 2017
[3] E.C. Faye, M.D. Fall and N. Dobigeon. « Regularization by denoising: Bayesian model and Langevin-within-split Gibbs sampling », IEEE Transactions on Image Processing, vol 34, pages 221-234, 2024
[4] E.C. Faye, M.D. Fall, N. Dobigeon and É. Barat « Bregman geometry-aware split Gibbs sampling for Bayesian Poisson inverse problems », Submitted.

14:00 - 14:45: Thomas POCK (TU Graz, Austria) - Sampling versus optimization (online) -
Abstract
Motivated by Bayesian formulations of inverse problems in imaging, this talk explores connections and differences between sampling and optimization. In particular, we show how modern sampling algorithms can be designed by borrowing key ideas from efficient optimization methods. The first, called the Gaussian Latent Machine, builds on ideas from half-quadratic optimization to design efficient samplers for non-smooth and non-convex models. The second, the Inertial Langevin Algorithm, incorporates inertial and accelerated gradient ideas into Langevin sampling, leading to significantly faster convergence.
14:45 - 15.30: Gersende FORT (LAAS, Toulouse) - Stochastic Approximation beyond Gradient for Machine Learning: Taming Noise with Guarantees -
Abstract
Stochastic Approximation (SA) is a classical iterative algorithm that has a long history of over 70 years. The goal of the SA scheme is to determine the roots of a nonlinear system when the mean field cannot be explicitly computed but a random oracle exists. Recently, the spectrum of use of SA schemes has widened considerably with the applications to statistical machine learning, due to the necessity to deal with a large amount of data observed with uncertainties. An extensive literature on stochastic optimization is devoted to the stochastic (sub)gradient (SG) algorithm, which is by far the most popular application of the SA scheme. The stochastic gradient algorithms are characterized by having an update recursion featuring an unbiased mean field which is the gradient of a loss function to be minimized.
However, a lesser known fact is that SA scheme also includes non-gradient stochastic algorithms whose expected oracles are not the gradients of any function and whose oracles are possibly biased. We will discuss examples among compressed stochastic Gradient, stochastic Expectation-Maximization and more generally stochastic Majorize-Minimization, algorithms in Reinforcement Learning for solving Bellman equations under constraints.
Most of the existing overviews on the convergence for SA schemes are restricted to the Gradient-case. In addition, they are essentially limited to asymptotic convergence for algorithms with decreasing step sizes. Possible reasons behind this, are the lack of a proper Lyapunov function to set the convergence analysis framework, and the potential bias which may destabilize the SA recursion. The second step of the talk is to present a design guideline of SA algorithms backed by theories: propose a general framework that unifies existing theories of SA and applies to non-gradient SA, and develop convergence results with an emphasis on non-asymptotic convergence analysis.
Finally, in the last part, we will review recent advances in SA, such as variance reduction within SA.

The talk is based on joint works with Aymeric Dieuleveut (CMAP, Ecole Polytechnique, France), Eric Moulines (ML Dpt, MBZUAI, UAE), and Hoi-To Wai (Dpt of SEEM, Chinese University of Hong-Kong).

16:00 - 16:45: Julián TACHELLA (ENS Lyon) - Equivariant Splitting: Self-supervised learning from incomplete data -
Abstract
Self-supervised learning for inverse problems allows training a reconstruction network from noise and/or incomplete data alone. These methods have the potential of enabling learning-based solutions when obtaining ground-truth references for training is expensive or even impossible. In this talk, I’ll present a new self-supervised learning strategy devised for the challenging setting where measurements are observed via a single incomplete observation model. I will also introduce a new definition of equivariance in the context of reconstruction networks, and show that the combination of self-supervised splitting losses and equivariant reconstruction networks results in unbiased estimates of the supervised loss.

Paper: https://openreview.net/forum?id=upMIVpe467 (ICLR'26)
16:45 - 17:30 : Andrés ALMANSA (MAP5, Paris)