PEPR PDE-AI

Mon, 01 Jan 0001 00:00:00 +0000

Geometric and algorithmic issues in control and learning

PDE-AI projet kickoff meeting - Jussieu, January 23-24 2024

Nice team

Jean-Baptiste Caillau, UniCA (PR) - Optimal control

Thibaud Kloczko, Inria (IR) - Scientitic computing, dev

Ludovic Rifford, UniCA (PR) - Control, SR geometry, transport

Samuel Vaiter, CNRS (CR) - Optimisation, machine learning

Theme 1 – The dynamics of Neural Networks training

Objective 1.2 – Control and machine learning: there and back again (participants: Strasbourg, Nice)
Task: Control for neural network analysis (1 postdoc)
Objective 1.3 – Scalable solvers and softwares (participants: U. Paris Cité, Nice)
Task: Automatic differentiation and control (1 postdoc + 1 engineer)

ResNets as discretised linear control systems

After Agrachev (SISSA), Sarychev (Florence), Scagliotti (TUM) et al:

Mon, 01 Jan 0001 00:00:00 +0000

Control for neural network analysis

Postdoc position, 22 months

Université Côte d’Azur, J. A. Dieudonné CNRS Lab. (Nice)

Control has recently proved very instrumental for the analysis of neural networks, and for their use in classification. When dealing with very deep residual networks (ResNets), for instance, a relevant approximation is to assume that there is a continuum of layers, indexed by time. In this framework, the composition of a finite number of cells can be interpreted as discretising a continuous ODE, called the neural ODE. In turn, for a large number of layers, the continuous model is meaningful to understand the network properties. The weights, labeled by time, are the controls of the system, and there are several issues that can be efficiently addressed through this point of view. First, in the case of supervised learning: learning a classifier is just learning some input-output map, a standard task in control theory. And meeting the requirements tied to the known data is associated with controllability issues, namely ensemble controllability, ultimately related to controllability on the group of diffeomorphisms on the ambient manifold (Agrachev and Sarychev, 2022; Scagliotti, 2023). Lie bracket techniques are in order for these questions, as well as other more constructive approaches that exploit the structure of the nonlinear activation functions of the original network (Li, Lin and Chen, 2023; Zuazua, 2022; Ruiz-Balet and Zuazua, 2022). Another line of active search to contribute to is the analysis of the convergence of training of neural networks, modelled by neural ODEs, and its relation to the well known turnpike phenomenon in optimal control (Geshkovski and Zuazua, 2022).

Mon, 01 Jan 0001 00:00:00 +0000

Automatic differentiation and control (2 positions)

Postdoc position, 22 months, and

Research engineer position, 18 months

Inria Centre at Université Côte d’Azur (Sophia)

More and more innovative algorithms in the field of AI and its declinations in optimal control or transport require the use of automatic differentiation (AD): schematically, the minimization of a loss function in machine learning necessitates the computation of a direction of descent, generally associated with a more or less weak notion of derivative (see, for instance, the notion of path differentiability and its applications in Bolte, Pauwels and Vaiter, 2023). The estimation of this derivative classically requires to differentiate the calculation produced by an algorithm. Automatically generating the algorithm for calculating this derivative often has many virtues (efficiency, numerical robustness, etc.) Although well established, the technical ecosystem of AD has undergone an extremely important revival over the last decade with the explosion of problems and techniques associated with ML. A quick review allows us to cite historical state-of-the-art codes such as Tapenade, developed at Inria Sophia (Hascoët and Pascual, 2013), and numerous more recent achievements such as TensorFlow, Pytorch, JAX, ForwardDiff, ReverseDiff or Zygote, directly resulting from the work of the ML community. We also note the recent importance for AD of approaches that operate in a language-agnostic way, directly at the level of an Intermediate Representation (LLVM code in the case of the Enzyme project, for example). In our academic context, it could also be relevant to cross this theme with another discipline well represented in Nice-Sophia: proof of code in language theory. The numerical aspects being crucial for the quality of the results provided by an algorithm, proving that these differentiation methods are correct would give an important added value to this work. These postdoc + engineer recruitments will allow to review and evaluate AD tools in close connection with the fundamental algorithms in optimisation / optimal control / optimal transport, with a strong focus on the Julia language. The performance of these tools will be benchmarked on use cases stemming from ongoing developments of the control-toolbox project (Caillau, Cots and Martinon, 2022) and applications in learning.

2e Rencontres PEPR PDE-AI

Mon, 01 Jan 0001 00:00:00 +0000

Location

The workshop will be hosted at the MAP5, Université Paris-Cité, 7th floor.

Map of 7th floor

Programme

12 mars Salle Vieussens B (7e étage)

10:00-10:40 Cyril Letrouit (Paris-Saclay) Quantitative stability of optimal transport maps

10:40-11:20 Christophe Vauthier (Paris Saclay) Critical points of sliced-Wasserstein distances

11:20-12:00 Amaury Belières-Frendo (Univ. Strasbourg) Symplectic neural networks for shape optimization

12:00-14:00 déjeuner, discussions

14:00-14:40 Erell Gachon (Bordeaux) Scalable and consistent embedding of probability measures into Hilbert spaces via measure quantization

Events

Mon, 01 Jan 0001 00:00:00 +0000

As recommended by the CNRS scientific council, PDE-AI promotes respect for gender parity (currently 20 to 25% women in our communities) on scientific committees, organizing committees and in the lists of plenary speakers for the conferences it supports. The ecological footprint of conferences is also a point of attention.

When applying to PDE-AI for support and sponsorship, we advise you to integrate these elements at the very beginning of the creation of programs and committees for your scientific events.

First (scientific) meeting of the PDE-AI network.

Mon, 01 Jan 0001 00:00:00 +0000

This workshop is organized with the support of the PEPR IA (France 2030, ANR), project PDE-AI.

Location

The workshop will be hosted at the
Laboratoire Jacques-Louis Lions
Jussieu, Sorbonne Université
Barre 15-16, 3ème étage, salle 09 (15-16-3-09)

(Métro Jussieu, 4 place Jussieu, Paris 5e).

Abstract

This two-days event will give to the participants to the project the possibility to present their current research and projetcs and foster collaborations with other teams.

Groupe de travail PDE-AI.

Mon, 01 Jan 0001 00:00:00 +0000

Certains jeudis à 11h ou 14h, durée 1h

Upcoming talk(s)

Some resources

Katia Meziani’s slides: Deep Learning démystifié : Petit guide (Intranet CEREMADE), with many references.

Job offers

Mon, 01 Jan 0001 00:00:00 +0000

Current offers:

PhD Position on optimization: aspects of neural differential equations, in Paris-Dauphine PSL starting September/October 2024

A three-year PhD position is available starting Sept/October 2024 under the supervision of Antonin Chambolle and Clément Royer. Applicants should hold a Masters degree with a strong component in applied mathematics, including optimization and partial differential equations. A full description of the PhD topic is available here.

Interested should send a CV along with contact information for two references to Clément Royer.

Local Events

Mon, 01 Jan 0001 00:00:00 +0000

This page lists the local events organized internally by each group of the PDE-AI network (working groups, seminars)

Events to come

CEREMADE, Université Paris-Dauphine: Groupe de travail PDE-AI, Next: Thu. 25 April 2024, 11h.
LJAD, I3S, Université Côte d’Azur: Séminaire OptAzur (monthly seminar on optimisation)

Previous events

Séminaire McTAO (Sophia) : Alexis Montoison (GERAD) - 13 mars 2024. Enhancing nonlinear optimization through GPU computing (see here)
Séminaire McTAO (Sophia) : Alessandro Scagliotti (Technical University of Munich) – 21 novembre 2024. Optimal control of ODEs with dynamics uncertainty (see here)
Séminaire McTAO (Sophia) : Alexis Montoison (Argonne National Lab.) - 8 janvier 2025. Sparse automatic differentiation (see here)

Mathematical Models for Plug-and-play Image Restoration

Mon, 01 Jan 0001 00:00:00 +0000

This workshop is organized with the support of RT-MIA, ANR PostProdLeap, MAP5, IMB.

Location

The workshop will be hosted at the
Centre Culturel Irlandais
5 Rue des Irlandais
75005 Paris

The CCI is accessible by bus or metro. (RER B “Luxembourg”, or Metro 7 “Place Monge” or “Cardinal Lemoine”).

Abstract

Designing efficient algorithms for ill-posed inverse problems in imaging science crucially relies on the choice of a prior on the solution. It has been a long-standing issue to design regularity priors that allow to use optimization algorithms having good convergence guarantees and producing sharp visual results. The Plug-and-Play framework implicitly regularizes inverse problems in optimization algorithms by substituting the regularizer, its gradient or its proximal operator by learned representations thereof. State-of-the-art restoration results have been obtained with denoisers parameterized by deep neural networks, and by generative models such as GANs, VAEs, Normalizing Flows or diffusion models. More recently the PnP framework has also been exploited in Bayesian imaging where it has been succesfully applied to MMSE estimation, uncertainty quantification among other Bayesian computations. However, these successes raise many issues related to the convergence of these schemes or their recovery guarantees. Moreover, the design of accurate denoising neural network architecture or the tuning of the hyperparameters involved are open problems that strongly affect the PnP restoration performance.

Members 2023 - 2027

Mon, 01 Jan 0001 00:00:00 +0000

Direction and consortium

Antonin Chambolle (CNRS, CEREMADE, Université Paris-Dauphine-PSL), main coordinator;
Jean-François Aujol (Prof. U. Bordeaux)
Jean-Baptiste Caillau (Prof. U. Côte d’Azur)
Elsa Cazelles (CNRS, IRIT, U. Toulouse)
Julie Delon (Prof. U. Paris-Cité)
Bruno Desprès (Prof. Sorbonne U.)
Julie Digne (CNRS, LIRIS, Lyon)
Anna Korba (Ass. Prof. CREST, GENES, ENSAE)
Quentin Mérigot (Prof. U. Paris-Saclay)
Emmanuel Franck (Inria, IRMA, Strasbourg) et Yannick Privat (Prof. U. Nancy) (Strasbourg)

Members

Université Paris-Dauphine PSL
- Antonin Chambolle, CNRS
- Pierre Cardaliaguet, Prof. Dauphine
- Thomas Gallouët, CR INRIA (Mokaplan)
- Idriss Mazari, MC. Dauphine
- Zhenjie Ren (任振杰), MC. Dauphine
- Clément Royer, MC. Dauphine
- Irène Waldspurger, CNRS
LIRIS, CNRS et Université de Lyon
- Julie Digne, CR CNRS LIRIS
- Nicolas Bonneel, CR CNRS LIRIS
- Élie Bretin, MC, Institut Camille Jordan (ICJ)
- Raphaëlle Chaine, Prof. LIRIS
- Simon Masnou, Prof. ICJ
Institut de Mathématiques de Bordeaux
- Jean-François Aujol, Prof.
- Jérémie Bigot, Prof.
- Bernard Bercu, Prof.
- Arthur Leclaire, MC
- Nicolas Papadakis DR CNRS
Université Paris-Cité
- Julie Delon, Prof MAP5
- Andrés Almansa, DR CNRS MAP5,
- Joan Glaunés, MC MAP5,
- Agnès Desolneux, DR CNRS Centre Borelli, ENS Paris-Saclay,
- Nathaël Gozlan, Prof. MAP5,
- Marcela Szopos, Prof. MAP5,
- Jonathan Vacher, MC MAP5
Sorbonne Université, LJLL
- Bruno Després, Prof
- Albert Cohen, Prof
- Frédéric Nataf, DR CNRS
- Katharina Schratz, Prof
Université Paris-Saclay
- Quentin Mérigot, Prof.
- Anna Kazeykina, MC
- Blanche Buet, MC
- Luca Nenna MC
- Hugo Leclerc, IR CNRS
Université de Toulouse
- Elsa Cazelles, CR CNRS IRIT
- Edouard Pauwels, Prof. TSE
- Jérôme Bolte, Prof. TSE
- Alice Le Brigant, MC SAMM Université Paris 1
- Thierry Klein, Prof. IMT, ENAC
Université Nice-Côte d’Azur / INRIA
- Jean-Baptiste Caillau, Prof Laboratoire J.A. Dieudonné (LJAD)
- Thibaud Kloczko, IR SED
- Ludovic Rifford, Prof LJAD
- Samuel Vaiter, CR LJAD
CREST/ENSAE (Institut Polytechnique de Paris)
- Anna Korba, Ass. Prof.
- Arnak Dalalyan, Prof.
Université de Strasbourg
- Yannick Privat, Prof (U. Nancy)
- Emmanuel Franck, CR INRIA
- Joubine Aghili, MC Institut de recherche mathématique (IRMA)
- Clémentine Courtès, MC IRMA
- Philippe Helluy, Prof. IRMA
- Victor Michel-Dansac, ISFP INRIA
- Laurent Navoret, MC IRMA
- Christophe Prud’homme, Prof. IRMA
- Vincent Vigon, MC IRMA

PhD and postdocs

Institut de Mathématiques de Bordeaux
- Julien Hermant, started 15th October 2023, co-advised by Jean-François Aujol, Charles Dossal, Aude Rondepierre. “Non-convex optimization.”
- Erell Gachon, started 1st November 2023, co-advised by Jérémie Bigot and Elsa Cazelles. “Applications statistiques du transport optimal pour l’analyse de données de cytométrie en flux et application à l’évaluation de la maladie résiduelle dans les leucémies aiguës.”
- Valentine Tosel, started 1st October 2025, co-advised by Julie Delon and Nicolas Papadakis. “Regularization of inverse problems with generative models”
Institut de Recherche en Informatique de Toulouse
- Léo Portales, started 1st December 2023, co-advised by Elsa Cazelles and Édouard Pauwels. “Density approximation using Lloyd’s algorithm.”
Université Nice-Côte d’Azur / INRIA
- Chih-Kang Huang, started January 1st 2026, co-advised by J.-B. Caillau and L. Rifford. “Control for machine learning”
- Marco Rando, started November 1st 2025, advised by S. Vaiter. “AD and machine learning”

Publications

Mon, 01 Jan 0001 00:00:00 +0000

(Toutes les publications doivent être déposées sur HAL avec licence CCBY)

Workshop on "Optimization and sampling for inverse problems"

Mon, 01 Jan 0001 00:00:00 +0000

Program

10:30 - 11:15: Antonin CHAMBOLLE (CEREMADE, Paris)
11:15 - 12:00: Mame Diarra FALL (LITIS, Rouen)
14:00 - 14:45: Thomas POCK (TU Graz, Austria)
14:45 - 15.30: Gersende FORT (LAAS, Toulouse)
16:00 - 16:45 : Julián TACHELLA (ENS Lyon)
16:45 - 17:30 : Andrés ALMANSA (MAP5, Paris)

Venue

Salle de Conférence
Institut de mathématiques de Bordeaux - Batiment A33
351 Cours de la liberation, 33405 Talence Cedex, France.

Location

Registration

Registration is free but mandatory. Due to room constraints, the number of participants will be limited.