A scientific breakthrough reveals why generative AI learns so effectively

A study conducted by Tony Bonnaire (IAS, CNRS & Université Paris-Saclay) together with Raphaël Urfin and Giulio Biroli (ENS Paris, CNRS & PSL) and Marc Mézard (Bocconi University) sheds new light on a central question in generative AI research: how diffusion models — the key technology behind modern generative AI — manage to generalize efficiently before memorizing the data on which they are trained.
 
The rapid rise of AI has brought a new generation of models capable of producing images, sounds, or videos of striking realism. Among them, diffusion models hold a prominent place: by learning from large datasets, they are able to generate content often indistinguishable from real data. But behind this achievement lies a fundamental challenge: how do these systems manage to create new data — in other words, to generalize — rather than simply memorize and reproduce exactly what they were trained on?
 
Through an interdisciplinary approach combining statistical physics, computer science, and numerical experiments, Tony Bonnaire and his collaborators have made a key discovery about how diffusion models learn: the quantitative identification of two distinct and predictable timescales — an initial phase of generalization that is independent of the training data, followed much later by a memorization phase that depends on the size of the dataset.
The team shows that the memorization timescale shifts further and further as the number of training samples increases, explaining why generative AIs based on diffusion models remain in a regime where they continue to create genuinely new data.
 
By demonstrating that the empirical performance and practical success of diffusion models rely on a demonstrable and measurable mechanism that naturally delays overfitting, the work of Tony Bonnaire and his collaborators provides a deep and actionable understanding of the mechanisms that govern modern generative AI. It provides a new foundation for the scientific analysis of generalization in artificial intelligence, as highlighted by the award of the NeurIPS 2025 Best Paper Prize — a truly exceptional distinction, with only 4 papers selected among 21,575 submissions (5,290 accepted) at the field’s most prestigious conference.
 
Article: "Why Diffusion Models Don’t Memorize: The Role of Implicit Dynamical Regularization in Training", Tony Bonnaire, Raphaël Urfin, Giulio Biroli et Marc Mézard
See : https://blog.neurips.cc/author/alexlu/
Contact:  Tony Bonnaire