Machine Learning Research Blog

Francis Bach

Menu
  • Home
  • About
  • Home page
Menu

Author: Loucas Pillaud-Vivien and Scott Pesme

Rethinking SGD’s noise – II: Implicit Bias

Posted on September 18, 2022September 22, 2022 by Loucas Pillaud-Vivien and Scott Pesme

In the previous post, we showed (or at least tried to!) how the inherent noise of the stochastic gradient descent algorithm (SGD), in the context of modern overparametrised architectures, is structured and carries two important features: (i) it vanishes for interpolating solutions and (ii) it belongs to a low-dimensional manifold spanned by the gradients. Building…

Read more

Recent Posts

  • Unraveling spectral properties of kernel matrices – II
  • My book is (at last) out!
  • Scaling laws of optimization
  • Unraveling spectral properties of kernel matrices – I
  • Revisiting the classics: Jensen’s inequality

About

I am Francis Bach, a researcher at INRIA in the Computer Science department of Ecole Normale Supérieure, in Paris, France. I have been working on machine learning since 2000, with a focus on algorithmic and theoretical contributions, in particular in optimization. All of my papers can be downloaded from my web page or my Google Scholar page. I also have a Twitter account.

Recent Posts

  • Unraveling spectral properties of kernel matrices – II
  • My book is (at last) out!
  • Scaling laws of optimization
  • Unraveling spectral properties of kernel matrices – I
  • Revisiting the classics: Jensen’s inequality

Recent Comments

  • Francis Bach on Unraveling spectral properties of kernel matrices – II
  • Chanwoo Chun on Unraveling spectral properties of kernel matrices – II
  • Antonio Horta Ribeiro on Unraveling spectral properties of kernel matrices – II
  • Francis Bach on My book is (at last) out!
  • Francis Bach on Unraveling spectral properties of kernel matrices – I

Archives

  • March 2025
  • December 2024
  • October 2024
  • January 2024
  • March 2023
  • February 2023
  • December 2022
  • November 2022
  • September 2022
  • July 2022
  • April 2022
  • March 2022
  • February 2022
  • July 2021
  • June 2021
  • April 2021
  • March 2021
  • February 2021
  • January 2021
  • December 2020
  • November 2020
  • October 2020
  • September 2020
  • August 2020
  • July 2020
  • June 2020
  • May 2020
  • April 2020
  • March 2020
  • February 2020
  • January 2020
  • December 2019
  • November 2019
  • October 2019
  • September 2019
  • August 2019
  • July 2019
  • June 2019

Categories

  • Machine learning
  • Opinions
  • Optimization
  • Tools

Meta

  • Log in
  • Entries feed
  • Comments feed
  • WordPress.org
©2025 Machine Learning Research Blog | WordPress Theme by Superbthemes.com