In my previous post, I described various (potentially non-smooth) functions that have quadratic (and thus smooth) variational formulations, a possibility that I referred to as the η-trick. For example, in its simplest formulation, we have \( \displaystyle |w| = \min_{ \eta \geq 0} \frac{1}{2} \frac{w^2}{\eta} + \frac{1}{2} \eta\). While it seems most often used for…
Author: Francis Bach
The “η-trick” or the effectiveness of reweighted least-squares
Optimizing a quadratic function is often considered “easy” as it is equivalent to solving a linear system, for which many algorithms exist. Thus, reformulating a non-quadratic optimization problem into a sequence of quadratic problems is a natural idea. While the standard generic way is Newton method, which is adapted to smooth (at least twice-differentiable) functions,…
I am starting a blog!
I have finally found time to start a blog. Although I don’t know if I will find the energy to do all this in the next few months, I am planning to use this blog in a variety of ways, with one post every first monday of the month: To develop in reasonable depth some…