Stepsize Hedging: an Alternative Mechanism for Accelerating Gradient Descent
Theory of Computing Report
June 1, 2026
Authors: Jason M. Altschuler, Pablo A. Parrilo
Can gradient descent be accelerated by just choosing better stepsizes? Surprisingly, the answer is yes. This short expository article provides an accessible introduction to this phenomenon of stepsize hedging.
Discussion in the ATmosphere