← Blog
Gradient Descent
ML/dictionary/gradient-descent
Definition
The optimisation method behind most ML training: nudge each parameter in the direction that reduces the loss, by a small step. Repeat a few billion times and the model converges.
Posts that use this term
- How a model learns: training and inference
Training is the expensive one-time event where a model's numbers get tuned. Inference is the cheap repeated use afterwards. The gap in cost is enormous, and it shapes the whole industry.