Editing Support vector machine (section)

==== Sub-gradient descent ====
[[Subgradient method|Sub-gradient descent]] algorithms for the SVM work directly with the expression

<math display="block">f(\mathbf{w}, b) = \left[\frac 1 n \sum_{i=1}^n \max\left(0, 1 - y_i(\mathbf{w}^\mathsf{T} \mathbf{x}_i - b)\right) \right] + \lambda \|\mathbf{w}\|^2.</math>

Note that <math>f</math> is a [[convex function]] of <math>\mathbf{w}</math> and <math>b</math>. As such, traditional [[gradient descent]] (or [[Stochastic gradient descent|SGD]]) methods can be adapted, where instead of taking a step in the direction of the function's gradient, a step is taken in the direction of a vector selected from the function's [[Subderivative|sub-gradient]]. This approach has the advantage that, for certain implementations, the number of iterations does not scale with <math>n</math>, the number of data points.<ref>{{Cite journal |title=Pegasos: primal estimated sub-gradient solver for SVM |journal=Mathematical Programming |date=2010-10-16 |issn=0025-5610 |pages=3–30 |volume=127 |issue=1 |doi=10.1007/s10107-010-0420-4 |first1=Shai |last1=Shalev-Shwartz |first2=Yoram |last2=Singer |first3=Nathan |last3=Srebro |first4=Andrew |last4=Cotter |citeseerx=10.1.1.161.9629 |s2cid=53306004 }}</ref>