Editing Neural network (machine learning) (section)

==== Learning rate ====
{{main|Learning rate}}
The learning rate defines the size of the corrective steps that the model takes to adjust for errors in each observation.<ref>{{cite arXiv|last=Wei|first=Jiakai|date=26 April 2019|title=Forget the Learning Rate, Decay Loss|class=cs.LG|eprint=1905.00094}}</ref> A high learning rate shortens the training time, but with lower ultimate accuracy, while a lower learning rate takes longer, but with the potential for greater accuracy. Optimizations such as [[Quickprop]] are primarily aimed at speeding up error minimization, while other improvements mainly try to increase reliability. In order to avoid [[oscillation]] inside the network such as alternating connection weights, and to improve the rate of convergence, refinements use an [[adaptive learning rate]] that increases or decreases as appropriate.<ref>{{Cite book|last1=Li|first1=Y.|last2=Fu|first2=Y.|last3=Li|first3=H.|last4=Zhang|first4=S. W.|title=2009 International Conference on Computational Intelligence and Natural Computing |chapter=The Improved Training Algorithm of Back Propagation Neural Network with Self-adaptive Learning Rate |s2cid=10557754|date=1 June 2009|isbn=978-0-7695-3645-3|volume=1|pages=73–76|doi=10.1109/CINC.2009.111}}</ref> The concept of momentum allows the balance between the gradient and the previous change to be weighted such that the weight adjustment depends to some degree on the previous change. A momentum close to 0 emphasizes the gradient, while a value close to 1 emphasizes the last change.{{citation needed|date=October 2024}}