Editing Neural network (machine learning) (section)

===Capacity===
A model's "capacity" property corresponds to its ability to model any given function. It is related to the amount of information that can be stored in the network and to the notion of complexity.
Two notions of capacity are known by the community. The information capacity and the VC Dimension. The information capacity of a perceptron is intensively discussed in [[David J. C. MacKay|Sir David MacKay]]'s book<ref name="auto">{{cite book| last=MacKay| first=David J.C.| author-link=David J.C. MacKay| year=2003| publisher=[[Cambridge University Press]]| isbn=978-0-521-64298-9| title=Information Theory, Inference, and Learning Algorithms| url=http://www.inference.phy.cam.ac.uk/itprnn/book.pdf| access-date=11 June 2016| archive-date=19 October 2016| archive-url=https://web.archive.org/web/20161019163258/http://www.inference.phy.cam.ac.uk/itprnn/book.pdf| url-status=live}}</ref> which summarizes work by [[Thomas M. Cover|Thomas Cover]].<ref>{{cite journal|last=Cover|first=Thomas|author-link=Thomas M. Cover|year=1965|publisher=[[IEEE]]|url=http://www-isl.stanford.edu/people/cover/papers/paper2.pdf|title=Geometrical and Statistical Properties of Systems of Linear Inequalities with Applications in Pattern Recognition|journal=IEEE Transactions on Electronic Computers|issue=3|pages=326–334|volume=EC-14|doi=10.1109/PGEC.1965.264137|access-date=10 March 2020|archive-date=5 March 2016|archive-url=https://web.archive.org/web/20160305031348/http://www-isl.stanford.edu/people/cover/papers/paper2.pdf|url-status=live}}</ref> The capacity of a network of standard neurons (not convolutional) can be derived by four rules<ref>{{cite book| last=Gerald | first=Friedland| title=Proceedings of the 27th ACM International Conference on Multimedia| chapter=Reproducibility and Experimental Design for Machine Learning on Audio and Multimedia Data| author-link=Gerald Friedland|year=2019|publisher=[[Association for Computing Machinery|ACM]]| pages=2709–2710| doi=10.1145/3343031.3350545| isbn=978-1-4503-6889-6| s2cid=204837170}}</ref> that derive from understanding a neuron as an electrical element. The information capacity captures the functions modelable by the network given any data as input. The second notion, is the [[VC dimension]]. VC Dimension uses the principles of [[measure theory]] and finds the maximum capacity under the best possible circumstances. This is, given input data in a specific form. As noted in,<ref name="auto"/> the VC Dimension for arbitrary inputs is half the information capacity of a perceptron. The VC Dimension for arbitrary points is sometimes referred to as Memory Capacity.<ref>{{cite web| url=http://tfmeter.icsi.berkeley.edu/| title=Stop tinkering, start measuring! Predictable experimental design of Neural Network experiments| website=The Tensorflow Meter| access-date=10 March 2020| archive-date=18 April 2022| archive-url=https://web.archive.org/web/20220418025904/http://tfmeter.icsi.berkeley.edu/| url-status=dead}}</ref>