Editing Q-learning (section)

== Implementation ==
''Q''-learning at its simplest stores data in tables. This approach falters with increasing numbers of states/actions since the likelihood of the agent visiting a particular state and performing a particular action is increasingly small.

=== Function approximation ===
''Q''-learning can be combined with [[function approximation]].<ref>{{cite book|chapter-url={{google books |plainurl=y |id=YPjNuvrJR0MC|pp= 207-251}}|title=Reinforcement Learning: State-of-the-Art|editor-last1=Wiering|editor-first1=Marco|editor-last2=Otterlo|editor-first2=Martijn van|date=5 March 2012|publisher=Springer Science & Business Media |first=Hado van |last=Hasselt |chapter=Reinforcement Learning in Continuous State and Action Spaces |pages= 207–251 |isbn=978-3-642-27645-3}}</ref> This makes it possible to apply the algorithm to larger problems, even when the state space is continuous.

One solution is to use an (adapted) [[artificial neural network]] as a function approximator.<ref name="CACM">{{cite journal|last=Tesauro|first=Gerald|date=March 1995|title=Temporal Difference Learning and TD-Gammon|url=http://www.bkgm.com/articles/tesauro/tdl.html|journal=Communications of the ACM|volume=38|issue=3|pages=58–68|doi=10.1145/203330.203343|s2cid=8763243|access-date=2010-02-08|doi-access=free}}</ref> Another possibility is to integrate Fuzzy Rule Interpolation (FRI) and use sparse [[Fuzzy rule|fuzzy rule-bases]]<ref>{{Cite book |last=Vincze |first=David |title=2017 IEEE 15th International Symposium on Applied Machine Intelligence and Informatics (SAMI) |chapter=Fuzzy rule interpolation and reinforcement learning |date=2017 |chapter-url=http://users.iit.uni-miskolc.hu/~vinczed/research/vinczed_sami2017_author_draft.pdf |publisher=IEEE |pages=173–178 |doi=10.1109/SAMI.2017.7880298|isbn=978-1-5090-5655-2 |s2cid=17590120 }}</ref> instead of discrete Q-tables or ANNs, which has the advantage of being a human-readable knowledge representation form. Function approximation may speed up learning in finite problems, due to the fact that the algorithm can generalize earlier experiences to previously unseen states.

=== Quantization ===
Another technique to decrease the state/action space quantizes possible values. Consider the example of learning to balance a stick on a finger. To describe a state at a certain point in time involves the position of the finger in space, its velocity, the angle of the stick and the [[angular velocity]] of the stick. This yields a four-element vector that describes one state, i.e. a snapshot of one state encoded into four values. The problem is that infinitely many possible states are present. To shrink the possible space of valid actions multiple values can be assigned to a bucket. The exact distance of the finger from its starting position (-Infinity to Infinity) is not known, but rather whether it is far away or not (Near, Far).<ref>{{cite arXiv |last1=Krishnan |first1=Srivatsan |last2=Lam |first2=Maximilian |last3=Chitlangia |first3=Sharad |last4=Wan |first4=Zishen |last5=Barth-Maron |first5=Gabriel |last6=Faust |first6=Aleksandra |last7=Reddi |first7=Vijay Janapa |title=QuaRL: Quantization for Fast and Environmentally Sustainable Reinforcement Learning |date=13 November 2022 |class=cs.LG |eprint=1910.01055 }}</ref>