Editing Handwriting recognition (section)

==Results since  2009==

Since 2009, the [[recurrent neural network]]s and deep [[feedforward neural network|feedforward]] neural networks developed in the research group of [[Jürgen Schmidhuber]] at the [[IDSIA|Swiss AI Lab IDSIA]] have won several international handwriting competitions.<ref>[http://www.kurzweilai.net/how-bio-inspired-deep-learning-keeps-winning-competitions 2012 Kurzweil AI Interview] {{Webarchive|url=https://web.archive.org/web/20180831075249/http://www.kurzweilai.net/how-bio-inspired-deep-learning-keeps-winning-competitions |date=31 August 2018 }} with [[Jürgen Schmidhuber]] on the eight competitions won by his Deep Learning team 2009-2012</ref> In particular, the bi-directional and [[multi-dimensional]] [[Long short-term memory]] (LSTM)<ref>Graves, Alex; and Schmidhuber, Jürgen; ''Offline Handwriting Recognition with Multidimensional Recurrent Neural Networks'', in Bengio, Yoshua; Schuurmans, Dale; Lafferty, John; Williams, Chris K. I.; and Culotta, Aron (eds.), ''Advances in Neural Information Processing Systems 22 (NIPS'22), December 7th–10th, 2009, Vancouver, BC'', Neural Information Processing Systems (NIPS) Foundation, 2009, pp. 545–552</ref><ref>A. Graves, M. Liwicki, S. Fernandez, R. Bertolami, H. Bunke, [[Jürgen Schmidhuber|J. Schmidhuber]]. A Novel Connectionist System for Improved Unconstrained Handwriting Recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 31, no. 5, 2009.</ref> of Alex Graves et al. won three competitions in connected handwriting recognition at the 2009 International Conference on Document Analysis and Recognition (ICDAR), without any prior knowledge about the three different languages (French, Arabic, [[Persian language|Persian]]) to be learned.  Recent [[GPU]]-based [[deep learning]] methods for feedforward networks by Dan Ciresan and colleagues at [[IDSIA]] won the ICDAR 2011 offline Chinese handwriting recognition contest; their neural networks also were the first artificial pattern recognizers to achieve human-competitive performance<ref>D. C. Ciresan, U. Meier, [[Jürgen Schmidhuber|J. Schmidhuber]]. Multi-column Deep Neural Networks for Image Classification. IEEE Conf. on Computer Vision and Pattern Recognition CVPR 2012.</ref> on the famous [[MNIST database|MNIST]] handwritten digits problem<ref>[[Yann LeCun|LeCun, Y.]], Bottou, L., Bengio, Y., & Haffner, P. (1998). Gradient-based learning applied to document recognition. Proc. IEEE, 86, pp. 2278–2324.</ref> of [[Yann LeCun]] and colleagues at [[NYU]].

Benjamin Graham of the [[University of Warwick]] won a 2013 Chinese handwriting recognition contest, with only a 2.61% error rate, by using an approach to [[convolutional neural networks]] that evolved (by 2017) into "sparse convolutional neural networks".<ref>{{cite news |title=Sparse Networks Come to the Aid of Big Physics |url=https://www.quantamagazine.org/sparse-neural-networks-point-physicists-to-useful-data-20230608/ |access-date=17 June 2023 |work=[[Quanta Magazine]] |date=June 2023}}</ref><ref>Graham, Benjamin. "Spatially-sparse convolutional neural networks." arXiv preprint arXiv:1409.6070 (2014).</ref>