Editing Agent-based model (section)

==Verification and validation==
[[Verification and validation]] (V&V) of simulation models is extremely important.<ref>{{Cite book |last1=Sargent |first1=R. G. |doi=10.1109/WSC.2000.899697 |chapter=Verification, validation and accreditation of simulation models |title=2000 Winter Simulation Conference Proceedings (Cat. No.00CH37165) |volume=1 |pages=50–59 |year=2000 |isbn=978-0-7803-6579-7 |citeseerx=10.1.1.17.438 |s2cid=57059217 }}</ref> Verification involves making sure the implemented model matches the conceptual model, whereas validation ensures that the implemented model has some relationship to the real-world. Face validation, sensitivity analysis, calibration, and statistical validation are different aspects of validation.<ref>{{Cite book |last1=Klügl |first1=F. |chapter=A validation methodology for agent-based simulations |doi=10.1145/1363686.1363696 |title=Proceedings of the 2008 ACM symposium on Applied computing - SAC '08 |pages=39–43 |year=2008 |isbn=9781595937537 |s2cid=9450992 }}</ref> A [[discrete-event simulation]] framework approach for the validation of agent-based systems has been proposed.<ref>{{cite journal |last1=Fortino |first1=G. |first2=A. |last2=Garro |first3=W. |last3=Russo |title=A Discrete-Event Simulation Framework for the Validation of Agent-Based and Multi-Agent Systems |year=2005 |url=http://www-lia.deis.unibo.it/books/woa2005/papers/11.pdf |access-date=September 27, 2009 |archive-date=June 26, 2011 |archive-url=https://web.archive.org/web/20110626211408/http://www-lia.deis.unibo.it/books/woa2005/papers/11.pdf |url-status=live }}</ref> A comprehensive resource on empirical validation of agent-based models can be found here.<ref>{{cite web |title=Empirical Validation: Agent-Based Computational Economics |url=https://www2.econ.iastate.edu/tesfatsi/empvalid.htm |last=Tesfatsion |first=Leigh |publisher=[[Iowa State University]] |access-date=June 24, 2020 |archive-date=June 26, 2020 |archive-url=https://web.archive.org/web/20200626090651/https://www2.econ.iastate.edu/tesfatsi/empvalid.htm |url-status=live }}</ref>

As an example of V&V technique, consider VOMAS (virtual overlay multi-agent system),<ref>{{cite journal |first1=Muaz |last1=Niazi |first2=Amir |last2=Hussain |first3=Mario |last3=Kolberg |title=Verification and Validation of Agent-Based Simulations using the VOMAS approach |journal=Proceedings of the Third Workshop on Multi-Agent Systems and Simulation '09 (MASS '09), as Part of MALLOW 09, Sep 7–11, 2009, Torino, Italy |url=http://www.cs.stir.ac.uk/~man/papers/VOMAS_CRV_aug_05_09_Muazv2.pdf |archive-url=https://web.archive.org/web/20110614052017/http://www.cs.stir.ac.uk/~man/papers/VOMAS_CRV_aug_05_09_Muazv2.pdf |archive-date=June 14, 2011 |url-status=dead}}</ref> a software engineering based approach, where a virtual overlay multi-agent system is developed alongside the agent-based model. Muazi et al. also provide an example of using VOMAS for verification and validation of a forest fire simulation model.<ref>{{cite book |first1=Muaz |last1=Niazi |first2=Qasim |last2=Siddique |first3=Amir |last3=Hussain |first4=Mario |last4=Kolberg |title=Proceedings of the 2010 Spring Simulation Multiconference |chapter=Verification & validation of an agent-based forest fire simulation model |pages=142–149 |date=April 11–15, 2010 |doi=10.1145/1878537.1878539 |isbn=978-1-4503-0069-8 |chapter-url=http://cs.stir.ac.uk/~man/papers/2_ADS_09_Final_Paper.pdf |archive-url=https://web.archive.org/web/20110725083937/http://cs.stir.ac.uk/~man/papers/2_ADS_09_Final_Paper.pdf |archive-date=July 25, 2011 |url-status=dead }}</ref><ref>{{cite journal |first=Muaz A. K. |last=Niazi |title=Towards A Novel Unified Framework for Developing Formal, Network and Validated Agent-Based Simulation Models of Complex Adaptive Systems |date=June 11, 2011 |publisher=[[University of Stirling]]|hdl=1893/3365 }} PhD Thesis</ref> Another software engineering method, i.e. [[test-driven development|Test-Driven Development]] has been adapted to for agent-based model validation.<ref>{{Cite journal |last1=Onggo |first1=B.S. |last2=Karatas |first2=M. |doi=10.1016/j.ejor.2016.03.050 |title=Test-driven simulation modelling: A case study using agent-based maritime search-operation simulation. |journal=European Journal of Operational Research |volume=254 |issue=2 |pages=517–531 |year=2016 |url= https://www.sciencedirect.com/science/article/abs/pii/S0377221716301965|archive-date=June 30, 2020 |archive-url=https://web.archive.org/web/20200630141434/https://www.sciencedirect.com/science/article/abs/pii/S0377221716301965 |url-status=live }}</ref> This approach has another advantage that allows an automatic validation using unit test tools.