Editing Unit testing (section)

== Limitations and disadvantages ==
Testing will not catch every error in the program, because it cannot evaluate every execution path in any but the most trivial programs. This [[decision problem|problem]] is a superset of the [[halting problem]], which is [[undecidable problem|undecidable]]. The same is true for unit testing. Additionally, unit testing by definition only tests the functionality of the units themselves. Therefore, it will not catch integration errors or broader system-level errors (such as functions performed across multiple units, or non-functional test areas such as [[Software performance testing|performance]]). Unit testing should be done in conjunction with other [[software testing]] activities, as they can only show the presence or absence of particular errors; they cannot prove a complete absence of errors. To guarantee correct behavior for every execution path and every possible input, and ensure the absence of errors, other techniques are required, namely the application of [[Formal verification|formal methods]] to prove that a software component has no unexpected behavior.{{Citation needed|date=September 2019}}

An elaborate hierarchy of unit tests does not equal integration testing. Integration with peripheral units should be included in integration tests, but not in unit tests.{{Citation needed|date=October 2010}} Integration testing typically still relies heavily on humans [[manual testing|testing manually]]; high-level or global-scope testing can be difficult to automate, such that manual testing often appears faster and cheaper.{{Citation needed|date=January 2010}}

Software testing is a combinatorial problem. For example, every Boolean decision statement requires at least two tests: one with an outcome of "true" and one with an outcome of "false". As a result, for every line of code written, programmers often need 3 to 5 lines of test code.{{cn|date=November 2023}} This obviously takes time and its investment may not be worth the effort. There are problems that cannot easily be tested at all &ndash; for example those that are [[Nondeterministic algorithm|nondeterministic]] or involve multiple [[Thread (computer science)|thread]]s. In addition, code for a unit test is as likely to be buggy as the code it is testing. [[Fred Brooks]] in ''[[The Mythical Man-Month]]'' quotes: "Never go to sea with two chronometers; take one or three."<ref>{{Cite book | isbn = 978-0-201-83595-3 | title = The Mythical Man-Month | last = Brooks | first = Frederick J. | author-link = Fred Brooks | year = 1995 | orig-year = 1975 | publisher = Addison-Wesley | page = [https://archive.org/details/mythicalmonth00broo/page/64 64] | title-link = The Mythical Man-Month }}</ref> Meaning, if two [[Marine chronometer|chronometers]] contradict, how do you know which one is correct?

=== Difficulty in setting up realistic and useful tests ===
Another challenge related to writing the unit tests is the difficulty of setting up realistic and useful tests. It is necessary to create relevant initial conditions so the part of the application being tested behaves like part of the complete system. If these initial conditions are not set correctly, the test will not be exercising the code in a realistic context, which diminishes the value and accuracy of unit test results.{{cn|date=November 2023}}

=== Requires discipline throughout the development process ===
To obtain the intended benefits from unit testing, rigorous discipline is needed throughout the software development process.

=== Requires version control ===
It is essential to keep careful records not only of the tests that have been performed, but also of all changes that have been made to the source code of this or any other unit in the software. Use of a [[version control]] system is essential. If a later version of the unit fails a particular test that it had previously passed, the version-control software can provide a list of the source code changes (if any) that have been applied to the unit since that time.{{Citation needed|date=September 2019}}

=== Requires regular reviews ===
It is also essential to implement a sustainable process for ensuring that test case failures are reviewed regularly and addressed immediately.<ref>{{cite web|last=daVeiga|first=Nada|author-link=Nada daVeiga| title=Change Code Without Fear: Utilize a regression safety net|date=2008-02-06| access-date=2008-02-08|url=http://www.ddj.com/development-tools/206105233}}</ref> If such a process is not implemented and ingrained into the team's workflow, the application will evolve out of sync with the unit test suite, increasing false positives and reducing the effectiveness of the test suite.

=== Limitations for embedded system software ===
Unit testing embedded system software presents a unique challenge: Because the software is being developed on a different platform than the one it will eventually run on, you cannot readily run a test program in the actual deployment environment, as is possible with desktop programs.<ref>{{cite web|last=Kucharski|first=Marek|author-link=Marek Kucharski| title=Making Unit Testing Practical for Embedded Development|date=2011-11-23| access-date=2020-07-20|url=https://www.electronicdesign.com/technologies/embedded-revolution/article/21794376/making-unit-testing-practical-for-embedded-development}}</ref>

=== Limitations for testing integration with external systems ===
Unit tests tend to be easiest when a method has input parameters and some output. It is not as easy to create unit tests when a major function of the method is to interact with something external to the application. For example, a method that will work with a database might require a mock up of database interactions to be created, which probably won't be as comprehensive as the real database interactions.<ref>{{cite web| url=http://wiki.c2.com/?UnitTestsAndDatabases | title=Unit Tests And Databases | access-date=2024-01-29}}</ref>{{better source needed|date=February 2019}}