Editing Reproducibility (section)

===Reproducible research method===
The term ''reproducible research'' refers to the idea that scientific results should be documented in such a way that their deduction is fully transparent. This requires a detailed description of the methods used to obtain the data<ref>{{Cite journal|last=King|first=Gary|date=1995|title=Replication, Replication|journal=PS: Political Science and Politics|volume=28|issue=3|pages=444–452|doi=10.2307/420301|jstor=420301|s2cid=250480339 |issn=1049-0965|url=http://nrs.harvard.edu/urn-3:HUL.InstRepos:4266312}}</ref><ref>{{cite journal|last1=Kühne |first1=Martin |last2=Liehr |first2=Andreas W. |year=2009 |title=Improving the Traditional Information Management in Natural Sciences |doi=10.2481/dsj.8.18 |journal=Data Science Journal |volume=8 |issue=1 |pages=18–27 |url=https://datascience.codata.org/jms/article/download/dsj.8.18/198 |doi-access=free}}</ref>
and making the full dataset and the code to calculate the results easily accessible.<ref>{{cite journal|last1=Fomel |first1=Sergey |author-link2=Jon Claerbout |last2=Claerbout |first2=Jon |year=2009 |title=Guest Editors' Introduction: Reproducible Research |journal=Computing in Science and Engineering |volume=11 |issue=1 |pages=5–7 |doi=10.1109/MCSE.2009.14 |bibcode=2009CSE....11a...5F}}</ref><ref name="buckheit1995" /><ref>{{cite journal|title=The Yale Law School Round Table on Data and Core Sharing: "Reproducible Research" |journal=Computing in Science and Engineering |volume=12 |issue=5 |pages=8–12 |doi=10.1109/MCSE.2010.113 |year=2010 |doi-access=}}</ref><ref>{{cite journal|last1=Marwick |first1=Ben |year=2016 |title=Computational reproducibility in archaeological research: Basic principles and a case study of their implementation |journal=Journal of Archaeological Method and Theory |volume=24 |issue=2 |pages=424–450 |doi=10.1007/s10816-015-9272-9 |s2cid=43958561 |url=https://ro.uow.edu.au/smhpapers/4034}}</ref><ref>{{cite journal|last1=Goodman|first1=Steven N.|last2=Fanelli|first2=Daniele|last3=Ioannidis|first3=John P. A.|title=What does research reproducibility mean?|journal=Science Translational Medicine|date=1 June 2016|volume=8|issue=341|pages=341ps12|doi=10.1126/scitranslmed.aaf5027|pmid=27252173|doi-access=free}}</ref><ref>{{Cite journal|last1=Harris J.K|last2=Johnson K.J|last3=Combs T.B|last4=Carothers B.J|last5=Luke D.A|last6=Wang X|date=2019|title=Three Changes Public Health Scientists Can Make to Help Build a Culture of Reproducible Research|journal=Public Health Rep. Public Health Reports|volume=134|issue=2|pages=109–111|issn=0033-3549|oclc=7991854250|doi=10.1177/0033354918821076|pmid=30657732|pmc=6410469}}</ref>
This is the essential part of [[open science]].

To make any research project computationally reproducible, general practice involves all data and files being clearly separated, labelled, and documented. All operations should be fully documented and automated as much as practicable, avoiding manual intervention where feasible. The workflow should be designed as a sequence of smaller steps that are combined so that the intermediate outputs from one step directly feed as inputs into the next step. Version control should be used as it lets the history of the project be easily reviewed and allows for the documenting and tracking of changes in a transparent manner.

A basic workflow for reproducible research involves data acquisition, data processing and data analysis. Data acquisition primarily consists of obtaining primary data from a primary source such as surveys, field observations, experimental research, or obtaining data from an existing source. Data processing involves the processing and review of the raw data collected in the first stage, and includes data entry, data manipulation and filtering and may be done using software. The data should be digitized and prepared for data analysis. Data may be analysed with the use of software to interpret or visualise statistics or data to produce the desired results of the research such as quantitative results including figures and tables. The use of software and automation enhances the reproducibility of research methods.<ref>{{cite book |last1=Kitzes |first1=Justin |last2=Turek |first2=Daniel |last3=Deniz |first3=Fatma |title=The practice of reproducible research case studies and lessons from the data-intensive sciences |date=2018 |publisher=University of California Press |location=Oakland, California |isbn=9780520294745 |pages=19–30 |jstor=10.1525/j.ctv1wxsc7 |url=http://www.jstor.org/stable/10.1525/j.ctv1wxsc7}}</ref>

There are systems that facilitate such documentation, like the [[R (programming language)|R]] [[Markdown]] language<ref>{{cite journal|last1=Marwick|first1=Ben|last2=Boettiger|first2=Carl|last3=Mullen|first3=Lincoln|title=Packaging data analytical work reproducibly using R (and friends)|journal=The American Statistician|volume=72|date=29 September 2017|pages=80–88|doi=10.1080/00031305.2017.1375986|s2cid=125412832|url=http://ro.uow.edu.au/cgi/viewcontent.cgi?article=6445&context=smhpapers}}</ref>
or the [[Jupyter]] notebook.<ref>{{cite conference|title=Jupyter Notebooks–a publishing format for reproducible computational workflows |url=https://eprints.soton.ac.uk/403913/1/STAL9781614996491-0087.pdf |archive-url=https://web.archive.org/web/20180110174609/https://eprints.soton.ac.uk/403913/1/STAL9781614996491-0087.pdf |archive-date=2018-01-10 |url-status=live |book-title=Positioning and Power in Academic Publishing: Players, Agents and Agendas |editor1-last=Loizides |editor1-first=F |editor2-last=Schmidt |editor2-first=B |publisher=IOS Press |last1=Kluyver |first1=Thomas |last2=Ragan-Kelley |first2=Benjamin |last3=Perez |first3=Fernando |last4=Granger |first4=Brian |last5=Bussonnier
|first5=Matthias |last6=Frederic |first6=Jonathan |last7=Kelley
|first7=Kyle |last8=Hamrick |first8=Jessica |last9=Grout |first9=Jason |last10=Corlay |first10=Sylvain |conference=20th International Conference on Electronic Publishing |pages=87–90 |year=2016|doi=10.3233/978-1-61499-649-1-87}}</ref><ref>{{cite journal |last1=Beg |first1=Marijan |last2=Taka |first2=Juliette |last3=Kluyver |first3=Thomas |last4=Konovalov |first4=Alexander |last5=Ragan-Kelley |first5=Min |last6=Thiery |first6=Nicolas M. |last7=Fangohr |first7=Hans |title=Using Jupyter for Reproducible Scientific Workflows |journal=Computing in Science & Engineering |date=1 March 2021 |volume=23 |issue=2 |pages=36–46 |doi=10.1109/MCSE.2021.3052101|arxiv=2102.09562 |bibcode=2021CSE....23b..36B |s2cid=231979203 }}</ref><ref>{{cite journal |last1=Granger |first1=Brian E. |last2=Perez |first2=Fernando |title=Jupyter: Thinking and Storytelling With Code and Data |journal=Computing in Science & Engineering |date=1 March 2021 |volume=23 |issue=2 |pages=7–14 |doi=10.1109/MCSE.2021.3059263|bibcode=2021CSE....23b...7G |s2cid=232413965 |doi-access=free }}</ref>
The [[Open Science Framework]] provides a platform and useful tools to support reproducible research.