Editing Delphi method (section)

==Applications==

===Use in forecasting===
First applications of the Delphi method were in the field of science and technology forecasting. The objective of the method was to combine expert opinions on likelihood and expected development time, of the particular technology, in a single indicator. One of the first such reports, prepared in 1964 by Gordon and Helmer, assessed the direction of long-term trends in science and technology development, covering such topics as scientific breakthroughs, [[population control]], [[automation]], space progress, war prevention and weapon systems. Other forecasts of technology were dealing with vehicle-highway systems, industrial robots, intelligent internet, broadband connections, and technology in education.

Later the Delphi method was applied in other places, especially those related to public policy issues, such as [[Economic indicator|economic trend]]s, health and education. It was also applied successfully and with high accuracy in business forecasting. For example, in one case reported by Basu and Schroeder (1977),<ref>{{cite journal| vauthors = Basu S, Schroeder RG |title=Incorporating Judgments in Sales Forecasts: Application of the Delphi Method at American Hoist & Derrick|journal=Interfaces|date=May 1977|volume=7|issue=3|pages=18–27|doi=10.1287/inte.7.3.18}}</ref> the Delphi method predicted the sales of a new product during the first two years with inaccuracy of 3–4% compared with actual sales. Quantitative methods produced errors of 10–15%, and traditional unstructured forecast methods had errors of about 20%. (This is only one example; the overall accuracy of the technique is mixed.)

The Delphi method has also been used as a tool to implement multi-stakeholder approaches for participative policy-making in developing countries. The governments of Latin America and the Caribbean have successfully used the Delphi method as an open-ended public-private sector approach to identify the most urgent challenges for their regional ICT-for-development [[eLAC Action Plans]].<ref name="HilbertMilesOthmer2009">{{cite journal | vauthors = Hilbert M, Miles I, Othmer J | year = 2009 | title = Foresight tools for participative policy-making in inter-governmental processes in developing countries: Lessons learned from the eLAC Policy Priorities Delphi | url = http://martinhilbert.net/Hilbert_etal.eLACdelphi.pdf | journal = [[Technological Forecasting and Social Change]] | volume = 15 | issue = 2| pages = 880–896 | doi = 10.1016/j.techfore.2009.01.001 | s2cid = 154784808 }}</ref> As a result, governments have widely acknowledged the value of collective intelligence from civil society, academic and private sector participants of the Delphi, especially in a field of rapid change, such as technology policies.

=== Use in patent participation identification ===
In the early 1980s Jackie Awerman of Jackie Awerman Associates, Inc. designed a modified Delphi method for identifying the roles of various contributors to the creation of a patent-eligible product. (Epsilon Corporation, Chemical Vapor Deposition Reactor) The results were then used by patent attorneys to determine bonus distribution percentage to the general satisfaction of all team members.{{Citation needed|date=July 2019}}

===Use in policy-making===
From the 1970s, the use of the Delphi technique in public policy-making introduces a number of methodological innovations. In particular:
* the need to examine several types of items (not only ''forecasting'' items but, typically, ''issue'' items, ''goal'' items, and ''option'' items) leads to introducing different evaluation scales which are not used in the standard Delphi. These often include ''desirability'', ''feasibility'' (technical and political) and ''probability'', which the analysts can use to outline different scenarios: the ''desired'' scenario (from desirability), the ''potential'' scenario (from feasibility) and the ''expected'' scenario (from probability);
* the complexity of issues posed in public policy-making tends to increase weighting of panelists’ arguments, such as soliciting pros and cons for each item along with new items for panel consideration;
* likewise, methods measuring panel evaluations tend to increase sophistication such as multi-dimensional scaling.

Further innovations come from the use of computer-based (and later web-based) Delphi conferences. According to Turoff and Hiltz,<ref>{{cite book | vauthors = Turoff M, Hiltz SR | chapter = Computer based Delphi processes. | veditors = Adler M, Ziglio E| title = Gazing into the oracle: The Delphi method and its application to social policy and public health. | date = 1996 | pages = 56–85 }}</ref> in computer-based Delphis: 
* the iteration structure used in the paper Delphis, which is divided into three or more discrete rounds, can be replaced by a process of continuous (roundless) interaction, enabling panelists to change their evaluations at any time;
* the statistical group response can be updated in real-time, and shown whenever a panelist provides a new evaluation.

According to Bolognini,<ref name="Bolognini">{{cite book | vauthors = Bolognini M |title=Democrazia elettronica. Metodo Delphi e politiche pubbliche (Electronic Democracy. Delphi Method and Public Policy-Making) |year=2001 |location=Rome |publisher=Carocci Editore |language=it |isbn=978-88-430-2035-5}}. A summary is also in {{cite book |title=The Millennium Project. Futures Research Methodology |year=2009 | veditors = Glenn JC, Gordon TJ |chapter=23 |location=New York |publisher=Amer Council for the United Nations |isbn=978-0-9818941-1-9 }}</ref> web-based Delphis offer two further possibilities, relevant in the context of interactive policy-making and [[e-democracy]]. These are: [[File:HYPERD.GIF|thumb|right|A web-based communication structure (Hyperdelphi)<ref name="Bolognini"/>]]
* the involvement of a large number of participants,
* the use of two or more panels representing different groups (such as policy-makers, experts, citizens), which the administrator can give tasks reflecting their diverse roles and expertise, and make them to interact within ad hoc communication structures. For example, the  ''policy community'' members (policy-makers and experts) may interact as part of the ''main conference'' panel, while they receive inputs from a ''virtual community'' (citizens, associations etc.) involved in a ''side conference''. These web-based variable communication structures, which he calls ''Hyperdelphi'' (HD), are designed to make Delphi conferences "more fluid and adapted to the hypertextual and interactive nature of digital communication".
One successful example of a (partially) web-based policy Delphi is the five-round Delphi exercise (with 1,454 contributions) for the creation of the [[eLAC Action Plans]] in Latin America. It is believed to be the most extensive online participatory policy-making foresight exercise in the history of intergovernmental processes in the developing world at this time.<ref name="HilbertMilesOthmer2009"/> In addition to the specific policy guidance provided, the authors list the following lessons learned: "(1) the potential of Policy Delphi methods to introduce transparency and accountability into public decision-making, especially in developing countries; (2) the utility of foresight exercises to foster multi-agency networking in the development community; (3) the usefulness of embedding foresight exercises into established mechanisms of representative democracy and international multilateralism, such as the United Nations; (4) the potential of online tools to facilitate participation in resource-scarce developing countries; and (5) the resource-efficiency stemming from the scale of international foresight exercises, and therefore its adequacy for resource-scarce regions."<ref name="HilbertMilesOthmer2009"/>

=== Use in health settings ===
The Delphi technique is widely used to help reach expert consensus in health-related settings.<ref name="Taylor2020"/> For example, it is frequently employed in the development of [[medical guideline]]s and [[Protocol (science)|protocol]]s.<ref name="Taylor2020">{{cite journal |vauthors=Taylor E |title=We Agree, Don't We? The Delphi Method for Health Environments Research |journal=HERD |volume=13 |issue=1 |pages=11–23 |date=2020 |pmid=31887097 |doi=10.1177/1937586719887709 |s2cid=209519275 |url=https://journals.sagepub.com/doi/pdf/10.1177/1937586719887709|url-access=subscription }}</ref>

==== Public health ====

Some examples of its application in [[public health]] contexts include [[metabolic dysfunction–associated steatotic liver disease|non-alcoholic fatty liver disease]],<ref>{{cite journal | vauthors = Lazarus JV, Mark HE, Anstee QM, Arab JP, Batterham RL, Castera L, Cortez-Pinto H, Crespo J, Cusi K, Dirac MA, Francque S, George J, Hagström H, Huang TT, Ismail MH, Kautz A, Sarin SK, Loomba R, Miller V, Newsome PN, Ninburg M, Ocama P, Ratziu V, Rinella M, Romero D, Romero-Gómez M, Schattenberg JM, Tsochatzis EA, Valenti L, Wong VW, Yilmaz Y, Younossi ZM, Zelber-Sagi S | display-authors = 6 | title = Advancing the global public health agenda for NAFLD: a consensus statement | journal = Nature Reviews. Gastroenterology & Hepatology | volume = 19 | issue = 1 | pages = 60–78 | date = January 2022 | pmid = 34707258 | doi = 10.1038/s41575-021-00523-4 | s2cid = 239891445 | doi-access = free | hdl = 11424/243850 | hdl-access = free }}</ref> iodine deficiency disorders,<ref>{{cite journal | vauthors = Schaffner M, Rochau U, Stojkov I, Qerimi Rushaj V, Völzke H, Marckmann G, Lazarus JH, Oberaigner W, Siebert U | display-authors = 6 | title = Barriers Against Prevention Programs for Iodine Deficiency Disorders in Europe: A Delphi Study | journal = Thyroid | volume = 31 | issue = 4 | pages = 649–657 | date = April 2021 | pmid = 32912084 | doi = 10.1089/thy.2020.0065 | s2cid = 221622474 }}</ref> building responsive health systems for communities affected by migration,<ref>{{cite journal | vauthors = Pottie K, Hui C, Rahman P, Ingleby D, Akl EA, Russell G, Ling L, Wickramage K, Mosca D, Brindis CD | display-authors = 6 | title = Building Responsive Health Systems to Help Communities Affected by Migration: An International Delphi Consensus | journal = International Journal of Environmental Research and Public Health | volume = 14 | issue = 2 | pages = 144 | date = February 2017 | pmid = 28165380 | pmc = 5334698 | doi = 10.3390/ijerph14020144 | doi-access = free }}</ref> the role of health systems in advancing well-being for those living with HIV,<ref>{{cite journal | vauthors = Lazarus JV, Safreed-Harmon K, Kamarulzaman A, Anderson J, Leite RB, Behrens G, Bekker LG, Bhagani S, Brown D, Brown G, Buchbinder S, Caceres C, Cahn PE, Carrieri P, Caswell G, Cooke GS, Monforte AD, Dedes N, Del Amo J, Elliott R, El-Sadr WM, Fuster-Ruiz de Apodaca MJ, Guaraldi G, Hallett T, Harding R, Hellard M, Jaffar S, Kall M, Klein M, Lewin SR, Mayer K, Pérez-Molina JA, Moraa D, Naniche D, Nash D, Noori T, Pozniak A, Rajasuriar R, Reiss P, Rizk N, Rockstroh J, Romero D, Sabin C, Serwadda D, Waters L | display-authors = 6 | title = Consensus statement on the role of health systems in advancing the long-term well-being of people living with HIV | journal = Nature Communications | volume = 12 | issue = 1 | pages = 4450 | date = July 2021 | pmid = 34272399 | pmc = 8285468 | doi = 10.1038/s41467-021-24673-w | bibcode = 2021NatCo..12.4450L }}</ref> on policies and interventions to reduce harmful [[gambling]],<ref>{{cite journal |last1=Regan |first1=Marguerite |last2=Smolar |first2=Maria |last3=Burton |first3=Robyn |last4=Clarke |first4=Zoe |last5=Sharpe |first5=Casey |last6=Henn |first6=Clive |last7=Marsden |first7=John |date=August 2022 |title=Policies and interventions to reduce harmful gambling: an international Delphi consensus and implementation rating study |journal=[[Lancet Public Health]] |volume=7 |issue=8 |pages=e705–e717 |doi=10.1016/S2468-2667(22)00137-2 |doi-access=free |pmid=35907421 }}</ref> on the regulation of [[electronic cigarettes]]<ref>{{cite journal |last1=Berlin |first1=Ivan |last2=Jacot-Sadowski |first2=Isabelle |last3=Humair |first3=Jean-Paul |last4=Cornuz |first4=Jacques |date=7 September 2021 |title=International expert consensus on electronic nicotine delivery systems and heated tobacco products: a Delphi survey |journal=[[BMJ Open]] |volume=11 |issue=9 |pages=e045724 |doi=10.1136/bmjopen-2020-045724 |doi-access=free |pmid=34493505 |pmc=8451280 }}</ref><ref>{{cite journal |last1=Velarde Crézé |first1=Camille |last2=Lebon |first2=Luc |last3=Duperrex |first3=Olivier |last4=Faivre |first4=Vincent |last5=Pasche |first5=Myriam |last6=Cornuz |first6=Jacques |date=1 February 2023 |title=New puff-like disposable electronic cigarettes: expert consensus on their regulation |url=https://www.revmed.ch/revue-medicale-suisse/2023/revue-medicale-suisse-812/nouvelles-cigarettes-electroniques-jetables-puffs-consensus-d-expert-e-s-sur-leur-reglementation |journal=[[Revue Médicale Suisse]] |volume=19 |issue=812 |pages=181–185 |doi=10.53738/REVMED.2023.19.812.181 |doi-access=free |pmid=36723643 |access-date=21 December 2024}}</ref><ref>{{cite journal |last1=Asfar |first1=Taghrid |last2=Schmidt |first2=Michael |last3=Oluwole |first3=Olusanya Joshua |last4=Casas |first4=Alejandra |last5=Friedman |first5=Lily  |last6=Ferdous |first6=Tarana  |last7=Noar |first7=Seth |last8=Eissenberg |first8=Thomas |last9=Maziak |first9=Wasim |date=2024 |title=Building consensus on a set of ENDS-specific pictorial health warnings: a Delphi study among a tobacco control expert panel |url=https://tobaccocontrol.bmj.com/content/early/2024/02/12/tc-2023-058384 |journal=[[Tobacco Control (journal)|Tobacco Control]] |doi=10.1136/tc-2023-058384 |pmid=38346872 |pmc=11317547 |pmc-embargo-date=August 12, 2025 |access-date=21 December 2024|url-access=subscription }}</ref> and on recommendations to end the [[COVID-19 pandemic]].<ref name=":0">{{cite journal | vauthors = Lazarus JV, Romero D, Kopka CJ, Karim SA, Abu-Raddad LJ, Almeida G, Baptista-Leite R, Barocas JA, Barreto ML, Bar-Yam Y, Bassat Q, Batista C, Bazilian M, Chiou ST, Del Rio C, Dore GJ, Gao GF, Gostin LO, Hellard M, Jimenez JL, Kang G, Lee N, Matičič M, McKee M, Nsanzimana S, Oliu-Barton M, Pradelski B, Pyzik O, Rabin K, Raina S, Rashid SF, Rathe M, Saenz R, Singh S, Trock-Hempler M, Villapol S, Yap P, Binagwaho A, Kamarulzaman A, El-Mohandes A | display-authors = 6 | title = A multinational Delphi consensus to end the COVID-19 public health threat | journal = Nature | volume = 611 | issue = 7935 | pages = 332–345 | date = November 2022 | pmid = 36329272 | pmc = 9646517 | doi = 10.1038/s41586-022-05398-2 | bibcode = 2022Natur.611..332L }}</ref>

==== Reporting guidelines====
Use of the Delphi method in the development of guidelines for the reporting of health research<ref name=Moher2010>{{cite journal | vauthors = Moher D, Schulz KF, Simera I, Altman DG | title = Guidance for developers of health research reporting guidelines | journal = PLOS Medicine | volume = 7 | issue = 2 | pages = e1000217 | date = February 2010 | pmid = 20169112 | pmc = 2821895 | doi = 10.1371/journal.pmed.1000217 | doi-access = free }}</ref> is recommended, especially for experienced developers.<ref name="EQUATOR">{{cite web |title=Developing your reporting guideline |url=https://www.equator-network.org/toolkits/developing-a-reporting-guideline/developing-your-reporting-guideline/ |website=www.equator-network.org |publisher=[[EQUATOR Network]] |archive-url=https://web.archive.org/web/20220929172737/https://www.equator-network.org/toolkits/developing-a-reporting-guideline/developing-your-reporting-guideline/ |archive-date=29 September 2022 |language=en |url-status=live}}</ref> Since this advice was made in 2010, two systematic reviews have found that fewer than 30% of published reporting guidelines incorporated Delphi methods into the development process.<ref>{{cite journal | vauthors = Wang X, Chen Y, Yang N, Deng W, Wang Q, Li N, Yao L, Wei D, Chen G, Yang K | display-authors = 6 | title = Methodology and reporting quality of reporting guidelines: systematic review | journal = BMC Medical Research Methodology | volume = 15 | issue = 74 | pages = 74 | date = September 2015 | pmid = 26395179 | pmc = 4579604 | doi = 10.1186/s12874-015-0069-z | doi-access = free }}</ref><ref>{{cite journal | vauthors = Banno M, Tsujimoto Y, Kataoka Y | title = The majority of reporting guidelines are not developed with the Delphi method: a systematic review of reporting guidelines | journal = Journal of Clinical Epidemiology | volume = 124 | pages = 50–57 | date = August 2020 | pmid = 32302679 | doi = 10.1016/j.jclinepi.2020.04.010 | s2cid = 215809096 }}</ref>

=== Online Delphi systems ===
{{main|Real-time Delphi}}
A number of Delphi forecasts are conducted using web sites that allow the process to be conducted in real-time. For instance, the TechCast Project uses a panel of 100 experts worldwide to forecast breakthroughs in all fields of science and technology. Another example is the [[Horizon Project]], where educational futurists collaborate online using the Delphi method to come up with the technological advancements to look out for in education for the next few years.