Editing Meta-analysis (section)

== Literature search ==
One of the most important steps of a meta-analysis is data collection. For an efficient database search, appropriate keywords and search limits need to be identified.<ref>{{Cite journal |last1=Grames |first1=Eliza M. |last2=Stillman |first2=Andrew N. |last3=Tingley |first3=Morgan W. |last4=Elphick |first4=Chris S. |date=2019 |editor-last=Freckleton |editor-first=Robert |title=An automated approach to identifying search terms for systematic reviews using keyword co-occurrence networks |journal=Methods in Ecology and Evolution |language=en |volume=10 |issue=10 |pages=1645–1654 |doi=10.1111/2041-210X.13268 |bibcode=2019MEcEv..10.1645G |issn=2041-210X|doi-access=free }}</ref> The use of Boolean operators and search limits can assist the literature search.<ref>{{Cite journal |last1=Sood |first1=Amit |last2=Erwin |first2=Patricia J. |last3=Ebbert |first3=Jon O. |date=2004 |title=Using Advanced Search Tools on PubMed for Citation Retrieval |journal=Mayo Clinic Proceedings |language=en |volume=79 |issue=10 |pages=1295–1300 |doi=10.4065/79.10.1295|pmid=15473412 |doi-access=free }}</ref><ref>{{Cite journal |last1=Vincent |first1=Beatriz |last2=Vincent |first2=Maurice |last3=Ferreira |first3=Carlos Gil |date=2006-03-01 |title=Making PubMed Searching Simple: Learning to Retrieve Medical Literature Through Interactive Problem Solving |journal=The Oncologist |language=en |volume=11 |issue=3 |pages=243–251 |doi=10.1634/theoncologist.11-3-243 |pmid=16549808 |issn=1083-7159|doi-access=free }}</ref> A number of databases are available (e.g., PubMed, Embase, PsychInfo), however, it is up to the researcher to choose the most appropriate sources for their research area.<ref name=":1">{{Cite journal |last=Quintana |first=Daniel S. |date=2015-10-08 |title=From pre-registration to publication: a non-technical primer for conducting a meta-analysis to synthesize correlational data |journal=Frontiers in Psychology |volume=6 |page=1549 |doi=10.3389/fpsyg.2015.01549 |issn=1664-1078 |pmc=4597034 |pmid=26500598 |doi-access=free}}</ref> Indeed, many scientists use duplicate search terms within two or more databases to cover multiple sources.<ref name=":6">{{Cite journal |last1=Bramer |first1=Wichor M. |last2=Giustini |first2=Dean |last3=de Jonge |first3=Gerdien B. |last4=Holland |first4=Leslie |last5=Bekhuis |first5=Tanja |date=2016 |title=De-duplication of database search results for systematic reviews in EndNote |journal=Journal of the Medical Library Association |volume=104 |issue=3 |pages=240–243 |doi=10.3163/1536-5050.104.3.014 |issn=1536-5050 |pmc=4915647 |pmid=27366130}}</ref> The reference lists of eligible studies can also be searched for eligible studies (i.e., snowballing).<ref name=":7">{{Cite journal |last1=Horsley |first1=Tanya |last2=Dingwall |first2=Orvie |last3=Sampson |first3=Margaret |date=2011-08-10 |editor-last=Cochrane Methodology Review Group |title=Checking reference lists to find additional studies for systematic reviews |journal=Cochrane Database of Systematic Reviews |language=en |volume=2011 |issue=8 |pages=MR000026 |doi=10.1002/14651858.MR000026.pub2 |pmc=7388740 |pmid=21833989}}</ref> The initial search may return a large volume of studies.<ref name=":7" /> Quite often, the abstract or the title of the manuscript reveals that the study is not eligible for inclusion, based on the pre-specified criteria.<ref name=":1" /> These studies can be discarded. However, if it appears that the study may be eligible (or even if there is some doubt) the full paper can be retained for closer inspection. The references lists of eligible articles can also be searched for any relevant articles.<ref name=":6" /><ref>{{Cite journal |last1=Bramer |first1=Wichor M. |last2=De Jonge |first2=Gerdien B. |last3=Rethlefsen |first3=Melissa L. |last4=Mast |first4=Frans |last5=Kleijnen |first5=Jos |date=2018-10-04 |title=A systematic approach to searching: an efficient and complete method to develop literature searches |url=http://jmla.pitt.edu/ojs/jmla/article/view/283 |journal=Journal of the Medical Library Association |volume=106 |issue=4 |pages=531–541 |doi=10.5195/jmla.2018.283 |issn=1558-9439 |pmc=6148622 |pmid=30271302}}</ref> These search results need to be detailed in a PRIMSA flow diagram<ref>{{Cite journal |last1=Moher |first1=David |last2=Tetzlaff |first2=Jennifer |last3=Tricco |first3=Andrea C |last4=Sampson |first4=Margaret |last5=Altman |first5=Douglas G |date=2007-03-27 |editor-last=Clarke |editor-first=Mike |title=Epidemiology and Reporting Characteristics of Systematic Reviews |journal=PLOS Medicine |language=en |volume=4 |issue=3 |pages=e78 |doi=10.1371/journal.pmed.0040078 |doi-access=free |issn=1549-1676 |pmc=1831728 |pmid=17388659}}</ref> which details the flow of information through all stages of the review. Thus, it is important to note how many studies were returned after using the specified search terms and how many of these studies were discarded, and for what reason.<ref name=":1" /> The search terms and strategy should be specific enough for a reader to reproduce the search.<ref>{{Cite journal |last1=Lakens |first1=Daniël |last2=Hilgard |first2=Joe |last3=Staaks |first3=Janneke |date=2016 |title=On the reproducibility of meta-analyses: six practical recommendations |journal=BMC Psychology |language=en |volume=4 |issue=1 |page=24 |doi=10.1186/s40359-016-0126-3 |doi-access=free |issn=2050-7283 |pmc=4886411 |pmid=27241618}}</ref> The date range of studies, along with the date (or date period) the search was conducted should also be provided.<ref>{{Cite journal |last1=Nguyen |first1=Phi-Yen |last2=McKenzie |first2=Joanne E. |last3=Hamilton |first3=Daniel G. |last4=Moher |first4=David |last5=Tugwell |first5=Peter |last6=Fidler |first6=Fiona M. |last7=Haddaway |first7=Neal R. |last8=Higgins |first8=Julian P. T. |last9=Kanukula |first9=Raju |last10=Karunananthan |first10=Sathya |last11=Maxwell |first11=Lara J. |last12=McDonald |first12=Steve |last13=Nakagawa |first13=Shinichi |last14=Nunan |first14=David |last15=Welch |first15=Vivian A. |date=2023 |title=Systematic reviewers' perspectives on sharing review data, analytic code, and other materials: A survey |journal=Cochrane Evidence Synthesis and Methods |language=en |volume=1 |issue=2 |doi=10.1002/cesm.12008 |issn=2832-9023|doi-access=free }}</ref>

A data collection form provides a standardized means of collecting data from eligible studies.<ref>{{Cite journal |last1=Page |first1=Matthew J. |last2=Altman |first2=Douglas G. |last3=Shamseer |first3=Larissa |last4=McKenzie |first4=Joanne E. |last5=Ahmadzai |first5=Nadera |last6=Wolfe |first6=Dianna |last7=Yazdi |first7=Fatemeh |last8=Catalá-López |first8=Ferrán |last9=Tricco |first9=Andrea C. |last10=Moher |first10=David |date=2018 |title=Reproducible research practices are underused in systematic reviews of biomedical interventions |url=https://linkinghub.elsevier.com/retrieve/pii/S0895435617305358 |journal=Journal of Clinical Epidemiology |language=en |volume=94 |pages=8–18 |doi=10.1016/j.jclinepi.2017.10.017|pmid=29113936 }}</ref> For a meta-analysis of correlational data, effect size information is usually collected as Pearson's ''r'' statistic.<ref>{{Cite journal |last=Aloe |first=Ariel M. |date=2014 |title=An Empirical Investigation of Partial Effect Sizes in Meta-Analysis of Correlational Data |url=http://www.tandfonline.com/doi/abs/10.1080/00221309.2013.853021 |journal=The Journal of General Psychology |language=en |volume=141 |issue=1 |pages=47–64 |doi=10.1080/00221309.2013.853021 |pmid=24838020 |issn=0022-1309}}</ref><ref>{{Cite journal |last1=Tracz |first1=Susan M. |last2=Elmore |first2=Patricia B. |last3=Pohlmann |first3=John T. |date=1992 |title=Correlational Meta-Analysis: Independent and Nonindependent Cases |url=https://journals.sagepub.com/doi/10.1177/0013164492052004007 |journal=Educational and Psychological Measurement |language=en |volume=52 |issue=4 |pages=879–888 |doi=10.1177/0013164492052004007 |issn=0013-1644}}</ref> Partial correlations are often reported in research, however, these may inflate relationships in comparison to zero-order correlations.<ref>{{Cite journal |last=Cramer |first=Duncan |date=2003 |title=A Cautionary Tale of Two Statistics: Partial Correlation and Standardized Partial Regression |url=http://www.tandfonline.com/doi/abs/10.1080/00223980309600632 |journal=The Journal of Psychology |language=en |volume=137 |issue=5 |pages=507–511 |doi=10.1080/00223980309600632 |pmid=14629080 |s2cid=37557674 |issn=0022-3980}}</ref> Moreover, the partialed out variables will likely vary from study-to-study. As a consequence, many meta-analyses exclude partial correlations from their analysis.<ref name=":1" /> As a final resort, plot digitizers can be used to scrape data points from scatterplots (if available) for the calculation of Pearson's ''r''.<ref>{{Cite journal |last1=Gross |first1=Arnd |last2=Schirm |first2=Sibylle |last3=Scholz |first3=Markus |date=2014 |title=Ycasd– a tool for capturing and scaling data from graphical representations |journal=BMC Bioinformatics |language=en |volume=15 |issue=1 |page=219 |doi=10.1186/1471-2105-15-219 |doi-access=free |pmid=24965054 |pmc=4085079 |issn=1471-2105}}</ref><ref>{{Citation |last1=Cliche |first1=Mathieu |title=Scatteract: Automated Extraction of Data from Scatter Plots |date=2017 |url=https://link.springer.com/10.1007/978-3-319-71249-9_9 |work=Machine Learning and Knowledge Discovery in Databases |volume=10534 |pages=135–150 |editor-last=Ceci |editor-first=Michelangelo |access-date=2023-12-26 |place=Cham |publisher=Springer International Publishing |language=en |doi=10.1007/978-3-319-71249-9_9 |isbn=978-3-319-71248-2 |last2=Rosenberg |first2=David |last3=Madeka |first3=Dhruv |last4=Yee |first4=Connie |arxiv=1704.06687 |s2cid=9543956 |editor2-last=Hollmén |editor2-first=Jaakko |editor3-last=Todorovski |editor3-first=Ljupčo |editor4-last=Vens |editor4-first=Celine}}</ref> Data reporting important study characteristics that may moderate effects, such as the mean age of participants, should also be collected.<ref>{{Cite journal |last1=Moreau |first1=David |last2=Gamble |first2=Beau |date=2022 |title=Conducting a meta-analysis in the age of open science: Tools, tips, and practical recommendations. |url=http://doi.apa.org/getdoi.cfm?doi=10.1037/met0000351 |journal=Psychological Methods |language=en |volume=27 |issue=3 |pages=426–432 |doi=10.1037/met0000351 |pmid=32914999 |s2cid=221619510 |issn=1939-1463}}</ref> A measure of study quality can also be included in these forms to assess the quality of evidence from each study.<ref name=":3">{{Cite journal |last1=McGuinness |first1=Luke A. |last2=Higgins |first2=Julian P. T. |date=2021 |title=Risk-of-bias VISualization (robvis): An R package and Shiny web app for visualizing risk-of-bias assessments |journal=Research Synthesis Methods |language=en |volume=12 |issue=1 |pages=55–61 |doi=10.1002/jrsm.1411 |pmid=32336025 |issn=1759-2879|doi-access=free |hdl=1983/e59b578e-1534-43d9-a438-8bc27b363a9a |hdl-access=free }}</ref> There are more than 80 tools available to assess the quality and risk of bias in observational studies reflecting the diversity of research approaches between fields.<ref name=":3" /><ref>{{Cite journal |last1=Sanderson |first1=S. |last2=Tatt |first2=I. D |last3=Higgins |first3=J. P. |date=2007-06-01 |title=Tools for assessing quality and susceptibility to bias in observational studies in epidemiology: a systematic review and annotated bibliography |journal=International Journal of Epidemiology |language=en |volume=36 |issue=3 |pages=666–676 |doi=10.1093/ije/dym018 |issn=0300-5771|doi-access=free |pmid=17470488 }}</ref><ref>{{Cite journal |last1=Haddaway |first1=Neal R. |last2=Macura |first2=Biljana |last3=Whaley |first3=Paul |last4=Pullin |first4=Andrew S. |date=2018 |title=ROSES RepOrting standards for Systematic Evidence Syntheses: pro forma, flow-diagram and descriptive summary of the plan and conduct of environmental systematic reviews and systematic maps |journal=Environmental Evidence |language=en |volume=7 |issue=1 |doi=10.1186/s13750-018-0121-7 |doi-access=free |bibcode=2018EnvEv...7....7H |issn=2047-2382}}</ref> These tools usually include an assessment of how dependent variables were measured, appropriate selection of participants, and appropriate control for confounding factors. Other quality measures that may be more relevant for correlational studies include sample size, psychometric properties, and reporting of methods.<ref name=":1" />

A final consideration is whether to include studies from the [[Grey literature|gray literature]],<ref>{{Cite journal |last=Paez |first=Arsenio |date=2017 |title=Gray literature: An important resource in systematic reviews |url=https://onlinelibrary.wiley.com/doi/10.1111/jebm.12266 |journal=Journal of Evidence-Based Medicine |language=en |volume=10 |issue=3 |pages=233–240 |doi=10.1111/jebm.12266 |pmid=28857505 |issn=1756-5383}}</ref> which is defined as research that has not been formally published.<ref>{{Cite journal |last1=Conn |first1=Vicki S. |last2=Valentine |first2=Jeffrey C. |last3=Cooper |first3=Harris M. |last4=Rantz |first4=Marilyn J. |date=2003 |title=Grey Literature in Meta-Analyses |url=http://journals.lww.com/00006199-200307000-00008 |journal=Nursing Research |language=en |volume=52 |issue=4 |pages=256–261 |doi=10.1097/00006199-200307000-00008 |pmid=12867783 |s2cid=27109643 |issn=0029-6562}}</ref> This type of literature includes conference abstracts,<ref>{{Cite journal |last1=Scherer |first1=Roberta W. |last2=Saldanha |first2=Ian J. |date=2019 |title=How should systematic reviewers handle conference abstracts? A view from the trenches |journal=Systematic Reviews |language=en |volume=8 |issue=1 |page=264 |doi=10.1186/s13643-019-1188-0 |doi-access=free |issn=2046-4053 |pmc=6836535 |pmid=31699124}}</ref> dissertations,<ref>{{Cite journal |last1=Hartling |first1=Lisa |last2=Featherstone |first2=Robin |last3=Nuspl |first3=Megan |last4=Shave |first4=Kassi |last5=Dryden |first5=Donna M. |last6=Vandermeer |first6=Ben |date=2017 |title=Grey literature in systematic reviews: a cross-sectional study of the contribution of non-English reports, unpublished studies and dissertations to the results of meta-analyses in child-relevant reviews |journal=BMC Medical Research Methodology |language=en |volume=17 |issue=1 |page=64 |doi=10.1186/s12874-017-0347-z |doi-access=free |issn=1471-2288 |pmc=5395863 |pmid=28420349}}</ref> and pre-prints.<ref>{{Cite journal |last1=Haddaway |first1=N.R. |last2=Woodcock |first2=P. |last3=Macura |first3=B. |last4=Collins |first4=A. |date=2015 |title=Making literature reviews more reliable through application of lessons from systematic reviews |url=https://www.jstor.org/stable/24761072 |journal=Conservation Biology |volume=29 |issue=6 |pages=1596–1605 |doi=10.1111/cobi.12541 |jstor=24761072 |pmid=26032263 |bibcode=2015ConBi..29.1596H |s2cid=20624428 |issn=0888-8892}}</ref> While the inclusion of gray literature reduces the risk of publication bias, the methodological quality of the work is often (but not always) lower than formally published work.<ref>{{Cite journal |last1=Egger |first1=M |last2=Jüni |first2=P |last3=Bartlett |first3=C |last4=Holenstein |first4=F |last5=Sterne |first5=J |date=2003 |title=How important are comprehensive literature searches and the assessment of trial quality in systematic reviews? Empirical study |journal=Health Technology Assessment |volume=7 |issue=1 |pages=1–82 |doi=10.3310/hta7010 |issn=1366-5278|doi-access=free |pmid=12583822 }}</ref><ref>{{Citation |last1=Lefebvre |first1=Carol |title=Searching for and selecting studies |date=2019-09-23 |url=https://onlinelibrary.wiley.com/doi/10.1002/9781119536604.ch4 |work=Cochrane Handbook for Systematic Reviews of Interventions |pages=67–107 |editor-last=Higgins |editor-first=Julian P.T. |access-date=2023-12-26 |edition=1 |publisher=Wiley |language=en |doi=10.1002/9781119536604.ch4 |isbn=978-1-119-53662-8 |last2=Glanville |first2=Julie |last3=Briscoe |first3=Simon |last4=Littlewood |first4=Anne |last5=Marshall |first5=Chris |last6=Metzendorf |first6=Maria-Inti |last7=Noel-Storr |first7=Anna |last8=Rader |first8=Tamara |last9=Shokraneh |first9=Farhad |s2cid=204603849 |editor2-last=Thomas |editor2-first=James |editor3-last=Chandler |editor3-first=Jacqueline |editor4-last=Cumpston |editor4-first=Miranda}}</ref> Reports from conference proceedings, which are the most common source of gray literature,<ref>{{Cite journal |last1=McAuley |first1=Laura |last2=Pham |first2=Ba' |last3=Tugwell |first3=Peter |last4=Moher |first4=David |date=2000 |title=Does the inclusion of grey literature influence estimates of intervention effectiveness reported in meta-analyses? |url=https://linkinghub.elsevier.com/retrieve/pii/S0140673600027860 |journal=The Lancet |language=en |volume=356 |issue=9237 |pages=1228–1231 |doi=10.1016/S0140-6736(00)02786-0|pmid=11072941 |s2cid=33777183 }}</ref> are poorly reported<ref>{{Cite journal |last1=Hopewell |first1=Sally |last2=Clarke |first2=Mike |date=2005 |title=Abstracts presented at the American Society of Clinical Oncology conference: how completely are trials reported? |url=http://dx.doi.org/10.1191/1740774505cn091oa |journal=Clinical Trials |volume=2 |issue=3 |pages=265–268 |doi=10.1191/1740774505cn091oa |pmid=16279150 |s2cid=3601317 |issn=1740-7745}}</ref> and data in the subsequent publication is often inconsistent, with differences observed in almost 20% of published studies.<ref>{{Cite journal |last1=Bhandari |first1=Mohit |last2=Devereaux |first2=P. J. |last3=Guyatt |first3=Gordon H. |last4=Cook |first4=Deborah J. |last5=Swiontkowski |first5=Marc F. |last6=Sprague |first6=Sheila |last7=Schemitsch |first7=Emil H. |date=2002 |title=An Observational Study of Orthopaedic Abstracts and Subsequent Full-Text Publications |url=http://dx.doi.org/10.2106/00004623-200204000-00017 |journal=The Journal of Bone and Joint Surgery-American Volume |volume=84 |issue=4 |pages=615–621 |doi=10.2106/00004623-200204000-00017 |pmid=11940624 |s2cid=8807106 |issn=0021-9355}}</ref>