Editing Hawthorne effect (section)

== Secondary observer effect ==

Despite the observer effect as popularized in the Hawthorne experiments being perhaps falsely identified (see above discussion), the popularity and plausibility of the observer effect in theory has led researchers to postulate that this effect could take place at a second level. Thus it has been proposed that there is a secondary observer effect when researchers working with secondary data such as survey data or various indicators may impact the results of their scientific research. Rather than having an effect on the subjects (as with the primary observer effect), the researchers likely have their own idiosyncrasies that influence how they handle the data and even what data they obtain from secondary sources. For one, the researchers may choose seemingly innocuous steps in their statistical analyses that end up causing significantly different results using the same data; e.g. weighting strategies, factor analytic techniques, or choice of estimation. In addition, researchers may use software packages that have different default settings that lead to small but significant fluctuations. Finally, the data that researchers use may not be identical, even though it seems so. For example, the [[OECD]] collects and distributes various socio-economic data; however, these data change over time such that a researcher who downloads the Australian [[Gross domestic product|GDP]] data for the year 2000 may have slightly different values than a researcher who downloads the same Australian GDP 2000 data a few years later. The idea of the secondary observer effect was floated by Nate Breznau in a thus far relatively obscure paper.<ref>{{Cite journal|last=Breznau|first=Nate|date=2016-05-03|title=Secondary observer effects: idiosyncratic errors in small-N secondary data analysis|journal=International Journal of Social Research Methodology|volume=19|issue=3|pages=301–318|doi=10.1080/13645579.2014.1001221|s2cid=145402768|issn=1364-5579|url=http://osf.io/hzm9n/}}</ref>

Although little attention has been paid to this phenomenon, the scientific implications are very large.<ref>{{Cite journal|last1=Shi|first1=Yuan|last2=Sorenson|first2=Olav|last3=Waguespack|first3=David|date=2017-01-30|title=Temporal Issues in Replication: The Stability of Centrality-Based Advantage|journal=Sociological Science|language=en-US|volume=4|pages=107–122|doi=10.15195/v4.a5|issn=2330-6696|doi-access=free}}</ref> Evidence of this effect may be seen in recent studies that assign a particular problem to a number of researchers or research teams who then work independently using the same data to try and find a solution. This is a process called [[crowdsourcing]] data analysis and was used in a groundbreaking study by Silberzahn, Rafael, Eric Uhlmann, Dan Martin and Brian Nosek et al. (2015) about red cards and player race in football (i.e. soccer).<ref>{{Cite web|url=https://osf.io/j5v8f/|title=Many analysts, one dataset: Making transparent how variations in analytical choices affect|last1=Silberzahn|first1=Raphael|last2=Uhlmann|first2=Eric L.|last3=Martin|first3=Daniel P.|last4=Nosek|first4=Brian A.|display-authors=etal|date=2015|website=OSF.io|access-date=2016-12-07}}</ref><ref>{{Cite news|url=https://policyandpoliticsblog.com/2015/03/26/crowdsourcing-data-to-improve-macro-comparative-research/|title=Crowdsourcing Data to Improve Macro-Comparative Research|date=2015-03-26|newspaper=Policy and Politics Journal|language=en-US|access-date=2016-12-07}}</ref>