Editing Content moderation (section)

=== Working conditions ===
Employees work by viewing, assessing and deleting disturbing content.<ref>{{Cite news|url=https://www.nytimes.com/2010/07/19/technology/19screen.html|title=Concern for Those Who Screen the Web for Barbarity|first=Brad|last=Stone|work=The New York Times |date=July 18, 2010}}</ref> ''[[Wired (magazine)|Wired]]'' reported in 2014, they may suffer psychological damage.<ref>{{cite magazine |author=Adrian Chen |author-link=Adrian Chen |date=23 October 2014 |title=The Laborers Who Keep Dick Pics and Beheadings Out of Your Facebook Feed |url=https://www.wired.com/2014/10/content-moderation/ |url-status=live |magazine=WIRED |archiveurl=https://web.archive.org/web/20150913025937/http://www.wired.com/2014/10/content-moderation/ |archivedate=2015-09-13}}</ref><ref>{{cite web|url=http://www.theawl.com/2014/10/the-internets-invisible-sin-eaters|title=The Internet's Invisible Sin-Eaters|work=The Awl |archiveurl=https://web.archive.org/web/20150908012259/http://www.theawl.com/2014/10/the-internets-invisible-sin-eaters |archivedate=2015-09-08 |url-status=live}}</ref><ref>{{Cite web |date=March 19, 2014 |title=Professor uncovers the Internet's hidden labour force |url=https://news.westernu.ca/2014/03/professor-uncovers-the-internets-hidden-labour-force/ |website=[[Western News]]}}</ref><ref name="idj">{{cite web|url=http://america.aljazeera.com/articles/2015/5/1/invisible-data-janitors-mop-up-top-websites-behind-the-scenes.html|title=Invisible Data Janitors Mop Up Top Websites - Al Jazeera America|work=aljazeera.com}}</ref><ref>{{cite magazine|url=https://www.wired.com/2015/08/facebook-block-offensive-videos-post/|title=Should Facebook Block Offensive Videos Before They Post?|date=26 August 2015|magazine=WIRED}}</ref> In 2017, the Guardian reported [[secondary trauma]] may arise, with symptoms similar to [[PTSD]].<ref name="guardian-2017-05-04">{{cite web|author=Olivia Solon|url=https://www.theguardian.com/technology/2017/may/04/facebook-content-moderators-ptsd-psychological-dangers|title=Facebook is hiring moderators. But is the job too gruesome to handle?|work=The Guardian|date=2017-05-04|access-date=2018-09-13}}</ref> Some large companies such as Facebook offer psychological support<ref name="guardian-2017-05-04" /> and increasingly rely on the use of [[artificial intelligence]] to sort out the most graphic and inappropriate content, but critics claim that it is insufficient.<ref>{{cite web|author=Olivia Solon|url=https://www.theguardian.com/news/2017/may/25/facebook-moderator-underpaid-overburdened-extreme-content|title=Underpaid and overburdened: the life of a Facebook moderator|work=The Guardian|date=2017-05-25|access-date=2018-09-13}}</ref> In 2019, NPR called it a job hazard.<ref name="NPR Job Hazard">{{Cite web|url=https://www.npr.org/2019/07/01/737498507/for-facebook-content-moderators-traumatizing-material-is-a-job-hazard|first=Terry|last=Gross|title=For Facebook Content Moderators, Traumatizing Material Is A Job Hazard|website=NPR.org}}</ref> [[Non-disclosure agreement]]s are the norm when content moderators are hired. This makes moderators more hesitant to speak up about working conditions or organize.<ref name=":0" />

Psychological hazards including stress and [[post-traumatic stress disorder]], combined with the [[precarity]] of [[algorithmic management]] and low wages make content moderation extremely challenging.<ref>{{Cite book |last=Gillespie |first=Tarleton |author-link=Tarleton Gillespie |url=https://cryptome.org/2021/09/custodians-internet.pdf |title=Custodians of the internet: platforms, content moderation, and the hidden decisions that shape social media |publisher=[[Yale University Press]] |year=2018 |isbn=978-0-300-23502-9 |location=}}</ref>{{Rp|page=123}} The number of tasks completed, for example [[Labeled data|labeling content]] as copyright violation, deleting a post containing hate-speech or reviewing graphic content are quantified for performance and [[quality assurance]].<ref name=":0" />

In February 2019, an investigative report by ''[[The Verge]]'' described poor working conditions at [[Cognizant]]'s office in [[Phoenix, Arizona]].<ref name="Trauma Floor">{{cite web |last1=Newton |first1=Casey |date=25 February 2019 |title=The secret lives of Facebook moderators in America |url=https://www.theverge.com/2019/2/25/18229714/cognizant-facebook-content-moderator-interviews-trauma-working-conditions-arizona |url-status=live |archive-url=https://web.archive.org/web/20210221020804/https://www.theverge.com/2019/2/25/18229714/cognizant-facebook-content-moderator-interviews-trauma-working-conditions-arizona |archive-date=21 February 2021 |access-date=20 June 2019 |website=[[The Verge]]}}</ref> Cognizant employees tasked with content moderation for Facebook developed [[mental health]] issues, including [[post-traumatic stress disorder]], as a result of exposure to [[graphic violence]], [[hate speech]], and [[conspiracy theories]] in the videos they were instructed to evaluate.<ref name="Trauma Floor" /><ref>{{cite web |last1=Feiner |first1=Lauren |date=25 February 2019 |title=Facebook content reviewers are coping with PTSD symptoms by having sex and doing drugs at work, report says |url=https://www.cnbc.com/2019/02/25/facebook-content-moderators-describe-a-stressful-work-environment.html |url-status=live |archive-url=https://web.archive.org/web/20190620141815/https://www.cnbc.com/2019/02/25/facebook-content-moderators-describe-a-stressful-work-environment.html |archive-date=20 June 2019 |access-date=20 June 2019 |website=[[CNBC]]}}</ref> Moderators at the Phoenix office reported [[drug abuse]], [[alcohol abuse]], and [[sexual intercourse]] in the workplace, and feared retaliation from [[Termination of employment|terminated]] workers who threatened to harm them.<ref name="Trauma Floor" /><ref>{{cite web |last=Silverstein |first=Jason |date=25 February 2019 |title=Facebook vows to improve content reviewing after moderators say they suffered PTSD |url=https://www.cbsnews.com/news/facebook-vows-to-improve-content-reviewing-after-moderators-say-they-suffered-ptsd/ |url-status=live |archive-url=https://web.archive.org/web/20190620141814/https://www.cbsnews.com/news/facebook-vows-to-improve-content-reviewing-after-moderators-say-they-suffered-ptsd/ |archive-date=20 June 2019 |access-date=20 June 2019 |website=[[CBS News]]}}</ref> In response, a Cognizant representative stated the company would examine the issues in the report.<ref name="Trauma Floor" />

''The Verge'' published a follow-up investigation of Cognizant's [[Tampa, Florida]], office in June 2019.<ref name="Bodies in Seats">{{cite web |last1=Newton |first1=Casey |date=19 June 2019 |title=Three Facebook moderators break their NDAs to expose a company in crisis |url=https://www.theverge.com/2019/6/19/18681845/facebook-moderator-interviews-video-trauma-ptsd-cognizant-tampa |url-status=live |archive-url=https://web.archive.org/web/20220912094501/https://www.theverge.com/2019/6/19/18681845/facebook-moderator-interviews-video-trauma-ptsd-cognizant-tampa |archive-date=12 September 2022 |access-date=20 June 2019 |website=[[The Verge]]}}</ref><ref>{{cite news |last1=Bridge |first1=Mark |date=20 June 2019 |title=Facebook worker who died of heart attack was under 'unworldly' pressure |url=https://www.thetimes.com/business-money/technology/article/facebook-worker-who-died-of-heart-attack-was-under-unworldly-pressure-9t5c8vrfs |url-access=subscription |url-status=live |archive-url=https://web.archive.org/web/20190620142628/https://www.thetimes.co.uk/article/facebook-worker-who-died-of-heart-attack-was-under-unworldly-pressure-9t5c8vrfs |archive-date=20 June 2019 |access-date=20 June 2019 |newspaper=[[The Times]] |issn=0140-0460}}</ref> Employees in the Tampa location described working conditions that were worse than the conditions in the Phoenix office.<ref name="Bodies in Seats" /><ref>{{cite web |last1=Carbone |first1=Christopher |date=20 June 2019 |title=Facebook moderator dies after viewing horrific videos, others share disturbing incidents: report |url=https://www.foxnews.com/tech/facebook-moderator-dies-horrific-videos-others-share-disturbing-incidents |url-status=live |archive-url=https://web.archive.org/web/20190621000151/https://www.foxnews.com/tech/facebook-moderator-dies-horrific-videos-others-share-disturbing-incidents |archive-date=21 June 2019 |access-date=21 June 2019 |website=[[Fox News]]}}</ref><ref>{{cite web |last1=Eadicicco |first1=Lisa |date=19 June 2019 |title=A Facebook content moderator died after suffering heart attack on the job |url=https://www.mysanantonio.com/technology/businessinsider/article/Facebook-content-moderator-died-after-heart-attack-14018228.php |url-status=live |archive-url=https://web.archive.org/web/20191130205600/https://www.mysanantonio.com/technology/businessinsider/article/Facebook-content-moderator-died-after-heart-attack-14018228.php |archive-date=30 November 2019 |access-date=20 June 2019 |website=[[San Antonio Express-News]]}}</ref> Similarly, Meta's outsourced moderation company in Kenya and Ghana reported mental illness, self-harm, attempted suicide, poor working conditions, low pay, and retaliation for advocating for better working conditions.<ref>{{Cite web |title=Meta’s content moderators face worst conditions yet at secret Ghana… |url=https://www.thebureauinvestigates.com/stories/2025-04-27/suicide-attempts-sackings-and-a-vow-of-silence-metas-new-moderators-face-worst-conditions-yet |access-date=2025-05-04 |website=TBIJ |language=en}}</ref>

Moderators were required to sign non-disclosure agreements with Cognizant to obtain the job, although three former workers broke the agreements to provide information to ''The Verge''.<ref name="Bodies in Seats" /><ref>{{cite web |last1=Feiner |first1=Lauren |date=19 June 2019 |title=Facebook content moderators break NDAs to expose shocking working conditions involving gruesome videos and feces smeared on walls |url=https://www.cnbc.com/2019/06/19/facebook-content-moderators-expose-disturbing-working-conditions.html |url-status=live |archive-url=https://web.archive.org/web/20190619160257/https://www.cnbc.com/2019/06/19/facebook-content-moderators-expose-disturbing-working-conditions.html |archive-date=19 June 2019 |access-date=20 June 2019 |website=[[CNBC]]}}</ref> In the Tampa office, workers reported inadequate mental health resources.<ref name="Bodies in Seats" /><ref>{{cite web |last1=Johnson |first1=O’Ryan |date=19 June 2019 |title=Cognizant Getting $200M From Facebook To Moderate Violent Content Amid Allegations Of 'Filthy' Work Conditions: Report |url=https://www.crn.com/news/channel-programs/cognizant-getting-200m-from-facebook-to-moderate-violent-content-amid-allegations-of-filthy-work-conditions-report |url-status=live |archive-url=https://web.archive.org/web/20190620141819/https://www.crn.com/news/channel-programs/cognizant-getting-200m-from-facebook-to-moderate-violent-content-amid-allegations-of-filthy-work-conditions-report |archive-date=20 June 2019 |access-date=20 June 2019 |website=[[CRN (magazine)|CRN]]}}</ref> As a result of exposure to videos depicting graphic violence, [[animal abuse]], and [[child sexual abuse]], some employees developed [[psychological trauma]] and post-traumatic stress disorder.<ref name="Bodies in Seats" /><ref>{{cite web |last=Bufkin |first=Ellie |date=19 June 2019 |title=Report reveals desperate working conditions of Facebook moderators — including death |url=https://www.washingtonexaminer.com/news/report-reveals-desperate-working-conditions-of-facebook-moderators-including-death |url-status=live |archive-url=https://web.archive.org/web/20190620041350/https://www.washingtonexaminer.com/news/report-reveals-desperate-working-conditions-of-facebook-moderators-including-death |archive-date=20 June 2019 |access-date=20 June 2019 |website=[[Washington Examiner]]}}</ref> In response to negative coverage related to its content moderation contracts, a Facebook director indicated that Facebook is in the process of developing a "global resiliency team" that would assist its contractors.<ref name="Bodies in Seats" />