Editing Artificial intelligence (section)

====Algorithmic bias and fairness====
{{Main|Algorithmic bias|Fairness (machine learning)}}

Machine learning applications will be [[algorithmic bias|biased]]{{Efn|In statistics, a [[Bias (statistics)|bias]] is a systematic error or deviation from the correct value. But in the context of [[Fairness (machine learning)|fairness]], it refers to a tendency in favor or against a certain group or individual characteristic, usually in a way that is considered unfair or harmful. A statistically unbiased AI system that produces disparate outcomes for different demographic groups may thus be viewed as biased in the ethical sense.<ref name="Samuel-2022"/>}} if they learn from biased data.{{Sfnp|Rose|2023}} The developers may not be aware that the bias exists.{{Sfnp|CNA|2019}} Bias can be introduced by the way [[training data]] is selected and by the way a model is deployed.{{Sfnp|Goffrey|2008|p=17}}{{Sfnp|Rose|2023}} If a biased algorithm is used to make decisions that can seriously [[harm]] people (as it can in [[health equity|medicine]], [[credit rating|finance]], [[recruitment]], [[public housing|housing]] or [[policing]]) then the algorithm may cause [[discrimination]].<ref>{{Harvtxt|Berdahl|Baker|Mann|Osoba|2023}}; {{Harvtxt|Goffrey|2008|p=17}}; {{Harvtxt|Rose|2023}}; {{Harvtxt|Russell|Norvig|2021|p=995}}</ref> The field of [[fairness (machine learning)|fairness]] studies how to prevent harms from algorithmic biases.

On June 28, 2015, [[Google Photos]]'s new image labeling feature mistakenly identified Jacky Alcine and a friend as "gorillas" because they were black. The system was trained on a dataset that contained very few images of black people,{{Sfnp|Christian|2020|p=25}} a problem called "sample size disparity".{{Sfnp|Russell|Norvig|2021|p=995}} Google "fixed" this problem by preventing the system from labelling ''anything'' as a "gorilla". Eight years later, in 2023, Google Photos still could not identify a gorilla, and neither could similar products from Apple, Facebook, Microsoft and Amazon.{{Sfnp|Grant|Hill|2023}}

[[COMPAS (software)|COMPAS]] is a commercial program widely used by [[U.S. court]]s to assess the likelihood of a [[defendant]] becoming a [[recidivist]]. In 2016, [[Julia Angwin]] at [[ProPublica]] discovered that COMPAS exhibited racial bias, despite the fact that the program was not told the races of the defendants. Although the error rate for both whites and blacks was calibrated equal at exactly 61%, the errors for each race were different—the system consistently overestimated the chance that a black person would re-offend and would underestimate the chance that a white person would not re-offend.{{Sfnp|Larson|Angwin|2016}} In 2017, several researchers{{Efn|Including [[Jon Kleinberg]] ([[Cornell University]]), Sendhil Mullainathan ([[University of Chicago]]), Cynthia Chouldechova ([[Carnegie Mellon]]) and Sam Corbett-Davis ([[Stanford]]){{Sfnp|Christian|2020|p=67–70}}}} showed that it was mathematically impossible for COMPAS to accommodate all possible measures of fairness when the base rates of re-offense were different for whites and blacks in the data.<ref>{{Harvtxt|Christian|2020|pp=67–70}}; {{Harvtxt|Russell|Norvig|2021|pp=993–994}}</ref>

A program can make biased decisions even if the data does not explicitly mention a problematic feature (such as "race" or "gender"). The feature will correlate with other features (like "address", "shopping history" or "first name"), and the program will make the same decisions based on these features as it would on "race" or "gender".<ref>{{Harvtxt|Russell|Norvig|2021|p=995}}; {{Harvtxt|Lipartito|2011|p=36}}; {{Harvtxt|Goodman|Flaxman|2017|p=6}}; {{Harvtxt|Christian|2020|pp=39–40, 65}}</ref> Moritz Hardt said "the most robust fact in this research area is that fairness through blindness doesn't work."<ref>Quoted in {{Harvtxt|Christian|2020|p=65}}.</ref>

Criticism of COMPAS highlighted that machine learning models are designed to make "predictions" that are only valid if we assume that the future will resemble the past. If they are trained on data that includes the results of racist decisions in the past, machine learning models must predict that racist decisions will be made in the future. If an application then uses these predictions as ''recommendations'', some of these "recommendations" will likely be racist.<ref>{{Harvtxt|Russell|Norvig|2021|p=994}}; {{Harvtxt|Christian|2020|pp=40, 80–81}}</ref> Thus, machine learning is not well suited to help make decisions in areas where there is hope that the future will be ''better'' than the past. It is descriptive rather than prescriptive.{{Efn|Moritz Hardt (a director at the [[Max Planck Institute for Intelligent Systems]]) argues that machine learning "is fundamentally the wrong tool for a lot of domains, where you're trying to design interventions and mechanisms that change the world."<ref>Quoted in {{Harvtxt|Christian|2020|p=80}}</ref>}}

Bias and unfairness may go undetected because the developers are overwhelmingly white and male: among AI engineers, about 4% are black and 20% are women.{{Sfnp|Russell|Norvig|2021|p=995}}

There are various conflicting definitions and mathematical models of fairness. These notions depend on ethical assumptions, and are influenced by beliefs about society. One broad category is [[Distributive justice|distributive fairness]], which focuses on the outcomes, often identifying groups and seeking to compensate for statistical disparities. Representational fairness tries to ensure that AI systems do not reinforce negative [[stereotype]]s or render certain groups invisible. Procedural fairness focuses on the decision process rather than the outcome. The most relevant notions of fairness may depend on the context, notably the type of AI application and the stakeholders. The subjectivity in the notions of bias and fairness makes it difficult for companies to operationalize them. Having access to sensitive attributes such as race or gender is also considered by many AI ethicists to be necessary in order to compensate for biases, but it may conflict with [[anti-discrimination law]]s.<ref name="Samuel-2022">{{Cite web |last=Samuel |first=Sigal |date=2022-04-19 |title=Why it's so damn hard to make AI fair and unbiased |url=https://www.vox.com/future-perfect/22916602/ai-bias-fairness-tradeoffs-artificial-intelligence |access-date=2024-07-24 |website=Vox |archive-date=5 October 2024 |archive-url=https://web.archive.org/web/20241005170153/https://www.vox.com/future-perfect/22916602/ai-bias-fairness-tradeoffs-artificial-intelligence |url-status=live }}</ref>

At its 2022 [[ACM Conference on Fairness, Accountability, and Transparency|Conference on Fairness, Accountability, and Transparency]] (ACM FAccT 2022), the [[Association for Computing Machinery]], in Seoul, South Korea, presented and published findings that recommend that until AI and robotics systems are demonstrated to be free of bias mistakes, they are unsafe, and the use of self-learning neural networks trained on vast, unregulated sources of flawed internet data should be curtailed.{{Dubious|date=July 2024|reason=Depending on what is meant by "free of bias", it may be impossible in practice to demonstrate it. Additionally, the study evaluates the priors (initial assumptions) of the robots, rather than their decision-making in scenarios where there is a correct choice. For example, it may not be sexist to have the prior that most doctors are males (it's actually an accurate statistical prior in the world we currently live in, so the bias may arguably be to not have this prior). If forced to choose which one is the doctor based solely on gender, a rational person seeking to maximize the number of correct answers would choose the man 100% of the time. The real issue arises when such priors lead to significant discrimination.}}{{Sfnp|Dockrill|2022}}