Editing Evolutionary game theory (section)

==Contests of selfish genes==
[[File:Spermophilus beldingi.jpg|thumb|upright|Female [[Belding's ground squirrel]]s risk their lives giving loud alarm calls, protecting closely related female colony members; males are less closely related and do not call.<ref>{{cite book |author=Dugatkin, Alan |title=Principles of Animal Behavior |date=2004 |publisher=WW Norton |pages=255–260 |isbn=978-0-393-97659-5}}</ref>]]

At first glance it may appear that the contestants of evolutionary games are the individuals present in each generation who directly participate in the game. But individuals live only through one game cycle, and instead it is the strategies that really contest with one another over the duration of these many-generation games.  So it is ultimately genes that play out a full contest – selfish genes of strategy.  The contesting genes are present in an individual and to a degree in all of the individual's kin.  This can sometimes profoundly affect which strategies survive, especially with issues of cooperation and defection. [[W. D. Hamilton|William Hamilton]],<ref>Sigmund, Karl, Institute of Mathematics University of Vienna,  "William D. Hamilton’s Work in Evolutionary Game Theory", Interim Report IR-02-019</ref> known for his theory of [[kin selection]], explored many of these cases using game-theoretic models. Kin-related treatment of game contests<ref name=Brembs>{{cite book |last=Brembs |first=B. |year=2001 |doi=10.1006/rwgn.2001.0581 |chapter=Hamilton's Theory |title=Encyclopedia of Genetics |pages=906–910 |publisher=Academic Press|isbn=978-0-12-227080-2|url=https://epub.uni-regensburg.de/28603/1/brembs.pdf }}</ref> helps to explain many aspects of the behaviour of [[social insects]], the altruistic behaviour in parent-offspring interactions, mutual protection behaviours, and co-operative [[parental care|care of offspring]]. For such games, Hamilton defined an extended form of fitness – ''[[inclusive fitness]]'', which includes an individual's offspring as well as any offspring equivalents found in kin.
 
{| class="wikitable"
|-
! The mathematics of kin selection
|-
| The concept of ''kin selection'' is that:
:''inclusive fitness=own contribution to fitness + contribution of all relatives''.

Fitness is measured relative to the average population; for example, fitness=1 means growth at the average rate for the population, fitness < 1 means having a decreasing share in the population (dying out), fitness > 1 means an increasing share in the population (taking over).

The inclusive fitness of an individual '''w<sub>i</sub>''' is the sum of its specific fitness of itself '''a<sub>i</sub>''' plus the specific fitness of each and every relative weighted by the degree of relatedness which equates to the ''summation'' of all '''r<sub>j</sub>*b<sub>j</sub>'''....... where '''r<sub>j</sub>''' is relatedness of a specific relative and '''b<sub>j</sub>''' is that specific relative's fitness – producing:

:<math>w_i=a_i+\sum_{j}r_jb_j. </math>

If individual a<sub>i</sub> sacrifices their "own average equivalent fitness of 1" by accepting a fitness cost C, and then to "get that loss back", w<sub>i</sub> must still be 1 (or greater than 1)...and using '''R*B''' to represent the summation results in:

:<big>'''''1< (1-C)+RB'''''</big>  ....or rearranging.....    <big>'''''R>C/B'''''.</big><ref name=Brembs/>
|}
Hamilton went beyond kin relatedness to work with [[Robert Axelrod (political scientist)|Robert Axelrod]], analysing games of co-operation under conditions not involving kin where [[reciprocal altruism]] came into play.<ref>{{cite journal |author1=Axelrod, R. |author2=Hamilton, W.D. |author2-link=W. D. Hamilton |date=1981 |title=The evolution of cooperation |journal=Science |volume=211 |issue=4489 |pages=1390–1396 |doi=10.1126/science.7466396 |pmid=7466396|bibcode=1981Sci...211.1390A }}</ref>

===Eusociality and kin selection===
[[File:Meat eater ant feeding on honey.jpg|thumb|[[Iridomyrmex purpureus|Meat ant]] workers (always female) are related to a parent by a factor of 0.5, to a sister by 0.75, to a child by 0.5 and to a brother by 0.25.
Therefore, it is significantly more advantageous to help produce a sister (0.75) than to have a child (0.5).]]
{{main|Eusociality}}

[[Eusocial]] insect workers forfeit reproductive rights to their queen. It has been suggested that kin selection, based on the genetic makeup of these workers, may predispose them to altruistic behaviours.<ref>{{cite journal | last1=Hughes | last2=Oldroyd | last3=Beekman | last4=Ratnieks | year=2008 | title=Ancestral Monogamy Shows Kin Selection Is Key to the Evolution of Eusociality | journal=Science | volume=320 | issue=5880| pages=1213–1216 | doi=10.1126/science.1156108 | pmid=18511689 |bibcode=2008Sci...320.1213H | s2cid=20388889 }}</ref> Most eusocial insect societies have [[haplodiploid]] sexual determination, which means that workers are unusually closely related.<ref>{{cite journal | last1=Thorne | first1=B. | year=1997 | title=Evolution of Eusociality in Termites | journal=Annual Review of Ecology and Systematics | volume=28 | issue=1| pages=27–54 | doi=10.1146/annurev.ecolsys.28.1.27 | pmc=349550 }}</ref>

This explanation of insect eusociality has, however, been challenged by a few highly-noted evolutionary game theorists (Nowak and Wilson)<ref>{{cite journal | last1=Nowak | first1=Tarnita | last2=Wilson | year=2010| title=The evolution of eusociality | journal=Nature | volume=466 | issue=7310| pages=1057–1062 | doi=10.1038/nature09205 | pmid=20740005 | pmc=3279739|bibcode=2010Natur.466.1057N }}</ref> who have published a controversial alternative game theoretic explanation based on a sequential development and group selection effects proposed for these insect species.<ref>{{cite journal | last1=Bourke | first1=Andrew | year=2011| title=The validity and value of inclusive fitness theory | journal= Proceedings of the Royal Society B: Biological Sciences| volume=278 | issue=1723| pages=3313–3320 | doi=10.1098/rspb.2011.1465| pmid=21920980 | pmc=3177639}}</ref>

===Prisoner's dilemma===
{{main|Prisoner's dilemma}}

A difficulty of the theory of evolution, recognised by Darwin himself, was the problem of [[altruism]]. If the basis for selection is at an individual level, altruism makes no sense at all. But universal selection at the group level (for the good of the species, not the individual) fails to pass the test of the mathematics of game theory and is certainly not the general case in nature.<ref>{{cite book |author=Okasha, Samir |title=Evolution and the Levels of Selection |date=2006 |publisher=Oxford University Press |isbn=978-0-19-926797-2 }}</ref> Yet in many social animals, altruistic behaviour exists. The solution to this problem can be found in the application of evolutionary game theory to the [[prisoner's dilemma]] game – a game which tests the payoffs of cooperating or in defecting from cooperation.  It is the most studied game in all of game theory.<ref>{{cite journal |author1=Pacheco, Jorge M. |author2=Santos, Francisco C. |author3=Souza, Max O. |author4=Skyrms, Brian  |title=Evolutionary dynamics of collective action in N-person stag hunt dilemmas |journal=Proceedings of the Royal Society |date=2009 |doi=10.1098/rspb.2008.1126 |pmid=18812288 |volume=276 |issue=1655 |pages=315–321|pmc=2674356 }}</ref>

The analysis of the prisoner's dilemma is as a repetitive game. This affords competitors the possibility of retaliating for defection in previous rounds of the game. Many strategies have been tested; the best competitive strategies are general cooperation, with a reserved retaliatory response if necessary.<ref>{{cite book |author=Axelrod, R. |date=1984 |title=The Evolution of Cooperation|publisher=Basic Books |isbn=0-465-02121-2|edition=1st}} {{cite book |author=Axelrod, R. |date=2009 |title=The Evolution of Cooperation|edition=Revised |isbn=<!--0-465-02121-2-->978-0-14-012495-8}}</ref> The most famous and one of the most successful of these is [[tit-for-tat]] with a simple algorithm.

<syntaxhighlight lang="python">
def tit_for_tat(last_move_by_opponent):
    """Defect if opponent defects, else cooperate."""
    if last_move_by_opponent == defect:
        defect()
    else:
        cooperate()
</syntaxhighlight>

The pay-off for any single round of the game is defined by the pay-off matrix for a single round game (shown in bar chart 1 below). In multi-round games the different choices – co-operate or defect – can be made in any particular round, resulting in a certain round payoff.  It is, however, the possible accumulated pay-offs over the multiple rounds that count in shaping the overall pay-offs for differing multi-round strategies such as tit-for-tat.

[[File:PrisonersPayoff.jpg|thumb|400px|Payoffs in two varieties of prisoner's dilemma game
<br>Prisoner's dilemma: co-operate or defect
<br>Payoff <sub>(temptation in defecting vs. co-operation)</sub> > Payoff <sub>(mutual co-operation)</sub> > Payoff<sub>(joint defection)</sub> > Payoff<sub>(sucker co-operates but opponent defects)</sub>]]

Example 1: The straightforward single round prisoner's dilemma game.  The classic prisoner's dilemma game payoffs gives a player a maximum payoff if they defect and their partner co-operates (this choice is known as ''temptation''). If, however, the player co-operates and their partner defects, they get the worst possible result (the suckers payoff). In these payoff conditions the best choice (a [[Nash equilibrium]]) is to defect.

Example 2: Prisoner's dilemma played repeatedly. The strategy employed is ''tit-for-tat'' which alters behaviours based on the action taken by a partner in the previous round – i.e. reward co-operation and punish defection.  The effect of this strategy in accumulated payoff over many rounds is to produce a higher payoff for both players' co-operation and a lower payoff for defection.  This removes the temptation to defect.  The suckers payoff also becomes less, although "invasion" by a pure defection strategy is not entirely eliminated.

===Routes to altruism===

Altruism takes place when one individual, at a cost (C) to itself, exercises a strategy that provides a benefit (B) to another individual. The cost may consist of a loss of capability or resource which helps in the battle for survival and reproduction, or an added risk to its own survival. Altruism strategies can arise through:

{| class="wikitable"
|-
! Type !! Applies to: !! Situation !! Mathematical effect
|-valign="top"
| '''Kin selection''' – (inclusive fitness of related contestants) || Kin – genetically related individuals || Evolutionary game participants are genes of strategy. The best payoff for an individual is not necessarily the best payoff for the gene.  In any generation the player gene is ''not'' only in one individual, it is in a kin-group.  The highest fitness payoff for the kin group is selected by natural selection.  Therefore, strategies that include self-sacrifice on the part of individuals are often game winners – the evolutionarily stable strategy. Animals must live in kin-groups during part of the game for the opportunity for this altruistic sacrifice ever to take place.  || Games must take into account inclusive fitness.  Fitness function is the combined fitness of a group of related contestants – each weighted by the degree of relatedness – relative to the total genetic population.  The mathematical analysis of this gene-centric view of the game leads to Hamilton's rule, that the relatedness of the altruistic donor must exceed the cost-benefit ratio of the altruistic act itself:<ref name="Nowak & Sigmund">{{cite journal | last1=Nowak | first1=Martin A. | last2=Sigmund | first2=Karl | year=2005 | title=Evolution of indirect reciprocity | journal=Nature | volume=437 | issue=7063| pages=1293–1295 | doi=10.1038/nature04131 | pmid=16251955 |bibcode=2005Natur.437.1291N | s2cid=3153895 | url=http://pure.iiasa.ac.at/7763/1/IR-05-079.pdf }}</ref>

:''R>c/b''  R is relatedness, c the cost, b the benefit
|- valign="top"
| '''Direct reciprocity''' || Contestants that trade favours in paired relationships || A game theoretic embodiment of "I'll scratch your back if you scratch mine".  A pair of individuals exchange favours in a multi-round game. The individuals are recognisable to one another as partnered.  The term "direct" applies because the return favour is specifically given back to the pair partner only. || The characteristics of the multi-round game produce a danger of defection and the potentially lesser payoffs of cooperation in each round, but any such defection can lead to punishment in a following round – establishing the game as a repeated prisoner's dilemma.  Therefore, the family of tit-for-tat strategies come to the fore.<ref>{{cite book |author=Axelrod, R. |date=1984 |title=The Evolution of Cooperation |pages=Chapters 1 to 4 |publisher=Penguin |isbn=978-0-14-012495-8}}</ref>

|- valign="top"
| '''Indirect reciprocity''' || Related or non related contestants trade favours but without partnering.  A return favour is "implied" but with no specific identified source who is to give it. ||  The return favour is not derived from any particular established partner. The potential for indirect reciprocity exists for a specific organism if it lives in a cluster of individuals who can interact over an extended period of time.
It has been argued that human behaviours in establishing moral systems as well as the expending of significant energies in human society for tracking individual reputations is a direct effect of societies' reliance on strategies of indirect reciprocation.<ref>{{cite book |author=Alexander R. |title=The Biology of Moral Systems |date=1987 |publisher=Aldine Transaction |isbn=978-0-202-01174-5 |url-access=registration |url=https://archive.org/details/biologyofmoralsy0000alex }}</ref>
  
 || The game is highly susceptible to defection, as direct retaliation is impossible.  Therefore, indirect reciprocity will not work without keeping a social score, a measure of past co-operative behaviour. The mathematics lead to a modified version of Hamilton's rule where:
:''q>c/b'' where q (the probability of knowing the social score) must be greater than the cost benefit ratio<ref>{{cite journal | last1=Nowak | first1=Martin A. | s2cid=4395576 | year=1998 | title=Evolution of indirect reciprocity by image scoring | journal=Nature | volume=393 | issue=6685| pages=573–575 | doi=10.1038/31225 | pmid=9634232|bibcode=1998Natur.393..573N }}</ref><ref>{{cite journal | last1=Nowak | first1=Martin A. | last2=Sigmund | first2=Karl | year=1998 | title=The Dynamics of Indirect Reciprocity | journal=Journal of Theoretical Biology | volume=194 | issue=4| pages=561–574 | doi=10.1006/jtbi.1998.0775| pmid=9790830 | bibcode=1998JThBi.194..561N | citeseerx=10.1.1.134.2590 }}</ref>

Organisms that use social score are termed Discriminators, and require a higher level of cognition than strategies of simple direct reciprocity. As evolutionary biologist David Haig put it – "For direct reciprocity you need a face; for indirect reciprocity you need a name".
|}

===The evolutionarily stable strategy===
[[File:AssessorGraph.jpg|thumb|300px| The payoff matrix for the hawk dove game, with the addition of the assessor strategy. This "studies its opponent", behaving as a hawk when matched with an opponent it judges "weaker", like a dove when the opponent seems bigger and stronger. Assessor is an ESS, since it can invade both hawk and dove populations, and can withstand invasion by either hawk or dove mutants.]]
{{main|Evolutionarily Stable Strategy}}

The [[evolutionarily stable strategy]] (ESS) is akin to the Nash equilibrium in classical game theory, but with mathematically extended criteria. Nash equilibrium is a game equilibrium where it is not rational for any player to deviate from their present strategy, provided that the others adhere to their strategies. An ESS is a state of game dynamics where, in a very large population of competitors, another mutant strategy cannot successfully enter the population to disturb the existing dynamic (which itself depends on the population mix). Therefore, a successful strategy (with an ESS) must be both effective against competitors when it is rare – to enter the previous competing population, and successful when later in high proportion in the population – to defend itself. This in turn means that the strategy must be successful when it contends with others exactly like itself.<ref>Taylor, P. D. (1979). ''Evolutionarily Stable Strategies with Two Types of Players'' J. Appl. Prob. 16, 76–83.</ref><ref>Taylor, P. D., and Jonker, L. B. (1978). ''Evolutionarily Stable Strategies and Game Dynamics'' Math. Biosci. 40, 145–156.</ref><ref>Osborn, Martin, Introduction to Game Theory, 2004, Oxford Press, p. 393-403 {{ISBN|0-19-512895-8}}</ref>

An ESS is not:

* An optimal strategy: that would maximize fitness, and many ESS states are far below the maximum fitness achievable in a fitness landscape. (See hawk dove graph above as an example of this.)
* A singular solution: often several ESS conditions can exist in a competitive situation. A particular contest might stabilize into any one of these possibilities, but later a major perturbation in conditions can move the solution into one of the alternative ESS states.
* Always present: it is possible for there to be no ESS. An evolutionary game with no ESS is "rock-scissors-paper", as found in species such as the side-blotched lizard (''[[Uta stansburiana]]'').
* An unbeatable strategy: the ESS is only an uninvadeable strategy.

[[File:Agelenopsis actuosa fem sp.jpg|thumb|Female funnel web spiders (Agelenopsis aperta) contest with one another for the possession of their desert spider webs using the assessor strategy.<ref>{{cite journal | last1=Riechert | first1=S.|author1-link= Susan Riechert |last2= Hammerstein | first2=P. |year=1995 |title= Putting Game Theory to the Test |doi= 10.1126/science.7886443 | pmid=7886443 |journal= Science |volume= 267 | issue=5204 |pages= 1591–1593 |bibcode= 1995Sci...267.1591P | s2cid=5133742 }}</ref>]]

The ESS state can be solved for by exploring either the dynamics of population change to determine an ESS, or by solving equations for the stable stationary point conditions which define an ESS.<ref>{{cite journal | last1=Chen | first1=Z | last2=Tan | first2=JY | last3=Wen | first3=Y | last4=Niu | first4=S | last5=Wong | first5=S-M | year=2012 | title=A Game-Theoretic Model of Interactions between Hibiscus Latent Singapore Virus and Tobacco Mosaic Virus | journal=PLOS ONE | volume=7 | issue=5| page=e37007 | doi=10.1371/journal.pone.0037007 |bibcode=2012PLoSO...737007C | pmid=22623970 | pmc=3356392| doi-access=free }}</ref>  For example, in the hawk dove game we can look for whether there is a static population mix condition where the fitness of doves will be exactly the same as fitness of hawks (therefore both having equivalent growth rates – a static point). 
  
Let the chance of meeting a hawk=p so therefore the chance of meeting a dove is (1-p)

Let Whawk equal the payoff for hawk...

Whawk=payoff in the chance of meeting a dove + payoff in the chance of meeting a hawk

Taking the payoff matrix results and plugging them into the above equation:

{{math|<var>Whawk</var>{{=}} <var>V·(1-p)+(V/2-C/2)·p</var>}}

Similarly for a dove:

{{math|<var>Wdove</var>{{=}} <var>V/2·(1-p)+0·(p)</var>}}

so....

{{math|<var>Wdove</var>{{=}} <var> V/2·(1-p) </var>}}

Equating the two fitnesses, hawk and dove

{{math|<var>V·(1-p)+(V/2-C/2)·p</var>{{=}} <var> V/2·(1-p) </var>}}

... and solving for p

{{math|<var>p</var>{{=}} <var>V/C</var>}}

so for this "static point" where the ''population percent'' is an ESS solves to be ESS<sub>(percent Hawk)</sub>=''V/C''

Similarly, using inequalities, it can be shown that an additional hawk or dove mutant entering this ESS state eventually results in less fitness for their kind – both a true Nash and an ESS equilibrium.  This example shows that when the risks of contest injury or death (the cost C) is significantly greater than the potential reward (the benefit value V), the stable population will be mixed between aggressors and doves, and the proportion of doves will exceed that of the aggressors.  This explains behaviours observed in nature.