Replicator equation

In mathematics, the replicator equation is a type of dynamical system used in evolutionary game theory to model how the frequency of strategies in a population changes over time. It is a deterministic, monotone, non-linear, and non-innovative dynamic that captures the principle of natural selection in strategic interactions.^[1]

The replicator equation describes how strategies with higher-than-average fitness increase in frequency, while less successful strategies decline. Unlike other models of replication—such as the quasispecies model—the replicator equation allows the fitness of each type to depend dynamically on the distribution of population types, making the fitness function an endogenous component of the system. This allows it to model frequency-dependent selection, where the success of a strategy depends on its prevalence relative to others.

Another key difference from the quasispecies model is that the replicator equation does not include mechanisms for mutation or the introduction of new strategies, and is thus considered non-innovative. It assumes all strategies are present from the outset and models only the relative growth or decline of their proportions over time.

Replicator dynamics have been widely applied in fields such as biology (to study evolution and population dynamics), economics (to analyze bounded rationality and strategy evolution), and machine learning (particularly in multi-agent systems and reinforcement learning).

Equation

The most general continuous form of the replicator equation is given by the differential equation:

${\dot {x_{i}}}=x_{i}[f_{i}(x)-\phi (x)],\quad \phi (x)=\sum _{j=1}^{n}{x_{j}f_{j}(x)}$

where $x_{i}$ is the proportion of type $i$ in the population, $x=(x_{1},\ldots ,x_{n})$ is the vector of the distribution of types in the population, $f_{i}(x)$ is the fitness of type $i$ (which is dependent on the population), and $\phi (x)$ is the average population fitness (given by the weighted average of the fitness of the $n$ types in the population). Since the elements of the population vector $x$ sum to unity by definition, the equation is defined on the n-dimensional simplex.

The replicator equation assumes a uniform population distribution; that is, it does not incorporate population structure into the fitness. The fitness landscape does incorporate the population distribution of types, in contrast to other similar equations, such as the quasispecies equation.

In application, populations are generally finite, making the discrete version more realistic. The analysis is more difficult and computationally intensive in the discrete formulation, so the continuous form is often used, although there are significant properties that are lost due to this smoothing. Note that the continuous form can be obtained from the discrete form by a limiting process.

To simplify analysis, fitness is often assumed to depend linearly upon the population distribution, which allows the replicator equation to be written in the form:

{\dot {x_{i}}}=x_{i}\left(\left(Ax\right)_{i}-x^{T}Ax\right)

where the payoff matrix $A$ holds all the fitness information for the population: the expected payoff can be written as $\left(Ax\right)_{i}$ and the mean fitness of the population as a whole can be written as $x^{T}Ax$ . It can be shown that the change in the ratio of two proportions $x_{i}/x_{j}$ with respect to time is: ${d \over {dt}}\left({x_{i} \over {x_{j}}}\right)={x_{i} \over {x_{j}}}\left[f_{i}(x)-f_{j}(x)\right]$ In other words, the change in the ratio is driven entirely by the difference in fitness between types.

Derivation of deterministic and stochastic replicator dynamics

Suppose that the number of individuals of type $i$ is $N_{i}$ and that the total number of individuals is $N$ . Define the proportion of each type to be $x_{i}=N_{i}/N$ . Assume that the change in each type is governed by geometric Brownian motion: $dN_{i}=f_{i}N_{i}dt+\sigma _{i}N_{i}dW_{i}$ where $f_{i}$ is the fitness associated with type $i$ . The average fitness of the types $\phi =x^{T}f$ . The Wiener processes are assumed to be uncorrelated. For $x_{i}(N_{1},...,N_{m})$ , Itô's lemma then gives us: ${\begin{aligned}dx_{i}(N_{1},...,N_{m})&={\partial x_{i} \over {\partial N_{j}}}dN_{j}+{1 \over {2}}{\partial ^{2}x_{i} \over {\partial N_{j}\partial N_{k}}}dN_{j}dN_{k}\\&={\partial x_{i} \over {\partial N_{j}}}dN_{j}+{1 \over {2}}{\partial ^{2}x_{i} \over {\partial N_{j}^{2}}}(dN_{j})^{2}\end{aligned}}$ The partial derivatives are then: ${\begin{aligned}{\partial x_{i} \over {\partial N_{j}}}&={1 \over {N}}\delta _{ij}-{x_{i} \over {N}}\\{\partial ^{2}x_{i} \over {\partial N_{j}^{2}}}&=-{2 \over {N^{2}}}\delta _{ij}+{2x_{i} \over {N^{2}}}\end{aligned}}$ where $\delta _{ij}$ is the Kronecker delta function. These relationships imply that: $dx_{i}={dN_{i} \over {N}}-x_{i}\sum _{j}{dN_{j} \over {N}}-{(dN_{i})^{2} \over {N^{2}}}+x_{i}\sum _{j}{(dN_{j})^{2} \over {N^{2}}}$ Each of the components in this equation may be calculated as: ${\begin{aligned}{dN_{i} \over {N}}&=f_{i}x_{i}dt+\sigma _{i}x_{i}dW_{i}\\-x_{i}\sum _{j}{dN_{j} \over {N}}&=-x_{i}\left(\phi dt+\sum _{j}\sigma _{j}x_{j}dW_{j}\right)\\-{(dN_{i})^{2} \over {N^{2}}}&=-\sigma _{i}^{2}x_{i}^{2}dt\\x_{i}\sum _{j}{(dN_{j})^{2} \over {N^{2}}}&=x_{i}\left(\sum _{j}\sigma _{j}^{2}x_{j}^{2}\right)dt\end{aligned}}$ Then the stochastic replicator dynamics equation for each type is given by: $dx_{i}=x_{i}\left(f_{i}-\phi -\sigma _{i}^{2}x_{i}+\sum _{j}\sigma _{j}^{2}x_{j}^{2}\right)dt+x_{i}\left(\sigma _{i}dW_{i}-\sum _{j}\sigma _{j}x_{j}dW_{j}\right)$ Assuming that the $\sigma _{i}$ terms are identically zero, the deterministic replicator dynamics equation is recovered.

Analysis

The analysis differs in the continuous and discrete cases: in the former, methods from differential equations are utilized, whereas in the latter the methods tend to be stochastic. Since the replicator equation is non-linear, an exact solution is difficult to obtain (even in simple versions of the continuous form) so the equation is usually analyzed in terms of stability. The replicator equation (in its continuous and discrete forms) satisfies the folk theorem of evolutionary game theory which characterizes the stability of equilibria of the equation. The solution of the equation is often given by the set of evolutionarily stable states of the population.

In general nondegenerate cases, there can be at most one interior evolutionary stable state (ESS), though there can be many equilibria on the boundary of the simplex. All the faces of the simplex are forward-invariant which corresponds to the lack of innovation in the replicator equation: once a strategy becomes extinct there is no way to revive it.

Phase portrait solutions for the continuous linear-fitness replicator equation have been classified in the two and three dimensional cases. Classification is more difficult in higher dimensions because the number of distinct portraits increases rapidly.

Relationships to other equations

The continuous replicator equation on $n$ types is equivalent to the Generalized Lotka–Volterra equation in $n-1$ dimensions.^[2]^[3] The transformation is made by the change of variables:

x_{i}={\frac {y_{i}}{1+\sum _{j=1}^{n-1}{y_{j}}}}\quad i=1,\ldots ,n-1

x_{n}={\frac {1}{1+\sum _{j=1}^{n-1}{y_{j}}}},

where $y_{i}$ is the Lotka–Volterra variable. The continuous replicator dynamic is also equivalent to the Price equation.^[4]

Discrete replicator equation

When one considers an unstructured infinite population with non-overlapping generations, one should work with the discrete forms of the replicator equation. Mathematically, two simple phenomenological versions---

x'_{i}=x_{i}+x_{i}\left[\left(Ax\right)_{i}-x^{T}Ax\right]\,({\rm {type~I),}}

x'_{i}=x_{i}\left[{\frac {\left(Ax\right)_{i}}{x^{T}Ax}}\right]\,({\rm {type~II),}}

---are consistent with the Darwinian tenet of natural selection or any analogous evolutionary phenomena. Here, prime stands for the next time step. However, the discrete nature of the equations puts bounds on the payoff-matrix elements.^[5] Interestingly, for the simple case of two-player-two-strategy games, the type I replicator map is capable of showing period doubling bifurcation leading to chaos and it also gives a hint on how to generalize^[6] the concept of the evolutionary stable state to accommodate the periodic solutions of the map.

Generalizations

A generalization of the replicator equation which incorporates mutation is given by the replicator-mutator equation, which takes the following form in the continuous version:^[7]

{\dot {x_{i}}}=\sum _{j=1}^{n}{x_{j}f_{j}(x)Q_{ji}}-\phi (x)x_{i},

where the matrix $Q$ gives the transition probabilities for the mutation of type $j$ to type $i$ , $f_{i}$ is the fitness of the $i^{th}$ and $\phi$ is the mean fitness of the population. This equation is a simultaneous generalization of the replicator equation and the quasispecies equation, and is used in the mathematical analysis of language.

The discrete version of the replicator-mutator equation may have two simple types in line with the two replicator maps written above:

x'_{i}=x_{i}+\sum _{j=1}^{n}{x_{j}f_{j}(x)Q_{ji}}-\phi (x)x_{i},

and

x'_{i}={\frac {\sum _{j=1}^{n}{x_{j}f_{j}(x)Q_{ji}}}{\phi (x)}},

respectively.

The replicator equation or the replicator-mutator equation can be extended^[8] to include the effect of delay that either corresponds to the delayed information about the population state or in realizing the effect of interaction among players. The replicator equation can also easily be generalized to asymmetric games. A recent generalization that incorporates population structure is used in evolutionary graph theory.^[9]

References

^ Hofbauer, Josef; Sigmund, Karl (2003). "Evolutionary game dynamics". Bulletin of the American Mathematical Society. 40 (4): 479–519. doi:10.1090/S0273-0979-03-00988-1. ISSN 0273-0979. {{cite journal}}: line feed character in |journal= at position 9 (help)
^ Bomze, Immanuel M. (1983-10-01). "Lotka-Volterra equation and replicator dynamics: A two-dimensional classification". Biological Cybernetics. 48 (3): 201–211. doi:10.1007/BF00318088. ISSN 1432-0770. S2CID 206774680.
^ Bomze, Immanuel M. (1995-04-01). "Lotka-Volterra equation and replicator dynamics: new issues in classification". Biological Cybernetics. 72 (5): 447–453. doi:10.1007/BF00201420. ISSN 1432-0770. S2CID 18754189.
^ Page, KAREN M.; Nowak, MARTIN A. (2002-11-07). "Unifying Evolutionary Dynamics". Journal of Theoretical Biology. 219 (1): 93–98. Bibcode:2002JThBi.219...93P. doi:10.1006/jtbi.2002.3112. ISSN 0022-5193. PMID 12392978.
^ Pandit, Varun; Mukhopadhyay, Archan; Chakraborty, Sagar (2018). "Weight of fitness deviation governs strict physical chaos in replicator dynamics". Chaos. 28 (3): 033104. arXiv:1703.10767. Bibcode:2018Chaos..28c3104P. doi:10.1063/1.5011955. PMID 29604653. S2CID 4559066.
^ Mukhopadhyay, Archan; Chakraborty, Sagar (2020). "Periodic Orbit can be Evolutionarily Stable: Case Study of Discrete Replicator Dynamics". Journal of Theoretical Biology. 497: 110288. arXiv:2102.11034. Bibcode:2020JThBi.49710288M. doi:10.1016/j.jtbi.2020.110288. PMID 32315673. S2CID 216073761.
^ Nowak, Martin A. (2006). Evolutionary Dynamics: Exploring the Equations of Life. Belknap Press. pp. 272–273. ISBN 978-0674023383.
^ Alboszta, Jan; Miękisz, Jacek (2004). "Stability of evolutionarily stable strategies in discrete replicator dynamicswithtimedelay". Journal of Theoretical Biology. 231 (2): 175–179. arXiv:q-bio/0409024. Bibcode:2004JThBi.231..175A. doi:10.1016/j.jtbi.2004.06.012. PMID 15380382. S2CID 15308310.
^ Lieberman, Erez; Hauert, Christoph; Nowak, Martin A. (2005). "Evolutionary dynamics on graphs". Nature. 433 (7023): 312–316. Bibcode:2005Natur.433..312L. doi:10.1038/nature03204. ISSN 1476-4687. PMID 15662424. S2CID 4386820.

Topics of game theory
Definitions	Congestion game Cooperative game Determinacy Escalation of commitment Extensive-form game First-player and second-player win Game complexity Graphical game Hierarchy of beliefs Information set Normal-form game Perfect recall Preference Sequential game Simultaneous game Simultaneous action selection Solved game Succinct game Mechanism design
Equilibrium concepts	Bayes correlated equilibrium Bayesian Nash equilibrium Berge equilibrium Core Correlated equilibrium Coalition-proof Nash equilibrium Epsilon-equilibrium Evolutionarily stable strategy Gibbs equilibrium Mertens-stable equilibrium Markov perfect equilibrium Nash equilibrium Pareto efficiency Perfect Bayesian equilibrium Proper equilibrium Quantal response equilibrium Quasi-perfect equilibrium Risk dominance Satisfaction equilibrium Self-confirming equilibrium Sequential equilibrium Shapley value Strong Nash equilibrium Subgame perfection Trembling hand equilibrium
Strategies	Appeasement Backward induction Bid shading Collusion Cheap talk De-escalation Deterrence Escalation Forward induction Grim trigger Markov strategy Pairing strategy Dominant strategies Pure strategy Mixed strategy Strategy-stealing argument Tit for tat
Classes of games	Auction Bargaining problem Differential game Global game Intransitive game Mean-field game n-player game Perfect information Large Poisson game Potential game Repeated game Screening game Signaling game Strictly determined game Stochastic game Symmetric game Zero-sum game
Games	Go Chess Infinite chess Checkers All-pay auction Prisoner's dilemma Gift-exchange game Optional prisoner's dilemma Traveler's dilemma Coordination game Chicken Centipede game Lewis signaling game Volunteer's dilemma Dollar auction Battle of the sexes Stag hunt Matching pennies Ultimatum game Electronic mail game Rock paper scissors Pirate game Dictator game Public goods game Blotto game War of attrition El Farol Bar problem Fair division Fair cake-cutting Bertrand competition Cournot competition Stackelberg competition Deadlock Diner's dilemma Guess 2/3 of the average Kuhn poker Nash bargaining game Induction puzzles Trust game Princess and monster game Rendezvous problem Pursuit game
Theorems	Aumann's agreement theorem Folk theorem Minimax theorem Nash's theorem Negamax theorem One-shot deviation principle Purification theorem Revelation principle Sprague–Grundy theorem Zermelo's theorem
Key figures	Albert W. Tucker Amos Tversky Antoine Augustin Cournot Ariel Rubinstein Claude Shannon Daniel Kahneman David K. Levine David M. Kreps Donald B. Gillies Drew Fudenberg Eric Maskin Harold W. Kuhn Herbert Simon Hervé Moulin John Conway Jean Tirole Jean-François Mertens Jennifer Tour Chayes John Harsanyi John Maynard Smith John Nash John von Neumann Kenneth Arrow Kenneth Binmore Leonid Hurwicz Lloyd Shapley Melvin Dresher Merrill M. Flood Olga Bondareva Oskar Morgenstern Paul Milgrom Peyton Young Reinhard Selten Robert Axelrod Robert Aumann Robert B. Wilson Roger Myerson Samuel Bowles Suzanne Scotchmer Thomas Schelling William Vickrey
Search optimizations	Alpha–beta pruning Aspiration window Principal variation search max^n algorithm Paranoid algorithm Lazy SMP
Miscellaneous	Bounded rationality Combinatorial game theory Confrontation analysis Coopetition Evolutionary game theory Glossary of game theory List of game theorists List of games in game theory No-win situation Topological game Tragedy of the commons

Conheça Walt Disney World