Cooperation in structured populations/ Mixed strategies

In well-mixed populations the equilibrium configuration does not depend on whether the individuals adopt pure strategies (to cooperate or to defect) or mixed strategies (to cooperate with a certain probability). All that matters is the frequency of the two behavioral patterns when averaging over many random encounters. This ambiguity is resolved in spatially structured populations where individuals interact only within a limited neighborhood. Spatial structures are modeled by arranging individuals on a lattice. The lattice geometry can be either square with a neighborhood size of $N=4$ or $N=8$, triangular with $N=3$ or hexagonal with $N=6$.

Updating of the lattice can be synchronous referring to populations with discrete non-overlapping generations or asynchronous for populations with overlapping generations in continuous time. For synchronous updates, all individuals interact and accumulate payoffs and then everbody attempts to reproduce with successrates relative to each individuals payoff within its neighborhood. For asynchronous updates only a single randomly selected focal site gets updated at a time: first the payoffs of the selected individual and all its neighbors are determined and then they compete to re-populate the focal site as outlined above.

Dynamical scenarios

The following examples illustrate and highlight different relevant scenarios but at the same time they are meant as suggestions and starting points for further exploring and experimenting with the dynamics of the system. If your browser has JavaScript enabled, the following links open a new window containing a running lab that has all necessary parameters set as appropriate.

Color code:	Maximum	Minimum	Mean

Strategies:	Defect Cooperate

Time evolution of the propensity to cooperate in structured populations with individuals adopting mixed strategies and engaging in prisoner's dilemma and snowdrift interactions.

File:Spatial Prisoner's Dilemma (mixed strategies, t=250).png

Space promotes cooperation in the Prisoner's Dilemma - mixed strategies

In spatially structured populations playing the Prisoner's Dilemma significant propensities to cooperate can evolve which contrasts with well-mixed populations where the levels of cooperation converge to zero. In spatial settings individuals with a higher readiness to cooperate can thrive by forming clusters and thereby reducing exploitation by less cooperative individuals.

The snapshot on the left shows a typical lattice configuration with an average probability to cooperate of 20%. The population has not yet reached the equilibrium and the readiness to cooperate would further increase.

File:Spatial Snowdrift Game (mixed strategies, synchronous, t=250).png

Space inhibits cooperation in the Snowdrift Game - mixed strategies, synchronous updates

As in the case of [struct.pure.html pure strategies], for the Snowdrift and Hawk-Dove game spatial structure reduces the readiness to cooperate for mixed strategies if the population is updated synchronously, i.e. with discrete non-overlapping generations.

The snapshot on the left show a typical configuration of a lattice population in equilibrium. The average probability to cooperate is roughly 10% for parameters where well-mixed populations sustain levels of 30% cooperation.

File:Spatial Snowdrift Game (mixed strategies, asynchronous, t=250).png

Space promotes cooperation in the Snowdrift Game - mixed strategies, asynchronous updates

The situation changes considerably, if the Snowdrift or Hawk-Dove game is played in spatially structured populations that are updated asynchronously, i.e. overlapping generations in continuous time. In that case space again promotes cooperation and results in average propensities to cooperate that lie above those obtained for well-mixed populations.

The snapshot on the left shows a typical lattice configuration with an average probability to cooperate of 20%. The population has not yet reached the equilibrium and the readiness to cooperate would further increase. The parameters are the same as in the synchronous case above where the equilibrium has already been reached at much lower levels of cooperation. This contrasts with the established view that any form of stochasticity (such as asynchronous updates) should favor defection.

Comparing this snapshot to the one for the Prisoner's Dilemma above (with the same average probability to cooperate) shows that the variance of strategies is considerably smaller in the Snowdrift and Hawk-Dove game than in the Prisoner's Dilemma. In the simulations this can be clearly seen when switching to the histogram view depicting the frequency distribution of the strategies.

Spatial dynamics

Whenever a site xgets updated, the focal individual and all its neighbors compete to re-populate the site. Their reproductive success depends on their performance, i.e. their average payoff, in interactions with their respective neighbors. If the individual in x cooperates with probability$p$ and the neighboring y with probability$q$, then the payoff for x amounts to$P_{p}=pqR+p(1-q)S+(1-p)qT+(1-p)(1-q)P$ and similarly for y to $P_{q}=pqR+p(1-q)S+(1-p)qT+(1-p)(1-q)P$. Note that this calculation of the payoffs assumes that individuals interact frequently such that payoffs from single interactions average out. In contrast, if the performance depends only on a single interaction, then an individual with strategy $p$ would obtain against strategy $q$ the payoffs $R,S,T$ or$P$ with probabilities $pq,p(1-q),(1-p)q$ and$(1-p)(1-q)$, respectively. In that case the fluctuations arising from the probabilistic payoffs destroy all spatial correlations and yield the same results as well-mixed populations.

The performance $P_{x}$ of x is then determined by averaging the payoffs from interactions with all its neighbors. The neighbor y succeeds in populating the focal sitex with a probability proportional to

$w_{y}=(1+\exp[(P_{y}-P_{x})/k])^{-1}$

where $k$ denotes a noise term which introduces an interesting form of errors since worse performing individuals may still manage to reproduce with a small probability. The offspring inherits the parental strategy $p$but with a small probability a mutation occurs changing the offspring strategy to$p+s$ where $s$ is a Gaussian distributed random variable with mean zero and small standard deviation. For small mutations, the difference in payoffs $P_{y}-P_{x}$ becomes small and the generalization of the replicator dynamics to spatial settings as introduced for structured populations with pure strategies no longer performs well. For this reason we chose the above, slighlty more complicated rule to determine the reproductive success for mixed strategies. With this rule, small differences in payoffs are amplified - at least for reasonably small $k$. The remaining part of the update procedure is identical to the pure strategy case: with probability $p_{x}=(1-w_{y_{1}})(1-w_{y_{2}})\cdots (1-w_{y_{N}})$ all neighborsy_i fail to reproduce and the focal individual succeeds in placing its own offspring in site x. Otherwise, with probability Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle 1 - p_x} , one neighbor takes over the focal site. The relative probability for success of neighbors y is$w_{y}/w$ where $w=w_{1}+w_{2}+\cdots +w_{N}$.