Spatial social dilemmas promote diversity

Author: Christoph Hauert

Cooperative investments in social dilemmas can spontaneously diversify into stably co-existing high and low contributors in well-mixed populations. Here we extend the analysis to emerging diversity in (spatially) structured populations. Using pair approximation we derive analytical expressions for the invasion fitness of rare mutants in structured populations, which then yields a spatial adaptive dynamics framework. This allows us to predict changes arising from population structures in terms of existence and location of singular strategies, as well as their convergence and evolutionary stability as compared to well-mixed populations. Based on spatial adaptive dynamics and extensive individual based simulations, we find that spatial structure has significant and varied impacts on evolutionary diversification in continuous social dilemmas. More specifically, spatial adaptive dynamics suggests that spontaneous diversification through evolutionary branching is suppressed, but simulations show that spatial dimensions offer new modes of diversification that are driven by an interplay of finite-size mutations and population structures. Even though spatial adaptive dynamics is unable to capture these new modes, they can still be understood based on an invasion analysis. In particular, population structures alter invasion fitness and can open up new regions in trait space where mutants can invade, but that may not be accessible to small mutational steps. Instead, stochastically appearing larger mutations or sequences of smaller mutations in a particular direction are required to bridge regions of unfavourable traits. The net effect is that spatial structure tends to promote diversification, especially when selection is strong.

Social dilemmas with continuous traits

Social dilemmas are important mathematical metaphors for studying the problem of cooperation. The best studied models of social dilemmas are the prisoner's dilemma and the snowdrift game. Traditionally, such models are often restricted to the two distinct strategies of cooperate, \(C\), and defect, \(D\). However, these models can easily be extended to describe a continuous range of cooperative investments into a public good. In such continuous games, the aim is to study the evolutionary dynamics of the level of investment.

For example, in the donation game, which is the most prominent version of the prisoner's dilemma, cooperators confer a benefit \(b\) to their interaction partners at a cost \(c\) to themselves. Defection entails neither costs nor confers benefits. With \(b>c>0\), both individuals prefer mutual cooperation, which pays \(b-c\), over a zero payoff for mutual defection. However, the temptation to shirk costs and free ride on benefits provided by the partner undermines cooperation to the detriment of all. In fact, defection is the dominant strategy because it results in higher payoffs regardless of the partner's strategy. This conflict of interest between the individual and the group represents the hallmark of social dilemmas.

Continuous Donation game

A natural translation of the donation game to continuous traits is based on cost and benefit functions, \(C(x)\) and \(B(x)\), where the trait \(x\in[0,x_\text{max}]\) represents a level of cooperative investment that can vary continuously. In the spirit of the donation game, an individual with strategy \(x\) interacting with an \(y\)-strategist obtains a payoff \(P(x,y)=B(y)-C(x)\). Assuming that (i) zero investments incur no costs and provide no benefits, \(B(0)=C(0)=0\), (ii) benefits exceed costs, \(B(x)>C(x)\geq0\), and (iii) are increasing functions, \(B^\prime(x), C^\prime(x)\geq0\), recovers the social dilemma of the donation game for continuous investment levels: the level of investment invariably evolves to zero, which corresponds to pure defection, despite the fact that both players would be better off at non-zero investment levels. The reason is that an actor can only influence the cost \(C(x)\) of an interaction, but not the received benefit \(B(y)\), and hence selection can only act to minimize costs.

Continuous Snowdrift game

In a weaker form of a social dilemma, the snowdrift game, cooperators also provide benefits, \(b\), at a cost, \(c\). However, benefits are now accessible to both individuals and accumulate in a discounted manner. For example, yeast secretes enzymes for extra cellular digestion of sucrose. While access to nutrients is crucial, the marginal value of additional resources diminishes and may exceed the intake capacity. As a result, the social dilemma remains in effect, because the most favourable outcome of mutual cooperation remains prone to cheating, but the dilemma is relaxed because it pays to cooperate against defectors. Again, it is straightforward to formulate a continuous version of the snowdrift game. In contrast to the donation game, in the continuous snowdrift game the benefits depend on the strategies of both interacting individuals. For example, for simplicity one can assume that benefits depend on the aggregate investment levels of both players, so that the benefit is a function \(B(x+y)\). Assuming a cost function \(C(x)\), an \(x\) strategist interacting with strategy \(y\) obtains a payoff \(P(x,y)=B(x+y)-C(x)\), again with \(B(0)=C(0)=0\), \(B(x)>C(x)\geq0\) and \(B^\prime(x),C^\prime(x)\geq0\), as before, but with the additional constraint \(C(x)>B(x+y)-B(y)\) for sufficiently large \(y\) so that the increased return from investments of an individual do not outweigh its costs when interacting with high investors. Typically this is achieved by saturating benefit functions, \(B^{\prime\prime}(x)<0\).

The gradual evolution of continuous traits can be described using the framework of adaptive dynamics. Below we extend this framework to spatial settings by amalgamating adaptive dynamics and pair approximation into spatial adaptive dynamics. This provides the toolbox to investigate the impact of spatial structures on the evolution of cooperation in the prisoner's dilemma as well as the snowdrift games.

Evolution in the continuous prisoner's dilemma

In the continuous prisoner's dilemma the payoff to an individual with strategy \(y\) interacting with an \(x\)-strategist is \(P(y,x)=B(x)-C(y)\). This implies that the selection gradient in well-mixed populations is proportional to \(-C'(x)\), and hence, assuming monotonously increasing costs, no singular strategy exists apart from the pure defection state \(x=0\), and the population always evolves to that state, regardless of how large the benefits of cooperation are. In contrast, for death-birth updating in structured populations the selection gradient, \eq{d:dB}, becomes

\begin{align} \label{eq:d:pd:dB} D_\text{db}(x) &= w\frac{k-2}{k(k-1)}(B^\prime(x)-k C^\prime(x)). \end{align}

Thus, in structured populations a singular strategy, \(x^\ast\), may exist as a solution to \(D_\text{db}(x^\ast)=0\). If \(x^\ast\) exists and is convergence stable then it is also evolutionarily stable because the two stability conditions are identical (the mixed derivatives on the right hand side of \eqs{css:dB} and (\ref{eq:ess:dB}) are zero). In particular, cooperation can be maintained in spatially structured populations, a result that is of course in line with classical theory. More specifically, cooperative investments can increase if the marginal benefits, \(B^\prime(x)\), exceed the \(k\)-fold marginal costs, \(C^\prime(x)\), which is reminiscent of the \(b>ck\)-rule in the traditional donation game (see Ohtsuki et al. 2006).

Conclusions & Discussion

In all scenarios considered here we find that population structures can promote and facilitate spontaneous diversification in social dilemmas into high and low investors, especially when selection is strong. However, at the same time, classical evolutionary branching tends to be inhibited, but compensated for by other modes of diversification. We derive an extension of adaptive dynamics for continuous games in (spatially) structured populations based on pair approximation, which tracks the frequencies of mutant-resident pairs during invasion. It turns out that predictions derived from this spatial adaptive dynamics framework are independent of selection strength. More precisely, selection strength only scales the magnitude of the selection gradient as well as that of convergence and evolutionary stability but neither affects the location of singular strategies nor their stability. Nevertheless, from the invasion analysis of mutant traits \(y\) into resident populations \(x\), as well as from individual-based simulations, it is clear that selection strength has a significant impact on the dynamics in general, and on diversification in particular. During the invasion process the local configuration probability of mutant pairs \(q_{m|m}\) is of crucial importance and changes with selection strength, see \eq{qmm:dB:sol}. However, in the adaptive dynamics limit \(y\to x\) the pair configuration probability reduces to a constant \(q_{m|m}\to1/(k-1)\). Even though this is the same value as obtained in the limit of weak selection, \(w\to0\), it is important to note that the limit \(y\to x\) does not necessarily imply weak selection. Thus, the different modes of diversification cannot be understood based on spatial adaptive dynamics alone but require a more comprehensive analysis of invasion dynamics.

Structured populations offer new modes of diversification that are driven by an interplay of finite-size mutations and population structures. Trait variation is more easily maintained in structured populations due to the slower spreading of advantageous traits as compared to well-mixed populations. Spatial adaptive dynamics is unable to capture these new modes of diversification because of the underlying assumption that the resident population is composed of discrete traits (monomorphic before branching and dimorphic or polymorphic after branching). Nevertheless, invasion analysis and pairwise invasibility plots, \(PIP\), in particular, provide an intuitive interpretation for additional pathways to diversification that are introduced by spatial structures and further promoted for increasing selection strength. Even in the vicinity of evolutionarily stable investments, trait combinations exist that admit mutual invasion and hence can coexist. However, such states cannot evolve through adaptive dynamics but are nevertheless accessible to trait distributions around the evolutionarily stable trait and can drive a degenerate form of branching. Moreover, spatial invasion fitness can open up new regions in trait space for mutant invasion. However, those regions need not be accessible by small mutational steps, and instead require stochastically appearing larger mutations or sequences of smaller mutations that allow to bridge regions of unfavourable traits.

Previous attempts at amalgamating adaptive dynamics and spatial structure have not observed spontaneous diversification or evolutionary branching. In particular, Allen et al. (2013) augment adaptive dynamics by structure coefficients (Tarnita et al. 2011), which restrict the analysis to weak selection. Moreover, their framework is fundamentally different from ours because it is based on fixation probabilities rather than invasion fitness. More specifically, their analysis is based on the fixation probabilities \(p_x\) and \(p_y\) of two types \(x\) and \(y\) in a population consisting of \(x\) and \(y\): \(x\) is favoured over \(y\) if \(p_x>p_y\) (see Eq.~4 and formulas following Eq.~7 in Allen et al. 2013). Clearly, it is then impossible to have both types be favoured in an \((x,y)\)-population, but this is exactly the requirement for mutual coexistence. In fact, Allen et al. acknowledge that fixation probabilities are "less appropriate" for studying evolutionary branching and that instead "establishment probabilities" should be considered for studying diversification. As a consequence their adaptive dynamics derivation precludes its most intriguing feature, that of evolutionary branching. However, in the limit of weak selection the two approaches are compatible in some aspects. For example, the selection gradient and the condition for convergence stability reported for the continuous prisoner's dilemma are identical (c.f. \eqs{d:csd:dB},~\ref{eq:css:csd:dB} and their Eqs.~11,~13). In our framework, the use of invasion fitness allows to determine evolutionary stability, and the corresponding \(PIP\)'s reveal the potential for degenerate forms of branching – but only for stronger selection (see \fig{kdk:dB}). Another attempt at spatial adaptive dynamics considers spatial structure in the form of demes (Parvinen et al. 2017). However, quite naturally such deme structures suppress branching because in the long run it is impossible to maintain multiple co-existing traits in small demes, which quickly become homogeneous. Instead, branching is only possible if demes are large enough to support branching in every single one.

Interestingly, in well-mixed populations evolutionary branching is only observed for the continuous snowdrift game, where two distinct traits of high and low investors can co-exist and essentially engage in a classical (discrete) snowdrift game. In contrast, in structured populations with death-birth updating, evolutionary branching is only observed for prisoner's dilemma type interactions where lower investments invariably dominate higher ones, which applies both in the continuous prisoner's dilemma as well as the continuous snowdrift game with sufficiently high costs. The reason for this surprising difference can be understood intuitively by considering the preferred spatial configurations in the two classical (discrete) games: in the prisoner's dilemma cooperators form compact clusters to reduce exploitation by defectors (minimize surface), while in the snowdrift game filament like clusters form because it is advantageous to adopt a strategy that is different from that of the interaction partners (maximize surface). In the continuous variants of those games it is naturally much harder to maintain and spread distinct traits in fragmented filament-like structures because they are more prone to effects of noise than compact clusters. Effectively this fragmentation inhibits evolutionary branching because diverging traits tend to trigger further fragmentation and as a consequence do not survive long enough to get established and form their own branch. In contrast, the compact clusters promoted by the prisoner's dilemma provide structural protection for higher investors and thus help drive diversification.

Because of global competition the spatial dynamics for birth-death updating is (unsurprisingly) much closer to results for well-mixed populations. For example, evolutionary branching was again only observed for continuous snowdrift game. Also because of global competition, structured populations are updated in a non-uniform manner. In particular, regions of high payoffs experience a much higher turnover than regions of low payoffs. For strong selection this can result in almost frozen parts of the population. As a consequence unsuccessful traits are able to stay around for long times and, in some cases, those traits turn out to be advantageous again at later times when the surroundings have sufficiently changed, so that the stragglers then contribute to diversification. This mode of diversification, however, introduces historical contingencies where the evolutionary end state can sensitively depend on the initial configuration.

Overall, we find that evolutionary diversification is a robust feature of continuous spatial games, and that spatial structure can sometimes hinder, but generally promotes diversification through modes of diversification that complement traditional evolutionary branching.

Spatial invasion fitness and adaptive dynamics

In the following, we assume that the total population size is constant, and that spatially structured populations are represented by lattices in which each site is occupied by one individual. Each individual interacts with a limited number of local neighbours, and we assume this number, \(k\), to be the same for all individuals. We first consider a case where there are two types of players in the structured population: a mutant type with trait value \(y\), and a resident type with trait value \(x\) (where \(x\) and \(y\) denote investment strategies in a continuous game). If the mutant has \(j\) other mutants among its \(k\) neighbours, the mutant payoff is \begin{align} \pi_m(j) = [(k-j)P(y,x)+j P(y,y)]/k . \end{align} Similarly, the payoff of a resident with \(j\) mutant neighbours is given by \begin{align} \pi_r(j) = [j P(x,y)+(k-j)P(x,x)]/k . \end{align} The payoffs, \(\pi_m(j), \pi_r(j)\), of mutants and residents from interactions with their \(k\) neighbours determines the birth rates as \(b_m(j)=\exp(w\pi_m(j))\) and \(b_r(j)=\exp(w\pi_r(j))\), where \(w>0\) denotes the strength of selection. The birth rate is proportional to the probability of taking over an empty site for which a given mutant or resident individual competes. For \(w\ll1\) selection is weak and differences in payoffs result in minor differences in birthrates, and hence in small differences in probabilities of winning competition for an empty site. With strong selection, \(w\gg1\), payoff differences are amplified in the corresponding birthrates. This exponential payoff-to-birthrate map has several convenient features:

1. ensures positive birthrates,
2. admits easy conversion to probabilities for reproduction,
3. selection can be arbitrarily strong,
4. for weak selection the more traditional form of birthrates \(b_i(j)\approx1+w\pi_i(j)\), \(i=m,r\), is recovered.

Note that in the limit \(y\to x\) differences in birthrates vanish. However, this does not imply weak selection. Instead, selection strength is determined by the magnitude of the invasion fitness gradient at \(x\), which is proportional to \(w\).

In well-mixed populations the current state is simply given by the frequency of mutants and residents, respectively. In contrast, in structured populations the state space is immense because it involves all possible configurations. Pair approximation offers a convenient framework to account for corrections arising from spatial arrangements. Instead of simply tracking the frequencies of mutants and residents, pair approximation considers the frequencies of neighbouring strategy pairs.

We denote the frequencies of mutant-mutant, mutant-resident, resident-mutant and resident-resident pairs by \(p_{mm}, p_{mr}, p_{rm}\), and \(p_{rr}\), respectively. This reduces to two dynamical equations because \(p_{mr}=p_{rm}\) and \(p_{mm}+p_{mr}+p_{rm}+p_{rr}=1\) must hold. The most informative quantities are the global mutant frequency, \(p_m=p_{mm}+p_{mr}\), and the local mutant density \(q_{m|m}=p_{mm}/p_m\), i.e. the conditional probability that a neighbour of a mutant is also a mutant. Note that for rare mutants, \(p_m\ll1\), their local densities need not be small. The derivation of the corresponding dynamical equations depends on the details of the microscopic updating.

Death-birth process

The death-birth process with selection on birth in structured populations results in local competition: an individual is selected to die uniformly randomly from the whole population, then its \(k\) neighbours compete to repopulate the newly vacated site. They succeed with a probability proportional to their birthrates. Note that payoffs, and hence birth rates, are based on interactions with all neighbours, including the neighbour that may subsequently be chosen to die (uniformly at random) and its vacant site subject to recolonization by the offspring of one of its neighbours. To determine the dynamics of \(p_m\) and \(q_{m|m}\), we first note that configurations only change when a resident is replaced by a mutant, or when a mutant is replaced by a resident. The dynamical equations for the mutant dynamics in structured populations based on pair approximation with death-birth updating are derived in SI~Text~S1. %\app{SAD:dB}.

Invasion fitness

In order to obtain the invasion fitness, we note that the dynamics of spatial invasion unfolds in two stages: mutants quickly establish a local (pseudo) equilibrium and then gradually increase (or decrease) in frequency. Formally, this is reflected in a separation of time scales in the limit of rare mutants, \(p_m\ll1\), between the slow global frequency dynamics and the fast local pair density (see \eq{dB:pm} and \eq{dB:qmm}). As a consequence, to calculate invasion fitness we assume that the local pair density of the mutant is at its equilibrium \(q^\ast_{m|m}\). The invasion fitness of mutants, \(f(x,y)\), defined as their per capita growth rate, \(\dot p_m/p_m\), in the limit \(p_m\to0\), then becomes:

\begin{align} \label{eq:ifit:dB} f(x,y) = &\ \frac{k (1-q^\ast_{m|m}) b_m\left((k-1) q^\ast_{m|m}\right)}{b_m\left((k-1) q^\ast_{m|m}\right)+(k-1) b_r(0)}-\sum_{j=0}^k \binom{k}{j} (q^\ast_{m|m})^j (1-q^\ast_{m|m})^{k-j}\frac{(k-j) b_r(1)}{j b_m\left(1+(k-1) q^\ast_{m|m}\right)+(k-j) b_r(1)}, \end{align}

where \(b_m(v)\) and \(b_r(v)\) denote the birth rates of mutants and residents, respectively, with an average number of \(v\) mutants in their neighbourhood.

Even though the solution to \(\dot q_{m|m}=0\) is analytically inaccessible, in general, the equilibrium \(q^\ast_{m|m}\) can be approximated using a Taylor expansion if \(|y-x|\ll1\):

\begin{align} \label{eq:qmm:dB:sol} q^\ast_{m|m} =&\ \frac1{k-1}+w (y-x) \frac{k^2-4}{2 (k-1)^2 k^2}\big(\partial_z P(x,z)+k \partial_z P(z,x) \big)\Big|_{z=x}+O\left((y-x)^2\right). \end{align}

It follows that in the limit \(y\to x\), mutants with at least one resident neighbour have, on average, one mutant neighbour among their \(k-1\) other neighbours. Note, mutants with no resident neighbours are uninteresting because they are unable to initiate a change in the population configuration. Interestingly, this limit of the local pair configuration is fairly robust with respect to changes in the updating process (c.f. Eq.~S25 in SI~Text~S4 for birth-death updating). Moreover, in this limit a rare mutation with positive invasion fitness is guaranteed to eventually take over.

Using \eqs{ifit:dB} and \ref{eq:qmm:dB:sol} the selection gradient, \(D_\text{db}(x)=\frac{\partial f(x,y)}{\partial y}\vert_{y=x}\), as well as its Jabobian, \(CS_\text{db}(x^\ast)=\frac{dD_\text{db}(x)}{dx}\vert_{x=x^\ast}\), and the Hessian of fitness, \(ES_\text{db}(x^\ast)=\frac{\partial^2f(^\ast,y)}{\partial y^2}\vert_{y=x^\ast}\), at a singular point \(x^\ast\) can be calculated as:

\begin{align} \label{eq:d:dB} D_\text{db}(x) = &\ w\frac{k-2}{k(k-1)}\big(k \partial_y P(y,x) + \partial_y P(x,y)\big)\Big|_{y=x}\\ \label{eq:css:dB} CS_\text{db}(x^\ast) = &\ w\frac{k-2}{k(k-1)}\Big(k \partial_y^2 P(y,x^\ast) + \partial_y^2 P(x^\ast,y) + (k+1)\partial_{y,z} P(y,z)\Big)\Big|_{z=y=x^\ast}\\ \label{eq:ess:dB} ES_\text{db}(x^\ast) = &\ C\!S_\text{db}(x^\ast)-w\frac{(k-2)^2(k+1)}{k^2(k-1)} \partial_{y,z} P(y,z)\big|_{z=y=x^\ast}. \end{align}

Publications

1. Hauert, C. & Doebeli, M. (2021) Spatial social dilemmas promote diversity, Proc. Natl. Acad. Sci. USA 118 42 e2105252118 doi: 10.1073/pnas.2105252118

References

1. Ohtsuki, H., Hauert, C., Lieberman, E., & Nowak, M.A. (2006) A simple rule for the evolution of cooperation on graphs. Nature 441 502-505.
2. Allen, B., Nowak, M.A., & Dieckmann, U. (2013) Adaptive dynamics with interaction structure. Am. Nat. 181 (6):E139--E163.
3. Tarnita, C.E., Wage, N., & Nowak, M.A. (2011) Multiple strategies in structured populations. Proc. Natl. Acad. Sci. USA 108:2334--2337.
4. Parvinen, K., Ohtsuki, H., & Wakano, J.Y. (2017) The effect of fecundity derivatives on the condition of evolutionary branching in spatial models. J. Theor. Biol. 416:129--143.