Spatial social dilemmas promote diversity

Author: Christoph Hauert

Cooperative investments in social dilemmas can spontaneously diversify into stably co-existing high and low contributors in well-mixed populations. Here we extend the analysis to emerging diversity in (spatially) structured populations. Using pair approximation we derive analytical expressions for the invasion fitness of rare mutants in structured populations, which then yields a spatial adaptive dynamics framework. This allows us to predict changes arising from population structures in terms of existence and location of singular strategies, as well as their convergence and evolutionary stability as compared to well-mixed populations. Based on spatial adaptive dynamics and extensive individual based simulations, we find that spatial structure has significant and varied impacts on evolutionary diversification in continuous social dilemmas. More specifically, spatial adaptive dynamics suggests that spontaneous diversification through evolutionary branching is suppressed, but simulations show that spatial dimensions offer new modes of diversification that are driven by an interplay of finite-size mutations and population structures. Even though spatial adaptive dynamics is unable to capture these new modes, they can still be understood based on an invasion analysis. In particular, population structures alter invasion fitness and can open up new regions in trait space where mutants can invade, but that may not be accessible to small mutational steps. Instead, stochastically appearing larger mutations or sequences of smaller mutations in a particular direction are required to bridge regions of unfavourable traits. The net effect is that spatial structure tends to promote diversification, especially when selection is strong.

Social dilemmas with continuous traits

Social dilemmas are important mathematical metaphors for studying the problem of cooperation. The best studied models of social dilemmas are the prisoner's dilemma and the snowdrift game. Traditionally, such models are often restricted to the two distinct strategies of cooperate, \(C\), and defect, \(D\). However, these models can easily be extended to describe a continuous range of cooperative investments into a public good. In such continuous games, the aim is to study the evolutionary dynamics of the level of investment.

For example, in the donation game, which is the most prominent version of the prisoner's dilemma, cooperators confer a benefit \(b\) to their interaction partners at a cost \(c\) to themselves. Defection entails neither costs nor confers benefits. With \(b>c>0\), both individuals prefer mutual cooperation, which pays \(b-c\), over a zero payoff for mutual defection. However, the temptation to shirk costs and free ride on benefits provided by the partner undermines cooperation to the detriment of all. In fact, defection is the dominant strategy because it results in higher payoffs regardless of the partner's strategy. This conflict of interest between the individual and the group represents the hallmark of social dilemmas.

Continuous Donation game

A natural translation of the donation game to continuous traits is based on cost and benefit functions, \(C(x)\) and \(B(x)\), where the trait \(x\in[0,x_\text{max}]\) represents a level of cooperative investment that can vary continuously. In the spirit of the donation game, an individual with strategy \(x\) interacting with an \(y\)-strategist obtains a payoff \(P(x,y)=B(y)-C(x)\). Assuming that (i) zero investments incur no costs and provide no benefits, \(B(0)=C(0)=0\), (ii) benefits exceed costs, \(B(x)>C(x)\geq0\), and (iii) are increasing functions, \(B^\prime(x), C^\prime(x)\geq0\), recovers the social dilemma of the donation game for continuous investment levels: the level of investment invariably evolves to zero, which corresponds to pure defection, despite the fact that both players would be better off at non-zero investment levels. The reason is that an actor can only influence the cost \(C(x)\) of an interaction, but not the received benefit \(B(y)\), and hence selection can only act to minimize costs.

Continuous Snowdrift game

In a weaker form of a social dilemma, the snowdrift game, cooperators also provide benefits, \(b\), at a cost, \(c\). However, benefits are now accessible to both individuals and accumulate in a discounted manner. For example, yeast secretes enzymes for extra cellular digestion of sucrose. While access to nutrients is crucial, the marginal value of additional resources diminishes and may exceed the intake capacity. As a result, the social dilemma remains in effect, because the most favourable outcome of mutual cooperation remains prone to cheating, but the dilemma is relaxed because it pays to cooperate against defectors. Again, it is straightforward to formulate a continuous version of the snowdrift game. In contrast to the donation game, in the continuous snowdrift game the benefits depend on the strategies of both interacting individuals. For example, for simplicity one can assume that benefits depend on the aggregate investment levels of both players, so that the benefit is a function \(B(x+y)\). Assuming a cost function \(C(x)\), an \(x\) strategist interacting with strategy \(y\) obtains a payoff \(P(x,y)=B(x+y)-C(x)\), again with \(B(0)=C(0)=0\), \(B(x)>C(x)\geq0\) and \(B^\prime(x),C^\prime(x)\geq0\), as before, but with the additional constraint \(C(x)>B(x+y)-B(y)\) for sufficiently large \(y\) so that the increased return from investments of an individual do not outweigh its costs when interacting with high investors. Typically this is achieved by saturating benefit functions, \(B^{\prime\prime}(x)<0\).

The gradual evolution of continuous traits can be described using the framework of adaptive dynamics. Below we extend this framework to spatial settings by amalgamating adaptive dynamics and pair approximation into spatial adaptive dynamics. This provides the toolbox to investigate the impact of spatial structures on the evolution of cooperation in the prisoner's dilemma as well as the snowdrift games.

Evolution in the continuous prisoner's dilemma

In the continuous prisoner's dilemma the payoff to an individual with strategy \(y\) interacting with an \(x\)-strategist is \(P(y,x)=B(x)-C(y)\). This implies that the selection gradient in well-mixed populations is proportional to \(-C'(x)\), and hence, assuming monotonously increasing costs, no singular strategy exists apart from the pure defection state \(x=0\), and the population always evolves to that state, regardless of how large the benefits of cooperation are. In contrast, for death-birth updating in structured populations the selection gradient, \eq{d:dB}, becomes

\begin{align} \label{eq:d:pd:dB} D_\text{db}(x) &= w\frac{k-2}{k(k-1)}(B^\prime(x)-k C^\prime(x)). \end{align}

Thus, in structured populations a singular strategy, \(x^\ast\), may exist as a solution to \(D_\text{db}(x^\ast)=0\). If \(x^\ast\) exists and is convergence stable then it is also evolutionarily stable because the two stability conditions are identical (the mixed derivatives on the right hand side of \eqs{css:dB} and (\ref{eq:ess:dB}) are zero). In particular, cooperation can be maintained in spatially structured populations, a result that is of course in line with classical theory. More specifically, cooperative investments can increase if the marginal benefits, \(B^\prime(x)\), exceed the \(k\)-fold marginal costs, \(C^\prime(x)\), which is reminiscent of the \(b>ck\)-rule in the traditional donation game (see Ohtsuki et al. 2006).

Evolution in the continuous snowdrift game

In the continuous snowdrift game the payoff to an individual with strategy \(y\) interacting with an \(x\)-strategist is \(P(y,x)=B(x+y)-C(y)\). Note that this generalization to continuous strategies does not imply that every interaction between an individual with strategy \(x\) and another with strategy \(y\) invariably results in a snowdrift game. In fact, snowdrift games in this narrower sense appear only if the two traits \(x, y\) straddle the singular strategy \(x^\ast\). Depending on the cost and benefit functions, as well as on the strategies \(x\) and \(y\), any kind of \(2\times2\) game can arise, including a prisoner's dilemma, in which the lower investing strategy dominates (Doebeli et al. 2004, Doebeli et al. 2013). In this sense the framework of the continuous snowdrift game actually encompasses the gist of the continuous prisoner's dilemma for sufficiently high costs -- just as the classical snowdrift game turns into a prisoner's dilemma (see Hauert & Doebeli, 2004).

The selection gradient for the continuous snowdrift game in structured populations, \eq{d:dB}, is given by \begin{align} \label{eq:d:csd:dB} D_\text{db}(x) &= w\frac{k-2}{k(k-1)}\left((k+1)B^\prime(2x)-k C^\prime(x)\right). \end{align} Thus, singular strategies \(x^\ast\) may exist, and if \(x^\ast\) exists, it is convergence stable if \begin{align} \label{eq:css:csd:dB} CS_\text{db}(x^\ast) &= w\frac{k-2}{k(k-1)}\left(2(k+1)B^{\prime\prime}(2x^\ast)-k C^{\prime\prime}(x^\ast)\right)<0, \end{align} and evolutionarily stable if \begin{align} \label{eq:ess:csd:dB} ES_\text{db}(x^\ast) &= w\frac{k-2}{k^2(k-1)}\left((k+2)(k+1)B^{\prime\prime}(2x^\ast)-k^2 C^{\prime\prime}(x^\ast)\right)<0. \end{align} In particular, the conditions for convergence and evolutionary stability are different, which indicates the potential for evolutionary branching and hence the evolutionary emergence and co-existence of high investing cooperators and low investing defectors. The above conditions are the spatial analogs of the analysis for well-mixed continuous snowdrift games reported in Doebeli et al. 2004. Also note that for linear cost and benefit functions it is not possible to satisfy all constraints for the continuous snowdrift game and hence the simplest case is given by quadratic costs and benefits.

Quadratic costs and benefits

For suitable cost and benefit functions the (spatial) adaptive dynamics is analytically accessible. Here we focus on the quadratic cost and benefit functions used in Doebeli et al. 2004 for the well-mixed case: \(B(x)=b_1 x+b_2 x^2\) and \(C(x)=c_1 x+c_2 x^2\). We assume that the evolving trait is confined to the interval \([0,1]\). The benefit and cost functions satisfy the criteria \(B(0)=C(0)=0\), \(B(x),C(x)\geq0\) and \(B^{\prime\prime}(x)<0\) for \(x\in[0,1]\) provided that \(b_1,c_1>0\), \(b_2,c_2<0\) as well as \(b_1<-4b_2\) and \(c_1<-2c_2\). The first two conditions ensure that costs and benefits are increasing yet saturating, while the latter two ensure that \(B(2x)\) and \(C(x)\) are increasing over the entire trait interval, i.e. their respective maximum occurs for \(x\geq1\).

For small resident values \(x\), the selection gradient \eq{d:csd:dB} becomes \(D_\text{db}(x)\approx w\frac{k-2}{k(k-1)}\left((k+1)b_1-k c_1\right)\), and hence cooperative investments evolve away from zero provided that \(b_1>\frac{k}{k+1}c_1\), which is slightly less restrictive than the corresponding condition \(b_1>c_1\) in the well-mixed case.

For the above quadratic costs and benefits the singular strategy is given by \begin{align} \label{eq:xs:csd:dB} x^\ast_\text{db} &= \frac{b_1(k+1)-c_1 k}{2c_2k-4b_2(k+1)}. \end{align} The numerator of \eq{xs:csd:dB} is positive if and only if the condition for evolution away from zero, \(b_1>\frac{k}{k+1}c_1\), is satisfied. Similarly, the denominator of \eq{xs:csd:dB} is positive if and only if \(2b_2(k+1)<c_2k\) (recall \(b_2, c_2<0\)). It is clear from \eq{css:csd:dB} that this is also the condition for convergence stability. Thus, if a singular point, \(x^\ast\), exists, cooperative traits either evolve away from zero and \(x^\ast\) is convergence stable, or the trait cannot evolve away from zero and \(x^\ast\) is a repellor.

We note that if the singular point exists, \(x^\ast\), it is shifted towards smaller investments for a given set of parameters, as compared to well-mixed populations with the same parameters: \(x^\ast = \frac{b_1-c_1}{2c_2-4b_2}\). Furthermore, the condition for convergence stability, \(2b_2(k+1)<c_2k\), is less restrictive than the corresponding condition \(2b_2<c_2\) in the well-mixed case. Finally, from \eq{ess:csd:dB} we see that the condition for evolutionary stability is \(b_2(k+2)(k+1)<c_2 k^2\), which is again less restrictive than the corresponding condition \(b_2<c_2\) in the well-mixed case. Combining the two stability conditions shows that evolutionary branching occurs for \(2b_2(k+1)/k<c_2<b_2(k+1)(k+2)/k^2\). Thus, the analytical approach based on pair approximation suggests that population structures tend to inhibit evolutionary diversification by decreasing the range of parameters for which the singular point is an evolutionary branching point (c.f. \fig{csd:dB}a \& c).

Conclusions & Discussion

In all scenarios considered here we find that population structures can promote and facilitate spontaneous diversification in social dilemmas into high and low investors, especially when selection is strong. However, at the same time, classical evolutionary branching tends to be inhibited, but compensated for by other modes of diversification. We derive an extension of adaptive dynamics for continuous games in (spatially) structured populations based on pair approximation, which tracks the frequencies of mutant-resident pairs during invasion. It turns out that predictions derived from this spatial adaptive dynamics framework are independent of selection strength. More precisely, selection strength only scales the magnitude of the selection gradient as well as that of convergence and evolutionary stability but neither affects the location of singular strategies nor their stability. Nevertheless, from the invasion analysis of mutant traits \(y\) into resident populations \(x\), as well as from individual-based simulations, it is clear that selection strength has a significant impact on the dynamics in general, and on diversification in particular. During the invasion process the local configuration probability of mutant pairs \(q_{m|m}\) is of crucial importance and changes with selection strength, see \eq{qmm:dB:sol}. However, in the adaptive dynamics limit \(y\to x\) the pair configuration probability reduces to a constant \(q_{m|m}\to1/(k-1)\). Even though this is the same value as obtained in the limit of weak selection, \(w\to0\), it is important to note that the limit \(y\to x\) does not necessarily imply weak selection. Thus, the different modes of diversification cannot be understood based on spatial adaptive dynamics alone but require a more comprehensive analysis of invasion dynamics.

Structured populations offer new modes of diversification that are driven by an interplay of finite-size mutations and population structures. Trait variation is more easily maintained in structured populations due to the slower spreading of advantageous traits as compared to well-mixed populations. Spatial adaptive dynamics is unable to capture these new modes of diversification because of the underlying assumption that the resident population is composed of discrete traits (monomorphic before branching and dimorphic or polymorphic after branching). Nevertheless, invasion analysis and pairwise invasibility plots, \(PIP\), in particular, provide an intuitive interpretation for additional pathways to diversification that are introduced by spatial structures and further promoted for increasing selection strength. Even in the vicinity of evolutionarily stable investments, trait combinations exist that admit mutual invasion and hence can coexist. However, such states cannot evolve through adaptive dynamics but are nevertheless accessible to trait distributions around the evolutionarily stable trait and can drive a degenerate form of branching. Moreover, spatial invasion fitness can open up new regions in trait space for mutant invasion. However, those regions need not be accessible by small mutational steps, and instead require stochastically appearing larger mutations or sequences of smaller mutations that allow to bridge regions of unfavourable traits.

Previous attempts at amalgamating adaptive dynamics and spatial structure have not observed spontaneous diversification or evolutionary branching. In particular, Allen et al. (2013) augment adaptive dynamics by structure coefficients (Tarnita et al. 2011), which restrict the analysis to weak selection. Moreover, their framework is fundamentally different from ours because it is based on fixation probabilities rather than invasion fitness. More specifically, their analysis is based on the fixation probabilities \(p_x\) and \(p_y\) of two types \(x\) and \(y\) in a population consisting of \(x\) and \(y\): \(x\) is favoured over \(y\) if \(p_x>p_y\) (see Eq.~4 and formulas following Eq.~7 in Allen et al. 2013). Clearly, it is then impossible to have both types be favoured in an \((x,y)\)-population, but this is exactly the requirement for mutual coexistence. In fact, Allen et al. acknowledge that fixation probabilities are "less appropriate" for studying evolutionary branching and that instead "establishment probabilities" should be considered for studying diversification. As a consequence their adaptive dynamics derivation precludes its most intriguing feature, that of evolutionary branching. However, in the limit of weak selection the two approaches are compatible in some aspects. For example, the selection gradient and the condition for convergence stability reported for the continuous prisoner's dilemma are identical (c.f. \eqs{d:csd:dB},~\ref{eq:css:csd:dB} and their Eqs.~11,~13). In our framework, the use of invasion fitness allows to determine evolutionary stability, and the corresponding \(PIP\)'s reveal the potential for degenerate forms of branching – but only for stronger selection (see \fig{kdk:dB}). Another attempt at spatial adaptive dynamics considers spatial structure in the form of demes (Parvinen et al. 2017). However, quite naturally such deme structures suppress branching because in the long run it is impossible to maintain multiple co-existing traits in small demes, which quickly become homogeneous. Instead, branching is only possible if demes are large enough to support branching in every single one.

Interestingly, in well-mixed populations evolutionary branching is only observed for the continuous snowdrift game, where two distinct traits of high and low investors can co-exist and essentially engage in a classical (discrete) snowdrift game. In contrast, in structured populations with death-birth updating, evolutionary branching is only observed for prisoner's dilemma type interactions where lower investments invariably dominate higher ones, which applies both in the continuous prisoner's dilemma as well as the continuous snowdrift game with sufficiently high costs. The reason for this surprising difference can be understood intuitively by considering the preferred spatial configurations in the two classical (discrete) games: in the prisoner's dilemma cooperators form compact clusters to reduce exploitation by defectors (minimize surface), while in the snowdrift game filament like clusters form because it is advantageous to adopt a strategy that is different from that of the interaction partners (maximize surface). In the continuous variants of those games it is naturally much harder to maintain and spread distinct traits in fragmented filament-like structures because they are more prone to effects of noise than compact clusters. Effectively this fragmentation inhibits evolutionary branching because diverging traits tend to trigger further fragmentation and as a consequence do not survive long enough to get established and form their own branch. In contrast, the compact clusters promoted by the prisoner's dilemma provide structural protection for higher investors and thus help drive diversification.

Because of global competition the spatial dynamics for birth-death updating is (unsurprisingly) much closer to results for well-mixed populations. For example, evolutionary branching was again only observed for continuous snowdrift game. Also because of global competition, structured populations are updated in a non-uniform manner. In particular, regions of high payoffs experience a much higher turnover than regions of low payoffs. For strong selection this can result in almost frozen parts of the population. As a consequence unsuccessful traits are able to stay around for long times and, in some cases, those traits turn out to be advantageous again at later times when the surroundings have sufficiently changed, so that the stragglers then contribute to diversification. This mode of diversification, however, introduces historical contingencies where the evolutionary end state can sensitively depend on the initial configuration.

Overall, we find that evolutionary diversification is a robust feature of continuous spatial games, and that spatial structure can sometimes hinder, but generally promotes diversification through modes of diversification that complement traditional evolutionary branching.

Publications

1. Hauert, C. & Doebeli, M. (2021) Spatial social dilemmas promote diversity, Proc. Natl. Acad. Sci. USA 118 42 e2105252118 doi: 10.1073/pnas.2105252118

References

1. Allen, B., Nowak, M.A., & Dieckmann, U. (2013) Adaptive dynamics with interaction structure. Am. Nat. 181 (6) E139–E163.
2. Doebeli M, Hauert C, Killingback T (2004) The evolutionary origin of cooperators and defectors. Science 306 (5697) 859–62.
3. Doebeli M, Hauert C, Killingback T (2013) A comment on "Towards a rigorous framework for studying 2-player continuous games" by Shade T. Shutters, Journal of Theoretical Biology 321, 40--43, 2013. J. Theor. Biol. 336 240–241.
4. Hauert C, Doebeli M (2004) Spatial structure often inhibits the evolution of cooperation in the snowdrift game. Nature 428 643–646.
5. Ohtsuki, H., Hauert, C., Lieberman, E., & Nowak, M.A. (2006) A simple rule for the evolution of cooperation on graphs. Nature 441 502–505.
6. Parvinen, K., Ohtsuki, H., & Wakano, J.Y. (2017) The effect of fecundity derivatives on the condition of evolutionary branching in spatial models. J. Theor. Biol. 416:129--143.
7. Tarnita, C.E., Wage, N., & Nowak, M.A. (2011) Multiple strategies in structured populations. Proc. Natl. Acad. Sci. USA 108 2334–2337.