That cognitive mechanisms drive the evolution of social behaviour is hardly surprising. Cognition includes perception, learning, memory and decision-making, and other ways in which animals can take in information about the world through the senses, process, retain and decide to act on it (Shettleworth, 2001). Having cognitive abilities hence allows an individual to carry out quick and sophisticated assessment of the pay-offs of social interactions. Humphrey (1976) and Dunbar (1998) suggested that the complexities of social living require complex cognition, and this was the driving force for the evolution of intelligence and enlarged brains. Suggestions have also been made about how the evolution of cognitive abilities had driven the evolution of abilities to navigate challenges of social life.
The ability to carry out individual recognition and to retain information about other parties are hallmarks of cognitive abilities that can be considered prerequisites for complex social behaviour. Discriminating between individuals requires the capacity to integrate past information with incoming sensory information, while retaining a mental representation of the individual as a unique entity (Tibetts and Dale, 2007). These allow individuals to carry out complex social behaviours that will ultimately provide them with a fitness advantage.
However, the significance of cognition in the evolution of social behavior is reduced when we consider how seemingly 'complex' social behaviors can be better explained by simpler mechanisms such as conditioning and associative learning. Indeed, many behavioural studies aimed at studying animal cognition have found that many social behaviors can be explained without the need to invoke cognitive mechanisms.
Discussions and studies of the relationship between cognition and social behavior have focused primarily on complex social behaviors of higher animals. While cognitive mechanisms can be the main driver of some social behaviours, this may not be a generalizable theory.
Social behavior is defined as a set of interactions among individuals of the same species with potentially separate evolutionary trajectories (Bourke, 2011). In many cases, natural selection acting on an individual at the gene level is sufficient to explain many instances of social behaviour. These are driven by natural selection, and maintained by the fitness advantage it gives to the actor. Considering the definition by Bourke, we are also reminded that social behaviour also exists in many non-animals that do not have any form of cognitive abilities e.g. Slime moulds, bacteria and plants.
It has been shown in many studies that having cognitive mechanisms account for the evolution of certain social behavior. In this essay, I aim to discuss how that often may not be the most parsimonious explanation for the evolution and maintenance of social behaviour, and also provide a variety of examples where social behaviour has evolved, and is evolving without cognitive abilities.
There are many social behaviours seem to entail cognitive complexity that it may be easy to assume that they only exist in human beings or our closest primate relatives. One of the prerequisites for many social behaviors is the ability to recognise individuals, and the ability to retain the information in the memory (Emery et al., 2007). Individual recognition can be considered a relatively complex form of recognition. Other than requiring a multi-component mental representation of a specific individual independent of the context, it also requires flexible learning, and memory (Tibbetts and Dale, 2007).
Individual recognition and information retention are especially beneficial when repeat interactions are likely. They allow one to assign a value to an individual, predict the likely future behaviour of that individual and anticipate or determine the pay-offs of interactions. The ability of individual recognition is hence beneficial in social interactions such as aggression, competition, parental care, maintenance of social hierarchies, retaining in-group and out-group classifications etc (Sheehan and Tibetts, 2011). This allows the reduction of the cost associated with competition. Individual recognition is also crucial in some social behaviours such as direct and indirect reciprocity, maintaining cooperation between non-kin (Tebbich et al., 2002).
For example, hooded warblers are able to remember specific neighbours from the previous breeding season even after the overwintering period (Godard, 1991). This provides reproductive benefits since re-establishment of territories would take up time that could be spent on courting and mating (Godard, 1991). Hermit crabs can distinguish and associate distinctive scents of an individual to their competitive abilities and the quality of their shell (Gherardi and Atema, 2005), lowering the cost of their social interactions associated with agonistic competitions.
Tibbetts (2002) found that the paper wasps (Polistes fuscatus) use highly variable facial and abdominal markings to identify nest mates at the individual level. This species of wasp has also been found to have long term social memory - they are able to remember the identity of social partners for more than a week despite interactions with other individuals in that period of time (Sheehan and Tibbetts, 2011). The ability to adjust their behaviour based on previous outcome is beneficial since the wasps are likely to re-encounter the same competitors.
In addition, the ability to attribute mental states to others is another prerequisite for a range of complex social behavior. Theory of Mind (ToM) is the ability to attribute a mental state to others and to infer knowledge held or deduce intentions in others (Premack and Woodruff, 1978). This requires complex cognitive mechanism, and ToM has been suggested to be a prerequisite for more advanced forms of social behaviour such as certain forms of cooperation and tactical deception (Premack and Woodruff, 1978).
Tactical deception is often associated with intelligence (Byrne and Whiten, 1985), and would be beneficial when social manipulation will lead to higher pay-offs than direct confrontation or competition with another individual (Byrne and Whiten, 1992).
Ravens and western scrub jays also seem to show deliberate deception that can be attributed to ToM. Ravens are also able to 'take note of' individuals who have seen them make caches. They alter their behavior towards their food they had hidden based on whether or not their competitors also know where the food is, by making 'false caches' when competitors are watching (Bugnyar, 2011).
Other than having deceptive intention themselves, the ability to attribute deceptive intentions to others suggest a high level of cognition. The proverbial 'It takes a thief to catch a thief' applies well to the Western scrub jays. Western scrub jays cache surplus food, and are also able to observe other scrub jays cache food, and pilfer their caches in private. However, only the scrub jays that had experiences of pilfering will pilfer and recache the food of other individuals (Emery and Clayton, 2001). This ability to extrapolate from own's previous experience seems best explained using mental state attribution.
Other than to deduce what others know or see, inferential ability can also be useful in other contexts. Another example of inferential ability is in transitive inference - Using known relationships to deduce unknown ones. Guillermo et al. (2004) showed that Pinyon jays are able to carry out transitive inference, whereby they are able to predict their social dominance based on observations of others and the interactions between them and individuals of unknown ranking. Such sophisticated inference about one's own relative dominance status has also been found in the African cichlids, Astatotilapia burtoni (Grosenick et al., 2007), in experiments that involve by-stander watching of fights between rival males.
Cognitive mechanisms of various complexity levels are required for individual recognition, memory, mental state attribution and inferential ability. The ability to make facultative decisions based on these can give rise to a plethora of other social behaviours. However, in many cases, seemingly complex social behavior may not require complex cognitive mechanisms. While it is tempting to attribute the exhibition of complex social behavior in to cognitive abilities, this may be overly presumptuous.
Indeed, many studies on animal intelligence have shown that 'complex' social behaviour can actually be explained more with simpler processes. Rather than being results of cognitive mechanisms such as individual recognition, memory, mental state attribution or inferential abilities, they could happen by chance due to associative learning, conditioning, or be a result of making inference based on behaviour (behavioural-reading rather than mind-reading) (Heyes, 1993; Kummer et al., 1990; Timberlake, 1994; Shettleworth, 2001).
The ability to understand the relationships between other individuals will allow an individual to better estimate the pay-offs of social interaction with other individuals. For example, a male with the ability to track the relationship between mother and offsprings will be better able to know when the female is receptive of a new mate. The ability to recognise third party relationships has been studied in spotted hyenas. After fights, individuals are more likely to attack relatives of their ex-opponents, rather than other individuals (Engh et al., 2005). This can be attributed to knowing third-party relationships, a relatively cognitively demanding ability. However, we cannot rule out that the behavior of the hyenas are guided by simple rules such as attacking individuals that are in close proximity to or smell like the former opponent.
The cooperative hunting behavior of hyenas may seem to require high-level coordination and division of labour (Guggisberg, 1962; Stander, 1992). However, such behavior can also be explained by simple rule of thumb such as 'keeping selected prey animals between you and another hunter' (Holekamp et al., 2000). Busse (1976) also suggested that cooperative-hunting wild chimpanzees may be following one's 'individually best strategy'.
Having a concept of 'social image' is one of the reasons behind apparently altruistic acts in human society. The 'altruistic' deed allows one to build up a good reputation based on which benefits can be obtained from other individuals. Cleaner fish can cooperate and remove ectoparasites on their client fish, or cheat on feeding client mucus. Bshary and Grutter (2006) tested whether client fish observing the interaction between cleaner fish and other client fish are able notice the act of the different cleaner fish and carry out 'image-scoring'. It was found that after observing interactions of others, bystander clients did spent more time next to cooperative cleaners. While this could be attributed to the fish's ability to remember observed interactions and make decisions about who to interact with, the writers cautioned that the client fish may not have retained a memory of the observation, since they react immediately to observed interactions (Bshary and Grutter, 2006).
Another instance of social behavior thought to require cognition is reciprocity. Reciprocity involves repeated interactions between the same individuals, a time delay between initial and reciprocal act, and hence requires individual recognition, memory, and facultative alteration of behavior based on behavior or partner. However, Suchak and de Waal (2012) showed that Brown capuchin monkeys (Cebus apella) have a tendency to benefit a partner, even following a selfish choice, i.e. they are more likely to make a 'prosocial' choice. This result deviates from the expected evolutionarily stable tit-for-tat strategy. In this case, there is no individual recognition or mental scorekeeping that is required in this social interaction between the capuchin monkeys, but the outcomes are similar to what is expected from a reciprocal interaction. This suggests that prosocial act could actually occur even without individual recognition and scorekeeping.
While we cannot disregard the importance of complex cognitive mechanisms in the evolution of many social behaviours, these and many other examples (e.g. Zuberbuhler, 2000; Menzel, 1990; de Waal and Luttrell, 1989; Flack et al., 2006; Proops et al., 2009) show how apparently complex social behaviour could take place without the need for complex mechanisms, and caution should be exercised when invoking cognitive mechanisms to explain social behaviour.
Back to basics
Most of the social behaviours mentioned so far are what have been described as 'complex' behaviours. While it is important to study the significance of cognitive mechanisms in the evolution of social behaviour, it distracts us from fundamental aspects of social interactions. Most fundamentally, social interaction can be classified into four categories: Cooperation, selfishness, altruism and spite (Hamilton, 1964). The evolution and maintenance of these behaviours are driven by natural selection acting at the level of the genes. For example, selfish behaviours benefit the actor by allowing the propagation of the actor's genes at the expense of the recipent, and would hence be favoured by natural selection.
Kin selection is also particularly relevant in the discussion of social interactions - Other than acting to maximize their own direct fitness, animals also act to maximize their inclusive fitness through their kin (Maynard-Smith, 1964). Natural selection hence leads organisms to 'appear' designed as if to maximise their inclusive fitness. Kin selection, driven by the relatedness between individuals provides explanation for the evolution of many social behaviours, especially those involving cooperation, altruism and spite.
Many parental care traits provide relevant examples of how the relatedness between individuals determines the extent they can gain indirect fitness benefits via the reproduction of kin. A classic example of this is in the cooperative breeding behaviour in long-tailed tits (Russell and Hatchwell, 2001). Individuals who fail to breed tend to help closer relatives, whom they are able to recognize through learning through direct association (Sharp et al., 2005).
Relatedness hence provides a measure of the value of another individual without the need to ascribe a value to an individual based on individual recognition and memory. In this case, the value is assigned based on whether the frequency of the genes of a focal individual will be increased or decreased by the reproductive success of the other individual.
Furthermore, when we look at social behaviour more broadly, we realize that there are many other social behaviours around that are very simple, and may not require cognitive mechanisms at all. In fact relatedness structure, driven by genes, is the key factor essential for the maintenance of eusocial and cooperative breeding behavior.
The relative importance of cognitive mechanisms in the evolution and maintenance of social behaviour would also be reduced if we extend the discussion beyond animals. As defined by Bourke (2011), social behaviour is a set of interactions among individuals with potentially separate evolutionary trajectories, and such interactions are common in non-animals as well.
Cooperative behaviour exists between the soil bacteria Rhizobia and their host legumes, where the bacteria fix nitrogen in the nodules of the roots of the host plant, and the host plant provides nutrients for the growth of the bacteria (Kiers at al., 2003). This cooperative behaviour between them is maintained by 'punishment' by the plant. Another classic example of punishment in the interaction between the Yucca plant and moth (Pellmyr & Huth, 1993) also illustrates how plants appear to balance the pay-offs of interactions and facultatively adjust the level of contribution.
Spiteful behaviour is observed in bacteria that produce bacteriocins into the environment, killing other strains of bacteria, but not its family member. This interaction evolved through the indirect fitness obtained by the bacteriocin-producing cell (Gardner et al., 2004). The bacteriocin-producing cells pay substantial reproductive costs as it is killed by cell lysis in the process, and do not gain any direct benefit. However, it receives indirect fitness benefits from the survival of its relative.
Dictyostelium purpureum, is a social amoebae that aggregate when starved and subsequently differentiate to form fruiting bodies. The differentiation into spore-producing cells and cells that die and form the base and the stalk. Non-spore producing cells are performing an altruistic act as only cells that produce spores are able to pass on their genes. Moreover, D. purpureum cells aggregate preferentially with its own kin (Mehdiabadi et al., 2006). This example of altruistic social behaviour requires kin recognition, but not require complex cognition.
Examples of social behaviour in non-animals are aplenty, and they show how individual selection at the gene level in many cases is sufficient to explain many instances of social behaviour. It would indeed be hard to argue that these organisms and cells have cognitive abilities.
It is exciting to discover complex social behavior in non-human animals or even non-animals that would be thought as 'uniquely human' from a layman's point of view. However, one must be wary of extrapolating mechanisms driving human behaviour on other organisms. While we still do not know the specific mechanisms that are involved, we should not abandon cognitive mechanisms as a driving force for social behaviour, and especially more complex social behaviours. Cognitive mechanisms drive abilities such as individual recognition and memory, and there are social behaviours in animals that are driven by cognitive mechanisms of different levels of complexity.
However, for many social behaviours, it is difficult to disentangle the effects of cognitive mechanisms and simpler processes that do not require cognition. As elucidated in this essay, many studies have shown that social behaviour invokes cognitive mechanisms as an explanation can be explained using simpler rules and operational mechanisms. Individual selection at the gene level in many cases is sufficient to explain many instances of social behaviour. These are driven by natural selection, and maintained by the fitness advantage it gives to the actor. Furthermore, when we look beyond animals, we realize that what can be undoubtedly described as social behavior, are found in organisms that do not have cognitive abilities.
A caveat to this discussion on cognitive mechanisms and social behaviour is that while cognitive mechanism is important in the evolution of some social behaviours in animals, the evolution of cognitive mechanisms can also be driven by social behaviors. The advantage of intelligence imposes selection on the evolution of intelligence on the partner and the competition between individuals engaged in social interactions with each other generates a mental arms race. Indeed, given the fitness advantage of both cognitive abilities and social living, it is not surprising that the evolution of both will feedback on each other.