Choosing the choice: Reflections on modelling decisions and behaviour in demographic agent-based models

This paper investigates the issues associated with choosing appropriate models of choice for demographic agent-based models. In particular, we discuss the importance of context, time preference, and dealing with uncertainty in decision modelling, as well as the heterogeneity between agents in their decision-making strategies. The paper concludes by advocating empirically driven, modular, and multi-model approaches to designing simulations of human decision-making, given the lack of an agreed strategy for dealing with any of these issues. Furthermore, we suggest that an iterative process of data collection and simulation experiments, with the latter informing future empirical data collection, should form the basis of such an endeavour. The discussion is illustrated with reference to selected demographic agent-based models, with a focus on migration.


Introduction
As demonstrated throughout this Supplement, agent-based models have become important tools with which demographers can study social reality. In these models, the analysis of population can be enhanced through the inclusion of processes that span different levels of aggregation, from individuals (micro) to societies (macro). Social networks, interactions between agents, and feedback effects can all easily be included in agent-based models. Still, the related methodology remains under development, with the actions of simulated agents often limited to passive responses to various stimuli. In particular, as evident in the nascent demographic agent-based literature, there is hardly any discussion of operationalizing actual decision-making processes.
One of the most pervasive features of making decisions in the social realm is their uncertainty: the actors involved often do not-and cannot-know the full consequences of their decisions, or even the full circumstances under which the decisions are made. However, in agent-based models, designed to reflect the complex features of social reality, uncertainty is typically being underplayed. As a result, the simulated agents lack psychological realism with respect to how they deal with ambiguity. Similarly, an analysis of additional layers of uncertainty, such as those related to model choice, model parameters, the stochastic nature of the computer code, and so on, are often missing in model-building endeavours. On the other hand, there is an acute need to bridge the gap between agent-based modelling and statistical inference, to provide an appropriate language for dealing with uncertainty (see, e.g., Heard et al. 2015).
In this paper, we discuss selected challenges of choosing an appropriate model of choice in the context of decision-making in agent-based demography. The argument is illustrated by examples related to demographic agent-based models, with a focus on migration. Specifically, in the next section, 'Context matters', we elaborate on the role of the context of the decision processes. In 'Auxiliary factors in modelling decisions', we discuss some specific aspects of the mechanics of choice: time, uncertainty, and the heterogeneity of agents. Finally, in the section 'A way forward', we conclude with a list of tentative recommendations for further work in the area, focusing on an empirically grounded modelling process that embraces multi-disciplinary approaches, multi-model frameworks, and modularity in simulation design.

Context matters
This section stresses the axiomatic status of the current models of choice in agent-based simulation, and discusses the numerous interfaces between choice and context for which modelling decisions must be made.
That the context in which a decision is made plays a role in the choice is inarguable: context can strongly influence the decision process itself (Ben-Akiva et al. 2012). The challenge arises in determining what the context is, and having done so, in deciding which parts of it to consider, and how to operationalize them. The former is the easier task, since the context is the universe up to the moment of decision. The latter tasks are more problematic and the 'state of the art' is, for now, just that-an art.
As an example of how choice and context interface, we can consider social context and how it is treated in modelling a specific demographic process. The social context of a decision is a broad sphere, touching as it does on social norms, the role of the decision-maker's social network, and so forth. Social context, and indeed any context, may interact with decision-making by being represented as a parameter to the decision model, or through impacting on the outcome of enacting a decision. Naturally, where the context is an input to the decision model, the modeller must consider whether the process it arises from is within the scope of the model, or should be externalized. Here, we follow the lead of Klabunde and Willekens (2016) and examine how social context is operationalized when modelling migration.
The prototypical approach to modelling choice is to use some form of random utility model. Baltas and Doyle (2001) have reviewed the key types in the context of marketing research, where all decision-makers consider the same factors in their decision-making and choose between the same options. Considering migration, social networks have a clear role (Epstein and Gang 2006), both through social influence on decision-making and in the process, by impacting how far a migration decision succeeds. Naturally, the two are interdependent, since success or failure carries implications for how that actor influences others.
If we limit ourselves further, and consider only how social influence may be modelled, then there are two key questions: (1) what is the unit of influence? and (2) how does influence interact with decision-making? It will be useful to address these one at a time, although the two are, of course, inter-twined, since influence can be defined as interpreted information that affects behaviour. This is also demonstrated by Mason et al. (2007) in their review of models of social influence, where many models treat these aspects in combination.
The flow of influence and the flow of information are interdependent processes, and the latter has attracted considerable attention in the last few years, with an increasing appreciation of the role of social networks in social influence. Here, rather than focusing on the structure over which influence flows, we will focus on the influence itself. Where information is transacted, it may be about attitudes, actions, or outcomes, and can correspond weakly or strongly to the truth. The mechanics of flow can also vary considerably, for example, transmission may occur only between direct links or be passed along many links, and may pass in only a single direction or in many directions.
The distinction between acts and attitudes is an interesting one, largely unexplored in this arena. Typically, we observe the actions of others, rather than getting accurate insight into their reasoning.
Where actions are what agents observe, their function is usually socially normative; for example, in the model of rural-urban migration by Silveira et al. (2006), an agent's utility is partially a function of how similar their behaviour is to that of their neighbours. The contrasting approach is to communicate a representation of the attitudes of the agent, as used by Espindola et al. (2006) in an earlier simulation of rural-urban migration, where agents compare 'satisfaction'.
Neither approach is a complete account. While mimicry seems to be fundamental to many aspects of human behaviour (Chartrand and Bargh 1999), we are not simply impersonators and instead interpret the actions of others. However, in the absence of telepathy, we can only draw inferences based on observation of others' actions, which may well include communicating with them about their internal state. To explain how people might translate observed behaviour into preferences, one potential approach might be similar to that of Jern et al. (2011), who propose an inverted generative model that presupposes a utility-type mechanism of decision-making.
Here, a generative model produces decisions, given preferences over outcomes, and the inversion provides estimates of preferences, given observed decisions. This also has implications for higher-level models of the interaction between context and the decision-making process, for example, that of Ben-Akiva et al. (2012), where there are explicit models S86 Jonathan Gray et al. of the interaction between context and choice. An inverted model that does not sufficiently capture the context in which an agent's decision-making took place-that is, the observing agent is underinformed about inputs to the other's decision process or constraints on their actions-may well lead to faulty inferences about what drove their behaviour. Although this is realistic, in that our own inferences about the motivations of others are not always accurate, it may not be desirable in all models.
The other usual interpretation of social information flow is as transmission of information about the state of the world, for example, information about expected income, as in models by Filho et al. (2011) and Klabunde (2014). Relating this back to our previous mention of faulty inference, this raises the issue of decision-making in an uncertain world, since there is no guarantee that information is accurate. In both models, agents take the average incomes of others in their social network as their expected income, were they to relocate. Although some distortion may be introduced by the limited sample, the reported income of other agents is perfectly accurate. A potential extension might be to consider inaccuracy arising from cognitive bias; for example, Mather et al. (2000) find that people distort their recollections in favour of the choices they have made, suggesting that agents may exaggerate their success. In contrast, McKenzie et al. (2013) find that potential migrants have unrealistically negative expectations about earnings and employment. They suggest this may partly arise from attempting to reduce demands for remittances, that is, agents under-report their incomes. In either case, this suggests that information transmission is not always simply a passive process. Influence may not be exclusively limited to information and could, for example, take the form of the provision of help, emotional assistance, or material support-transactions where information is present, but secondary. McKenzie et al. (2013) also suggest that overweighting of negative experiences plays a role in explaining migrants' overly negative expectations about future earnings. This demonstrates the importance of considering how information is incorporated into the decision process. In this case, the suggestion is that another cognitive bias, loss aversion, is a salient feature. This could be reflected by using a decision model that explicitly incorporates loss aversion, for example, the Prospect Theory (Kahneman and Tversky 1979) or Cumulative Prospect Theory (Tversky and Kahneman 1992). Both variants of Prospect Theory are intended to mimic human decisionmaking, by distorting the perception of high and low probabilities towards certainty, and shifting the value of gains and losses such that large gains are progressively less significant and losses more so. The wider implication is that social information flow may be distorted at the source, or through interpretation, and that agent-based models are uniquely suited to capture this. We focus here on descriptive theories of decision-making, precisely because the aim is to model human behaviours as they are manifested, rather than how they should be. Other approaches to decision models include heuristic methods (Gigerenzer and Goldstein 1996), which emphasize the bounded rationality of human decision-making, and Bayesian approaches, which address learning as a component of the decision process (Robbins 1964). In an agent-based modelling context, Gray et al. (2016) contrast Bayesian, heuristic, and Cumulative Prospect Theory approaches in a simulation of alcohol misuse disclosure among pregnant women.
The diversity of approaches to this single issue reflects the open-ended nature of agent-based modelling. A recurring feature, however, is that an appreciation of the multiple roles of contexts in decision-making is needed. Interpersonal context, for example, is a critical feature of information transmission, but also plays a role in the subjective value of outcomes. This is a challenge, since it is not clear that the two are readily separable and our understanding of interpersonal context is incomplete. However, this presents us with a great opportunity to use simulation to expand our understanding both of the role of social processes and their mechanics; and, in following the example of Tversky and Kahneman (1992), to forge interdisciplinary collaborations with experts in the microdomain of behaviour.

Auxiliary factors in modelling decisions
This section focuses on three auxiliary model elements that are often overlooked, understandably, in implementations of agent-based decision models, but that are also key assumptions. The first is the treatment of time in models of decision-making, the second is the all-encompassing uncertainty under which the decisions are made, and the third is the extent to which individual agents differ in their approaches to making decisions in the same context.

Time
Time is represented explicitly in almost all agent-based models, either in discrete or (pseudo) continuous form, in order to allow the dynamic unfolding of the modelled processes by 'stepping' from one time step to the next (or between time-ordered events in continuous-time discrete-event simulation). However, time does not often enter into the decision-making process of the simulated individuals. This results in the exclusion of a number of factors that may potentially be significant in eventual model outcomes. First, individuals rarely make decisions instantaneously, especially important ones with potentially far-ranging and long-lasting consequences. These include most decisions related to demographic behaviour, such as entering into or ending partnerships, having children, and migrating. As Klabunde et al. (2017) describe elsewhere in this Supplement, the Theory of Planned Behaviour (Ajzen 1991) can be used to describe the gradual process of intention forming, influence absorption, information gathering, and planning that terminate in a migration event. This drawn-out decision-making process allows for the possibility that agents may change their minds. Second, past experiences affect decisions in the present. This brings up questions about how the passage of time affects memory and attitudes. Finally, agents differ in their preferences between rewards realized immediately and equivalent rewards realized at some point in the future.
There are many good reasons why ignoring time in models of decision-making may be valid or even desirable (and, as modellers, we have often done so). As noted earlier, the context and research question must, of course, inform modelling decisions. It may be that the decisions in question have only short-term consequences and so a detailed consideration of how agents treat time in their decisionmaking may be completely superfluous.
That said, as demographers, we are generally concerned with decisions that affect the life course. These necessarily have consequences which extend far beyond the present (Willekens 2001). For example, the decision to have a child may affect not only a parent's immediate satisfaction with life, labour market participation, income streams, consumption, and so forth, but will continue to do so for the rest of their life. Future 'opportunity costs' must also be borne in mind (Butz and Ward 1979): taking such a decision now may widen or narrow future options in various other areas of life. Similar considerations are salient for migration and partnership decisions, and indeed for many other demographic and social phenomena for which the questions of time and sequencing of events tend too often to be ignored (see Abbott 2001 for an excellent overview).
This assumed sensitivity of demographic decisionmaking to the horizons over which the resultant costs and benefits are realized suggests that we should at least acknowledge what assumptions we make about how people value present vs. future rewards. The flexibility of agent-based models suggests that there is also scope to examine the effect of decision models that include time preference. Considering migration models, agents may learn about the potential rewards from migration from their social environment, and such information will influence a prediction about their future, conditional on their migration decision. The influence of this prediction on the final decision may then depend on the time horizon over which rewards are realized. A summary of current thinking on time dependence in decision-making follows; including such tenets in demographic modelling will certainly be challenging, and in many cases impossible. However, being aware of the potential problems with making simplifying assumptions about the treatment of time in decision models is valuable, and at least helps a modeller to anticipate when and why such assumptions may fall down.
Many economic models do, of course, include time preference in decision-making through the use of discounting; in migration modelling, this element is already present in the neoclassical framework, whereby the potential future gains from migration are discounted (see Massey et al. 1993 for an overview). This approach assumes that rewards received in the present are valued more highly by decisionmakers than those that will only be realized later. The most common functional form for such discounting is the exponential (O'Donoghue and Rabin 2000): Here, U t , denoting present utility at time t, is a function of the stream of individual rewards u t at each time point, weighted according to d t , with d being the discounting factor. This form has a useful property in that it is time-consistent; agents who prefer one option over another today will do so until the reward has been realized. As an example, consider a utility-maximizing individual who saves money to fund a migration attempt and must choose between two destinations: one that is closer and thus cheaper to travel to, and an alternative that is more distant and thus requires a longer saving period, but is also more attractive. If such a migrant discounts future rewards exponentially, then once they have S88 Jonathan Gray et al. made a choice, such a decision will remain optimal until the trip is made, assuming the material circumstances do not change.
However, experimental evidence for both humans and animals suggests that this is not how time discounting is actually practised (Boyer 2008). While we do prefer rewards now to those later, we do not do so in a consistent manner; preference orderings may change as we move forward in time. The functional form suggested by this sort of behaviour is hyperbolic (Benhabib et al. 2010): Such behaviour can be approximated by the use of a quasi-hyperbolic function, also known as presentweighted exponential discounting, which, as the latter name suggests, accounts for time-inconsistency by including a weight-either fixed or variable-on present rewards in the function (Benhabib et al. 2010). To return to the example of the migrant above, a hyperbolically or quasi-hyperbolically discounting agent may begin to save with the more distant, more attractive destination in mind, but actually move to the closer destination once they have saved enough for it to be an immediate option, even without receiving any new information about either option. Unfortunately, this is not the limit of potential complications around time. A large number of other contextual factors affect the extent of discounting. For instance, it has been found that smaller rewards are more heavily discounted than larger rewards; that people treat reductions in time to rewards differently to equivalent delays; and that future losses are treated as more significant than equivalent gains (Read and Loewenstein 2000).
Another factor to consider is human reflexivity; often, we know that we are weak-willed and we therefore take steps to restrain the compulsiveness of our future selves (O'Donoghue and Rabin 2000). Such 'pre-commitment' may take the form of imposing extra costs on ourselves, should the planned course of action be deviated from. For instance, making a promise or public commitment to a colleague or friend introduces the risk of social costs, should we renege.
To give a demographic example of how these models of time preference could be important, the work of Wrede (2011) is instructive. He investigates the effect of hyperbolic discounting on fertility in a simple three-period model, where mothers can have both immediate rewards from motherhood and delayed rewards in the form of being looked after in old age. He finds that the introduction of hyperbolic discounting reduces the number of births if this second, investment, motive is dominant, but that the ability to offset the immediate costs of motherhood through financial tools may mitigate these effects. The idea of pre-commitment is also mentioned (although not modelled); in this context, an individual might limit their future freedom of action through sterilization. Agent-based models are flexible enough to embed such time preference models in their wider social context (as discussed in the 'Context matters' section).
A crucial question is around how much of these empirical and theoretical considerations with respect to time should be included in simulation in a particular case. A model of migration may include exponential or hyperbolical discounting of future income streams associated with migration, as is common in many analytic models (Carrington et al. 1996). This may be crucial to understanding behaviour, as perhaps an adjustment period of obtaining country-specific human capital leads to initially poor earnings (Dustmann et al. 2011), and so the payoff to migration is disproportionately located at a longer time horizon. Alternatively, as a simplification, agents may be constructed to consider only a choice between current earnings and earnings that could be achieved on arrival. It may be that this type of simplification with respect to time-dependent decisions is sufficient to capture the essence of the process at hand. However, in general we would suggest that an awareness of the effect of the choices one is making about time and decision-making is invaluable, given that many demographic decisions have long-term consequences, and therefore testing of different models of time preference should be considered by modellers.
In order to allow for discounting of future reward streams, we must allow for agents to predict either their future earnings or their ability to save conditional on migration, or both. Of course, in practice, predictions of such quantities are rarely perfect and can easily turn out to be incorrect (Dustmann 1997). Sozou (1998) describes how the hyperbolic discounting scheme can be derived from an exponential scheme where the reward may be lost in the intervening period between initial consideration of the choice and realization of the outcome; as the agent becomes more certain about their eventual receipt of any given reward, the attractiveness of the option increases. The possibility for error in prediction is just one of the many sources of uncertainty Choosing the choice S89 surrounding the agent's decision-making process and the attempt to model it.

Uncertainty
Uncertainty is a pervasive feature of any social reality, and thus also any demographic reality. This uncertainty stems both from a lack of knowledge about processes and phenomena (epistemic uncertainty) and from the inherent randomness of the world (aleatory uncertainty; see, e.g., O'Hagan 2004). This means not only that social actors make decisions under conditions of imperfect knowledge, but also that the reality is random and unpredictable, and can yield many surprises. In agent-based modelling, these features need to be reflected in the simulated representations of the world, both with respect to agents themselves and to features of their environment.
As discussed in the 'Context matters' section, human decisions are made under the conditions of bounded rationality and loss aversion. In the context of uncertainty, risk aversion is equally important. The underpinning theory, dating back at least to Pratt (1964) and Arrow (1965), looks at departures from the expected utility decision framework, and stipulates that risk-averse decision-makers are willing to choose options with lower expected utility, as long as they involve less or no risk. Both aleatory and epistemic uncertainty are important here: through learning, agents can try to reduce epistemic uncertainty but, especially in the context of timedependent agent-based models, some of the uncertainty is irreducible, particularly with respect to decisions about the future. This constitutes one of the main arguments for the importance of time, as argued in the previous subsection. Besides, varying attitudes to risk are an important source of heterogeneity of the agents, as discussed in more detail in the next subsection.
In the context of agent-based modelling of migration-itself one of the most uncertain demographic processes-there are some aspects that make it crucial for uncertainty to be included in the models. For example, the uncertain nature of future benefits from migration has already been reflected in the neoclassical theories of migration, whereby earnings are weighted by the probability of employment, and future income streams are discounted to reflect the inter-temporal nature of decision-making (Massey et al. 1993). In the New Economic Theory of migration, uncertainty comes to the fore of the decision-making process at the family or household level, and migration becomes one of the important tools of managing labour market risks (Stark and Blum 1985). Uncertainty is at its most acute when we consider migration caused by extreme events, such as armed conflict or environmental catastrophes. Under such circumstances, decisions need to be made rapidly, sometimes with very limited insight into available options and the potential trade-offs between costs, benefits, and risks.
At the model level, there are different layers of uncertainty: in the parameters, model specification, computer code, and observations, not to mention the inherent residual variability (Kennedy and O'Hagan 2001). There are different ways to treat these manifestations of uncertainty in agent-based models. The use of statistical emulators or metamodels (Kleijnen and Sargent 2000)-statistical models approximating the key uncertain relationships between the parameters of the underlying complex computational models-has been suggested by Kennedy and O'Hagan (2001) and Oakley and O'Hagan (2004), and more recently, specifically in the context of agent-based modelling, by Heard et al. (2015) and Hilton and Bijak (2016). Alternatives include the use of approximate Bayesian computation, as advocated by Grazzini et al. (2017), or hybrid approaches, whereby emulators are enhanced by direct samples from the output of the underlying agent-based models (Kamiński 2015).
All these different approaches aim to describe uncertainty in complex models in a coherent way, and to allow meaningful (and ideally computationally efficient) inference from observed data. The methodological developments in this area are likely to continue: Heard et al. (2015) note a need to further enhance the statistical methods specifically designed for dealing with agent-based models. However, even now, the available tools enable a statistically rigorous (if at times approximate, as in the case of emulators) analysis of uncertainty in selected aspects of the models at hand. Coupled with the explicit acknowledgement of uncertainty in human decision-making, an honest account of the aleatory and epistemic limitations of agent-based models seems indispensable for their further methodological advancement and uptake in demography and beyond.

Heterogeneity in decision-making
In the context of the study of expert decision-making, psychologist James Shanteau has criticized the discipline of psychology for its focus on the study of the S90 Jonathan Gray et al.
'Generalized Normal Adult Human Mind' while ignoring differences between people and contexts (Shanteau and Edwards 2015). Agent-based models provide an opportunity to study two potential ways in which individuals differ in methods of decisionmaking. The first type of heterogeneity-somewhat easier to operationalize and analyse-can be described as parametric differences, in which individuals are thought of as having the same underlying model of decision, but are different only in the way that these are parameterized. Many agent-based models include such differentiation. As an example, in the Epstein (2002) model of civil disobedience, individuals differ in the extent of their aversion to risk, but for a given level of risk aversion, agents will behave in the same way (holding all else constant). Notably, this heterogeneity in risk aversion is crucial to the behaviour of the model. Without it, the 'revolts' against authority, a central feature of the model, would never be triggered.
Second, individuals may differ in the methods they use to come to decisions in the same situations. Sociological models regarding frames and scripts provide some justification for thinking that this may be the case (Kronenberg 2014). They suggest that, first, individuals attempt to select a frame for the situation they encounter, where frames are mental models that answer the question: 'What kind of situation is this?' Conditional on the selected frame, individuals then look to choose a relevant script in answer to the hypothetical question: 'What am I expected to do in this situation?' (Kronenberg 2014, p. 99). Thus, individuals may differ widely in their behaviour, should they frame the situation differently, or if their cultural experiences suggest a different script. This second type of heterogeneity is potentially more difficult to operationalize, yet is more fundamental to the way the decision processes are described.
To put such discussion in a migration context, Bijwaard (2010) shows the importance of considering both temporary and permanent migration within a single migration stream; we can interpret this as migrants adopting different decision-making strategies or, equivalently, adopting different frames and scripts. The use of agent-based models allows a range of potential scripts to be explored, with fewer constraints on what these scripts must look like.
As well as differences between agents, theories of frame and script selection also suggest differences within single agents over time; different frames and scripts may be selected in response to different contexts, developing experiences, or influences from peers. This leads to discussion of another form of difference in decision-making that may be important to demographers: that of differences over the life course. Established migrants who were once intending to save for a period and return home to their family may change their mind and settle permanently in response to a new interpretation of their identity and circumstances (Constant and Massey 2002).
As with representations of time, the relevance of heterogeneity in decision-making is likely to be very dependent on context. If heterogeneity exists, however, different mixes of agents could produce different macro-level results where interactions between agents feature heavily in the simulation, because of the possibility of non-linear interactions and emergence. Thus, researchers may wish to consider whether decision-making methods are likely to differ between individuals in their real-world system of interest.
A systematic review of various aspects of decisionmaking in agent-based models of land use has recently been published by Groeneveld et al. (2017). The review looks at a very general level at some of the aspects discussed in this paper, such as uncertainty and heterogeneity, and provides useful insights into the current modelling practice in a specific area of application. In particular, only 6 out of the 118 models studied in the review were found to include some uncertainty analysis in the decision model, and only 17 out of 128 incorporated uncertainty in the decision processes of agents (Groeneveld et al. 2017). There is certainly room for further improvement in that respect, and this conclusion holds for demographic models as much as it does for studies of land use.

Multi-model approaches, modularity, and iterative model building
In beginning to answer the question of which model of choice behaviour is appropriate for a given problem, we would do well to borrow an insight from computational neuroscience and recognize that a productive understanding of decision-making in a demographic context requires understanding at multiple levels of theoretical abstraction or generality (Marr and Poggio 1976;Marr 1982). High-level models like the Theory of Planned Behaviour (Ajzen 1991) can assist in understanding what decisions are being made, and why, and the sequencing of the underpinning processes. Models of process, and the integration of context like that of Ben-Akiva et al. (2012), can inform us about how Choosing the choice S91 decision-making takes place and the processes involved. Finally, these must be coupled with appropriate implementation.
All these problems are far from trivial, particularly as there is little consensus about which models of decision-making are best, or how they should draw information from the context of the decision. How then should we proceed with modelling, when the fundamental questions about how individuals make decisions, how they treat time, and how they differ between themselves in these factors are still very uncertain? One way forward is to use the same simulation set-up to analyse multiple models of behaviour Epstein 2013;Rossiter et al. 2014). This multi-model approach allows the modeller to attempt to distinguish between more and less plausible models of behaviour in their particular research context. Conditional on the simulation setup being appropriate, more plausible models may be better able to reproduce multiple empirical patterns at varying scales (Werker and Brenner 2004;Bianchi et al. 2008).
This commitment to analysing multiple models of behaviour within a single simulation project is facilitated by a modular approach to simulation design, so that different models of behaviour can be swapped in and out without changing the underlying conditions of the simulation (Epstein 2013), as can be seen in Gray et al. (2016). Such an approach reflects standard object-oriented design principles, but is not always reflected in an academic context (see also Rossiter 2015 for discussion of the application of software engineering principles to social simulation). Modularity also allows for the possibility of easy extension by other researchers with access to the code, allowing further behavioural models to be examined within the same simulation framework. On this theme, Bell et al. (2015) highlight the atomistic nature of the agent-based modelling discipline, and advocate the introduction of Agent-based Modelling Primitives (AMPs) to enable general components of agentbased models to be packaged and reused.
So, in the light of the above arguments, how do we choose the appropriate representation of human decisions and behaviour to be used in the agentbased modelling of demographic processes? First, we need to remember that agent-based models are a very convenient tool with which we can integrate behavioural theory with social theory and data. This implies that-as suggested in the 'Manifesto of computational social science' (Conte et al. 2012)such models would ideally need to collect additional information on human decision-making through bespoke cognitive experiments, instead of merely relying on hypotheses or assumptions (Courgeau et al. 2016). In this way, the selection of a particular choice model and its underlying choice theory becomes largely an empirical issue, being driven to a great extent by the empirical observations and experimental data on decision-making in a particular context, and specific to the decision problem of interest. Following this approach would allow for the inclusion of empirical micro-foundations in computational demographic models in an open and transparent way, and provide a natural, bottom-up, and empirical way of evaluating the suitability of particular choice theories for the problem at hand. Such a method would also take forward and expand on some of the suggestions mentioned by Groeneveld et al. (2017), including the call for a fuller use of knowledge on human decision-making and greater cross-disciplinary collaboration in modelling.
Second, the existing methods for the statistical design of experiments (e.g., Chaloner and Verdinelli 1995;Kleijnen and Sargent 2000) would allow for the adoption of a systematic approach to model design and data collection. Following the suggestions of Courgeau et al. (2016), the simulations would be built iteratively, by applying the experimental design principles to identify the elements of the model that require additional empirical information, and then by collecting this information. In this way, the important aspects of the decision process discussed above-context, uncertainty, time, and heterogeneity-can be built explicitly into the model through its specification or parameters. Then, they can be learnt about through repeated collection of information, including through experiments. The modularity of the model would help this process, as it would allow for fine-tuning of the different aspects of the model independently, before they are assembled together. An outline of this process is illustrated in Figure 1.
Following proposals from Cioffi-Revilla (2010) and from the ecological literature on agent-based models Schmolke et al. 2010), an iterative process of development and evaluation against empirical data is required in order to best facilitate an approach that revolves around experimentation with multiple models.  describe how patterns in observations encode information about the underlying structure of the real-world target process, and state that these patterns can be used to inform development of the model structure, compare the plausibility of different models, and reduce uncertainty about parameters. For , patterns can be quantitative or qualitative, but crucially should be examined on multiple S92 Jonathan Gray et al.
scales (for instance, at the individual and macro levels). This ensures structural realism, and provides more information with which to discriminate between competing models and choose parameters. Iteration allows feedback from the comparison between the simulated and empirical data to be incorporated into the model design . Cioffi-Revilla (2010) takes a similar approach in recommending a process of movement from simple to complex models, where at each stage an additional component of the phenomenon under study can be added, until a satisfactory model is found.
It is interesting to note commonalities between these approaches and the more general process described by Billari (2015). He describes a broad programme for demographic research that separates the 'discovery' phase from the phase focused on explanation. Agentbased modelling belongs to the second stage, while one could consider that the discoveries uncovered during the first phase constitute the patterns to be matched during the development process. Following Coleman (1986), Billari describes how demographic processes are driven by interactions between and within levels, which underlines the potential power of matching patterns at multiple levels of analysis.
The iterative approach to model building places considerable demands on the modeller in terms of developing the model, but also in exercising their judgement as to what should be included, and in identifying where a component should be iterated out. This is, therefore, an expensive exercise, both because augmenting a model is time-consuming and because evaluating complex models poses considerable challenges. The challenge of evaluation can be somewhat ameliorated using techniques like sensitivity analysis (see Thiele et al. 2014 for a review of several techniques, and Oakley and O'Hagan 2004 for an alternative approach), which can support the modeller in identifying how the individual parameters of a model contribute to overall outputs. In this context, sensitivity analysis refers to a broad range of techniques that enable understanding of how the various model inputs affect the output (result) variables and how important the model parameters are relative to one another (Oakley and O'Hagan 2004;Thiele et al. 2014). One important benefit of sensitivity analysis is that parameters that do not impact the results of a model significantly can be removed. This too has limitations, because as discussed in relation to heterogeneity in agents (see the 'Heterogeneity in decision-making' section), system properties of interest can take the form of both process and parameters, and sensitivity analysis is directly informative only about the latter. As a result, the onus remains on the modeller to use their best judgement.
The application of judgement in the process of model development should be documented: Schmolke et al. (2010) suggest that developmental cycles are recorded in a standardized format, the Transparent and Comprehensive Framework for Ecological Modelling (TRACE). This framework sets out a structure in which the model development, testing, and application phases of a project can be described to the reader, making it possible for the reader to reconstruct the process through which the final model was chosen. The authors suggest that TRACE documentation should be a living document resembling a lab notebook. Such documentation of the model development process provides transparency about the decisions made and the justifications for them, and also facilitates interaction with decision-makers.

A workflow for demographic agent-based modelling
How might the recommendations in the previous sections be incorporated in the process of conducting We recommend beginning the model-building process by identifying the key features and the context of the phenomenon being modelled. In the example of migration, the key aspects of the model would include not only the different types of heterogeneous agent-migrants, non-migrants, institutions, intermediaries, policymakers, and so on-but also the available contextual information: the known and relevant features of the geographic, social, and economic environments in which the agents operate. The selection of these features would be informed by the theoretical literature on migration (see Massey et al. 1993 for examples) and made realistic to the greatest degree possible by including empirical information-quantitative as well as qualitative-on what is known about migrant decision processes. The staged nature of decisions, as stipulated by the Theory of Planned Behaviour, could also be incorporated at this stage.
The second step of the model-building process would involve constructing a prototype, where the features that cannot be benchmarked to real-world data would be parameterized. Specifically, in the context of decision-making, such parameters could describe preferences related to risk or utility, aspects of decisions related to uncertainty or imperfect knowledge, or time discounting. The choice of a model of decision-making could be also parameterized: there may be many candidate frameworks that can apply to different situations or types of agent (such as various heuristics, the Cumulative Prospect Theory, or models of Bayesian reasoning; see Gray et al. 2016), each with their idiosyncratic features. Crucially, for the purpose of constructing the agent-based models, all unknown parameters would need to be described in probabilistic terms and embedded within a framework of statistical design of experiments (Chaloner and Verdinelli 1995). In this way, the prototype agent-based model could be executed for different combinations of parameters, yielding different outputs.
The third step of the process would consist of calibrating the outputs of the model against the selected aspects of social reality, ideally in a probabilistic manner (Hilton and Bijak 2016). Once the model is calibrated, its statistical properties can be analysed. In particular, the sensitivity of the outputs to the different parameters can be assessed (Oakley and O'Hagan 2004) and the parameters that do not influence model behaviour can be removed for the sake of parsimony. Conversely, we may want to find the empirical basis for those parameters that appear to be highly influential for the chosen outputs, for example, by collecting new data on the specific aspects of the agent-based model.
In particular, by calibrating the model, we are able to choose the most appropriate representation of the decision processes of different agents. If the results of calibration do not point to choosing a specific set of decision rules and models for various agents, this aspect of modelling can be enhanced by conducting cognitive experiments under controlled conditions. The sensitivity analysis would help to identify which aspects of the decision-making process are the most important from the point of view of generating the observed outcomes-these could be the treatment of time, uncertainty, or other parameters.
In the fourth step of the process, the decision problems faced by the agents can be reproduced under laboratory conditions in a series of tasks to be solved by human volunteers participating in such experiments. The experiments would be designed in such a way, so as to correspond structurally to the decision problem at hand. For example, in the context of the neoclassical theory of migration, the participants could decide whether or not to play a lottery game, whereby they would either lose money instantaneously (corresponding to an unsuccessful migration outcome), or win a promise of a monetary payment at some time in the future (corresponding to a successful migration outcome). The decisions of participants on whether or not to play the game could then be operationalized in the simulated decision-making of the modelled agents by using the particular decision theory that is found to correspond most closely to the patterns observed in the experiment.
The additional insights from the experimental data collection exercise would enable fine-tuning and reparameterizing of the agent-based model in light of the additional findings, as proposed in the 'Manifesto of computational social science' (Conte et al. 2012). The second, third, and fourth steps above can then be iterated until no more improvements can reasonably be made. What will be left is the residual uncertainty that is an irreducible feature of all models, and especially such complex ones.
Several examples of agent-based models presented in this Supplement already contain some of the building blocks of the proposed approach. The multistage nature of the decision-making processes in the context of migration are explored by Klabunde et al. (2017) and Kley (2017), both drawing from the Theory of Planned Behaviour (Ajzen 1991) and grounded in available empirical information. In addition, Warnke et al. (2017) provide a stochastic description of the underlying decision-making processes, and propose a corresponding programming language designed to facilitate the implementation S94 Jonathan Gray et al. and execution of agent-based models. The natural next step would be to use these building blocks, and others-such as the experimental design, cognitive psychological experiments on decisionmaking, or statistical analysis of uncertainty-for constructing more robust and empirically grounded models of human decisions, which could then be embedded within agent-based models of human populations.
As argued throughout this Supplement, all these features, and more, make agent-based modelling a very attractive analytical proposition for demographers. The potential gains in understanding of the underlying population processes cannot be overstated, and further links connecting agent-based modelling and statistical demography are yet to be explored (see, e.g., Bijak and Bryant 2016). The current lack of methodological consensus on modelling agency and human decisions is not an obstacle, but rather a challenge to guide further work in this area. Given that over the last 15 years agent-based models have slowly begun to enter the demographic mainstream (Billari and Prskawetz 2003;Van Bavel and Grow 2016), the gaps in knowledge in the existing approaches have become clearly visible. In our view one of the important gaps relates to the agency and decision-making processes of the simulated agents and related aspects, some of whichcontext, time, uncertainty, and a variety of forms of decision-making-are discussed in this paper. Empirically driven, modular, and multi-model approaches to designing simulations of human decision-making would certainly go some way towards filling these gaps. Now is the time to initiate the discussion about choosing the choice.

Notes and acknowledgements
1 Please direct all correspondence to Jason Hilton by Email: J.D.Hilton@soton.ac.uk