Active inference tries to understand action and perception as working hand-in-hand to minimize a (free energy bound on) surprise. When surprise is quantified by prediction error we get predictive coding or processing. So why minimize surprise? The answer is simple: If we minimize surprise, we restrict ourselves to the embodied states that we expect to frequent. This single imperative has construct validity from several perspectives. For example, in statistics, surprise is the negative logarithm of Bayesian model evidence. This means that minimizing surprise is equivalent to Bayesian inference; hence, the Bayesian brain (Gregory, 1980; Dayan, Hinton, Neal, & Zemel, 1995). From the perspective of self-organization, autopoiesis, and allostasis, the long-term average of surprise is entropy. This means minimizing surprise allows biological agents to resist a natural tendency to disorder (Maturana & Varela, 1980; Sterling & Eyer, 1988). Finally, if we associate surprise with cost or negative value, then minimizing surprise maximizes expected value or utility (Friston et al., 2013).

To minimize prediction error, one clearly needs a (hierarchical) model that can generate predictions of sensory input. The hierarchical aspect is crucial here, in the sense that the nature of inferences or expectations is determined by the predictions they generate. For example, predictions may be limited to the visual domain, or could be proprioceptive predictions that engage classical reflex arcs. The key point here is that at some hierarchical level, expectations will furnish bilateral predictions of exteroceptive (sensory) and proprioceptive (motor) sensations. These amodal expectations can be construed as sensorimotor constructs—that rest upon sensorimotor contingencies for their acquisition or learning. In short, sensorimotor contingencies are a necessary part of hierarchical inference (Gibson, 1977; Noë, 2004). However, this does not mean that all expectations are sensorimotor; there are necessarily some perceptual expectations (e.g., qualia) that are purely sensory in nature (see Figure 1). Technically, these are sequestered from sensorimotor constructs by a Markov blanket (Pearl, 1988). This implies that not all perception is sensorimotor in nature (e.g., listening to music); for example, synesthetic concurrents that appear to be conditionally independent of beliefs about motor control:

As noted by Seth, prediction errors can be resolved by adjusting hierarchical expectations to improve the predictions (i.e., perception)—or sensations can be re-sampled to comply with predictions (i.e., action). This necessarily entails predictions of what would be sampled under allowable actions. In other words, our generative models have to entertain fictive or counterfactual outcomes to enable action to resolve prediction errors—or avoid surprises. This is a central tenet of active inference, as demonstrated in the counterfactual predictive processing that underlies oculomotor control and visual searches (Friston, Adams, Perrinet, & Breakspear, 2012). The same basic idea underlies recent formulations of decision making and choice, in which action is sampled from prior beliefs about (future) control states, given the current sensory state (Friston et al., 2013): This expression says that the least surprising policy or control sequence (over times ) minimizes the difference (Kullback-Leibler divergence) between the final outcomes , given the current state and a policy, and outcomes that are, a priori, anticipated. This formulation of prior beliefs about controlled outcomes is closely related to KL-control (Ortega & Braun, 2013; van den Broek, Wiegerinck, & Kappen, 2010) and can be related directly to classical utility theory (Friston et al., 2013).

From our point of view, it says two important things. First, it casts behavior explicitly in terms of prior beliefs about sensorimotor contingencies: “PPSMC requires that counterfactual predictions be explicitly incorporated as part of the priors in a hierarchical generative model.” (Seth, this issue)

Second, it shows that when expectations about sensory states do not inform beliefs about control, there is no mutual information between the sensory expectation and the final outcome: In other words, there are purely sensory expectations that do not inform future or counterfactual outcomes—and have no sensorimotor contingency. One can imagine quantifying the degree to which sensory beliefs inform controlled outcomes in terms of the mutual information between the current and future sensory state—a metric of counterfactual richness: “My specific claim is that the subjective veridicality (or perceptual presence) of normal perception depends precisely on the counterfactual richness of the corresponding generative models.” (Seth, this issue)