Augmented action observation: Theory and practical applications in sensorimotor rehabilitation

Sensory feedback is a fundamental aspect of e ﬀ ective motor learning in sport and clinical contexts. One way to provide this is through sensory augmentation, where extrinsic sensory information are associated with, and modulated by, movement. Traditionally, sensory augmentation has been used as an online strategy, where feedback is provided during physical execution of an action. In this article, we argue that action observation can be an additional e ﬀ ective channel to provide augmented feedback, which would be complementary to other, more traditional, motor learning and sensory augmentation strategies. Given these similarities between observing and executing an action, action observation could be used when physical training is di ﬃ cult or not feasible, for example during immobilization or during the initial stages of a rehabilitation protocol when peripheral fatigue is a common issue. We review the bene ﬁ ts of observational learning and preliminary evidence for the e ﬀ ectiveness of using augmented action observation to improve learning. We also highlight current knowledge gaps which make the transition from laboratory to practical contexts di ﬃ cult. Finally, we highlight the key areas of focus for future research.


Introduction
Effective motor learning requires the integration of sensorimotor abilities with the goal of the action, and knowledge of the environment (including the ways in which one can interact with it).Feedback is fundamental to this sensorimotor learning to inform both the state of the environment and the performer's own body (Sigrist et al., 2013).This feedback can be intrinsic (i.e., inherently bound to the performance, such as the sound arising from the interaction between one's shoes and the ground) or extrinsic (i.e., feedback that is not associated to the interaction between the individual and the environment, such as the verbal advice typically provided by practitioners).Increasingly, sensory augmentation is being used as a form of extrinsic feedback to support the rehabilitation process.Here, extrinsic sensory feedback is associated with, and modulated by, movement characteristics, to aid sensorimotor learning, and alter subsequent actions.For example, providing the sound of walking on gravel at different speeds has been shown to improve walking speed in people with Parkinson's Disease (Young et al., 2013).However, feedback can also disrupt motor behaviour as seen by the slower running times and disruption to the overall kinematics of the action when the sound of the steps during hurdling was delayed (Kennel et al., 2015).These studies highlight the importance of feedback in motor control and learning (see Subramanian et al., 2010 for a systematic review on the provision of extrinsic feedback following stroke).
Sensory augmentation builds on well-established evidence that perception is a multisensory process that integrates different sensory sources within a context-dependent hierarchy based on the epistemic value of the sensory information (Parr & Friston, 2017).In this context, epistemic value refers to the ability of sensory information to reduce sensorimotor uncertainty (Parr & Friston, 2017); when sensory sources are conflicting, the brain attenuates the streams with the least epistemic value in favour of the one with the higher epistemic value (Limanowski, 2021).In practice, this means that sensory information continuously biases our perception, decision-making and actions (Cisek & Kalaska, 2010).Sensory augmentation can exploit this mechanism by associating movement-related information with sensory feedback to induce plasticity.
To date, sensory augmentation has mostly been provided during the physical execution of actions, either concurrently or at the end of the action.However, the well-established literature on mirror neurons (see Kilner & Lemon, 2013 for a review) provides a theoretical grounding for the provision of feedback during the mere observation of these actions.Evidence shows that observing an action activates a similar network of areas to the physical execution of that same action (Rizzolatti & Craighero, 2004).This functional equivalence (Holmes & Collins, 2001) has been used as a learning strategy in sport and clinical settings, with positive performance enhancement and plasticity (Guidali et al., 2020;Lepage et al., 2012).These findings highlight the potential benefits of using biofeedback during action observation to provide multisensory stimuli to the learner.This is important because multisensory inputs have been shown to enhance learning compared to unisensory input (Shams & Seitz, 2008).
In this article, we discuss emerging evidence in this field, and provide a theoretical and practical framework for augmented Action Observation (aAO).The proposed aAO represents a mapping between an observed action and extrinsic sensory information (i.e., sensory information that does not arise from the interaction between the body and the environment per se, but that becomes associated to some aspects of the action such as the pitch of a sound that increases/decreases with arm abduction/adduction).We argue that aAO can provide critical additional feedback to athletes and patients, and can extend the use of sensory augmentation to conditions where movement is not immediately possible (e.g., during immobilization).We begin by drawing similarities between the perception of actions and sensory augmentation, based on active inference.We then summarize research on the effectiveness of action observation as a learning strategy in rehabilitation, followed by a summary of research on augmented action observation.We conclude by discussing opportunities for further development of the techniques, as well as the challenges to overcome to apply augmented action observation to clinical practice.

From observation to internal representation of actions
When we observe an action, our sensorimotor system is activated in a similar way to the physical execution of the same action.This has been attributed to mirror neurons; a class of neurons that encode both the perception and execution of an action.Originally discovered in the macaque monkey (di Pellegrino et al., 1992), later studies demonstrate their presence in humans (Mukamel et al., 2010), spanning occipitotemporal, parietal and premotor cortices (Caspers et al., 2010;Iacoboni & Dapretto, 2006;Nelissen et al., 2011;Rizzolatti & Craighero, 2004;Saygin, 2007Saygin, , 2012)).Key mirror neuron areas are located in bilateral ventral and dorsal premotor cortex, bilateral inferior and superior parietal lobule, right superior occipital gyrus, bilateral pre-supplementary motor area, parietal-occipital areas, and the extrastriate body area (Hardwick et al., 2018).Additionally, the superior temporal sulcus is commonly reported to be key for action understanding (Caspers et al., 2010;Kilner et al., 2007aKilner et al., , 2007b;;Urgen & Saygin, 2020).There is also evidence suggestive of mirror neuron activity in M1 in both primates and humans (Casile, 2013).In humans, Transcranial Magnetic Stimulation (TMS) research demonstrates an increase in motor-evoked potentials (an index to measure the excitability of the neural population projecting to the effector) of the relevant effector in M1 during both action observation and execution (Naish et al., 2014).This sensorimotor activation is thought to occur in two phases.First, there is generalized activation within the early motor processing area of M1 (Lepage et al., 2010), followed by an effectorspecific facilitation of the motor cortex (which has similar temporal, spatial and contextual similarities with the execution of the same action; Cavallo et al., 2014).Due to their involvement in both the observation and execution of actions, these neurons are thought to have a prominent role in processing bodily representation, and the interaction between the body and the environment (Kilner et al., 2007a(Kilner et al., , 2007b)), as well as enabling action understanding (Rizzolatti & Craighero, 2004).
Computationally, mirror neurons enable the brain to infer the goal of the action by predicting the sensory consequences of the (observed) action.One of the main theories explaining the functioning of the mirror neurons (and more generally brain functioning) is predictive coding accounts (Friston et al., 2011;Kilner et al., 2007aKilner et al., , 2007b;;Urgen & Saygin, 2020), according to which sensorimotor predictions are based on hierarchical messaging between higherlevel areas (which process multisensory information and progressively abstract representationsacting on a slower time scale compared to other representations; Mumford, 1991Mumford, , 1992;;Shipp, 2016), and lower-level areas (which process information on a faster temporal scale, with a focus on progressively unimodal information; Kilner, 2011).Within this framework, higher level areas provide predictions (based on prior knowledge and experience of the action and, if relevant, the objects and/or people involved; e.g., Bach et al., 2014;Bach & Schenke, 2017;Joyce et al., 2016;Schenke et al., 2016Schenke et al., , 2020) ) to lower level areas about the causes of the sensory input.If these predictions do not fully explain these sensations, a feed-forward message containing this prediction error is projected from lower level areas to higher level areas.This error is processed by the higher level areas and is either explained away or adapted predictions are then projected back to lower level areas.Whilst there will always be some unexplained predictive error, perceptual systems work to reduce this error by combining all sensory input into a continuously updating feedforward loop (Walsh et al., 2020).Thus, each computational level of this hierarchy has its own, progressively abstracted, representation of the sensations, and recursive feed-forward and feed-back messaging ensures that the causes of sensations are fully explained (Shipp, 2016).In other words, the brain uses prior knowledge (higher level, abstracted representations) to create a "best guess" of what is processed by the lower level areas.When this best guess does not explain the incoming sensation well, it is refined by a recursive intra-and inter-cortical processing (Shipp, 2016).This matching process is thought to underly both action execution and action observation (Kilner et al., 2007a(Kilner et al., , 2007b)).Indeed, as Friston et al. (2011, p. 137)

argues, "[…]
mirror neurons represent motor intentions (goals) and generate predictions about the proprioceptive and exteroceptive (e.g., visual) consequences of action, irrespective of agency (self or other)."In this case, the forward model remains the same, but the difference between the execution or observation of an action is the context (agency).
Taken together, action understanding and imitation result from feed-back and feed-forward connections between areas that contain prior knowledge of the brain, body and environment, and incoming sensory information.These computational accounts are supported by empirical research evidencing activation of relevant multisensory areas (PMv, PPC, STS) and lower-level sensory areas (e.g., primary and secondary somatosensory cortexes).This evidence can inform theory and practice for the use of (augmented) action observation within motor rehabilitation by highlighting the mechanisms by which sensory information can be effectively integrated into multi-level internal representations of the body, task and environment to enhance sensorimotor leaning.

Learning by observing in rehabilitation; action observation treatment
The mapping between observed actions and the observer's own sensorimotor system is a fundamental aspect of observational learning, a major way, and arguably the earliest form, of human learning.The repeated and structured observation of actions not only provides information of what to do, but also how to perform an action (i.e., which neural strategies are required; Mattar & Gribble, 2005).Moreover, evidence suggests that observational learning can induce computational and neurophysiological changes to the internal representation of the action equivalent to those resulting from actual physical training (Mattar & Gribble, 2005;McGregor et al., 2016McGregor et al., , 2018)).The sustained activity within the Action Observation Network during observational learning strengthens its connectivity both endogenously (i.e., within the network) and exogenously (i.e., with other networks) in a Hebbian-like fashion (Guidali et al., 2020).This Hebbian-like plasticity (which is critical given the importance of plasticity for functional recovery after neurological events such as stroke; (Takeuchi & Izumi, 2015)) during observational learning is similar to that seen during physical training (Lepage et al., 2012).Observational learning, when integrated with traditional physical rehabilitation, is particularly effective during early learning stages (Gatti, 2013).Whilst physical execution remains the foundation of sensorimotor (re)learning, integrating cognitive strategies such as observational learning provides additional benefits and variety to the rehabilitative regime.Within this, Action Observation Treatment is one such protocol that has shown particular promise for functional recovery (Buccino, 2014).
During Action Observation Treatment patients observe an action (often related to activities of daily living)typically via a videoand then physically execute the same action (for a review of this methodology see Buccino, 2014).This treatment has been shown to improve performance of activities of daily living, which are key to helping patients re-gain independence.For example, in stroke rehabilitation, 4 weeks of Action Observation Treatment significantly improved performance in activities of daily living, compared to the same protocol with physical training alone (Ertelt et al., 2007).Importantly, the group who underwent Action Observation Treatment showed increased activity in ventral premotor cortex, supplementary motor area and superior temporal sulcus, which are key areas within the Action Observation Network.This is important because connectivity strength in this network is directly related to learning effectiveness (McGregor & Gribble, 2017).
In essence, Action Observation Treatment represents a strategy to provide visual guidance to the observer on what to do, and how to do it.The effectiveness of Action Observation Treatment has been demonstrated in a variety of conditions such as in children with cerebral palsy (Buccino et al., 2018), and for Muskoloskeletal and orthopaedic injuries (Bellelli et al., 2010).A recent metanalysis of 748 patients suggest that Action Observation Treatment improved various functional tests, suggesting increased bodily function and daily activity in upper and lower limbs (Buchignani et al., 2019).A more recent metanalysis reported strong evidence for the effectiveness of Action Observation Treatment in stroke and Parkinson's Disease, and moderate evidence for orthopaedic injuries and multiple sclerosis (Ryan, 2021).Thus, Action Observation Treatment represents an evidence-based strategy for motor (re)learning within clinical settings.The finding that physical practice typically promotes mainly explicit learning, whereas observational learning promotes implicit learning (Bird et al., 2005) strengthens the recommendation for their joint use to provide increased learning opportunities to patients, resulting in functional improvement compared to physical training alone.The addition of observational learning could be particularly useful in populations where implicit learning is typically more effective than explicit learning such as in those with high levels of anxiety.Interestingly, recent research has even shown the promise of Action Observation Treatment as a pre-emptive measure.For example, less mobility deterioration was found in the initial days after hip arthroplasty surgery in patients who underwent preoperative action observation and motor imagery compared to a control group that received only standard care (Temporiti et al., 2022).These results are in line with research on hypomobility showing that Action Observation Treatment, unlike other cognitive strategies such as motor imagery, is effective in maintaining brain activation during immobilization and movement restraints, and can prevent corticomotor depressiona key issue often resulting from immobilization (Bassolino et al., 2014).Indeed, Action Observation Treatment prevented brain activity suppression or functional detriment after 20 h or 16 h of immobilization respectively (Bassolino et al., 2014;De Marco et al., 2021).This corroborates the view that Action Observation is an effective strategy to prevent motor decay during immobilization (see also Rannaud Monany et al., 2022).This is important because a patient may be immobilized, or need to build up more muscle mass before the specific actions for physical training can be performed.
Whilst accumulating evidence supports the effectiveness, efficiency and costeffectiveness of Action Observation Treatment in a range of conditions and settings with relatively little training required, further research is needed to optimize this methodology.Whilst originally patients were instructed to observe an action for two minutes, and then physically execute this action for three minutes, a recent systematic review highlights that the effectiveness of Action Observation Treatment in Parkinson's Disease is affected by both the intervention dose and the characteristics of the visual stimulus (Giannakopoulos et al., 2022).Moreover, despite its effectiveness, the current recommendation is to use action observation in conjunction with physical training, not as a replacement because during action observation no movement-related afference is processed, which may result in a suboptimal learning rate compared to physical execution of the action.Thus, it is important to consider what sensory information is processed during mere action observation, and whether the "missing" movement-related afference could be simulated to create a more effective motor (re)learning strategy.

Sensory processing during action observation
Evidence suggests that key sensory areas are not just activated during action execution, but also during the mere observation of the same action (Caspers et al., 2010).For example, observation of touch has been linked to activation of both primary (SI) and secondary (SII) somatosensory cortices, though these areas are differentially activated when observing actions with or without extrinsic sensory information.For example, Avikainen et al. (2002) found increased activity in SI and decreased activity of SII when participants observed small object manipulations whilst receiving peripheral nerve stimulation.On the other hand, Keysers et al. (2004) reported activity in the left SII, but not SI, when participants observed being touched, and both right and left SII and SI activation when participants were touched on the left and right leg, respectively.SI is excited by the physical domain of the somatosensory stimulation such as peripheral nerve stimulation (Hashimoto et al., 1990), whilst SII is thought to be higher in the cortical hierarchy, processing the more abstract nature of the sensation (including sensory memory) and multisensory integration (Adams et al., 2013).Thus, the contradictory results may be driven by the lack of extrinsic sensory information in the latter study.This is an important avenue for further research as it could provide a solid basis upon which to judge the validity of augmented Action Observation.During action observation the dominant sensory feedback is visual, though information from other senses are also integrated.Computationally, the brain is thought to use forward generative models to predict the causes of incoming sensory information (Adams et al., 2013;Friston et al., 2011;Kilner et al., 2007aKilner et al., , 2007b)).Active inference highlights the important role of connectivity between somatosensory cortices.Here, SII is thought to process more abstract representation resulting from multisensory integration, providing predictions to SI as to the causes of the incoming sensory information.Whereas, SI is thought to process sensation in the physical domain (i.e., resulting from sensory stimulation) providing prediction errors to SII in a feed-forward fashion (Adams et al., 2013).These dynamics could also explain the seemingly contrasting results in brain activity by Avikainen et al. (2002) and Keysers et al. (2004).
Behavioural studies have investigated the forward model during action observation using sensory attenuation, an experimental phenomenon whereby participants report self-generated sensation (e.g., a touch with one hand on the other) as less intense than externally-induced ones (Blakemore et al., 2000;Shergill et al., 2003).Sensory attenuation results from the predictive nature of the brain, whereby incoming sensory information are "subtracted" from internal sensorimotor models of the action (Brown et al., 2013).This is, for example, the reason why one cannot tickle oneselfincoming somatosensory information from touch are predicted and removed from the sensorimotor model (Blakemore et al., 2000).There is conflicting evidence on sensory attenuation during action observation.For auditory and tactile tasks, there is no evidence of sensory attenuation (Kilteni et al., 2021;Weiss et al., 2011).However, a study by Thomas (2022) suggests that while the motor component of sensory attenuation is not modulated by action observation, there is an enhanced perceptual processing, which is in line with the neuroimaging studies discussed earlier reporting activity in somatosensory areas.The extent to which sensory attenuation occurs may be dependent on how sensory information are linked to motor activation, or the strength of these associations.Indeed, when participants practiced a task beforehand, audio-visual attenuation was found for action observation (Sato, 2008).This is an important area for further research because it highlights the potential for action observation to be an effective delivery channel for extrinsic sensory information, thus strengthening multisensory integration, especially in relation to the visual modality.Indeed, aAO could represent one way to induce a more sophisticated representation of the action (see Shams & Seitz, 2008 for a discussion about the benefits of multisensory practice).That is, providing movement-related feedback via sensory augmentation during action observation could result in improved performance and plasticity, strengthening connectivity between sensory and motor areas.Indeed, research has shown that motor skill rehabilitation is more difficult when sensory perception is affected in stroke (Smania et al., 2003), highlighting the key role of sensory feedback during rehabilitation.

Augmented action observation (aAO)
aAO is a strategy to provide extrinsic feedback by mapping sensory stimuli (e.g., the pitch of a sound or a vibration pattern) to movement parameters (i.e., kinematics of a particular body part), such that the observed movement modulates the sensory information conveyed to the observer.This could provide an additional strategy to more conventional online and offline paradigms.Indeed, aAO could be considered as an hybrid between the two: sensory information is provided online with respect of the observed movement, but offline with respect to the actual execution of the action (Figure 1).Research on aAO to date has focused on the use of proprioceptive, somatosensory and auditory sources to provide movement-related biofeedback.There has also been some research into the use of visual information to augment visual focus during action observation (D'Innocenzo et al., 2016).While these are interesting and suitable interventions to direct participants' attention to important aspects of performance, they are not aAO strategies per se, as there is no mapping between a movement and sensory modality.Furthermore, visual augmentation is prone to the guidance effect, a detriment in performance seen when the augmented feedback is removed (Ronsse et al., 2011), which further decreases the usefulness of vision as an augmented modality.
To the best of our knowledge, the auditory modality (particularly sonification) is the most used augmented modality during aAO research.Sonification refers to an auditory augmentation strategy whereby a sound (or sound characteristics such as brightness, pitch, etc.) is associated with, and modulated by, silent movement characteristics (aspects of a movement that would not produce feedback per se as they are not directly related to the interaction between the body and the environment; Dubus & Bresin, 2013).Sonified Action Observation has been found to increase the effectiveness of speed estimations of an observed swimming action (Schmitz et al., 2013).Here, participants who observed a stroke action (where the relative distance between wrists and ankles was mapped to two sounds) estimated the avatar speed much more closely to the actual actions compared to a control group (who observed the same action without any feedback mapping).This "sonification" group also showed increased activity between multisensory regions (such as the STS and frontal regions), and subcortical regions involved in sensorimotor control (such as the basal ganglia and thalamus).Schmitz et al. (2013) also reported increased connectivity between BA6 and BA44 (part of the Action Observation Network; Caspers et al., 2010;Hardwick et al., 2018) and a number of frontal and parietal areas included in the sensorimotor network.Mezzarobba et al. (2018) suggest that Sonified AO could be a suitable addition to traditional physical practice in clinical contexts.Here, patients with Parkinson's Disease completed 16 rehabilitative sessions (totalling 16 h over 8 weeks) where they observed common daily tasks with sonification, followed by the imitation of the same task.Compared to the control group, who performed a traditional cue-based practice session, those in the Sonified Action Observation group showed improvements in functional tests for quality of life, which were retained up to 3 months post intervention.Whilst this is a promising paradigm, the research is still in its infancy and it remains to be seen what the optimal session should involve, and how frequently these sessions are required to be effective.Furthermore there are questions about whether the effectiveness of such protocols is limited to more complex movements rather than to simpler movements whereby augmented feedback may not be providing useful information and so does not become optimally integrated (Bigliassi et al., 2018;Limanowski, 2021).For example, a single practice session where participants observed and imagined sonification of simple actions, such as index-thumb finger grasping, did not affect corticospinal excitability compared to the same practice protocol completed without sonification (Castro et al., 2021;Castro et al., 2021).Furthermore, practicing with sonification did not result in audiomotor resonance (an increase in cortical activation in motor areas when participants hear an action sound), which could suggest that the auditory information was not integrated in the internal representation of the action (Castro et al., 2021).A further key consideration for Sonified Action Observation research is the potential difference between sonification during observation and execution of actions.For example, Bresin et al. (2020) found evidence that sonification for executed movements may be more focused on the functional support of movement execution, whereas sonification during observation may be more focused on the communication of certain movement qualities.These differences warrant further investigation, particularly for research that combines observation and execution of actions.
Another modality that has been used to provide movement-related feedback during aAO is somatosensory stimulation.For example, Bisio et al. (2015) reported that observing a simple action (a hand opening/closing) with the addition of peripheral nerve stimulation significantly increased corticospinal excitability of the abductor pollicis brevis muscle (prime movement for the action) up to 45 min after the protocol.These changes were not seen if either action observation of the same action or peripheral nerve stimulation were delivered in isolation.In a follow-up study Bisio et al. (2017) showed that combining action observation with peripheral nerve stimulation induced plasticity, thus making the protocol suitable as a neurophysiological strategy for rehabilitation.Importantly, however, this facilitation is only found when the stimulation is mapped to an action such that it is able to convey meaningful information to the observer.McGregor et al. (2016) did not map their stimulation to the observed action (in this case, a reach towards a target using a manipulandum with clockwise force fields), opting for a constant peripheral nerve stimulation at 3 Hz during practice, and found a decrease in performance after the protocol compared to pre-learning tests.These studies demonstrate that it is not just a case of providing additional information to facilitation performance; for aAO to be effective the augmenting information needs to be meaningfully mapped onto the observed action to convey useful information directly relating to the action.Importantly, multisensory integration occurs most strongly when inputs are perceived to be connected spatially, concurrently (Shams & Seitz, 2008) and/or are of similar quality (Winkielman et al., 2015).When these perceived connections are present, multimodal integration represents a viable interpretation of events (Talsma, 2015).However, whilst having information from multiple modalities can enhance ones interpretation, it can also result in interpretative errors such as the McGurk effect (Mcgurk & Macdonald, 1976).This demonstrates the unification of multisensory information into a single model, rather than the presence of multiple separate unimodal models (Land, 2014;Simon et al., 2016).Importantly for motor (re)learning, multisensory experience is thought to result in multimodal memory such that any related modal path can reactivate the associated information in the other modalities (Talsma, 2015), and could also increase overall attentional capacity through resource pooling (Wickens, 2008).
Recently, tendon vibration has also been used to provide feedback during action observation.Tendon vibration has been shown to induce a vivid illusion of movement, usually contralaterally to the vibration site as the vibrated muscle is reported lengthening (Goodwin, 1972).A TMS study reported that motor cortex excitability was significantly higher at the end of a practice block where tendon vibration occurred alongside the observation of the hand opening/closing, and this upregulation was retained up to 60 min after the practice (Bisio et al., 2019).However, this was only seen when action observation and vibration were congruent; no changes in corticospinal excitability were seen if the pattern of stimulation was reversed.Again, this supports the need for meaningful mapping of the stimulation to the observed movement.Together studies on aAO show that auditory somatosensory and proprioceptive stimulation can be a suitable addition to observational learning to induce longlasting changes in the brain similar to those seen during physical practice (Bisio et al., 2017).However, more research is needed to determine the clinical relevance of such protocols, and whether the findings can be generalized to more complex movements.

Augmented action observation: Foundational challenges and opportunities
Optimizing protocol efficiency in motor (re)learning protocols is important to improve patient quality of life and to reduce the cost of patient care for health services.In this article we highlighted theoretical bases for the use of action observation to deliver extrinsic augmenting feedback.However, there are several challenges for this novel protocol before it can be applied more widely within clinical settings.
One crucial challenge for the development of aAO concerns the sensorimotor mapping between movement and sensory dimensions (Sigrist et al., 2013, Figure 1).Here, consideration is needed as to the specifics of movement and sensory dimensions (including the feedback that should be provided, and the delivery method).Associating feedback to action is rather vague, as action control is multidimensional and hierarchical (Adams et al., 2013;Friston, 2011;Grafton & Hamilton, 2007).Thus, research must be clear as to which movement dimensions need to be used.Well established theories on action control conceptualize action in four levels: (i) Kinematics, (ii) Motor commands, (iii) Goals and (iv) Intentions (Grafton & Hamilton, 2007;Hamilton & Grafton, 1993).
Here, the kinematic level refers to the visible motion of the body in space, including the trajectory and velocity of body parts, whereas the motor level refers to the signals that leave M1, directed to the muscle.On the other hand, goal and intention levels refer to the immediate and overall purpose of the action.Other accounts also introduce the level of "posture", to provide further finesse to the model (Proietti et al., 2021).A framework such as this has direct implications for aAO, as it highlights which of action-dependent dimension could be most intuitive and useful to map.Whilst it is relatively straightforward to use kinematic and motor command levels to provide feedback, it is harder to conceptualize the usefulness of sensory augmentation for the intention and goal levels.Using a brain-computer interface, it is possible to track participants' action intentions and goals, and to associate feedback to these intentions (Müller-Putz et al., 2016).However, it is hard to conceptualize the epistemic value of such feedback during a learning protocol.Thus, based on the current knowledge, we suggest clinical applications of augmented action observation research should focus on the kinematic and motor command levels until further research has been conducted on the applicability of hierarchically higher levels.
The kinematic level could represent an opportunity for practitioners to stress certain aspects of the action that are fundamental for effective performance.Biomechanical analyses highlight how postures and movement patterns predict performance success, both in sport and clinical sciences.These analyses can be used to define which movement aspect to augment to maximize treatment effectiveness.To do so, different established technologies can be used to track movements with a strong degree of accuracy at various levels of affordability, from research-grade (e.g., VICON systems) to the more commercially available systems (e.g., Microsoft Kinect).Whilst "traditional" motion tracking using markers has received ample research, the presence of reflective markers could represent a visual cue during action observation and, depending on the goal of protocol, could bias the learning process.This is important as visual cues modulate motor resonance (Puglisi et al., 2018).This problem is less pressing with whole body movements, where the camera zoom out and the marker is proportionally less visible.However, reflections from the markers could nevertheless disrupt processing.To avoid any potential bias, recent developments in markerless video analysis could represent a more viable way to track kinematics.In addition, the reduced setup time of this technology increases the viability for remote interaction with a single camera (Wade et al., 2022).Markerless technology has been used to provide biofeedback in clinical conditions, although to the best of our knowledge no studies have specifically applied it to augmented Action Observation (Scott et al., 2022).However, authors have questioned the comparative reliability of markerless technologies, suggesting further research is needed (Cronin, 2021;Wade et al., 2022).Whilst kinematics are heavily used in sensory augmentation research (Dubus & Bresin, 2013 for a review focused on sonification), knowledge of motor commands could, in principle, provide more useful information, as it would be harder for the observer to infer such information compared to kinematics.At the motor level, motor commands could be decoded using electromyography (EMG).
When designing sensorimotor mapping for aAO (but a similar analysis could be made for sensory information during executed movements), particular attention should be paid to the sensory modality and parameters used in the mapping (Figure 1).In principle, any sensory modality can be used as augmenting feedback, although olfactory and gustatory information would be difficult to conceptualize and use for this purpose.Therefore, the most useful sensory information is likely to be visual, auditory, somatosensory and proprioceptive modalities (Sigrist et al., 2013 for a review).These sensory modalities can be split into different parameters to be manipulated to provide feedback.For example, within the auditory modality, one could base the feedback upon the pitch, brightness or volume of a tone (or even use a combination of these to be modulated by different parameters).For example, Schulz (2016) divided 3D space into different tones, and then the brightness, pitch and volume of these tones were modulated by the participant's arm position within the sonified 3D space.Here, improved motor abilities (including movement smoothness and perception of effort) were seen after stroke in those that completed a training protocol where they had to recreate a reference melody by moving the arm within the 3D space.Taken together, the multidimensionality of sensory augmentation can unleash research creativity, and can afford the creation of innovative and engaging system that can improve patient engagement and adherence to the rehabilitation programme.However, further research is needed to establish best practices in sensorimotor mapping and feedback delivery.

Conclusion and recommendations
The goal of research in clinical sciences in ultimately to create new therapies that can be used within clinical settings.Historically, motor rehabilitation has been focused on physical therapy and maximizing the neuromechanical system.However, advances in neuroscience have provided cognitive-based protocols to improve sensorimotor performance.In this article, we advanced the proposal that Action Observation Treatment, which has already been shown to be effective in rehabilitation of different conditions, could be further improved and used as a vehicle to deliver extrinsic sensory information.The evidence discussed highlights the potential for aAO to provide critical "missing" information (in the form of movement-related feedback) within augmented Action Observation.However, whilst there is a large theoretical base for such an adapted protocol, there is only limited research into the application of this to date.Thus, the key recommendation from this review is the need for further research on the foundational elements of aAO, with a specific focus on the sensorimotor mapping, the delivery method and the periodization with other therapies.Research should then consider best practices to translate basic and clinical studies into usable recommendations to clinicians.

Disclosure statement
No potential conflict of interest was reported by the author(s).

Figure 1 .
Figure 1.When designing sensory augmentation strategies, three fundamental dimensions need to be taken into account; (a) hierarchical level within the action control; (b) sensory feedback; (c) Feedback delivery.In this article we suggest that augmented action observation can be considered as an additional way to provide feedback, as it takes elements to both online and offline delivery methods.