Adaptation for multisensory relative timing
Introduction
Determining the relative timing between two signals arising from different sensory modalities — whether the signals are simultaneous or the order in which they occur — might be an important perceptual operation to determine whether the two signals should be causally connected and integrated into a single perceptual event [1].
The perception of relative timing for multisensory signals has been studied since the origins of psychology (see [2]), but only about a decade ago was it discovered that relative timing perception is not fixed, but depends on recently experienced asynchronies [3, 4]. In such studies, audiovisual stimuli are repeatedly presented with a fixed asynchrony (e.g. vision leads audition by 235 ms). In subsequent test trials, presentations of a stimulus with a smaller asynchrony (e.g. vision leads audition by 100 ms) are apparently perceived as closer in time and more likely to be reported as having occurred simultaneously than they were before the period of exposure to a fixed asynchrony. (Figure 1a). A corresponding change in subjective simultaneity occurs after repeated exposure to an auditory signal leading a visual signal. The effect of asynchrony exposure on perceived relative timing, lag exposure effects for short, also occurs for other tasks and combinations of signals (Box 1).
Little is known about the mechanisms underlying lag exposure effects (Box 2), but recent studies indicate that lag exposure effects might have properties similar to the classic perceptual after-effects described for visual attributes such as lightness, contrast, color, spatial frequency, orientation, speed or motion direction [5, 6]. Consequently, the effect of sensory history on temporal and non-temporal attributes may be described by similar principles.
Section snippets
Lag exposure effects are caused by adaptation
For non-temporal attributes, there is solid evidence that after-effects are indeed perceptual, caused by sensory adaptation, rather than changes in decision processes [7]. First, after-effects have neural correlates in sensory areas [6]. Second, after-effects are behaviorally associated with changes in the discriminability of the attribute [6, 8, 9]. For relative timing, to our knowledge only one study has examined the neural correlates of lag exposure effects [10], and until recently there was
Function
As described in the introduction, lag exposure effects reduce the perceived asynchrony for relative timings in which the order of presentation of the signals matches the order of the adaptor (Figure 1a). Phenomenologically, this reduction is similar to the reduction caused by adaptation to non-temporal attributes such as color or contrast [5, 6]. For example, adaption to a high contrast grating causes a subsequently presented lower contrast grating to be perceived as even lower contrast (Figure
Rapid adaptation
Perceptual after-effects for non-temporal attributes are traditionally measured by presenting adaptors for a long time — on the order of seconds — before each test presentation [23, 24]. However, some studies have also shown that adaptors presented for just tens or hundreds of milliseconds are sufficient to produce after-effects [26, 27, 28, 29], a result consistent with the rapid changes in neural response caused by adaptation [6, 28]. If the principles governing adaptation are similar for
Attractive biases
For non-temporal attributes, a different class of perceptual after-effects have also been reported in which perception of the attribute is attracted toward the previously experienced value of the attribute, that is, biased in the direction opposite to classic after-effects [34, 35, 36, 37•, 38, 39, 40]. Attractive biases have also been reported for relative timing [41, 42•]. In one study, for example, a series of pairs of tactile stimuli were delivered one to each hand with a distribution of
Mechanisms
Classic after-effects are often described by simple population codes in which adaptation reduces the gain of the neurons responding to the adaptor, and in which the decoder is unaware that adaptation has taken place [44]. Recalibration (for contrast, e.g.) and repulsion (for orientation, e.g.) effects could be obtained using band-pass and high-pass filters respectively [44]. A simple population code model with band-pass filters has been proposed to explain lag exposure effects in terms of
Conclusions
Audiovisual and sensorimotor lag exposure effects have been the subject of many recent investigations. Although it remains unclear whether different mechanisms underlie their operation, and whether they differ from those for other multisensory combinations, the results of recent studies suggest that the associated perceptual changes are similar to those caused by adaptation to non-temporal attributes.
For relative timing, it is often proposed that the function of adaptation is the alignment of
Conflict of interests statement
Nothing declared.
References and recommended reading
Papers of particular interest, published within the period of review, have been highlighted as:
• of special interest
•• of outstanding interest
Acknowledgements
We would like to thank Shin’ya Nishida for comments and discussions regarding the ideas contained in this review. DL was funded by the Catalan government (2014SGR-79) and Ministry of Economy and Competition of the Spanish government (PSI2013-41568-P). WR was supported by EU FET Proactive grant TIMESTORM — Mind and Time: Investigation of the Temporal Traits of Human-Machine Convergence. IC was supported by a grant of Ville de Paris, in the context of the HABOT project.
References (85)
- et al.
Recalibration of temporal order perception by exposure to audio-visual asynchrony
Cogn Brain Res
(2004) - et al.
Shifts of criteria or neural timing? The assumptions underlying timing perception studies
Conscious Cogn
(2011) - et al.
Sight and sound out of synch: fragmentation and renormalisation of audiovisual integration and subjective timing
Cortex
(2013) - et al.
Asynchrony adaptation reveals neural population code for audio-visual timing
Proc Roy Soc B-Biol Sci
(2011) - et al.
Sensory adaptation within a Bayesian framework for perception
Adv Neural Inform Process Syst
(2006) Attention-dependent brief adaptation to contour orientation: a high-level after effect for convexity?
Vision Res
(2001)- et al.
Perceptual and neural consequences of rapid motion adaptation
Proc Natl Acad Sci
(2011) - et al.
Rapid, generalized adaptation to asynchronous audiovisual speech
Proc Biol Sci Roy Soc
(2015) - et al.
Serial dependence in visual perception
Nat Neurosci
(2014) - et al.
Serial dependence in the perception of faces
Curr Biol
(2014)