Abstract
People often move in synchrony with auditory rhythms (e.g., music), whereas synchronization of movement with purely visual rhythms is rare. In two experiments, this apparent attraction of movement to auditory rhythms was investigated by requiring participants to tap their index finger in synchrony with an isochronous auditory (tone) or visual (flashing light) target sequence while a distractor sequence was presented in the other modality at one of various phase relationships. The obtained asynchronies and their variability showed that auditory distractors strongly attracted participants' taps, whereas visual distractors had much weaker effects, if any. This asymmetry held regardless of the spatial congruence or relative salience of the stimuli in the two modalities. When different irregular timing patterns were imposed on target and distractor sequences, participants' taps tended to track the timing pattern of auditory distractor sequences when they were approximately in phase with visual target sequences, but not the reverse. These results confirm that rhythmic movement is more strongly attracted to auditory than to visual rhythms. To the extent that this is an innate proclivity, it may have been an important factor in the evolution of music.
Similar content being viewed by others
Notes
The IOI was described nominally as 500 ms in Repp and Penel (2002), with a footnote pointing out that it was in fact 2.4% shorter.
After Experiment 1 had been completed, two studies in which relative phase was varied came to our attention. Peryer, Sloboda, and Nte (2002) used isochronous visual target sequences in combination with faster or slower auditory distractor sequences. The auditory sequences significantly increased the variability of synchronization performance. Aschersleben and Bertelson (Gisa Aschersleben, personal communication, June 2002) have conducted an experiment apparently similar to ours. However, we have not yet seen a written report.
That is, we were prepared to interpret effects of relative phase as effects of absolute temporal distance. For reasons of convenience, however, we used normalized relative phase (ranging from −.5 to .5) rather than temporal separation in milliseconds as the independent variable. Whether relative phase as such plays a role in governing the magnitude of distractor effects is a question we did not address directly in the present study because this would have required varying the IOI duration, resulting in an excessively large design.
We are assuming that peripherally visible movements of the finger contribute little to error correction in synchronization. If they do play a role, this would work in favor of the visual modality.
The data of two additional participants were excluded, in one case (not analyzed) because of recording artifacts and in the other case because of atypical and highly variable results that were difficult to make sense of. For example, that participant tapped after rather than before visual stimuli (i.e., with positive asynchronies), which is quite unusual.
It is known from earlier waveform measurements that the output is 2.4% faster than specified in MAX (cf. Footnote 1). Thus, the actual IOI was 625 ms. (Note that relative phases are independent of this scaling factor.) All millisecond values reported in the Method and Results sections are nominal, as specified or recorded by the MAX program. Actual values were 2.4% shorter. Apart from this scaling factor, input and output timing are believed to be accurate within ±1 ms.
According to the first author's informal impressions, the deviations from isochrony were detectable but not particularly salient in auditory sequences. They were undetectable in visual sequences (cf. Repp & Penel, 2002).
We are not aware of any observations that deaf individuals synchronize more readily with visual stimuli than hearing individuals, although this would be worth investigating.
References
Aschersleben, G. (2002). Temporal control of movements in sensorimotor synchronization. Brain and Cognition, 48, 66–79.
Bartlett, N. R., & Bartlett, S. C. (1959). Synchronization of a motor response with an anticipated sensory event. Psychological Review, 66, 203–218.
Bertelson, P., & Aschersleben, G. (1998). Automatic visual bias of perceived auditory location. Psychonomic Bulletin & Review, 5, 482–489.
Brass, M., Bekkering, H., & Prinz, W. (2001). Movement observation affects movement execution in a simple response task. Acta Psychologica, 106, 3–22.
Chen, Y., Repp, B. H., & Patel, A. D. (2002). Spectral decomposition of variability in synchronization and continuation tapping: Comparisons between auditory and visual pacing and feedback conditions. Human Movement Science, 21, 515–532.
Condon, W. S., & Sander, L. W. (1974). Neonate movement is synchronized with adult speech: Interactional participation and language acquisition. Science, 183, 99–101.
Drake, C. (1997). Motor and perceptually preferred synchronisation by children and adults: Binary and ternary ratios. Polish Quarterly of Developmental Psychology, 3, 43–61.
Drake, C., Jones, M. R., & Baruch, C. (2000). The development of rhythmic attending in auditory sequences: attunement, referent period, focal attending. Cognition, 77, 251–288.
Driver, J., & Spence, C. (2000). Multisensory perception: Beyond modularity and convergence. Current Biology, 10, R731–R735.
Fendrich, R., & Corballis, P. M. (2001). The temporal cross-capture of audition and vision. Perception & Psychophysics, 63, 719–725.
Fraisse, P. (1948). Rythmes auditifs et rythmes visuels. [Visual and auditory rhythms.] L'Année Psychologique, 49, 21–41.
Fraisse, P., Pichon, P., & Clairouin, G. (1949). Les aptitudes rythmiques: Etude comparée des oligophrènes et des enfants normaux. [Rhythmical aptitudes: A comparative study of mentally retarded and normal children.] Journal de Psychologie Normale et Pathologique, 42, 309–330.
Gault, R. H., & Goodfellow, L. D. (1938). An empirical comparison of audition, vision, and touch in the discrimination of temporal patterns and ability to reproduce them. Journal of General Psychology, 18, 41–47.
Glenberg, A., & Jona, M. (1991). Temporal coding in rhythm tasks revealed by modality effects. Memory & Cognition, 19, 514–522.
Glenberg, A., Mann, S., Altman, L., Forman, T., & Procise, S. (1989). Modality effects in the coding and reproduction of rhythms. Memory & Cognition, 17, 373–383.
Goldstone, S., Boardman, W. K., & Lhamon, W. T. (1959). Intersensory comparisons of temporal judgments. Journal of Experimental Psychology, 57, 243–248.
Goodfellow, L. D. (1934). An empirical comparison of audition, vision, and touch in the discrimination of short intervals of time. American Journal of Psychology, 46, 243–258.
Grondin, S. (1993). Duration discrimination of empty and filled intervals marked by auditory and visual signals. Perception & Psychophysics, 54, 383–394.
Grondin, S., & Rousseau, R. (1991). Judging the relative duration of multimodal short empty time intervals. Perception & Psychophysics, 49, 245–256.
Grondin, S., Meilleur-Wells, G., Ouellette, C., & Macar, F. (1998). Sensory effects on judgments of short time-intervals. Psychological Research, 61, 261–268.
Grondin, S., Ouellet, B., & Roussel, M.-E. (2001). About optimal timing and stability of Weber fraction for duration discrimination. Acoustical Science and Technology, 22, 370–372.
Haken, H., Kelso, J. A. S., & Bunz, H. (1985). A theoretical model of phase transitions in human hand movements. Biological Cybernetics, 51, 347–356.
Hargreaves, D. J. (1986). The developmental psychology of music. Cambridge, U.K.: Cambridge University Press.
Harrington, L. K., & Peck, C. K. (1998). Spatial disparity affects visual-auditory interactions in human sensorimotor processing. Experimental Brain Research, 122, 247–252.
Kelso, J. A. S. (1995). Dynamic patterns: The self-organization of brain and behavior. Cambridge, MA: MIT Press.
Kelso, J. A. S., DelColle, J. D., & Schöner, G. (1990). Action-perception as a pattern formation process. In M. Jeannerod (Ed.), Attention and performance XIII (pp. 138–169). Hillsdale, NJ: Erlbaum.
Kolers, P. A., & Brewster, J. M. (1985). Rhythms and responses. Journal of Experimental Psychology: Human Perception and Performance, 11, 150–167.
Lewald, J., & Guski, R. (2003). Cross-modal perceptual integration of spatially and temporally disparate auditory and visual stimuli. Cognitive Brain Research, 16, 468–478.
Lewald, J., Ehrenstein, W. H., & Guski, R. (2001). Spatio-temporal constraints for auditory-visual integration. Behavioral Brain Research, 121, 69–79.
Michon, J. A. (1967). Timing in temporal tracking. Assen, NL: Van Gorcum
Morein-Zamir, S., Soto-Faraco, S., & Kingstone, A. (2003). Auditory capture of vision: Examining temporal ventriloquism. Cognitive Brain Research, 17, 154–163.
Peryer, G., Sloboda, & Nte, S. (2002). How is the synchronisation of tapping to a visual isochronous pulse affected by an interfering auditory pulse? In C. Stevens, D. Burnham, G. McPherson, E. Schubert, & J. Renwick (Eds.), Proceedings of the 7th International Conference on Music Perception and Cognition, Sydney, 2002 (pp. 783–786). Adelaide, Australia: Causal Productions (CD-ROM).
Pressing, J. (1998). Error correction processes in temporal pattern production. Journal of Mathematical Psychology, 42, 63–101.
Rainbow, E. L., & Owen, D. (1979). A progress report on a three year investigation of the rhythmic ability of pre-school aged children. Bulletin of the Council for Research in Music Education, 59, 84–86.
Recanzone, G. H. (2003). Auditory influences on visual temporal rate perception. Journal of Neurophysiology, 89, 1078–1093.
Repp, B. H. (1997). Acoustics, perception, and production of legato articulation on a computer-controlled grand piano. Journal of the Acoustical Society of America, 102, 1878–1890
Repp, B. H. (2000). Compensation for subliminal timing perturbations in perceptual-motor synchronization. Psychological Research, 63, 106–128.
Repp, B. H. (2001). Phase correction, phase resetting, and phase shifts after subliminal timing perturbations in sensorimotor synchronization. Journal of Experimental Psychology: Human Perception and Performance, 27, 600–621.
Repp, B. H. (2002a). Automaticity and voluntary control of phase correction following event onset shifts in sensorimotor synchronization. Journal of Experimental Psychology: Human Perception and Performance, 28, 410–430.
Repp, B. H. (2002b). The embodiment of musical structure: Effects of musical context on sensorimotor synchronization with complex timing patterns. In W. Prinz & B. Hommel (Eds.), Common mechanisms in perception and action: Attention and Performance XIX (pp. 245–265). Oxford, U.K.: Oxford University Press.
Repp, B. H. (2002c). Perception of timing is more context sensitive than sensorimotor synchronization. Perception & Psychophysics, 64, 703–716.
Repp, B. H. (2003). Phase attraction in sensorimotor synchronization with auditory sequences: Effects of single and periodic distractors on synchronization accuracy. Journal of Experimental Psychology: Human Perception and Performance, 29, 290–309.
Repp, B. H. (in press). Rate limits in sensorimotor synchronization with auditory and visual sequences: The synchronization threshold and the benefits and costs of interval subdivision. Journal of Motor Behavior.
Repp, B. H., & Penel, A. (2002). Auditory dominance in temporal processing: New evidence from synchronization with simultaneous visual and auditory sequences. Journal of Experimental Psychology: Human Perception and Performance, 28, 1085–1099.
Rousseau, R., Poirier, J., & Lemyre, L. (1983). Duration discrimination of empty time intervals marked by intermodal pulses. Perception & Psychophysics, 34, 541–548.
Semjen, A., & Ivry, R. B. (2001). The coupled oscillator model of between-hand coordination in alternate-hand tapping: A reappraisal. Journal of Experimental Psychology: Human Perception and Performance, 27, 251–265.
Shams, L., Kamitani, Y., & Shimojo, S. (2000). What you see is what you hear. Nature, 408, 788.
Slutsky, D. A., & Recanzone, G. H. (2001). Temporal and spatial dependency of the ventriloquism effect. NeuroReport, 12, 7–12.
Spence, C., Shore, D. I., & Klein, R. M. (2001). Multisensory prior entry. Journal of Experimental Psychology: General, 130, 799–832.
Spence, C., Baddeley, R., Zampini, M., James, R., & Shore, D. I. (2003). Multisensory temporal order judgments: When two locations are better than one. Perception & Psychophysics, 65, 318–328.
Stein, B. E. (1998). Neural mechanisms for synthesizing sensory information and producing adaptive behaviors. Experimental Brain Research, 123, 124–135.
Trevarthen, C. (1999–2000). Musicality and the intrinsic motive pulse: evidence from human psychobiology and infant communication. Musicae Scientiae (special issue), 155–215.
Tuller, B., & Kelso, J. A. S. (1989). Environmentally-specified patterns of movement coordination in normal and split-brain subjects. Experimental Brain Research, 75, 306–316.
Walker, J. T., & Scott, K. J. (1981). Auditory-visual conflicts in the perceived duration of lights, tones, and gaps. Journal of Experimental Psychology: Human Perception and Performance, 7, 1327–1339.
Yamanishi, Y., Kawato, M., & Suzuki, R. (1980). Two coupled oscillators as a model for the coordinated finger tapping by both hands. Biological Cybernetics, 37, 219–225.
Acknowledgements
This research was supported by NIH grant MH-51230 awarded to the first author, and by a Cold Spring Harbor Laboratory post-doctoral fellowship awarded to the second author. Thanks are due to Helen Sayward and Susan Holleran for help with data analysis, and to Alex Aksentijevic, Peter Keller, and Ani Patel for helpful comments on an earlier version of the manuscript.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Repp, B.H., Penel, A. Rhythmic movement is attracted more strongly to auditory than to visual rhythms. Psychological Research 68, 252–270 (2004). https://doi.org/10.1007/s00426-003-0143-8
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00426-003-0143-8