Audiovisual-Haptic Simultaneity Across the Body in Gameplay Viewing Experiences

This paper explores human performance on simultaneity judgments between audiovisual and haptic stimuli across the body for use in future real-time haptic applications. Three representative body sites, the torso, fingertip, and foot, were stimulated with vibration, and a media clip was used as an audiovisual stimulus. The results showed that: (1) the timing delay that humans can tolerate in the audiovisual leadinghaptic following case was 55 ms at the fingertip, 65 ms at the chest, and 45 ms at the foot, and these values were significantly different from each other, (2) the regression curves shifted toward the haptic-leading direction from the chest down to the foot, showing significantly different points of subjective synchrony (PSS) between the body sites but similar window widths of simultaneity, (3) the PSSs were obtained between 20 and 40 ms in the haptic leading-audiovisual following case, and (4) a significant asymmetry was observed in the curves between haptic-leading and audiovisual-leading stimuli, with a higher temporal sensitivity in the audiovisual-leading case. We expect our results can provide essential information for multisensory applications.

Jiwan Lee, Gyeore Yun, and Seungmoon Choi
Download Paper