音にも影響を与えるジェスチャー

Beat gestures influence which speech sounds you hear

Beat gestures – spontaneously produced biphasic movements of the hand – are among the most frequently encountered co-speech gestures in human communication. They are closely temporally aligned to the prosodic characteristics of the speech signal, typically occurring on lexically stressed syllables. Despite their prevalence across speakers of the world’s languages, how beat gestures impact spoken word recognition is unclear. Can these simple ‘flicks of the hand’ influence speech perception? Across six experiments, we demonstrate that beat gestures influence the explicit and implicit perception of lexical stress (e.g., distinguishing OBject from obJECT ), and in turn, can influence what vowels listeners hear. Thus, we provide converging evidence for a manual McGurk effect : even the simplest ‘flicks of the hands’ influence which speech sounds we hear. SIGNIFICANCE STATEMENT Beat gestures are very common in human face-to-face communication. Yet we know little about their behavioral consequences for spoken language comprehension. We demonstrate that beat gestures influence the explicit and implicit perception of lexical stress, and, in turn, can even shape what vowels we think we hear. This demonstration of a manual McGurk effect provides some of the first empirical support for a recent multimodal, situated psycholinguistic framework of human communication, while challenging current models of spoken word recognition that do not yet incorporate multimodal prosody. Moreover, it has the potential to enrich human-computer interaction and improve multimodal speech recognition systems. ### Competing Interest Statement The authors have declared no competing interest.

Beat gestures influence which speech sounds you hear

　日本人はあまりジェスチャーというのを多用する文化ではないですがビートジェスチャーと言われる会話のテンポや拍子を誇張するジェスチャーでは聞いている人がその影響を受けて同音異義語などを音を判断するのに有効に活用されていることがわかってきたようです。新規の「手は口ほどにものをいう」と言ってしまってよいのかもしれません。ただ人間の脳としてよりイメージが付きやすいのはやはりアイコンジェスチャーと言われる何かをまねするような仕草とのこと（例えばゴルフの素振りとか釣りとか）の方が脳は活性化しているようで理解させるという意味ではこっちを重視すべきなのかもしれません