Mind Hacks

Hack #59. Hear with Your Eyes: The McGurk Effect

Listen with your eyes closed and you’ll hear one sound; listen and watch the speaker at the same time and you’ll hear another.

If there were ever a way of showing that your senses combine to completely change your ultimate experience, it’s the McGurk Effect. This classic illusion, invented by Harry McGurk (and originally published in 1976 ¹ ), makes you hear different sounds being spoken depending on whether or not you can see the speaker’s lips. Knowing what’s going to happen doesn’t help: the effect just isn’t as strong.

In Action

Watch Arnt Maasø’s McGurk Effect video ( http://www.media.uio.no/personer/arntm/McGurk_english.html ; QuickTime with sound). You can see a freeze frame of the video in Figure 5-3.

Figure 5-3. Arnt Maasø’s McGurk Effect video

When you play it with your eyes closed, the voice says “ba ba.” Play the video again, and watch the mouth: the voice says “da da.” Try to hear “ba ba” while you watch the lips move. It can’t be done.

How It Works

The illusion itself can’t happen in real life. McGurk made it by splicing the sound of someone saying “ba ba” over a video of him making a different sound, “ga ga.” When you’re not watching the video, you hear what’s actually being spoken. But when you see the speaker too, the two bits of information clash. The position of the lips is key in telling what sound someone’s making, especially ...