Concert of sound and vision helps predict new sights
A new study has shown extra levels of complexity in the way sound and vision tell us about the world.
Researchers looking at the processes involved in sight have found that the visual cortex also uses information from the ears.
They suggest this auditory input enables the visual system to predict incoming visual information, and could confer a survival advantage.
Professor Lars Muckli, who led the research, says sound can lend extra value to movement.
“Sounds create visual imagery, mental images, and automatic projections,” he said.
“So, for example, if you are in a street and you hear the sound of an approaching motorbike, you expect to see a motorbike coming around the corner. If it turned out to be a horse, you'd be very surprised.”
The findings were made in a study from the Institute of Neuroscience and Psychology at the University of Glasgow.
The study, published in the journal Current Biology, involved conducting five different experiments using functional Magnetic Resonance Imaging (fMRI) to examine the activity in the early visual cortex in 10 volunteer subjects.
In one experiment they asked the blindfolded volunteers to listen to three different natural sounds – birdsong, traffic noise and a talking crowd.
Using an algorithm for identifying unique patterns in brain activity, the researchers were able to discriminate between the different sounds being processed in early visual cortex activity.
A second experiment revealed even imagined images, in the absence of both sight and sound, evoked activity in the early visual cortex.
“This research enhances our basic understanding of how interconnected different regions of the brain are,” Dr Muckli said.
“The early visual cortex hasn't previously been known to process auditory information, and while there is some anatomical evidence of interconnectedness in monkeys, our study is the first to clearly show a relationship in humans.
“In future we will test how this auditory information supports visual processing, but the assumption is it provides predictions to help the visual system to focus on surprising events which would confer a survival advantage.
“This might provide insights into mental health conditions such as schizophrenia or autism and help us understand how sensory perceptions differ in these individuals,” he said.
The researchers will now explore more sounds to find out how precise predictive coding in the brain can be.