By Mark Schier, Adjunct Associate Professor, School of Education, University of New England

Creating a rich sensory environment will increase our students’ potential for learning. Knowledge retention will be enhanced with simultaneous sensory stimuli, provided these are balanced and appropriate to face to face, blended or fully online environments.

Here are some examples of what you can do, or what you may already have in your teaching:

• Add relevant sound to complement videos and presentations
• Encourage learners to visualise written words – use narrations to supplement slides
• Where appropriate, link to familiar everyday activities (kitchen, garden, urban, beach, sports ground)
• Include references to other senses: touch, smell, taste, balance for learners to explore

The science supporting this relates to the complex world in which we exist and our way of interpreting it. The five most known senses are sight, hearing, smell, taste, and touch; however, we also have sensory neurons and receptors that respond to nuances of touch, temperature, acceleration, velocity, balance, and proprioception (internal sense of position), trigeminal (a further chemical sense separate to taste and smell) and a sense of pain. There are at least 17 separate senses, and when more are experienced together, we gain a better picture of the world. We call this multisensory integration: where sensory inputs form together to create a single sensory understanding.

Because our brains deal with multiple inputs, and each stimulus input varies in reliability, each brain constructs its best coherent representation of the world. It does this by weighting the perceived importance of each input. The process is unconscious and continuous, and means that we update the probability for a scenario as more information comes in. We begin by inferring what we are perceiving and modify that as the situation changes.

To add to the complexity, as different senses are processed by different regions in the brain, the critical locations for improving our view of the world are those where the different sensory inputs are close to each other. These regions contain neurons which are active when simultaneous inputs in more than one sensory system occur. The main regions in the brain where this occurs are the temporo-parieto-occipital junction and the fronto-temporal junction. At the temporo-parieto-occipital junction, we have language, visuo-spatial, writing, reading, music, face recognition and symbol processing among others. At the fronto-temporal junction, we have executive understanding, error correction and planning.

Human neural pathways learn and optimise operation in multisensory modes, and some regions of the brain only develop when exposed to multisensory inputs. A study investigating visual and auditory stimuli has revealed that congruent (matched) visual and auditory stimuli showed faster and more sustained learning over single-sensory stimuli. Additionally. the benefit disappeared when the visual and auditory stimuli were not congruent (Kim, Seitz, & Shams, 2008). There are other studies utilising different sensory inputs also showing similar results (Shimojo & Shams, 2001).

Be careful— there are also times when multisensory integration can confuse us. One example of combined auditory and visual input misleading our view of the world is known as the McGurk Effect (Tiippana, 2014). It occurs when an auditory input is paired with two different visual inputs, but the sound changes to match the visual input. If you are watching a video and soundtrack with a person saying: ‘ba ba ba’ this is clearly heard. However, if we change the video to a person saying: ‘fa fa fa’ (with identical soundtrack as the first), then instead of ‘ba ba ba’, we hear ‘fa fa fa’ (Sixesfullofnines, 2021).

How can we capitalise on this knowledge? Our best results are achieved by providing congruent sensory stimuli for students within the learning environment. This needs to be tailored for face to face, blended and fully online environments, but the richer the sensory environment, the greater the potential for learning. However, don’t add stimuli that are unrelated or distracting to the situation.

Finally, some take away principles for enhancing teaching:

• Multisensory stimulation is good for the experience as it enhances learning and retention
• The greater the alignment between senses, the more realistic the experience
• All learners are different and providing multisensory input provides more potential connections
• Don’t be afraid to try multisensory learning items — they can always be modified for next time.

 References

Kim, R. S., Seitz, A. R., & Shams, L. (2008). Benefits of stimulus congruency for multisensory facilitation of visual learning. PloS one, 3(1), e1532. https://doi.org/10.1371/journal.pone.0001532

Shimojo, S., & Shams, L. (2001). Sensory modalities are not separate modalities: plasticity and interactions. Current opinion in neurobiology, 11(4), 505-509. https://doi.org/10.1016/S0959-4388(00)00241-5

Sixesfullofnines (2021, September 13). McGurk effect – Auditory Illusion – BBC Horizon Clip. [Video]. YouTube. https://youtu.be/2k8fHR9jKVM

Tiippana, K. (2014). What is the McGurk effect? Frontiers in Psychology, 5(725). https://doi:10.3389/fpsyg.2014.00725