41 |
Seeing and Hearing a Word: Combining Eye and Ear Is More Efficient than Combining the Parts of a Word
|
|
|
|
Abstract:
To understand why human sensitivity for complex objects is so low, we study how word identification combines eye and ear or parts of a word (features, letters, syllables). Our observers identify printed and spoken words presented concurrently or separately. When researchers measure threshold (energy of the faintest visible or audible signal) they may report either sensitivity (one over the human threshold) or efficiency (ratio of the best possible threshold to the human threshold). When the best possible algorithm identifies an object (like a word) in noise, its threshold is independent of how many parts the object has. But, with human observers, efficiency depends on the task. In some tasks, human observers combine parts efficiently, needing hardly more energy to identify an object with more parts. In other tasks, they combine inefficiently, needing energy nearly proportional to the number of parts, over a 60∶1 range. Whether presented to eye or ear, efficiency for detecting a short sinusoid (tone or grating) with few features is a substantial 20%, while efficiency for identifying a word with many features is merely 1%. Why? We show that the low human sensitivity for words is a cost of combining their many parts. We report a dichotomy between inefficient combining of adjacent features and efficient combining across senses. Joining our results with a survey of the cue-combination literature reveals that cues combine efficiently only if they are perceived as aspects of the same object. Observers give different names to adjacent letters in a word, and combine them inefficiently. Observers give the same name to a word’s image and sound, and combine them efficiently. The brain’s machinery optimally combines only cues that are perceived as originating from the same object. Presumably such cues each find their own way through the brain to arrive at the same object representation.
|
|
Keyword:
Research Article
|
|
URL: https://doi.org/10.1371/journal.pone.0064803 http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3667182 http://www.ncbi.nlm.nih.gov/pubmed/23734220
|
|
BASE
|
|
Hide details
|
|
42 |
Physiological evidence for auditory modulation filterbanks: Cortical responses to concurrent modulations
|
|
|
|
BASE
|
|
Show details
|
|
46 |
Neuronal oscillations and speech perception: critical-band temporal envelopes are the essence
|
|
|
|
In: ISSN: 1662-5161 ; Frontiers in Human Neuroscience, Vol. 6 (2012) P. 340 (2012)
|
|
BASE
|
|
Show details
|
|
47 |
Cortical oscillations and speech processing: emerging computational principles and operations
|
|
|
|
In: ISSN: 1097-6256 ; Nature Neuroscience, Vol. 15, No 4 (2012) pp. 511-7 (2012)
|
|
BASE
|
|
Show details
|
|
49 |
Cortical Oscillations in Auditory Perception and Speech: Evidence for Two Temporal Windows in Human Auditory Cortex
|
|
|
|
BASE
|
|
Show details
|
|
50 |
Mental imagery of speech: linking motor and perceptual systems through internal simulation and estimation
|
|
|
|
BASE
|
|
Show details
|
|
51 |
The maps problem and the mapping problem: Two challenges for a cognitive neuroscience of speech and language
|
|
|
|
BASE
|
|
Show details
|
|
56 |
A mutual information analysis of neural coding of speech by low-frequency MEG phase information.
|
|
|
|
BASE
|
|
Show details
|
|
57 |
A mutual information analysis of neural coding of speech by low-frequency MEG phase information
|
|
|
|
BASE
|
|
Show details
|
|
58 |
Evidence for Early Morphological Decomposition: Combining Masked Priming with Magnetoencephalography
|
|
|
|
BASE
|
|
Show details
|
|
59 |
Genetics and language: a neurobiological perspective on the missing link (-ing hypotheses)
|
|
|
|
BASE
|
|
Show details
|
|
60 |
MEG, PSYCHOPHYSICAL AND COMPUTATIONAL STUDIES OF LOUDNESS, TIMBRE, AND AUDIOVISUAL INTEGRATION
|
|
|
|
BASE
|
|
Show details
|
|
|
|