Kaiser Multisensory Object Perception in the Primate Brain

1. Auflage 2010
ISBN: 978-1-4419-5615-6
Verlag: Springer-Verlag
Format: PDF
Kopierschutz: Adobe DRM (»Systemvoraussetzungen)

E-Book, Englisch, 383 Seiten

It should come as no surprise to those interested in sensory processes that its research history is among the longest and richest of the many systematic efforts to understand how our bodies function. The continuing obsession with sensory systems is as much a re?ection of the fundamental need to understand how we experience the physical world as it is to understand how we become who we are based on those very experiences. The senses function as both portal and teacher, and their individual and collective properties have fascinated scientists and philosophers for millennia. In this context, the attention directed toward specifying their properties on a sense-by-sense basis that dominated sensory research in the 20th century seems a prelude to our current preoccupation with how they function in concert. Nevertheless, it was the concentrated effort on the operational principles of in- vidual senses that provided the depth of understanding necessary to inform current efforts to reveal how they act cooperatively. We know that the information provided by any individual sensory modality is not always veridical, but is subject to a myriad of modality-speci?c distortions. Thus, the brain’s ability to compare across the senses and to integrate the information they provide is not only a way to examine the accuracy of any individual sensory channel but also a way to enhance the collective information they make available to the brain.

Autoren/Hrsg.

Kaiser, Jochen

Inhaltsverzeichnis

1;Foreword;5
2;Contents;7
3;Contributors;9
4;1 General Introduction;12
5;Part I Mechanisms;16
5.1;2 Corticocortical Connectivity Subserving Different Forms of Multisensory Convergence;17
5.1.1;2.1 Introduction;17
5.1.2;2.2 Bimodal Properties: The Superior Temporal Sulcus;17
5.1.2.1;2.2.1 Bimodal Properties: The Anterior Ectosylvian Sulcus;18
5.1.2.2;2.2.2 Bimodal Properties: The Superior Colliculus;20
5.1.3;2.3 Non-bimodal Forms of Multisensory Processing;21
5.1.3.1;2.3.1 Subthreshold Multisensory Processing and Crossmodal Cortical Connectivity;23
5.1.3.2;2.3.2 Crossmodal Cortical Connections and Multisensory Integration;23
5.1.4;2.4 Multisensory Convergence: A Continuum?;25
5.1.4.1;2.4.1 Cortical Diversity: Differential Distributions of Multisensory Neuron Types?;25
5.1.5;2.5 Conclusions;28
5.1.6;References;28
5.2;3 Computational Modeling of Multisensory Object Perception;31
5.2.1;3.1 Introduction;31
5.2.2;3.2 Empirical Evidence for Crossmodal Object Perception;32
5.2.3;3.3 Computational Principles Evident from the Experimental Data;33
5.2.3.1;3.3.1 Sensory Integration;34
5.2.3.2;3.3.2 Sensory Combination;34
5.2.3.3;3.3.3 Integration of Measurements with Prior Knowledge;35
5.2.3.4;3.3.4 Explaining Away;35
5.2.3.5;3.3.5 Using the Appropriate Model;36
5.2.3.6;3.3.6 Decision Making;36
5.2.3.7;3.3.7 Learning of Cue Reliabilities and Priors;37
5.2.3.8;3.3.8 Task Dependence;37
5.2.3.9;3.3.9 Developmental Learning;37
5.2.4;3.4 Models of Multimodal Object Perception;38
5.2.4.1;3.4.1 Ideal Observer Models of Multimodal Object Perception;39
5.2.4.2;3.4.2 Intermediate Models of Multimodal Object Perception;48
5.2.4.3;3.4.3 Neural Models Implementing Multimodal Perception;49
5.2.5;3.5 Open Questions;52
5.2.5.1;3.5.1 How Are Model Parameters Learned?;52
5.2.5.2;3.5.2 How Are the Likelihood Functions Learned?;53
5.2.5.3;3.5.3 Where Does the Prior Come From?;53
5.2.5.4;3.5.4 How Does the Brain Learn the Appropriate Generative Model, If at All?;54
5.2.5.5;3.5.5 How Are Different Models Combined?;56
5.2.5.6;3.5.6 Does the Brain Represent Full Probability Distributions or Implicit Measures of Uncertainties?;56
5.2.5.7;3.5.7 How Ideal Is Human Learning?;57
5.2.5.8;3.5.8 How Do Laboratory Experiments Transfer to Natural Tasks?;57
5.2.5.9;3.5.9 The Role of Approximations;58
5.2.6;3.6 Conclusion;58
5.2.7;References;59
5.3;4 Methodological Considerations: Electrophysiology of Multisensory Interactions in Humans;64
5.3.1;4.1 Introduction;64
5.3.1.1;4.1.1 Specificities of the Electrophysiological Techniques;64
5.3.2;4.2 The Additive Model in Human Electrophysiology;65
5.3.2.1;4.2.1 Potential Biases and Artifacts Generated by the Additive Model ;66
5.3.2.1.1;4.2.1.1 Activities Common to All (Bimodal and Unimodal) Stimuli;66
5.3.2.1.2;4.2.1.2 Unimodal Deactivation of Sensory Cortices: Block vs Random Designs;67
5.3.2.1.3;4.2.1.3 More About Attentional Effects in the Additive Model;68
5.3.2.2;4.2.2 Suitability and Necessity of the Additive Model in Human Electrophysiology;69
5.3.2.3;4.2.3 Topographic Interpretation of : Nature of the Cross-Modal Interactions;71
5.3.3;4.3 The Additive Model in EEG/MEG, Hemodynamic, and Single-Neuron Data: One Model, Three Interpretations;72
5.3.3.1;4.3.1 Comparison with Hemodynamic Data;72
5.3.3.2;4.3.2 Comparison with Single-Unit Data;73
5.3.4;4.4 Statistical Assessment of Cross-Modal Interactions;74
5.3.4.1;4.4.1 At a Group Level;74
5.3.4.2;4.4.2 At an Individual Subject Level;74
5.3.4.3;4.4.3 Correction for Multiple Testing;76
5.3.5;4.5 Conclusion;77
5.3.6;References;77
5.4;5 Cortical Oscillations and Multisensory Interactions in Humans;80
5.4.1;5.1 Introduction;80
5.4.2;5.2 Topographically Unspecific or Widespread Activity;81
5.4.3;5.3 Localized Activity;82
5.4.4;5.4 Cortico-cortical Interactions;86
5.4.5;5.5 Conclusions;87
5.4.6;References;88
5.5;6 Multisensory Functional Magnetic Resonance Imaging;92
5.5.1;6.1 Introduction;92
5.5.2;6.2 Limitations and Strengths of fMRI;93
5.5.3;6.3 Detection of Multisensory Integration Regions;94
5.5.4;6.4 Connectivity;96
5.5.4.1;6.4.1 Anatomical Connectivity;97
5.5.4.2;6.4.2 Functional Connectivity;97
5.5.4.3;6.4.3 Effective Connectivity;97
5.5.4.4;6.4.4 Representational Connectivity;98
5.5.5;6.5 Conclusion and Outlook;98
5.5.6;References;99
6;Part II Audio-Visual Integration;102
6.1;7 Audiovisual Temporal Integration for Complex Speech, Object-Action, Animal Call, and Musical Stimuli;103
6.1.1;7.1 Introduction;103
6.1.2;7.2 Multisensory Integration and Temporal Synchrony;104
6.1.3;7.3 The Mechanisms Underlying Multisensory Temporal Perception;106
6.1.4;7.4 Measuring Temporal Perception;107
6.1.5;7.5 Audiovisual Temporal Perception for Simple Stimuli;108
6.1.6;7.6 Audiovisual Temporal Perception for Complex Stimuli;110
6.1.7;7.7 Factors Affecting the Audiovisual Temporal Perception of Complex Speech and Nonspeech Stimuli;115
6.1.7.1;7.7.1 How Does Stimulus Type Affect Audiovisual Temporal Perception?;116
6.1.7.2;7.7.2 How Do the Physical Characteristics in the Articulation of a Speech Stimulus Affect Audiovisual Temporal Perception?;118
6.1.7.3;7.7.3 Does the Visual Orientation of a Stimulus Affect Audiovisual Temporal Perception?;120
6.1.7.4;7.7.4 What Role Does the 'Unity Effect' Play in Audiovisual Temporal Perception?;120
6.1.8;7.8 Conclusions;123
6.1.9;References;124
6.2;8 Imaging Cross-Modal Influences in Auditory Cortex;130
6.2.1;8.1 Introduction;130
6.2.2;8.2 Cross-Modal Influences in Auditory Cortex Revealed by fMRI;131
6.2.3;8.3 A Functional Parcellation of Auditory Cortex;134
6.2.4;8.4 Localizing Cross-Modal Influences to Individual Fields;136
6.2.5;8.5 Extrapolation from Functional Imaging to Neuronal Responses;139
6.2.6;8.6 Conclusions;140
6.2.7;References;140
6.3;9 The Default Mode of Primate Vocal Communication and Its Neural Correlates;145
6.3.1;9.1 Introduction;145
6.3.2;9.2 Faces and Voices Are Inextricably Linked in Primates;145
6.3.3;9.3 Neocortical Bases for Integrating Faces and Voices;148
6.3.4;9.4 Auditory Cortical Interactions with the Superior Temporal Sulcus Mediates Face/Voice Integration;150
6.3.5;9.5 Beyond Visual Influences in Auditory Cortex;152
6.3.6;9.6 The Development of Multisensory Systems for Communication;153
6.3.7;9.7 Conclusion;155
6.3.8;References;155
6.4;10 Audio-Visual Perception of Everyday Natural Objects Hemodynamic Studies in Humans;160
6.4.1;10.1 Introduction;160
6.4.2;10.2 Overview of Audio-Visual Interaction Sites in Human Cortex;162
6.4.3;10.3 Background of System-Level Mechanisms for Multisensory Integration;167
6.4.4;10.4 Meta-analyses of Brain Networks Involved in Audio-Visual Interactions;170
6.4.4.1;10.4.1 Networks That Reflect Intermodal Invariant Audio-Visual Attributes;170
6.4.4.2;10.4.2 Networks for Audio-Visual Integration of Natural Objects and Faces in Motion;174
6.4.4.3;10.4.3 Networks for Audio-Visual Integration Using Static Pictures of Objects;177
6.4.4.4;10.4.4 Interactions When Using Artificial or Abstract Audio-Visual Pairings;178
6.4.4.5;10.4.5 Networks for Semantically Mis-matched Auditory and Visual Pairs;180
6.4.5;10.5 Category-Specific Audio-Visual Object Processing and Knowledge Representations;185
6.4.6;10.6 Conclusions;186
6.4.7;References;187
6.5;11 Single-Trial Multisensory Learning and Memory Retrieval;196
6.5.1;11.1 Background;196
6.5.2;11.2 Findings;198
6.5.2.1;11.2.1 Multisensory Experiences Impact Subsequent Visual Memory Performance;198
6.5.2.2;11.2.2 Effects on Memory Performance Are Dissociable from Encoding;201
6.5.2.3;11.2.3 The Role of Attention, Alerting, and Novelty;202
6.5.2.4;11.2.4 Visual Stimuli Are Rapidly Discriminated Within Lateral Occipital Cortices According to Past Multisensory Experiences;203
6.5.3;11.3 Implications;204
6.5.4;11.4 Conclusions and Future Directions;209
6.5.5;References;210
7;Part III Visuo-Tactile Integration;214
7.1;12 Multisensory Texture Perception;215
7.1.1;12.1 Introduction;215
7.1.2;12.2 Texture and Its Measurement;216
7.1.3;12.3 Haptic Roughness Perception;218
7.1.4;12.4 Visual and Visual/Haptic Texture Perception;222
7.1.5;12.5 Auditory Texture Perception;226
7.1.6;12.6 Brain Correlates of Texture Perception;229
7.1.7;12.7 Final Comments;230
7.2;References;231
7.3;13 Dorsal and Ventral Cortical Pathways for Visuo-haptic Shape Integration Revealed Using fMRI;235
7.3.1;13.1 Introduction;235
7.3.2;13.2 Visual Cortical Pathways for Action and Perception;236
7.3.3;13.3 Converging Visual and Somatosensory Pathways;237
7.3.4;13.4 Measuring Neuronal Convergence with BOLD fMRI;241
7.3.5;13.5 Sites of Visuo-haptic Neuronal Convergence;244
7.3.6;13.6 Conclusions;250
7.3.7;References;250
7.4;14 Visuo-haptic Perception of Objects and Scenes;255
7.4.1;14.1 Introduction;255
7.4.2;14.2 Evidence for Common Principles of Functional Organisation Across Vision and Touch;256
7.4.3;14.3 Evidence for Common Principles of Information Processing Across Vision and Touch for Object Recognition;260
7.4.3.1;14.3.1 Shape Constancy and the Visual Recognition of Objects Across Changes in Viewpoint;260
7.4.3.2;14.3.2 Shape Constancy and the Recognition of Static Objects in Vision and Touch;263
7.4.3.3;14.3.3 Shape Constancy and the Recognition of Dynamic Objects in Vision and Touch;265
7.4.4;14.4 Evidence for Common Information Processing of Where Information Across Vision and Touch;267
7.4.4.1;14.4.1 Visual and Haptic Spatial Updating of Scenes;267
7.4.4.2;14.4.2 Crossmodal Updating in Scene Perception;269
7.4.4.3;14.4.3 The Role of Noninformative Visual Information on Haptic Scene Perception;270
7.4.5;14.5 Conclusions and Future Directions;271
7.4.6;References;273
7.5;15 Haptic Face Processing and Its Relation to Vision;276
7.5.1;15.1 Overview;276
7.5.2;15.2 Facial Identity;277
7.5.2.1;15.2.1 Visual Perception of Facial Identity;277
7.5.2.1.1;15.2.1.1 How Does the Visual System Process Facial Identity?;277
7.5.2.1.2;15.2.1.2 How Does the Visual System R epresent Facial Identity?;279
7.5.2.1.3;15.2.1.3 What Are the Neural Mechan isms that Underlie Visual Perception of Facial Identity?;279
7.5.2.2;15.2.2 Haptic Perception of Facial Identity;281
7.5.2.2.1;15.2.2.1 How Does the Haptic System Process Facial Identity and How Does This Relate to Vision?;281
7.5.2.2.2;15.2.2.2 How Does the Haptic System Represent Facial Identity and How Does This Relate to Vision?;283
7.5.2.2.3;15.2.2.3 What Are the Neural Mechanisms that Underlie Haptic Perception of Facial Identity and How Does This Relate to Vision?;284
7.5.2.3;15.2.3 Summary;287
7.5.3;15.3 Facial Expressions of Emotion;288
7.5.3.1;15.3.1 Visual Perception of Emotion from Facial Expressions;288
7.5.3.1.1;15.3.1.1 How Does the Visual System Process Facial Expressions of Emotion?;288
7.5.3.1.2;15.3.1.2 How Does the Visual System Represen t Facial Expressions of Emotion?;289
7.5.3.1.3;15.3.1.3 What Are the Neural Mechanisms that Underlie Visual Perception of Facial Expressions of Emotion?;290
7.5.3.2;15.3.2 Haptic Processing of Emotion from Facial Expressions;291
7.5.3.2.1;15.3.2.1 How Does the Haptic System Process Facial Expressions of Emotion and How Does This Relate to Vision?;291
7.5.3.2.2;15.3.2.2 How Does the Haptic System Represent Facial Expressions of Emotion and How Does This Relate to Vision?;294
7.5.3.2.3;15.3.2.3 What Are the Neural Mechanisms that Underlie Haptic Perception of Facial Expressions of Emotion and How Does This Relate to Vision?;295
7.5.3.3;15.3.3 Summary;296
7.5.4;15.4 General Summary, Conclusions, and Future Directions;296
7.5.4.1;15.4.1 How Are Faces Processed?;296
7.5.4.2;15.4.2 How Are Faces Represented?;297
7.5.4.3;15.4.3 What Are the Underlying Neural Mechanisms and How Does this Relate to Vision?;297
7.5.5;References;298
8;Part IV Plasticity;304
8.1;16 The Ontogeny of Human Multisensory Object Perception: A Constructivist Account;305
8.1.1;16.1 Introduction;305
8.1.2;16.2 Setting the Theoretical Problem;306
8.1.3;16.3 Response to A-V Intensity Relations;308
8.1.4;16.4 Response to A-V Temporal Synchrony Relations;309
8.1.4.1;16.4.1 Infant Perception of A-V Synchrony Relations;310
8.1.4.2;16.4.2 Infant Perception of A-V Speech Synchrony and Effects of Experience;311
8.1.4.3;16.4.3 Binding of Multisensory Attributes;313
8.1.4.4;16.4.4 Binding of Nonnative Faces and Vocalizations;314
8.1.4.5;16.4.5 The Importance of Spatiotemporal Coherence in Object Perception;318
8.1.4.6;16.4.6 Summary of Effects of A-V Temporal Synchrony on Infant Perception;319
8.1.5;16.5 Response to A-V Colocation;320
8.1.6;16.6 Perception of Multisensory Sequences in Infancy;322
8.1.7;16.7 Conclusion;326
8.1.8;References;326
8.2;17 Neural Development and Plasticity of MultisensoryRepresentations;330
8.2.1;17.1 A Brief Introduction to Multisensory Processes;330
8.2.2;17.2 Neural Processing in Adult Multisensory Circuits;331
8.2.3;17.3 The Development of Subcortical Multisensory Representations;332
8.2.4;17.4 The Development of Cortical Multisensory Representations;335
8.2.5;17.5 The Maturation of the Multisensory Integrative Principles;336
8.2.6;17.6 Sensory Experience as a Key Determinant in Multisensory Development;337
8.2.7;17.7 The Mechanistic Bases for Multisensory Development?;340
8.2.8;17.8 Spatial Receptive Fields and Spatiotemporal Receptive Fields: New Tools to Evaluate Multisensory Representations and Their Development;341
8.2.9;17.9 Multisensory Studies in Awake and Behaving Preparations;344
8.2.10;17.10 Moving Toward the Development and Plasticity of Multisensory Object Representations;345
8.2.11;References;346
8.3;18 Large-Scale Brain Plasticity Following Blindness and the Use of Sensory Substitution Devices;351
8.3.1;18.1 Introduction;351
8.3.2;18.2 The Plastic Brain;353
8.3.2.1;18.2.1 Plasticity Across the Lifespan;354
8.3.3;18.3 Plastic Changes Following Sensory Loss;355
8.3.4;18.4 Developmental and Adult Plasticity Following Blindness;361
8.3.5;18.5 Rehabilitation in the Case of Blindness and Severe Visual Impairment;363
8.3.5.1;18.5.1 Sensory Substitution Devices;363
8.3.5.2;18.5.2 Sensory Restoration Approaches;369
8.3.6;18.6 Concluding Remarks;372
8.3.7;References;373
9;Index;381