Buch, Englisch, 512 Seiten
An Introduction to Speech, Audio and Psychoacoustics
Buch, Englisch, 512 Seiten
ISBN: 978-1-394-36727-6
Verlag: Wiley John + Sons
Master communication acoustics with AI, spatial audio, and VR
The newly revised second edition of Communication Acoustics provides graduate students and professionals with comprehensive foundations in auditory perception, signal processing, and modern computational methods. The authors integrate traditional psychoacoustic principles with cutting-edge developments in machine learning, sound and music computing, spatial audio, and immersive virtual reality audio systems.
This updated edition includes a broad and comprehensive introduction to machine learning techniques used in different branches of audio, speech, and technical audiology. Coverage includes recent advances in spatial audio rendering, VR audio systems, computational auditory modeling, sound quality measurement, electroacoustic transducers, and hearing aid processing. Python programming examples with comprehensive teaching materials support hands-on learning and practical application.
Readers will find: - Machine learning using neural architectures and methods directly applicable in audio and speech technologies
- Spatial audio for virtual reality including 6DOF rendering, head tracking, and advanced HRIR individualization techniques
- Comprehensive view to sound and music computing, from audio coding to sound synthesis, audio effects, and reverberation
- Updated content on loudspeakers and microphones, source directivity, musical instrument physics, and physical modeling of musical instruments
- Psychoacoustics including new imagery of the cochlea, perceptual cues, listening tests and psychoacoustic quantities
- Python programming examples with supplementary teaching slides and project instructions to support hands-on learning and course implementation
Communication Acoustics is an ideal resource for graduate students in electrical engineering, PhD researchers in acoustics, and industrial R&D professionals. By integrating psychoacoustic principles with advances in AI, spatial audio, and VR technologies, the book prepares readers for next-generation solutions.
Autoren/Hrsg.
Fachgebiete
- Technische Wissenschaften Sonstige Technologien | Angewandte Technik Signalverarbeitung, Bildverarbeitung, Scanning
- Sozialwissenschaften Medien- und Kommunikationswissenschaften Kommunikationswissenschaften
- Mathematik | Informatik EDV | Informatik Informatik Tonsignalverarbeitung
- Technische Wissenschaften Elektronik | Nachrichtentechnik Nachrichten- und Kommunikationstechnik
- Technische Wissenschaften Sonstige Technologien | Angewandte Technik Akustik, Tontechnik
Weitere Infos & Material
Preface to Second Edition xix
Introduction 1
1 How to Study and Develop Communication Acoustics 7
1.1 Domains of Knowledge 7
1.2 Methodology of Research and Development 8
1.3 Systems Approach to Modelling 10
1.4 Focus of the Book 12
1.5 Intended Audience 13
References 13
2 Physics of Sound 15
2.1 Vibration and Wave Behaviour of Sound 15
2.1.1 From Vibration to Waves 16
2.1.2 A Simple Vibrating System 16
2.1.3 Resonance 18
2.1.4 Complex Mass–Spring Systems 19
2.1.5 Modal Behaviour 19
2.1.6 Waves 21
2.2 Acoustic Measures and Quantities 22
2.2.1 Sound and Voice as Signals 23
2.2.2 Sound Pressure 23
2.2.3 Sound Pressure Level 24
2.2.4 Sound Power 24
2.2.5 Energetic Analysis of Sound Fields 25
2.2.6 Computation with Amplitude and Level Quantities 26
2.3 Wave Phenomena 26
2.3.1 Spherical Waves 27
2.3.2 Plane Waves and the Wave Field in a Tube 27
2.3.3 Wave Propagation in Solid Materials 29
2.3.4 Reflection, Absorption and Refraction 31
2.3.5 Scattering and Diffraction 32
2.3.6 Doppler Effect 32
2.4 Sound in Closed Spaces: Acoustics of Rooms and Halls 34
2.4.1 Sound Field in a Room 34
2.4.2 Reverberation 36
2.4.3 Sound Pressure Level in a Room 37
2.4.4 Modal Behaviour of Sound in a Room 37
2.4.5 Computational Modelling of Closed Space Acoustics 39
References 40
3 Signal Processing and Signals 43
3.1 Signals 43
3.1.1 Sounds as Signals 43
3.1.2 Typical Signals 45
3.2 Fundamental Concepts of Signal Processing 46
3.2.1 Linear and Time-invariant Systems 46
3.2.2 Convolution 47
3.2.3 Signal Transforms 48
3.2.4 Fourier Analysis and Synthesis 48
3.2.5 Spectrum Analysis 50
3.2.6 Time–Frequency Representations 51
3.2.7 Auto- and Cross-correlation 54
3.2.8 Cepstrum 55
3.3 Digital Signal Processing (DSP) 55
3.3.1 Sampling and Signal Conversion 55
3.3.2 Z Transform 56
3.3.3 Filters as LTI Systems 57
3.3.4 Digital Filtering 57
3.3.5 Linear Prediction 59
3.3.6 Adaptive Filtering 60
3.4 Hidden Markov Models 61
3.5 Concepts of Intelligent and Learning Systems 62
References 63
4 Electroacoustics and Responses of Audio Systems 65
4.1 Audio System Responses 65
4.1.1 Measurement of System Response 65
4.1.2 Ideal Reproduction of Sound 66
4.1.3 Impulse Response and Magnitude Response 67
4.1.4 Phase Response 68
4.1.5 Non-linear Distortion 69
4.1.6 Signal-to-Noise Ratio 71
4.1.7 Directional Responses of Electroacoustic Devices 71
4.1.8 Directivity Index 72
4.1.9 Power response 72
4.2 Electroacoustics 73
4.2.1 Electrodynamic Loudspeakers 73
4.2.2 MEMS Loudspeakers 76
4.2.3 Other Loudspeaker Types 76
4.2.4 Headphones 77
4.2.5 Pressure and Pressure-gradient Microphones 78
4.2.6 Microphone Directional Patterns 79
4.2.7 Condenser Microphone 81
4.2.8 Dynamic Microphone 81
4.2.9 MEMS Microphones 82
4.2.10 Other Microphone Types 83
4.2.11 Microphone Self-noise and Dynamic Range 83
4.3 Response Equalization 84
References 85
5 Human Voice 87
5.1 Speech Production 87
5.1.1 Speech Production Mechanism 88
5.1.2 Vocal Folds and Phonation 89
5.1.3 Vocal and Nasal Tract and Articulation 90
5.1.4 Voice Directivity 91
5.2 Units and Notation of Speech used in Phonetics 92
5.2.1 Vowels 94
5.2.2 Consonants 95
5.2.3 Prosody and Suprasegmental Features 97
5.3 Modelling of Speech Production 98
5.3.1 Glottal Modelling 100
5.3.2 Vocal Tract Modelling 101
5.3.3 Articulatory Synthesis 103
5.3.4 Formant Synthesis 103
5.4 Singing Voice 104
References 105
6 Acoustic Musical Instruments 107
6.1 Types of Acoustic Musical Instruments 107
6.2 Resonators in Instruments 108
6.3 Sources of Excitation 110
6.4 Controlling the Frequency of Vibration 111
6.5 Combining the Excitation and Resonant Structures 113
6.6 Envelope of Sounds 113
References 114
7 Physiology and Anatomy of Hearing 115
7.1 Global Structure of the Ear 115
7.2 External Ear 116
7.3 Middle Ear 117
7.4 Inner Ear 119
7.4.1 Structure of the Cochlea 119
7.4.2 Passive Cochlear Processing 123
7.4.3 Active Function of the Cochlea 123
7.4.4 Inner Hair Cells 126
7.4.5 Cochlear Non-linearities 127
7.5 Otoacoustic Emissions 128
7.6 Auditory Nerve 129
7.6.1 Information Transmission Using the Firing Rate 130
7.6.2 Phase Locking 130
7.7 Auditory Nervous System 132
7.7.1 Structure of the Auditory Pathway 133
7.7.2 Studying Brain Function 134
References 135
8 The Approach and Methodology of Psychoacoustics 137
8.1 Sound Events Versus Auditory Events 137
8.2 Psychophysical Functions 139
8.3 Generation of Sound Events 139
8.3.1 Synthesis of Sound Signals 139
8.3.2 Listening Setup and Conditions 140
8.4 Selection of Subjects for Listening Tests 141
8.5 What Are We Measuring? 142
8.5.1 Thresholds 142
8.5.2 Signal Detection Theory 143
8.5.3 Scales and Categorization of Percepts 144
8.5.4 Scales in Subjective Tests 145
8.6 Tasks for Subjects 145
8.7 Basic Psychoacoustic Test Methods 146
8.7.1 Method of Constant Stimuli 146
8.7.2 Method of Limits 147
8.7.3 Method of Adjustment 147
8.7.4 Method of Tracking 148
8.7.5 Direct Scaling Methods 148
8.7.6 Adaptive Staircase Methods 148
8.7.7 Adaptive Estimation of Distribution 149
8.8 Descriptive Sensory Analysis 149
8.8.1 Verbal Elicitation 151
8.8.2 Non-verbal Elicitation 152
8.8.3 Indirect Elicitation 152
8.9 Physiological Measurements 153
8.10 Psychoacoustic Tests from the Point of View of Statistics 153
References 154
9 Basic Function of Hearing 157
9.1 Effective Hearing Area 157
9.1.1 Equal Loudness Curves 159
9.1.2 Sound Level and Its Measurement 159
9.2 Spectral Masking 160
9.2.1 Masking by Noise 161
9.2.2 Masking by Pure Tones 163
9.2.3 Masking by Complex Tones 163
9.2.4 Other Masking Phenomena 163
9.3 Temporal Masking 165
9.4 Frequency Selectivity of Hearing 167
9.4.1 Psychoacoustic Tuning Curves 167
9.4.2 Bark Bandwidths 168
9.4.3 ERB Bandwidths 170
9.4.4 Bark, ERB and Greenwood Scales 171
References 172
10 Basic Psychoacoustic Quantities 175
10.1 Pitch 175
10.1.1 Pitch Strength and Frequency Range 175
10.1.2 JND of Pitch 176
10.1.3 Pitch Perception Versus Duration of Sound 176
10.1.4 Mel Scale 177
10.1.5 Logarithmic Pitch Scale and Musical Scale 179
10.1.6 Detection Threshold of Pitch Change and Frequency Modulation 180
10.1.7 Pitch of Coloured Noise 180
10.1.8 Repetition Pitch 181
10.1.9 Virtual Pitch 181
10.1.10 Pitch of Non-harmonic Complex Sounds 182
10.1.11 Pitch Theories 182
10.1.12 Absolute Pitch 182
10.2 Loudness 183
10.2.1 Loudness Determination Experiments 183
10.2.2 Loudness Level 184
10.2.3 Loudness of a Pure Tone 184
10.2.4 Loudness of Broadband Signals 185
10.2.5 Excitation Pattern, Specific Loudness and Loudness 187
10.2.6 Difference Threshold of Loudness 188
10.2.7 Loudness Versus Duration of Sound 190
10.3 Timbre 192
10.3.1 Timbre of Steady-state Sounds 192
10.3.2 Timbre of Sound Including Modulations 192
10.4 Subjective Duration of Sound 193
References 194
11 Further Analysis in Hearing 197
11.1 Sharpness 197
11.2 Detection of Modulation and Sound Onset 199
11.2.1 Fluctuation Strength 199
11.2.2 Impulsiveness 201
11.3 Roughness 201
11.4 Tonality 203
11.5 Discrimination of Changes in Signal Magnitude and Phase Spectra 204
11.5.1 Adaptation to the Magnitude Spectrum 204
11.5.2 Perception of Phase and Time Differences 205
11.5.3 Factors Leading to a Perception of Buzziness 206
11.5.4 Perception of Frequency-dependent Group Delay of Audio Devices 208
11.6 Psychoacoustic Concepts and Music 210
11.6.1 Sensory Consonance and Dissonance 210
11.6.2 Intervals, Scales and Tuning in Music 211
11.6.3 Rhythm, Tempo, Bar and Measure 214
11.7 Perceptual Organization of Sound 215
11.7.1 Segregation of Sound Sources 216
11.7.2 Sound Streaming and Auditory Scene Analysis 217
References 219
12 Spatial Hearing 223
12.1 Concepts and Definitions for Spatial Hearing 223
12.1.1 Basic Concepts 223
12.1.2 Coordinate Systems for Spatial Hearing 224
12.2 Head-related Acoustics 226
12.3 Localization Cues 230
12.3.1 Interaural Time Difference 231
12.3.2 Interaural Level Difference 233
12.3.3 Interaural Coherence 234
12.3.4 Monaural Cues 235
12.3.5 Dynamic Cues 237
12.3.6 Interaction Between Spatial Hearing and Vision 238
12.4 Localization Accuracy 238
12.4.1 Localization in the Horizontal Plane 238
12.4.2 Localization in the Median Plane 240
12.4.3 3D Localization 240
12.4.4 Perception of the Distribution of a Spatially Extended Source 241
12.5 Directional Hearing in Enclosed Spaces 243
12.5.1 Precedence Effect 243
12.5.2 Adaptation to the Room Effect in Localization 244
12.6 Binaural Advantages in Timbre Perception 245
12.6.1 Binaural Detection and Unmasking 245
12.6.2 Binaural Decolouration 246
12.7 Perception of Source Distance 247
12.7.1 Cues for Distance Perception 247
12.7.2 Accuracy of Distance Perception 248
References 249
13 Auditory Modelling 251
13.1 Simple Psychoacoustic Modelling in Time–Frequency Domain 252
13.1.1 Computation of the Auditory Spectrum Through DFT 252
13.1.2 Applications of DFT-based Auditory Models 254
13.2 Time-domain Auditory Models 257
13.2.1 Modelling the Outer and Middle Ear 257
13.2.2 Modelling Frequency Selectivity with Linear Filter Banks 258
13.2.3 Modelling Frequency Selectivity with Non-linear Filter Banks 258
13.2.4 Modelling Frequency Selectivity with Filter Banks with Frequency-channel Coupling 260
13.2.5 Hair-cell Models 260
13.3 Modelling of Higher-level Systemic Properties 262
13.3.1 Envelope Detection and Temporal Dynamics 262
13.3.2 Analysis of Pitch and Periodicity 265
13.3.3 Modelling of Loudness Perception 266
13.4 Models of Spatial Hearing 267
13.4.1 Delay-network-based Models of Binaural Hearing 267
13.4.2 Equalization-cancellation and ILD Models 269
13.4.3 Count-comparison Models 270
13.4.4 Models of Localization in Sagittal Planes 271
13.4.5 General Models of Sound-source Localization 271
13.5 Python Examples 272
13.5.1 Filter-bank Model with Autocorrelation-based Pitch Analysis 272
13.5.2 Binaural Filter-bank Model with Cross-correlation-based ITD Analysis 275
References 278
14 Time–Frequency-domain Processing of Audio 281
14.1 Basic Techniques and Concepts for Time–Frequency Processing 281
14.1.1 Frame-based Processing 281
14.1.2 Downsampled Filter-bank Processing 283
14.1.3 Modulation with Tone Sequences 285
14.1.4 Aliasing 286
14.2 Time–Frequency Transforms 287
14.2.1 Short-time Fourier Transform 288
14.2.2 Alias-free STFT 290
14.2.3 Modified Discrete Cosine Transform 290
14.2.4 Pseudo-quadrature Mirror Filter (PQMF) Bank 291
14.2.5 Complex QMF 292
14.2.6 Sub-sub-band Filtering of the Complex QMF Bands 293
14.2.7 Stochastic Measures of Time–Frequency Signals 295
14.2.8 Decorrelation 296
References 296
15 Machine learning 299
15.1 Training Deep Neural Networks 299
15.2 Neural Network Architectures 301
15.2.1 Multi-layer Perceptrons (MLPs) 301
15.2.2 Convolutional Neural Networks (CNNs) 303
15.2.3 Recurrent Neural Networks (RNNs) 304
15.2.4 Transformers and Attention 304
15.2.5 State-space Models 305
15.3 Feature Representations 305
15.3.1 Time-frequency Representations 306
15.3.2 Spatial Features 307
15.3.3 Deep Embeddings 307
15.4 Loss Functions 308
15.4.1 Spectral Loss Functions 309
15.4.2 Physics-informed Loss Functions 309
15.4.3 Adversarial Loss Functions 310
15.4.4 Contrastive Loss Functions 310
15.5 Generative models 311
15.5.1 Autoregressive Models 311
15.5.2 Variational Autoencoders 312
15.5.3 Diffusion Models 312
15.5.4 Generative Adversarial Networks (GANs) 313
15.6 Implicit Neural Representations 313
References 314
16 Sound Reproduction 319
16.1 Need for Sound Reproduction 319
16.2 Audio Content Production 320
16.3 Measuring and Controlling Programme Loudness 321
16.4 Listening Setups 322
16.4.1 Loudspeaker Setups 322
16.4.2 Listening Room Acoustics 324
16.4.3 Audiovisual Systems 325
16.4.4 Auditory-tactile Systems 326
16.5 Recording Techniques 327
16.5.1 Monophonic Techniques 327
16.5.2 Spot Microphone Technique 328
16.5.3 Coincident Microphone Techniques for Two-channel Stereophony 328
16.5.4 Spaced Microphone Techniques for Two-channel Stereophony 329
16.5.5 Spaced Microphone Techniques for Multi-channel Loudspeaker Systems 330
References 331
17 Spatial Audio 335
17.1 Basic concepts 335
17.2 Amplitude Panning 336
17.2.1 Amplitude Panning in a Stereophonic Setup 336
17.2.2 Amplitude Panning in Horizontal Multi-channel Loudspeaker setups 337
17.2.3 3D Amplitude Panning for Multi-channel Loudspeaker Setups 338
17.2.4 Uniform Spreading of Amplitude-panned Sources 339
17.3 Time Delay Panning 340
17.4 Wave Field Synthesis 340
17.5 Controlling the Distance of Virtual Sources 342
17.6 Spatial Audio Reproduction with Ambisonics 343
17.6.1 Circular and Spherical Harmonic Encoding of Sound Fields 344
17.6.2 Linear Decoding to Loudspeaker Arrays 345
17.6.3 Optimizing Loudspeaker Decoding with Vector-models 347
17.6.4 Microphone Arrays for Ambisonics 349
17.7 Parametric Time–Frequency-domain Reproduction 351
17.7.1 Analysis Principles 351
17.7.2 Synthesis Principles 352
17.7.3 Spatial Sound Recording and Reproduction 353
17.7.4 Directional Audio Coding 354
17.7.5 Auralization of Measured Room Impulse Responses 355
17.8 Spatial Effects and Super-hearing 356
17.9 Binaural Techniques 356
17.9.1 Listening to Binaural Recordings with Headphones 356
17.9.2 HRTF Processing for Headphone Listening 357
17.9.3 Binaural Listening of Virtual Loudspeakers 358
17.9.4 Binaural Listening to Two-channel Stereophonic Content 359
17.9.5 Binaural Techniques with Cross-talk-cancelled Loudspeakers 359
17.9.6 HRTF Individualization 361
17.9.7 HRTF Interpolation Methods 362
17.10 Beamforming 363
References 364
18 Virtual reality 369
18.1 Basic Concepts 369
18.2 Sound in VR 371
18.3 Modelling Sound Propagation Path 373
18.3.1 Source Directivity 373
18.3.2 Direct Sound Path 373
18.3.3 Early Response 374
18.3.4 Late Response 375
18.3.5 Connected Spaces 375
18.3.6 Wave-based Methods 376
18.4 Six-degrees-of-freedom Reproduction 376
18.5 Rendering Audio in VR 377
References 380
19 Sound and Music Computing 383
19.1 Coding of Audio Signals 383
19.1.1 Masking-based Audio Coding 383
19.1.2 Audio Coding with Spectral Band Replication 384
19.1.3 Neural Coding of Audio 385
19.2 Spatial Audio Coding and Up-mixing 386
19.2.1 Parametric Coding of Multichannel Audio 386
19.2.2 Immersive Audio Coding 387
19.2.3 Stereo Upmixing and Enhancement for Loudspeakers and Headphones 388
19.3 Sound Synthesis in Music 390
19.3.1 Synthesis Methods 390
19.3.2 Physical Modelling of Musical Instruments 391
19.3.3 Neural Methods for Musical Sound Synthesis 393
19.4 Differentiable DSP 394
19.5 Procedural Audio Synthesis 395
19.6 Digital Audio Effects 396
19.7 Reverberators 397
19.7.1 DSP Structures for Reverberators 398
19.7.2 Using Room Impulse Responses in Reverberators 399
19.8 Sonic Interaction Design 400
19.9 Computational Auditory Scene Analysis, CASA 401
19.10 Music Information Retrieval 402
19.11 Miscellaneous Applications 404
References 405
20 Speech Technologies 411
20.1 Speech Coding 411
20.1.1 Waveform Coding 412
20.1.2 Vocoders 412
20.1.3 Neural Coding of Speech 415
20.2 Text-to-Speech Synthesis 415
20.2.1 Early Knowledge-based Text-to-speech (TTS) Synthesis 416
20.2.2 Unit-selection Synthesis 417
20.2.3 Statistical Parametric Synthesis 419
20.2.4 Neural Text-to-speech Synthesis with Acoustic Representations 421
20.2.5 Neural Vocoders for Speech Synthesis 422
20.2.6 End-to-end Neural Text-to-speech Synthesis 423
20.3 Speech Recognition 424
References 426
21 Sound Quality 431
21.1 Historical Background of Sound Quality 432
21.2 The Many Facets of Sound Quality 433
21.3 Immersion 434
21.4 Systemic Framework for Sound Quality 435
21.5 Subjective Sound Quality Measurement 436
21.5.1 Mean Opinion Score 436
21.5.2 MUSHRA 437
21.6 Audio Quality 438
21.7 Monaural Quality 439
21.7.1 Signal-to-Interference and Signal-to-artefacts Ratios 439
21.7.2 Measures and Models for Monaural Audio Quality 440
21.8 Spatial Audio Quality 443
21.8.1 Subjective Directional Quality 443
21.8.2 Externalization and Distance Perception 444
21.8.3 Subjective Binaural Colouration 445
21.8.4 Instrumental Evaluation of Spatial Audio 446
21.9 Audio Quality in Virtual and Augmented Reality 446
21.10 Quality of Speech Communication 447
21.10.1 Subjective Methods and Measures 448
21.10.2 Objective Methods and Measures 449
21.11 Measuring Speech Understandability with the Modulation Transfer
Function 450
21.11.1 Modulation Transfer Function 450
21.11.2 Speech Transmission Index STI 454
21.11.3 STI and Speech Intelligibility 454
21.11.4 Practical Measurement of STI 455
21.12 Objective Speech Quality Measurement for Telecommunication 456
21.12.1 General Speech Quality Measurement Techniques 457
21.12.2 Measurement of the Perceptual Effect of Background Noise 459
21.12.3 Measurement of the Perceptual Effect of Echoes 459
21.13 Sound Quality in Auditoria and Concert Halls 460
21.13.1 Subjective Measures 461
21.13.2 Objective Measures 462
21.13.3 Percentage of Consonant Loss 464
21.14 Noise Quality 464
21.15 Product Sound Quality 465
References 466
22 Technical Audiology 471
22.1 Hearing Impairments and Disabilities 471
22.1.1 Key Terminology 472
22.1.2 Classification of Hearing Impairments 473
22.1.3 Causes of Hearing Impairments 474
22.2 Symptoms and Consequences of Hearing Impairments 474
22.2.1 Hearing Threshold Shift 474
22.2.2 Distortion and Decrease in Discrimination 476
22.2.3 Speech Communication Problems 477
22.2.4 Tinnitus 478
22.3 The Effect of Noise on Hearing 478
22.3.1 Noise 479
22.3.2 Formation of Noise-Induced Hearing Loss 480
22.3.3 Temporary Threshold Shift 480
22.3.4 Hearing Protection 480
22.4 Audiometry 483
22.4.1 Pure-tone Audiometry 483
22.4.2 Bone-conduction Audiometry 484
22.4.3 Speech Audiometry 484
22.4.4 Sound-field Audiometry 485
22.4.5 Tympanometry 485
22.4.6 Otoacoustic Emissions 486
22.4.7 Neural Responses 486
22.5 Hearing Aids 487
22.5.1 Types of Hearing Aids 487
22.5.2 Signal Processing in Hearing Aids 488
22.5.2.1 Gain Control 489
22.5.2.2 Directional Microphone Systems 490
22.5.2.3 Other Signal Processing Features 491
22.5.3 Transmission Systems and Assistive Listening Devices 492
22.6 Implantable Hearing Solutions 492
22.6.1 Cochlear Implants 492
22.6.2 Electric-acoustic Stimulation 495
22.6.3 Bone-anchored Hearing Aids 495
22.6.4 Middle-ear Implants 495
References 496
Index 499
