Wang / Zhao / Cheng Machine Learning for Vision-Based Motion Analysis

Theory and Techniques
1. Auflage 2010
ISBN: 978-0-85729-057-1
Verlag: Springer
Format: PDF
Kopierschutz: 1 - PDF Watermark

E-Book, Englisch, 372 Seiten

Reihe: Advances in Computer Vision and Pattern Recognition

Techniques of vision-based motion analysis aim to detect, track, identify, and generally understand the behavior of objects in image sequences. With the growth of video data in a wide range of applications from visual surveillance to human-machine interfaces, the ability to automatically analyze and understand object motions from video footage is of increasing importance. Among the latest developments in this field is the application of statistical machine learning algorithms for object tracking, activity modeling, and recognition.Developed from expert contributions to the first and second International Workshop on Machine Learning for Vision-Based Motion Analysis, this important text/reference highlights the latest algorithms and systems for robust and effective vision-based motion understanding from a machine learning perspective. Highlighting the benefits of collaboration between the communities of object motion understanding and machine learning, the book discusses the most active forefronts of research, including current challenges and potential future directions.Topics and features: provides a comprehensive review of the latest developments in vision-based motion analysis, presenting numerous case studies on state-of-the-art learning algorithms; examines algorithms for clustering and segmentation, and manifold learning for dynamical models; describes the theory behind mixed-state statistical models, with a focus on mixed-state Markov models that take into account spatial and temporal interaction; discusses object tracking in surveillance image streams, discriminative multiple target tracking, and guidewire tracking in fluoroscopy; explores issues of modeling for saliency detection, human gait modeling, modeling of extremely crowded scenes, and behavior modeling from video surveillance data; investigates methods for automatic recognition of gestures in Sign Language, and human action recognition from small training sets.Researchers, professional engineers, and graduate students in computer vision, pattern recognition and machine learning, will all find this text an accessible survey of machine learning techniques for vision-based motion analysis. The book will also be of interest to all who work with specific vision applications, such as surveillance, sport event analysis, healthcare, video conferencing, and motion video indexing and retrieval.

Autoren/Hrsg.

Wang, Liang

Zhao, Guoying

Cheng, Li

Pietikäinen, Matti

Inhaltsverzeichnis

1;Preface;5
1.1;Part I: Manifold Learning and Clustering/Segmentation;6
1.2;Part II: Tracking;7
1.3;Part III: Motion Analysis and Behavior Modeling;9
1.4;Part IV: Gesture and Action Recognition;10
1.4.1;Acknowledgements;11
2;Contents;12
3;Manifold Learning and Clustering/Segmentation;14
3.1;Practical Algorithms of Spectral Clustering: Toward Large-Scale Vision-Based Motion Analysis;15
3.1.1;Introduction;15
3.1.2;Spectral Clustering;16
3.1.2.1;Principle;16
3.1.2.2;Algorithm;18
3.1.2.3;Related Work;18
3.1.3;Dimensionality Reduction by Random Projection;19
3.1.3.1;Random Projection;19
3.1.3.1.1;Acceleration of Kernel Computation;20
3.1.3.1.2;Random Sampling as Random Projection;20
3.1.3.1.3;Using a Minority of Image Pixels;21
3.1.3.2;Efficient Random Projection;21
3.1.4;Size Reduction of Affinity Matrix by Sampling;23
3.1.4.1;Random Subsampling;24
3.1.4.2;Pre-clustering;25
3.1.5;Practical Ncut Algorithms;26
3.1.5.1;Randomized Ncut Algorithm;26
3.1.5.1.1;Invocation of Dimensionality Reduction;27
3.1.5.1.2;Relation to the Original Algorithm;27
3.1.5.1.3;Scale Selection;27
3.1.5.1.4;Number of Clusters;28
3.1.5.2;Ncut Algorithm with Pre-clustering;28
3.1.6;Experiments;29
3.1.6.1;Performance Tests;29
3.1.6.1.1;Error Analysis;29
3.1.6.1.2;Computational Cost;30
3.1.6.2;Image Segmentation;31
3.1.6.3;Motion Segmentation;32
3.1.6.4;Video Shot Segmentation;33
3.1.6.4.1;Segmentation Using Appearance-Based Similarities;33
3.1.6.4.2;Segmentation with Local Scaling;33
3.1.7;Conclusions;35
3.1.8;Appendix: Clustering Scores;37
3.1.9;References;37
3.2;Riemannian Manifold Clustering and Dimensionality Reduction for Vision-Based Analysis;39
3.2.1;Introduction;40
3.2.1.1;Chapter summary;42
3.2.2;Review of Local Nonlinear Dimensionality Reduction Methods in Euclidean Spaces;43
3.2.2.1;NLDR for a Nonlinear Manifold;43
3.2.2.1.1;Calculation of M in LLE;44
3.2.2.1.2;Calculation of M in LE;44
3.2.2.1.3;Calculation of M in HLLE;45
3.2.2.2;NLDR for a Single Subspace;45
3.2.3;Manifold Clustering and Dimensionality Reduction Using the Euclidean Metric;47
3.2.3.1;Manifold Clustering and Dimensionality Reduction for a k-Separated Union of k-Connected Nonlinear Manifolds;47
3.2.3.2;Degeneracies for a k-Separated Union of k-Connected Linear Manifolds;48
3.2.4;Manifold Clustering and Dimensionality Reduction Using the Riemannian Metric;50
3.2.4.1;Review of Riemannian Manifolds;50
3.2.4.2;Extending Manifold Clustering and Dimensionality Reduction to Riemannian Manifolds;53
3.2.4.2.1;Selection of the Riemannian kNN;53
3.2.4.2.2;Riemannian Calculation of M for LLE;53
3.2.4.2.3;Riemannian Calculation of M for LE;54
3.2.4.2.4;Riemannian Calculation of M for HLLE;54
3.2.4.2.5;Calculation of the Embedding Coordinates;54
3.2.4.2.6;Extending Manifold Clustering to Riemannian Manifolds;55
3.2.5;Experiments;55
3.2.5.1;Application and Experiments on SPSD(3) ;55
3.2.5.2;Application and Experiments on the Space of Probability Density Functions;58
3.2.6;Conclusion and Open Research Problems;62
3.2.7;References;63
3.3;Manifold Learning for Multi-dimensional Auto-regressive Dynamical Models;66
3.3.1;Introduction;66
3.3.2;Learning Pullback Metrics for Linear Models;68
3.3.2.1;Pullback Metrics;68
3.3.2.2;Fisher Metric for Linear Models;69
3.3.2.3;General Framework;69
3.3.2.4;Objective Functions: Classification Performance and Inverse Volume;71
3.3.3;Pullback Metrics for Multidimensional Autoregressive Models;72
3.3.3.1;The Basis Manifold;72
3.3.3.1.1;The Basis Manifold AR(2,1) in the Scalar Case;72
3.3.3.1.2;The Multidimensional Case;73
3.3.3.1.3;Product Metric;73
3.3.3.1.4;Geodesics;74
3.3.3.2;An Automorphism for the Scalar Case;75
3.3.3.3;Product and Global Automorphisms for AR(2,p);75
3.3.3.3.1;Volume Element for AR(2,p) Under Product Automorphism;76
3.3.4;Tests on Identity Recognition;77
3.3.4.1;Feature Representation;78
3.3.4.2;Identification of a AR(2,p) Model for Each Sequence;78
3.3.4.3;Performances of Optimal Pullback Metrics;80
3.3.4.4;Influence of Parameters;82
3.3.5;Perspectives and Conclusions;83
3.3.6;References;83
4;Tracking;86
4.1;Mixed-State Markov Models in Image Motion Analysis;87
4.1.1;Introduction;88
4.1.1.1;Outline of the Chapter;89
4.1.2;Related Work: Discrete-Continuous Models and Dynamic Textures;89
4.1.2.1;Discrete-Continuous Approaches;90
4.1.2.2;Dynamic Texture Characterization;91
4.1.3;The Mixed-State Nature of Motion Measurements;92
4.1.4;Mixed-State Markov Models;94
4.1.4.1;Mixed-State Random Variables;94
4.1.4.2;Mixed-State Markov Random Fields;96
4.1.4.2.1;The Mixed-State Gibbs Distribution;97
4.1.4.2.2;Mixed-State Automodels;98
4.1.4.3;Causal Mixed-State Markov Models;101
4.1.5;Sampling, Estimation and Inference in MS-MRF;102
4.1.5.1;Sampling;102
4.1.5.2;Parameter Estimation;103
4.1.5.3;Inference of Mixed-State Random Fields;104
4.1.6;Characterizing Motion Textures with MS-MRF;105
4.1.6.1;Defining the Set of Parameters;105
4.1.6.2;Recognition of Motion Textures;106
4.1.6.2.1;Application to Motion Texture Classification;107
4.1.6.2.2;Temporal Consistency;110
4.1.7;Mixed-State Causal Modeling of Motion Textures;111
4.1.7.1;Learning MS-MC Motion Texture Models;113
4.1.7.2;Model Matching;113
4.1.8;Mixed-State Markov Chain vs. Mixed-State Markov Random Field Motion Texture Models;114
4.1.9;Motion Texture Tracking;117
4.1.9.1;Experimental Results;118
4.1.10;Conclusions;123
4.1.11;References;123
4.2;Learning to Detect Event Sequences in Surveillance Streams at Very Low Frame Rate;126
4.2.1;Introduction;126
4.2.2;Approaches for Image Reviews;127
4.2.3;Surveillance for Nuclear Safeguards;129
4.2.4;Filtering Surveillance Streams by Combining Uninformed and Informed Search Strategies;130
4.2.5;Searching Events by Scene Change Detection;132
4.2.6;Searching Events by Sequence and Time Attributes;134
4.2.6.1;Modeling Nuclear Flask Processing with a HSMM;136
4.2.6.1.1;State Space;137
4.2.6.1.2;Transition Matrix;137
4.2.6.1.3;Sojourn Times;138
4.2.6.1.4;Emissions;138
4.2.6.2;Training the HSMM;139
4.2.6.3;The MM Image Review Tool;140
4.2.6.4;Discussion about the MM Review Tool;142
4.2.7;Benchmarking Image Review Filters;145
4.2.7.1;Image Sets;145
4.2.7.2;Performance Metrics;146
4.2.7.3;Experimental Results;147
4.2.8;Discussion;150
4.2.9;References;152
4.3;Discriminative Multiple Target Tracking;154
4.3.1;Introduction;154
4.3.2;Appearance and Motion Model of Multiple Targets;156
4.3.2.1;Metric Learning Framework;156
4.3.2.2;Joint Appearance Model Estimation;157
4.3.2.3;Motion Parameter Optimization;158
4.3.3;Online Matching and Updating Multiple Models;159
4.3.4;Discriminant Exclusive Principle;161
4.3.5;Experiments;161
4.3.5.1;Visualization of Learned Appearance Model;161
4.3.5.2;Multiple Target Tracking for Different Video Sequences;162
4.3.6;Discussions, Conclusion and Future Work;165
4.3.7;References;166
4.4;A Framework of Wire Tracking in Image Guided Interventions;168
4.4.1;Background;169
4.4.2;Guidewire Tracking Method;172
4.4.2.1;Method Overview;172
4.4.2.1.1;A Guidewire Model;172
4.4.2.1.2;A Probabilistic Guidewire Tracking Framework;173
4.4.2.2;Guidewire Measurement Models;175
4.4.2.2.1;Learning-Based Guidewire Measurements;175
4.4.2.2.2;Appearance-Based Measurements;176
4.4.2.2.3;Fusion of Multiple Measurements;177
4.4.2.3;Hierarchical and Multi-resolution Guidewire Tracking;178
4.4.2.3.1;Kernel-Based Measurement Smoothing;178
4.4.2.3.2;Rigid Tracking;178
4.4.2.3.3;Nonrigid Tracking;179
4.4.3;Experiments;181
4.4.3.1;Data and Evaluation Protocol;181
4.4.3.2;Quantitative Evaluations;183
4.4.4;Conclusion;184
4.4.5;References;185
5;Motion Analysis and Behavior Modeling;187
5.1;An Integrated Approach to Visual Attention Modeling for Saliency Detection in Videos;188
5.1.1;Introduction;189
5.1.1.1;Interest Point Detection;189
5.1.1.2;Visual Attention Modeling;190
5.1.1.3;Proposed Approach;191
5.1.2;Prior Work;193
5.1.2.1;Visual Attention Modeling Methods;194
5.1.2.1.1;Bottom-Up Saliency;194
5.1.2.1.2;Top-Down Saliency;196
5.1.2.1.3;Integrating Top-Down and Bottom-Up Saliency;196
5.1.2.2;Interest Point Detection Methods;197
5.1.2.3;Human Eye Movement as Indicators of User Interest;199
5.1.2.3.1;Use of Eye-Tracking in Related Work;200
5.1.2.4;Limitations of Existing Work;201
5.1.3;Learning Attention-Based Saliency: Conceptual Framework;202
5.1.3.1;Learning Context-Specific Saliency;203
5.1.3.2;Predicting Context-Specific Saliency;204
5.1.4;Experiments and Results;204
5.1.4.1;Experimental Setup;205
5.1.4.2;Implementation;206
5.1.4.3;Results;207
5.1.4.3.1;Eye Movement Prediction;208
5.1.4.3.2;Context Specific Saliency Detection;209
5.1.4.4;Discussion;211
5.1.4.5;Integrating Bottom-Up Approaches: A Possible Extension;213
5.1.4.5.1;Integration Using Probabilistic Framework;213
5.1.4.5.2;Results of the Integrated Framework;214
5.1.5;Conclusions and Future Work;217
5.1.5.1;Possible Applications;217
5.1.5.2;Future Work;218
5.1.6;References;219
5.2;Video-Based Human Motion Estimation by Part-Whole Gait Manifold Learning;222
5.2.1;Introduction;223
5.2.2;Related Works;224
5.2.2.1;Discriminative Approaches;225
5.2.2.1.1;Feature Representation;225
5.2.2.1.2;Inference Algorithms;225
5.2.2.2;Generative Approaches;226
5.2.2.2.1;Visual Observations;226
5.2.2.2.2;Human Shape Models;226
5.2.2.2.3;Inference Algorithms;227
5.2.2.2.4;Human Motion Models;227
5.2.2.2.5;Single Pose Manifold;228
5.2.2.2.6;Dual Pose Manifolds;228
5.2.2.2.7;Shared Pose Manifold;228
5.2.2.3;Our Research;228
5.2.3;Research Overview;229
5.2.3.1;Dual Gait Generative Models;229
5.2.3.2;Gait Manifolds;229
5.2.3.3;Inference for Gait Estimation;230
5.2.4;Dual Gait Generative Models;231
5.2.4.1;Kinematic Gait Generative Model (KGGM);231
5.2.4.2;Visual Gait Generative Model (VGGM);232
5.2.4.3;Two-Layer KGGM and VGGM;233
5.2.5;Gait Manifolds;234
5.2.5.1;Gait Manifold Learning;234
5.2.5.2;Gait Manifold Topology;236
5.2.5.2.1;Euclidean Distance Between Gait Vectors;236
5.2.5.2.2;Distance Between 3D Joint Positions;237
5.2.5.2.3;Fourier Analysis of Joint Angles;237
5.2.5.3;Part-Whole Gait Manifolds;239
5.2.5.4;Manifold Mapping Between KGGM and VGGM;240
5.2.5.4.1;Nonlinear Mapping Functions (MAP-1);241
5.2.5.4.2;Similarity-Preserving Mapping Functions (MAP-2);241
5.2.5.4.3;MAP-1 vs. MAP-2;242
5.2.6;Inference Algorithms;242
5.2.6.1;Graphical Models;242
5.2.6.2;Whole-Based Gait Estimation;244
5.2.6.2.1;Segmental Modeling;245
5.2.6.2.2;Mode-Based Gait Estimation;245
5.2.6.2.3;Segmental Jump-Diffusion MCMC Inference;246
5.2.6.3;Part-Based Gait Estimation;247
5.2.6.3.1;Part-Level Gait Priors;248
5.2.6.3.2;Part-Level Likelihood Functions;249
5.2.7;Experimental Results and Discussions;250
5.2.7.1;Experimental Setups;251
5.2.7.1.1;Training Data Collection;251
5.2.7.1.2;Testing Data Collection;253
5.2.7.1.3;Local Error Analysis;253
5.2.7.1.4;Global Error Analysis;255
5.2.7.2;Experiments on KGGM;255
5.2.7.3;Evaluation of Two-Stage Inference;256
5.2.7.3.1;Segmental Gait Modeling;256
5.2.7.3.2;Local Motion Estimation;257
5.2.7.3.3;Whole-Based Gait Estimation;258
5.2.7.3.4;Part-Whole Gait Estimation;259
5.2.7.4;Overall Performance Evaluation;260
5.2.7.4.1;Group-I;261
5.2.7.4.2;Group-II;263
5.2.7.4.3;Group-III;263
5.2.7.5;Limitations and Discussion;264
5.2.8;Conclusion and Future Research;265
5.2.9;References;265
5.3;Spatio-Temporal Motion Pattern Models of Extremely Crowded Scenes;269
5.3.1;Introduction;269
5.3.2;Related Work;271
5.3.3;Local Spatio-Temporal Motion Patterns;271
5.3.4;Prototypical Motion Patterns;273
5.3.5;Distribution-Based Hidden Markov Models;275
5.3.6;Experimental Results;276
5.3.7;Conclusion;279
5.3.8;References;280
5.4;Learning Behavioral Patterns of Time Series for Video-Surveillance;281
5.4.1;Introduction;281
5.4.2;Related Works;283
5.4.3;Low Level Processing and Initial Representation;286
5.4.3.1;Video Processing;286
5.4.3.2;The Choice of the Input Space;287
5.4.4;Temporal Series Representations;288
5.4.4.1;Curve Fitting;288
5.4.4.2;Probabilistic Models;289
5.4.4.3;String-Based Approach;290
5.4.5;Learning Behaviors;291
5.4.5.1;The Learning Phase;291
5.4.5.2;Kernels for Time-Series;293
5.4.5.2.1;Probability Product Kernel (PPK);293
5.4.5.2.2;Kernels for String-Based Representations;294
5.4.5.3;Run Time Analysis;294
5.4.6; Experimental Analysis;295
5.4.6.1;Data Collection and Semi-automatic Labeling;295
5.4.6.2;Model Selection;297
5.4.6.2.1;Curve Fitting;297
5.4.6.2.2;Probabilistic Model;298
5.4.6.2.3;String-Based Approach;299
5.4.6.2.4;Kernel Choice;299
5.4.6.2.5;RLS Classification;299
5.4.6.2.6;HMMs Likelihood Estimation;300
5.4.6.2.7;Spectral Clustering;300
5.4.6.3;Supervised Analysis;300
5.4.6.4;Unsupervised Analysis;302
5.4.7;Discussion and Open Problems;306
5.4.8;References;309
6;Gesture and Action Recognition;311
6.1;Recognition of Spatiotemporal Gestures in Sign Language Using Gesture Threshold HMMs;312
6.1.1;Introduction;312
6.1.1.1;Related Work;313
6.1.1.1.1;Isolated Gesture Recognition;313
6.1.1.1.2;Continuous Gesture Recognition;314
6.1.1.2;Chapter Outline;316
6.1.2;Hidden Markov Models;317
6.1.2.1;HMM Algorithms;318
6.1.2.2;Types of HMMs;318
6.1.3;Threshold HMM Model;319
6.1.4;GT-HMM Framework;320
6.1.4.1;GT-HMM Training;321
6.1.4.1.1;Gesture Subunit Initialization;322
6.1.4.2;GT-HMM for Gesture Recognition;326
6.1.4.2.1;Gesture Classification;326
6.1.4.2.2;Parallel Training;327
6.1.4.2.3;Parallel Gesture Classification;327
6.1.4.2.4;Continuous Recognition;328
6.1.4.2.5;Candidate Selection;329
6.1.5;Experiments;330
6.1.5.1;Feature Extraction;330
6.1.5.2;Evaluation of Techniques on Isolated Gestures;332
6.1.5.2.1;Manual Sign Experiments;333
6.1.5.2.2;Head Gesture Experiments;336
6.1.5.2.3;Eye Brow Gesture Experiments;338
6.1.5.2.4;Benchmark Data-Set: Marcel InteractPlay Database;340
6.1.5.3;Continuous Gesture Recognition Experiments;343
6.1.5.3.1;Continuous Experiment Results;344
6.1.5.3.2;Continuous User Independent Experiment Results;345
6.1.5.4;Multimodal Recognition Examples;348
6.1.6;Conclusion;349
6.1.7;References;351
6.2;Learning Transferable Distance Functions for Human Action Recognition;354
6.2.1;Introduction;354
6.2.2;Previous Work;356
6.2.2.1;Related Work in Learning;356
6.2.2.1.1;"Relatedness" via Features;357
6.2.2.1.2;"Relatedness" via Model Parameters;358
6.2.2.2;Related Work in Vision;358
6.2.3;Motion Descriptors and Matching Scheme;360
6.2.3.1;Motion Descriptors;360
6.2.3.2;Patch Based Action Comparison;360
6.2.4;Learning a Transferable Distance Function;362
6.2.4.1;Transferable Distance Function;363
6.2.4.2;Max-Margin Formulation;364
6.2.4.3;Solving the Dual;366
6.2.4.4;Hyper-Features;366
6.2.5;Experiments;367
6.2.5.1;Datasets;367
6.2.5.1.1;KTH Dataset;367
6.2.5.1.2;Weizmann Dataset;367
6.2.5.1.3;Cluttered Human Action Dataset;368
6.2.5.2;Experimental Results;368
6.2.5.2.1;Direct Comparison on KTH;368
6.2.5.2.2;Training on Weizmann and Testing on KTH;369
6.2.5.2.3;Direct Comparison on Weizmann;372
6.2.5.2.4;Direct Comparison on Cluttered Action Dataset;372
6.2.5.2.5;Training on KTH and Testing on Cluttered Action Dataset;373
6.2.6;Conclusion;373
6.2.7;References;373
7;Index;376