Bhanu / Han Human Recognition at a Distance in Video

1. Auflage 2010
ISBN: 978-0-85729-124-0
Verlag: Springer
Format: PDF
Kopierschutz: 1 - PDF Watermark

E-Book, Englisch, 253 Seiten

Reihe: Advances in Computer Vision and Pattern Recognition

Most biometric systems employed for human recognition require physical contact with, or close proximity to, a cooperative subject. Far more challenging is the ability to reliably recognize individuals at a distance, when viewed from an arbitrary angle under real-world environmental conditions. Gait and face data are the two biometrics that can be most easily captured from a distance using a video camera. This comprehensive and logically organized text/reference addresses the fundamental problems associated with gait and face-based human recognition, from color and infrared video data that are acquired from a distance. It examines both model-free and model-based approaches to gait-based human recognition, including newly developed techniques where the both the model and the data (obtained from multiple cameras) are in 3D. In addition, the work considers new video-based techniques for face profile recognition, and for the super-resolution of facial imagery obtained at different angles. Finally, the book investigates integrated systems that detect and fuse both gait and face biometrics from video data. Topics and features: discusses a framework for human gait analysis based on Gait Energy Image, a spatio-temporal gait representation; evaluates the discriminating power of model-based gait features using Bayesian statistical analysis; examines methods for human recognition using 3D gait biometrics, and for moving-human detection using both color and thermal image sequences; describes approaches for the integration face profile and gait biometrics, and for super-resolution of frontal and side-view face images; introduces an objective non-reference quality evaluation algorithm for super-resolved images; presents performance comparisons between different biometrics and different fusion methods for integrating gait and super-resolved face from video. This unique and authoritative text is an invaluable resource for researchers and graduate students of computer vision, pattern recognition and biometrics. The book will also be of great interest to professional engineers of biometric systems.

Autoren/Hrsg.

Bhanu, Bir

Han, Ju

Inhaltsverzeichnis

1;Preface;5
2;Contents;8
3;List of Figures;13
4;List of Tables;21
5;Introduction to Gait-Based Individual Recognition at a Distance;24
5.1;Introduction;25
5.1.1;Gait-Based Human Recognition;26
5.1.2;Face-Based Human Recognition;26
5.1.3;Key Ideas Described in the Book;27
5.1.4;Organization of the Book;29
6;Gait-Based Individual Recognition at a Distance;32
6.1;Gait Representations in Video;33
6.1.1;Human Motion Analysis and Representations;33
6.1.2;Human Activity and Individual Recognition by Gait;34
6.1.2.1;Human Recognition by Gait;35
6.1.2.1.1;Model-Based Approaches;35
6.1.2.1.2;Model-Free Approaches;35
6.1.2.2;Human Activity Recognition;37
6.1.2.2.1;Model-Based Approaches;37
6.1.2.2.2;Model-Free Approaches;37
6.1.3;Gait Energy Image (GEI) Representation;37
6.1.3.1;Motivation;38
6.1.3.2;Representation Construction;38
6.1.3.3;Relationship with MEI and MHI;38
6.1.3.4;Representation Justification;39
6.1.4;Framework for GEI-Based Recognition;41
6.1.4.1;Silhouette Extraction and Processing;41
6.1.4.2;Feature Extraction;42
6.1.5;Summary;44
6.2;Model-Free Gait-Based Human Recognition in Video;45
6.2.1;Statistical Feature Fusion for Human Recognition by Gait;45
6.2.1.1;Real and Synthetic Gait Templates;46
6.2.1.2;Human Recognition;48
6.2.1.3;Experimental Results;50
6.2.1.3.1;Data and Parameters;50
6.2.1.3.2;Performance Evaluation;52
6.2.2;Human Recognition Based on Environmental Context;53
6.2.2.1;Walking Surface Type Detection;54
6.2.2.2;Classifier Design;57
6.2.2.2.1;Probabilistic Classifier Combination;58
6.2.2.3;Experimental Results;59
6.2.3;View-Insensitive Human Recognition by Gait;60
6.2.3.1;View-Insensitive Gait Templates;60
6.2.3.2;Human Recognition;62
6.2.3.3;Experimental Results;63
6.2.4;Human Repetitive Activity Recognition in Thermal Imagery;65
6.2.4.1;Object Detection in Thermal Infrared Imagery;66
6.2.4.2;Human Repetitive Activity Representation and Recognition;67
6.2.4.3;Experimental Results;68
6.2.5;Human Recognition Under Different Carrying Conditions;70
6.2.5.1;Technical Approach;70
6.2.5.1.1;Gait Energy Image (GEI);70
6.2.5.1.2;Feature Extraction;71
6.2.5.1.3;Co-evolutionary Genetic Programming;72
6.2.5.1.4;Majority Voting;73
6.2.5.2;Experimental Results;73
6.2.5.2.1;Data;73
6.2.5.2.2;Experiments;74
6.2.5.2.3;Classifier Performance Comparison;74
6.2.6;Summary;75
6.3;Discrimination Analysis for Model-Based Gait Recognition;77
6.3.1;Predicting Human Recognition Performance;77
6.3.2;Algorithm Dependent Prediction and Performance Bounds;78
6.3.2.1;Body Part Length Distribution;78
6.3.2.2;Algorithm Dependent Performance Prediction;80
6.3.2.3;Upper Bound on PCR;81
6.3.3;Experimental Results;82
6.3.4;Summary;83
6.4;Model-Based Human Recognition-2D and 3D Gait;85
6.4.1;2D Gait Recognition (3D Model, 2D Data);85
6.4.1.1;3D Human Modeling;86
6.4.1.1.1;Human Kinematic Model;86
6.4.1.1.2;Human Model Parameter Selection;87
6.4.1.1.3;Camera Model and Coordinate Transformation;88
6.4.1.1.3.1;World Coordinate to Camera Coordinate;89
6.4.1.1.3.2;Camera Coordinate to Ideal Image Coordinate;89
6.4.1.1.3.3;Ideal Image Coordinate to Actual Image Coordinate;89
6.4.1.1.3.4;Actual Image Coordinate to Computer Image Coordinate;90
6.4.1.2;Human Recognition from Single Non-calibrated Camera;90
6.4.1.2.1;Silhouette Preprocessing;90
6.4.1.2.2;Matching Between 3D Model and 2D Silhouette;91
6.4.1.2.3;Human Model Parameter Estimation;91
6.4.1.2.3.1;Stationary Parameter Estimation;91
6.4.1.2.3.2;Kinematic Parameter Estimation;92
6.4.1.2.4;Recognition Based on Kinematic and Stationary Features;93
6.4.1.2.4.1;Kinematic and Stationary Feature Classifier;93
6.4.1.2.4.2;Classifier Combination Strategies;93
6.4.1.2.5;Performance Evaluation on Monocular Image Sequences;94
6.4.1.2.5.1;Performance of Stationary Feature Classifier;94
6.4.1.2.5.2;Performance of Kinematic Feature Classifier;95
6.4.1.2.5.3;Performance with Classifier Combination;96
6.4.1.3;Human Recognition from Multiple Calibrated Cameras;96
6.4.1.3.1;Human Model Parameter Selection;96
6.4.1.3.2;Matching Between 3D Human Model and Multiple 2D Silhouettes;97
6.4.1.3.3;Human Model Parameter Initialization and Estimation;97
6.4.1.3.4;Performance Evaluation on Data from Multiple Cameras;98
6.4.2;Gait Recognition in 3D;100
6.4.2.1;Individual Recognition by Gait in 3D;100
6.4.2.2;Related Work;101
6.4.2.3;Technical Approach;103
6.4.2.3.1;3D Human Body Data;103
6.4.2.3.2;3D Human Body Model;104
6.4.2.3.3;Model Fitting;105
6.4.2.3.4;Body Axes;105
6.4.2.3.5;Torso;106
6.4.2.3.6;Arms and Legs;107
6.4.2.3.7;Head and Neck;108
6.4.2.3.8;Gait Reconstruction;108
6.4.2.3.9;Feature Matching;108
6.4.2.4;Experimental Results;109
6.4.2.4.1;Gait Reconstruction;109
6.4.2.4.2;Training and Testing Data;110
6.4.2.4.3;Gait Recognition;112
6.4.3;Summary;114
6.5;Fusion of Color/Infrared Video for Human Detection;115
6.5.1;Related Work;117
6.5.2;Hierarchical Image Registration and Fusion Approach;119
6.5.2.1;Image Transformation Model;120
6.5.2.2;Preliminary Human Silhouette Extraction and Correspondence Initialization;121
6.5.2.3;Automatic Image Registration;122
6.5.2.3.1;Model Parameter Selection;122
6.5.2.3.2;Parameter Estimation Based on Hierarchical Genetic Algorithm;124
6.5.2.4;Sensor Fusion;127
6.5.2.5;Registration of EO/IR Sequences with Multiple Objects;128
6.5.3;Experimental Results;128
6.5.3.1;Image Registration Results;129
6.5.3.2;Sensor Fusion Results;132
6.5.4;Summary;133
7;Face Recognition at a Distance in Video;135
7.1;Super-Resolution of Facial Images in Video at a Distance;136
7.1.1;Closed-Loop Super-Resolution of Face Images in Video;137
7.1.1.1;Related Work;137
7.1.1.2;Technical Approach;138
7.1.1.2.1;Bilinear Basis Images Computation;139
7.1.1.2.2;Pose and Illumination Estimation;139
7.1.1.2.3;Super-Resolution Algorithm;139
7.1.1.3;Experimental Results;141
7.1.1.3.1;Synthetic Data;141
7.1.1.3.2;Real Video;142
7.1.2;Super-Resolution of Facial Images with Expression Changes in Video;143
7.1.2.1;Related Work;144
7.1.2.2;Technical Approach;145
7.1.2.2.1;Tracking of Facial Regions;146
7.1.2.2.2;Local Deformation;147
7.1.2.2.3;Free Form Deformation Formulation;147
7.1.2.2.4;Cost Function;148
7.1.2.2.5;Resolution Aware Local Deformation;148
7.1.2.2.5.1;Deform Local Motion on High Resolution Data;149
7.1.2.2.5.2;Super-Resolution Methodology Requires Sub-pixel Registration;150
7.1.2.2.6;Super-Resolution Algorithm;151
7.1.2.2.7;A Match Measure for Warping Errors;151
7.1.2.3;Experimental Results;151
7.1.2.3.1;Data and Parameters;151
7.1.2.3.2;Results of Resolution Aware FFD;152
7.1.2.3.3;Super-Resolution Results-Global Registration vs. Global + RAIFFD Local Deformation;152
7.1.2.3.3.1;Quantification of Performance;153
7.1.2.3.4;Proposed Approach with Two Different SR Algorithms;155
7.1.3;Constructing Enhanced Side Face Images from Video;156
7.1.3.1;Enhanced Side Face Image (ESFI) Construction;158
7.1.3.2;Technical Approach;158
7.1.3.2.1;Acquiring Moving Head of a Person in Video;158
7.1.3.2.2;Side Face Image Alignment;158
7.1.3.2.2.1;Elastic Registration Method;158
7.1.3.2.2.2;Match Statistic;160
7.1.3.2.3;Resolution Enhancement Algorithm;162
7.1.3.2.3.1;The Imaging Model;162
7.1.3.2.3.2;Algorithm for Resolution Enhancement;163
7.1.3.2.4;Side Face Normalization;163
7.1.4;Summary;167
7.2;Evaluating Quality of Super-Resolved Face Images;168
7.2.1;Image Quality Indices;168
7.2.2;Integrated Image Quality Index;169
7.2.2.1;Gray Scale Based Quality (Qg);171
7.2.2.2;Structure Based Quality (Qe);172
7.2.2.3;Similarity Between Input Images (Qi);173
7.2.2.4;Integrated Quality Measure (Qint);174
7.2.3;Experimental Results for Face Recognition in Video;174
7.2.3.1;Experiment 1: Influence of Pose Variation on the Super-Resolved Face Image;175
7.2.3.2;Experiment 2: Influence of Lighting Variation on the Super-Resolved Face Image;177
7.2.3.3;Experiment 3: Influence of Facial Expression Variation on the Super-Resolved Face Image;178
7.2.3.4;Experiment 4: Influence of the Number of Images Used for Constructing the Super-Resolved Face Image for Face Recognition;179
7.2.3.5;Discussion;182
7.2.4;Summary;183
8;Integrated Face and Gait for Human Recognition at a Distance in Video;184
8.1;Integrating Face Profile and Gait at a Distance;185
8.1.1;Introduction;185
8.1.2;Technical Approach;187
8.1.2.1;High-Resolution Image Construction for Face Profile;187
8.1.2.1.1;The Imaging Model;188
8.1.2.1.2;The Super Resolution Algorithm;189
8.1.2.2;Face Profile Representation and Matching;191
8.1.2.2.1;Face Profile Extraction;192
8.1.2.2.2;Curvature-Based Fiducial Extraction;193
8.1.2.2.3;Profile Matching Using Dynamic Time Warping;194
8.1.2.3;Gait Recognition;196
8.1.2.4;Integrating Face Profile and Gait for Recognition at a Distance;197
8.1.3;Experimental Results;197
8.1.3.1;Face Profile-Based Recognition;197
8.1.3.1.1;Static Face Database;197
8.1.3.1.2;Experimental Results;198
8.1.3.2;Integrating Face Profile With Gait;199
8.1.3.2.1;Video Data;199
8.1.3.2.2;Experimental Results;199
8.1.4;Summary;202
8.2;Match Score Level Fusion of Face and Gait at a Distance;203
8.2.1;Introduction;204
8.2.2;Related Work;205
8.2.3;Technical Approach;206
8.2.3.1;Enhanced Side Face Image Construction;207
8.2.3.2;Gait Energy Image Construction;208
8.2.3.3;Human Recognition Using ESFI and GEI;208
8.2.3.3.1;Feature Learning Using PCA and MDA Combined Method;208
8.2.3.3.2;Recognition by Integrating ESFI and GEI;209
8.2.4;Experimental Results and Performance Analysis;211
8.2.4.1;Experiments and Parameters;211
8.2.4.1.1;Experiment 1;212
8.2.4.1.2;Experiment 2;216
8.2.4.1.3;Experiment 3;218
8.2.4.2;Performance Analysis;219
8.2.4.2.1;Discussion on Experiments;219
8.2.4.2.2;Performance Characterization Statistic Q;222
8.2.5;Summary;224
8.3;Feature Level Fusion of Face and Gait at a Distance;226
8.3.1;Introduction;226
8.3.2;Technical Approach;229
8.3.2.1;Human Identification Using ESFI and GEI;231
8.3.2.1.1;Feature Learning Using PCA;231
8.3.2.1.2;Synthetic Feature Generation and Classification;232
8.3.3;The Related Fusion Schemes;233
8.3.3.1;Fusion at the Match Score Level ;234
8.3.3.2;Fusion at the Feature Level ;235
8.3.4;Experimental Results and Comparisons;235
8.3.4.1;Experiments and Parameters;235
8.3.4.1.1;Experiment 1;239
8.3.4.1.2;Experiment 2;241
8.3.4.2;Discussion on Experiments;247
8.3.5;Summary;249
9;Conclusions for Integrated Gait and Face for Human Recognition at a Distance in Video;250
9.1;Conclusions and Future Work;251
9.1.1;Summary;251
9.1.1.1;Gait-Based Human Recognition at a Distance;251
9.1.1.2;Video-Based Human Recognition at a Distance;252
9.1.1.3;Fusion of Face and Gait for Human Recognition at Distance;253
9.1.2;Future Research Directions;254
10;References;256
11;Index;266