Gibbon / Liu Introduction to Video Search Engines

1. Auflage 2008
ISBN: 978-3-540-79337-3
Verlag: Springer Berlin Heidelberg
Format: PDF
Kopierschutz: 1 - PDF Watermark

E-Book, Englisch, 276 Seiten

The evolution of technology has set the stage for the rapid growth of the video Web: broadband Internet access is ubiquitous, and streaming media protocols, systems, and encoding standards are mature. In addition to Web video delivery, users can easily contribute content captured on low cost camera phones and other consumer products. The media and entertainment industry no longer views these developments as a threat to their established business practices, but as an opportunity to provide services for more viewers in a wider range of consumption contexts. The emergence of IPTV and mobile video services offers unprecedented access to an ever growing number of broadcast channels and provides the flexibility to deliver new, more personalized video services. Highly capable portable media players allow us to take this personalized content with us, and to consume it even in places where the network does not reach. Video search engines enable users to take advantage of these emerging video resources for a wide variety of applications including entertainment, education and communications. However, the task of information extr- tion from video for retrieval applications is challenging, providing opp- tunities for innovation. This book aims to first describe the current state of video search engine technology and second to inform those with the req- site technical skills of the opportunities to contribute to the development of this field. Today's Web search engines have greatly improved the accessibility and therefore the value of the Web.

David Gibbon joined Bell Laboratories in 1985 and is currently a Lead Member of Technical Staff in the Video and Multimedia Services Research Department at AT&T Labs - Research. His research interests include multimedia processing for searching and browsing of video databases and real-time video processing for communications applications. David has written book chapters and encyclopedia articles as well as numerous technical papers; he has 40 US patent filings and holds 14 US patents in the areas of multimedia indexing, streaming, and video analysis; and he is a member of the ACM, and a senior member of the IEEE. David contributes to IPTV industry standards for metadata and in 2007 he was awarded the AT&T Science and Technology Medal for outstanding technical leadership and innovation in the field of Video and Multimedia Processing and Digital Content Management.Zhu Liu joined AT&T Labs - Research in 2000, and he is currently a Principal Member of Technical Staff in the Video and Multimedia Services Research Department. His research interests include multimedia content processing, multimedia databases, pattern recognition, and machine learning. Zhu holds 7 US patents and he is the inventor of more than 20 pending patents in the areas of multimedia service and content analysis. He has published more than 40 refereed papers in international leading journals and at key conferences in the areas of multimedia. He is a member of ACM and Tau Beta Pi, and a senior member of the IEEE.

Autoren/Hrsg.

Gibbon, David C.

Liu, Zhu

Inhaltsverzeichnis

1;Preface;5
1.1;Who should read this book?;7
1.2;How is this book organized?;7
1.3;Acknowledgements;8
2;Contents;9
3;1 Video Search;16
3.1;1.1 Introduction;16
3.2;1.2 Addressing the Opportunity;17
3.3;1.3 Classification of Web Video Sites;20
3.3.1;1.3.1 Content Originators and Traditional Broadcasters;20
3.3.2;1.3.2 Aggregators;21
3.3.3;1.3.3 Download;21
3.3.4;1.3.4 Sharing;21
3.3.5;1.3.5 Application Specific;22
3.3.6;1.3.6 Other Video Systems;22
3.4;1.4 Classification of Video Sources;23
3.4.1;1.4.1 Webcams / Security;24
3.4.2;1.4.2 Video Telephony / Teleconferencing;24
3.4.3;1.4.3 Industrial / Academic / Medical;24
3.4.4;1.4.4 User Generated Content;25
3.4.5;1.4.5 Public Access and Government (PEG) Content;25
3.4.6;1.4.6 Enterprise Content;25
3.4.7;1.4.7 Rushes, Raw Footage;26
3.4.8;1.4.8 News;26
3.4.9;1.4.9 Advertising;26
3.4.10;1.4.10 Episodic TV Programming;26
3.4.11;1.4.11 Feature Films;27
3.4.12;1.4.12 Content Value;27
3.5;1.5 Challenges of Video Search;28
3.5.1;1.5.1 Acquisition;29
3.5.2;1.5.2 Media File Formats;30
3.5.3;1.5.3 Data Transport;31
3.5.4;1.5.4 Browsing;31
3.5.5;1.5.5 Duplication;32
3.5.6;1.5.6 Ranking and Indexing;32
3.6;1.6 Advantages of Video Search over Text;33
3.6.1;1.6.1 Applications;33
3.6.2;1.6.2 Metadata;34
3.7;1.7 Metadata vs. Content;34
3.7.1;1.7.1 Content-based retrieval;34
3.8;1.8 Conclusion;35
3.9;References;36
4;2 Video Data Sources and Applications;38
4.1;2.1 Introduction;38
4.1.1;2.1.1 Evolution of Digital Media Metadata;38
4.1.2;2.1.2 Consumer Video Metadata;39
4.1.3;2.1.3 Metadata Loss;39
4.1.4;2.1.4 Metadata Standards;40
4.1.5;2.1.5 Dublin Core;41
4.1.6;2.1.6 MPEG-7;42
4.1.7;2.1.7 MPEG-21;42
4.2;2.2 Essential Media Metadata;44
4.2.1;2.2.1 Embed Global Metadata;44
4.2.2;2.2.2 Elementary Metadata;44
4.3;2.3 Metadata for Personal Media Collections;46
4.3.1;2.3.1 Consumer Media Libraries;46
4.3.2;2.3.2 UPnP Forum;48
4.3.3;2.3.3 MP3 ID3;48
4.3.4;2.3.4 3GP / QuickTime / MP4;49
4.3.5;2.3.5 Metadata Services;49
4.3.6;2.3.6 Content Identification;51
4.3.7;2.3.7 Recorded Television;52
4.4;2.4 Media Syndication: RSS Content Description;54
4.4.1;2.4.1 Content Syndication;54
4.4.2;2.4.2 Media Enclosures;54
4.4.3;2.4.3 Podcasts;56
4.4.4;2.4.4 RSS for Content Ingest;57
4.4.5;2.4.5 MediaRSS;58
4.5;2.5 Metadata for Broadcast Television;58
4.5.1;2.5.1 Electronic Programming Guide (EPG);59
4.5.2;2.5.2 Extended Data Service (XDS);61
4.5.3;2.5.3 Program and System Identifier Protocol (PSIP);62
4.6;2.6 Metadata for Video on Demand;62
4.6.1;2.6.1 Introduction;62
4.6.2;2.6.2 Cable Labs;64
4.7;2.7 Production Metadata;65
4.8;2.8 Timed Text Formats;66
4.8.1;2.8.1 Introduction;66
4.8.2;2.8.2 Synchronization Precision and Resolution;67
4.8.3;2.8.3 Transcripts;68
4.8.4;2.8.4 Closed Captions;69
4.8.5;2.8.5 Synchronized Accessible Media Interchange;70
4.8.6;2.8.6 Metadata from Social Sources;70
4.8.7;2.8.7 Metadata Issues;70
4.9;2.9 Conclusion;71
4.10;References;71
5;3 Internet Video;74
5.1;3.1 Introduction;74
5.2;3.2 Digital Video;74
5.2.1;3.2.1 Aspect Ratio;74
5.2.2;3.2.2 Luminance and Chrominance Resolution;76
5.2.3;3.2.3 Video Compression;77
5.3;3.3 Internet Protocol Media Systems;81
5.3.1;3.3.1 Transport;81
5.3.2;3.3.2 Searching VoD vs. Live;82
5.3.3;3.3.3 IPTV;83
5.3.4;3.3.4 Rights Management;85
5.3.5;3.3.5 Redirector Files;85
5.3.6;3.3.6 Layered Encoding;88
5.3.7;3.3.7 Illustrated Audio;88
5.4;3.4 Media Captioning;89
5.5;3.5 Conclusion;90
5.6;References;91
6;4 Video Search Engine Systems;92
6.1;4.1 Introduction;92
6.2;4.2 Content Acquisition;93
6.2.1;4.2.1 Metadata Normalization;93
6.2.2;4.2.2 User Contributed;94
6.2.3;4.2.3 Syndicated Contribution;95
6.2.4;4.2.4 Broadcast Acquisition;96
6.3;4.3 Content Processing;97
6.3.1;4.3.1 Asset Management;97
6.4;4.4 Retrieval;99
6.5;4.5 User Perspectives;100
6.5.1;4.5.1 Interaction States;100
6.5.2;4.5.2 Granularity of Search Results Representation;102
6.5.3;CID1;103
6.6;4.6 Factors Concerning Scalability;103
6.6.1;4.6.1 Introduction;103
6.6.2;4.6.2 Acquisition;104
6.6.3;4.6.3 Processing;104
6.6.4;4.6.4 Storage;105
6.6.5;4.6.5 Retrieval;106
6.7;4.7 Retrieval Interfaces;107
6.8;4.8 Typical System Features;108
6.9;4.9 Conclusion;109
6.10;References;109
7;5 Media Processing;112
7.1;5.1 Introduction;112
7.2;5.2 Feature Extraction;114
7.3;5.3 Media Segmentation;115
7.4;5.4 Clustering, Structure Generation;116
7.5;5.5 Real-Time Processing;118
7.6;5.6 Systems Issues and Architectures;118
7.7;5.7 Conclusion;119
7.8;References;120
8;6 Video Processing;122
8.1;6.1 Introduction;122
8.2;6.2 Shot Boundary Determination;123
8.2.1;6.2.1 Feature Extraction;125
8.2.2;6.2.2 Shot Boundary Detectors;126
8.2.3;6.2.3 Fusion of Detector Results;132
8.2.4;6.2.4 Evaluation Results;132
8.3;6.3 Representative Image Selection;133
8.4;6.4 Face Detection;136
8.5;6.5 Face Recognition;141
8.6;6.6 Video Optical Character Recognition;144
8.7;6.7 Concept Detection;146
8.7.1;6.7.1 Color Feature;148
8.7.2;6.7.2 Texture Feature;148
8.7.3;6.7.3 Edge Feature;150
8.8;6.8 Video Browsing;150
8.9;6.9 Conclusion;155
8.10;References;156
9;7 Audio Processing;160
9.1;7.1 Introduction;160
9.2;7.2 Audio Signal and Its Representation;161
9.3;7.3 Audio Features;163
9.3.1;7.3.1 Frame-Level Features;163
9.3.2;7.3.2 Clip-Level Features;169
9.4;7.4 Audio Segmentation;171
9.4.1;7.4.1 Speaker Segmentation;172
9.4.2;7.4.2 Audio Scene Segmentation;173
9.5;7.5 Audio Content Categorization;175
9.5.1;7.5.1 Speaker Recognition;175
9.5.2;7.5.2 Audio Scene Detection;177
9.5.3;7.5.3 Music Genre Classification;178
9.6;7.6 Speech Recognition;179
9.7;7.7 Audio Query and Browsing Techniques;181
9.7.1;7.7.1 SpeechLogger;182
9.7.2;7.7.2 Query by Example;186
9.8;7.8 Conclusion;187
9.9;References;188
10;8 Text Processing;192
10.1;8.1 Introduction;192
10.2;8.2 Story Segmentation;193
10.2.1;8.2.1 Cue Phrases;193
10.2.2;8.2.2 Cosine Similarity;194
10.2.3;8.2.3 Dynamic Programming;196
10.2.4;8.2.4 Topic Classification;198
10.3;8.3 Named Entity Extraction;198
10.3.1;8.3.1 Rule Based NEE;199
10.3.2;8.3.2 Data Driven NEE;200
10.3.3;8.3.3 NEE Tools;201
10.4;8.4 Part-of-Speech Tagging;202
10.5;8.5 Capitalization;204
10.5.1;8.5.1 Linguistic Processing Architecture;206
10.5.2;8.5.2 Web Document Collection;206
10.5.3;8.5.3 Text Capitalization Algorithm;207
10.6;8.6 Information Retrieval;209
10.6.1;8.6.1 Stemming;209
10.6.2;8.6.2 Term Weighting;210
10.6.3;8.6.3 Ranking;211
10.7;8.7 Text Summarization;212
10.7.1;8.7.1 Keyword Extraction;214
10.8;8.8 Conclusion;216
10.9;References;216
11;9 Multimodal Processing;218
11.1;9.1 Introduction;218
11.2;9.2 Case Studies;220
11.2.1;9.2.1 Closed Caption Alignment;220
11.2.2;9.2.2 Multimodal News Story Segmentation;224
11.2.3;9.2.3 Major Cast Detection;229
11.3;9.3 Conclusion;232
11.4;References;232
12;10 Research Systems;236
12.1;10.1 Introduction;236
12.2;10.2 Academic and Industrial Research;237
12.3;10.3 Early Internet Deployments;241
12.3.1;10.3.1 SpeechBot;241
12.3.2;10.3.2 StreamSage;242
12.3.3;10.3.3 SingingFish;242
12.4;10.4 Selected Commercial Systems;243
12.4.1;10.4.1 Virage and Convera;243
12.4.2;10.4.2 Nexidia (FastTalk);243
12.5;10.5 Resources: Datasets, Evaluations, Conferences;244
12.6;10.6 Media Monitoring Deployments;246
12.7;10.7 Case Study: AT&T MIRACLE;247
12.7.1;10.7.1 Introduction;247
12.7.2;10.7.2 System Architecture;247
12.7.3;10.7.3 Collections;248
12.7.4;10.7.4 Data Organization;250
12.7.5;10.7.5 Acquisition / Ingest;251
12.7.6;10.7.6 Content Processing;253
12.7.7;10.7.7 Real-time processing;254
12.7.8;10.7.8 Query Engine;254
12.7.9;10.7.9 Applications;255
12.7.10;10.7.10 Performance;255
12.8;10.8 Conclusion;257
12.9;References;257
13;11 Current Trends in Video Search;262
13.1;11.1 Introduction;262
13.2;11.2 Video Production;263
13.2.1;11.2.1 Metadata Retention;263
13.2.2;11.2.2 Multiple Distribution Channels;263
13.2.3;11.2.3 Mobisodes and Webisodes;264
13.3;11.3 Video Distribution;264
13.3.1;11.3.1 Streaming Protocols;265
13.3.2;11.3.2 Electronic Sell Through;265
13.3.3;11.3.3 Peer-to-peer Delivery;266
13.3.4;11.3.4 Managed Download;266
13.3.5;11.3.5 Syndication;267
13.4;11.4 The Video Web and User Interaction;267
13.4.1;11.4.1 Web-Based Editing;267
13.4.2;11.4.2 Media Browsing;267
13.4.3;11.4.3 Social Tagging;268
13.4.4;11.4.4 Dynamic Interfaces;268
13.4.5;11.4.5 Video Blogs (vlogs);269
13.4.6;11.4.6 Integrated Collections;269
13.5;11.5 Television Technology and Consumption;269
13.5.1;11.5.1 Proliferation of Channels;270
13.5.2;11.5.2 Live to Time Shifted;270
13.5.3;11.5.3 Mobile Consumption;270
13.6;11.6 Trends in Media Devices;271
13.6.1;11.6.1 Increased Media Capabilities;271
13.6.2;11.6.2 Increasing Accessibility;272
13.6.3;11.6.3 DRM;272
13.6.4;11.6.4 Home Media Systems;272
13.7;11.7 Media Processing Research;272
13.8;11.8 Deployments;275
13.9;11.9 Conclusion;276
13.10;References;276
14;Glossary;280
15;Index;286