Müller / Clough / Deselaers ImageCLEF

Experimental Evaluation in Visual Information Retrieval
1. Auflage 2010
ISBN: 978-3-642-15181-1
Verlag: Springer
Format: PDF
Kopierschutz: 1 - PDF Watermark

E-Book, Englisch, 544 Seiten

Reihe: Computer Science

The pervasive creation and consumption of content, especially visual content, is ingrained into our modern world. We're constantly consuming visual media content, in printed form and in digital form, in work and in leisure pursuits. Like our cave- man forefathers, we use pictures to record things which are of importance to us as memory cues for the future, but nowadays we also use pictures and images to document processes; we use them in engineering, in art, in science, in medicine, in entertainment and we also use images in advertising. Moreover, when images are in digital format, either scanned from an analogue format or more often than not born digital, we can use the power of our computing and networking to exploit images to great effect. Most of the technical problems associated with creating, compressing, storing, transmitting, rendering and protecting image data are already solved. We use - cepted standards and have tremendous infrastructure and the only outstanding ch- lenges, apart from managing the scale issues associated with growth, are to do with locating images. That involves analysing them to determine their content, clas- fying them into related groupings, and searching for images. To overcome these challenges we currently rely on image metadata, the description of the images, - ther captured automatically at creation time or manually added afterwards.

Autoren/Hrsg.

Müller, Henning

Clough, Paul

Deselaers, Thomas

Caputo, Barbara

Inhaltsverzeichnis

1;Foreword;6
2;Preface;8
3;Acknowledgements;10
4;Contents;12
5;List of Contributors;24
6;Introduction;28
6.1;Seven Years of Image Retrieval Evaluation;30
6.1.1;Introduction;30
6.1.2;Evaluation of IR Systems;32
6.1.2.1;IR Test Collections;33
6.1.2.2;Cross--Language Evaluation Forum (CLEF);36
6.1.3;ImageCLEF;36
6.1.3.1;Aim and Objectives;36
6.1.3.2;Tasks and Participants;38
6.1.3.3;Data sets;39
6.1.3.4;Contributions;39
6.1.3.5;Organisational Challenges;41
6.1.4;Conclusions;42
6.1.5;References;43
6.2;Data Sets Created in ImageCLEF;46
6.2.1;Introduction;46
6.2.1.1;Collection Creation;47
6.2.1.2;Requirements and Specification;48
6.2.1.3;Collection Overview;50
6.2.2;Image Collections for Photographic Retrieval;51
6.2.2.1;The St. Andrews Collection of Historic Photographs;51
6.2.2.2;The IAPR TC--12 Database;53
6.2.2.3;The Belga News Agency Photographic Collection;55
6.2.3;Image Collections for Medical Retrieval;56
6.2.3.1;The ImageCLEFmed Teaching Files;57
6.2.3.2;The RSNA Database;61
6.2.4;Automatic Image Annotation and Object Recognition;62
6.2.4.1;The IRMA Database;62
6.2.4.2;The LookThatUp (LTU) Data set;63
6.2.4.3;The PASCAL Object Recognition Database;64
6.2.4.4;The MIR Flickr Image Data Set;65
6.2.5;Image Collections in Other Tasks;65
6.2.5.1;The INEX MM Wikipedia Collection;66
6.2.5.2;The KTH--IDOL2 Database;67
6.2.6;Conclusions;68
6.2.7;References;69
6.3;Creating Realistic Topics for Image Retrieval Evaluation;71
6.3.1;Introduction;71
6.3.2;User Models and Information Sources;74
6.3.2.1;Machine--Oriented Evaluation;74
6.3.2.2;User Models;75
6.3.2.3;Information Sources for Topic Creation;76
6.3.3;Concrete Examples for Generated Visual Topics in Several Domains;79
6.3.3.1;Photographic Retrieval;79
6.3.3.2;Medical Retrieval;80
6.3.4;The Influence of Topics on the Results of Evaluation;81
6.3.4.1;Classifying Topics Into Categories;82
6.3.4.2;Links Between Topics and the Relevance Judgments;83
6.3.4.3;What Can Be Evaluated and What Can Not?;83
6.3.5;Conclusions;84
6.3.6;References;85
6.4;Relevance Judgments for Image Retrieval Evaluation;88
6.4.1;Introduction;88
6.4.2;Overview of Relevance Judgments in Information Retrieval;89
6.4.2.1;Test Collections;89
6.4.2.2;Relevance Judgments;90
6.4.3;Relevance Judging for the ImageCLEF Medical Retrieval Task;97
6.4.3.1;Topics and Collection;97
6.4.3.2;Judges;98
6.4.3.3;Relevance Judgment Systems and the Process of Judging;99
6.4.4;Conclusions and Future Work;103
6.4.5;References;104
6.5;Performance Measures Used in Image Information Retrieval;106
6.5.1;Evaluation Measures Used in ImageCLEF;106
6.5.2;Measures for Retrieval;107
6.5.2.1;Measuring at Fixed Recall;108
6.5.2.2;Measuring at Fixed Rank;110
6.5.2.3;Measures for Diversity;112
6.5.2.4;Collating Two Measures Into One;113
6.5.2.5;Miscellaneous Measures;113
6.5.2.6;Considering Multiple Measures;114
6.5.2.7;Measures for Image Annotation and Concept Detection;115
6.5.3;Use of Measures in ImageCLEF;116
6.5.4;Conclusions;117
6.5.5;References;117
6.6;Fusion Techniques for Combining Textual and Visual Information Retrieval;120
6.6.1;Introduction;120
6.6.1.1;Information Fusion and Orthogonality;122
6.6.2;Methods;123
6.6.3;Results;123
6.6.3.1;Early Fusion Approaches;123
6.6.3.2;Late Fusion Approaches;124
6.6.3.3;Inter--media Feedback with Query Expansion;129
6.6.3.4;Other Approaches;130
6.6.3.5;Overview of the Methods from 2004--2009;130
6.6.4;Justification for the Approaches and Generally Known Problems;130
6.6.5;Conclusions;133
6.6.6;References;133
7;Track Reports;140
7.1;Interactive Image Retrieval;142
7.1.1;Interactive Studies in Information Retrieval;142
7.1.2;iCLEF Experiments on Interactive Image Retrieval;144
7.1.2.1;iCLEF Image Retrieval Experiments: The Latin Square Phase;145
7.1.2.2;iCLEF Experiments with Flickr;148
7.1.2.3;The Target Collection: Flickr;149
7.1.2.4;Annotations;149
7.1.2.5;The Task;150
7.1.2.6;Experiments;152
7.1.3;Task Space, Technology and Research Questions;159
7.1.3.1;Use Cases for Interactive Image Retrieval;159
7.1.3.2;Challenges: Technology and Interaction;160
7.1.4;References;162
7.2;Photographic Image Retrieval;165
7.2.1;Introduction;165
7.2.2;Ad hoc Retrieval of Historic Photographs: ImageCLEF 2003--2005;166
7.2.2.1;Test Collection and Distribution;167
7.2.2.2;Query Topics;168
7.2.2.3;Relevance Judgments and Performance Measures;171
7.2.2.4;Results and Analysis;171
7.2.3;Ad hoc Retrieval of Generic Photographs: ImageCLEFphoto 2006-2007;173
7.2.3.1;Test Collection and Distribution;174
7.2.3.2;Query Topics;175
7.2.3.3;Relevance Judgments and Performance Measures;176
7.2.3.4;Results and Analysis;177
7.2.3.5;Visual Sub--task;178
7.2.4;Ad hoc Retrieval and Result Diversity: ImageCLEFphoto 2008--2009;179
7.2.4.1;Test Collection and Distribution;179
7.2.4.2;Query Topics;180
7.2.4.3;Relevance Judgments and Performance Measures;182
7.2.4.4;Results and Analysis;182
7.2.5;Conclusion and Future Prospects;184
7.2.6;References;185
7.3;The Wikipedia Image Retrieval Task;187
7.3.1;Introduction;187
7.3.2;Task Overview;188
7.3.2.1;Evaluation Objectives;188
7.3.2.2;Wikipedia Image Collection;189
7.3.2.3;Additional Resources;189
7.3.2.4;Topics;190
7.3.2.5;Relevance Assessments;191
7.3.3;Evaluation;193
7.3.3.1;Participants;193
7.3.3.2;Approaches;194
7.3.3.3;Results;199
7.3.4;Discussion;203
7.3.4.1;Best Practices;203
7.3.4.2;Open Issues;204
7.3.5;Conclusions and the Future of the Task;205
7.3.6;References;205
7.4;The Robot Vision Task;208
7.4.1;Introduction;208
7.4.2;The Robot Vision Task at ImageCLEF 2009: Objectives and Overview;210
7.4.2.1;The Robot Vision Task 2009;211
7.4.2.2;Robot Vision 2009: The Database;211
7.4.2.3;Robot Vision 2009: Performance Evaluation;212
7.4.2.4;Robot Vision 2009: Approaches and Results;215
7.4.3;Moving Forward: Robot Vision in 2010;217
7.4.3.1;The Robot Vision Task at ICPR2010;217
7.4.3.2;The Robot Vision Task at ImageCLEF2010;219
7.4.4;Conclusions;220
7.4.5;References;220
7.5;Object and Concept Recognition for Image Retrieval;222
7.5.1;Introduction;222
7.5.2;History of the ImageCLEF Object and Concept Recognition Tasks;223
7.5.2.1;2006: Object Annotation Task;224
7.5.2.2;2007: Object Retrieval Task;225
7.5.2.3;2008: Visual Concept Detection Task;226
7.5.2.4;2009: Visual Concept Detection Task;227
7.5.3;Approaches to Object Recognition;227
7.5.3.1;Descriptors;229
7.5.3.2;Feature Post--processing and Codebook Generation;230
7.5.3.3;Classifier;230
7.5.3.4;Post--Processing;231
7.5.4;Results;231
7.5.4.1;2006: Object Annotation Task;232
7.5.4.2;2007: Object Retrieval Task;232
7.5.4.3;2008: Visual Concept Detection Task;233
7.5.4.4;2009: Visual Concept Detection Task;234
7.5.4.5;Evolution of Concept Detection Performance;236
7.5.4.6;Discussion;237
7.5.5;Combinations with the Photo Retrieval Task;238
7.5.6;Conclusion;238
7.5.7;References;239
7.6;The Medical Image Classification Task;243
7.6.1;Introduction;243
7.6.2;History of ImageCLEF Medical Annotation;244
7.6.2.1;The Aim of the Challenge;244
7.6.2.2;The Database;245
7.6.2.3;Error Evaluation;249
7.6.3;Approaches to Medical Image Annotation;251
7.6.3.1;Image Representation;252
7.6.3.2;Classification Methods;252
7.6.3.3;Hierarchy;253
7.6.3.4;Unbalanced Class Distribution;253
7.6.4;Results;253
7.6.5;Conclusion;257
7.6.6;References;259
7.7;The Medical Image Retrieval Task;261
7.7.1;Introduction;261
7.7.2;Participation in the Medical Retrieval Task;262
7.7.3;Development of Databases and Tasks over the Years;264
7.7.3.1;2004;264
7.7.3.2;2005--2007;265
7.7.3.3;2008--2009;269
7.7.4;Evolution of Techniques Used by the Participants;271
7.7.4.1;Visual Retrieval;272
7.7.4.2;Textual Retrieval;272
7.7.4.3;Combining Visual and Textual Retrieval;273
7.7.4.4;Case--Based Retrieval Topics;273
7.7.5;Results;273
7.7.5.1;Visual Retrieval;274
7.7.5.2;Textual Retrieval;274
7.7.5.3;Mixed Retrieval;275
7.7.5.4;Relevance Feedback and Manual Query Reformulation;275
7.7.6;Main Lessons Learned;275
7.7.7;Conclusions;277
7.7.8;References;277
8;Participant reports;280
8.1;Expansion and Re--ranking Approaches for Multimodal Image Retrieval using Text--based Methods;282
8.1.1;Introduction;283
8.1.2;Integrated Retrieval Model;284
8.1.2.1;Handling Multi--modality in the Vector Space Model;285
8.1.3;Document and Query Expansion;286
8.1.4;Re--ranking;288
8.1.4.1;Level 1: Narrowing-down and Re-indexing;290
8.1.4.2;Level 2: Cover Coefficient Based Re--ranking;290
8.1.5;Results;292
8.1.6;Conclusions;294
8.1.7;References;295
8.2;Revisiting Sub--topic Retrieval in the ImageCLEF 2009 Photo Retrieval Task;297
8.2.1;Introduction;298
8.2.2;Background and Related Work;300
8.2.2.1;Sub--topic Retrieval;300
8.2.2.2;The Probability Ranking Principle;302
8.2.2.3;Beyond Independent Relevance;302
8.2.3;Document Clustering and Inter--Cluster Document Selection;304
8.2.3.1;Re--examining Document Clustering Techniques;304
8.2.3.2;Clustering for Sub--topic Retrieval;305
8.2.4;Empirical Study;307
8.2.5;Results;310
8.2.6;Conclusions;311
8.2.7;References;313
8.3;Knowledge Integration using Textual Information for Improving ImageCLEF Collections;315
8.3.1;Introduction;315
8.3.2;System Description;317
8.3.2.1;Photo Retrieval System;317
8.3.2.2;Medical Retrieval System;318
8.3.3;Photo Task;318
8.3.3.1;Using Several IR and a Voting System;321
8.3.3.2;Filtering;322
8.3.3.3;Clustering;325
8.3.4;The Medical Task;326
8.3.4.1;Metadata Selection using Information Gain;326
8.3.4.2;Expanding with Ontologies;328
8.3.4.3;Fusion of Visual and Textual Lists;331
8.3.5;Conclusion and Further Work;331
8.3.6;References;333
8.4;Leveraging Image, Text and Cross--media Similarities for Diversity--focused Multimedia Retrieval;334
8.4.1;Introduction;334
8.4.2;Content--Based Image Retrieval;336
8.4.2.1;Fisher Vector Representation of Images;337
8.4.2.2;Image Retrieval at ImageCLEF Photo;339
8.4.3;Text Representation and Retrieval;340
8.4.3.1;Language Models;340
8.4.3.2;Text Enrichment at ImageCLEF Photo;341
8.4.4;Text--Image Information Fusion;345
8.4.4.1;Cross--Media Similarities;346
8.4.4.2;Cross--Media Retrieval at ImageCLEF Photo;348
8.4.5;Diversity--focused Multimedia Retrieval;351
8.4.5.1;Re--ranking Top--Listed Documents to Promote Diversity;352
8.4.5.2;Diversity--focused Retrieval at ImageCLEF Photo;355
8.4.6;Conclusion;358
8.4.7;References;359
8.5;University of Amsterdam at the Visual Concept Detection and Annotation Tasks;362
8.5.1;Introduction;362
8.5.2;Concept Detection Pipeline;363
8.5.2.1;Point Sampling Strategy;364
8.5.2.2;Color Descriptor Extraction;365
8.5.2.3;Bag--of--Words model;366
8.5.2.4;Machine Learning;367
8.5.3;Experiments;368
8.5.3.1;Spatial Pyramid Levels;368
8.5.3.2;Point Sampling Strategies and Color Descriptors;369
8.5.3.3;Combinations of Sampling Strategies and Descriptors;370
8.5.3.4;Discussion;372
8.5.4;ImageCLEF 2009;372
8.5.4.1;Evaluation Per Image;374
8.5.4.2;Conclusion;374
8.5.5;ImageCLEF@ICPR 2010;375
8.5.6;Conclusion;375
8.5.7;References;376
8.6;Intermedia Conceptual Indexing;378
8.6.1;Introduction;378
8.6.2;Conceptual Indexing;380
8.6.2.1;Concept Usage and Definition in IR;380
8.6.2.2;Concept Mapping to Text;381
8.6.2.3;Mapping Steps;382
8.6.2.4;IR Models Using Concepts;385
8.6.2.5;Experiments using the ImageCLEF Collection;386
8.6.3;Image Indexing using a Visual Ontology;388
8.6.3.1;Image Indexing Based on VisMed Terms;389
8.6.3.2;FlexiTile Matching;392
8.6.3.3;Medical Image Retrieval Using VisMed Terms;393
8.6.3.4;Spatial Visual Queries;394
8.6.4;Multimedia and Intermedia Indexing;395
8.6.5;Conclusions;397
8.6.6;References;398
8.7;Conceptual Indexing Contribution to ImageCLEF Medical Retrieval Tasks;400
8.7.1;Introduction;401
8.7.2;Semantic Indexing Using Ontologies;401
8.7.3;Conceptual Indexing;402
8.7.3.1;Language Models for Concepts;402
8.7.3.2;Concept Detection;403
8.7.3.3;Concept Evaluation Using ImageCLEFmed 2005--07;404
8.7.4;From Concepts to Graphs;405
8.7.4.1;A Language Model for Graphs;405
8.7.4.2;Graph Detection;406
8.7.4.3;Graph Results on ImageCLEFmed 2005--07;407
8.7.5;Mixing Concept Sources;407
8.7.5.1;Query Fusion;408
8.7.5.2;Document Model Fusion;408
8.7.5.3;Joint Decomposition;409
8.7.5.4;Results on ImageCLEFmed 2005--07;411
8.7.6;Adding Pseudo--Feedback;412
8.7.6.1;Pseudo--Relevance Feedback Model;412
8.7.6.2;Results;413
8.7.7;Conclusions;414
8.7.8;References;414
8.8;Improving Early Precision in the ImageCLEF Medical Retrieval Task;416
8.8.1;Introduction;416
8.8.1.1;What is Early Precision?;417
8.8.1.2;Why Improve Early Precision?;418
8.8.2;ImageCLEF;418
8.8.3;Our System;419
8.8.3.1;User Interface;419
8.8.3.2;Image Database;420
8.8.3.3;Query Parsing and Indexing;421
8.8.4;Improving Precision;422
8.8.4.1;Modality Filtration;422
8.8.4.2;Using Modality Information for Retrieval;425
8.8.4.3;Using Interactive Retrieval;427
8.8.5;Conclusions;430
8.8.6;References;431
8.9;Lung Nodule Detection;433
8.9.1;Introduction;433
8.9.1.1;Lung Cancer --- Clinical Motivation;434
8.9.1.2;Computer--Aided Detection of Lung Nodules;436
8.9.1.3;Ground Truth for Lesions;437
8.9.2;Review of Existing Techniques;438
8.9.2.1;Gray--Level Threshold;439
8.9.2.2;Template Matching;439
8.9.2.3;Spherical Enhancing Filters;440
8.9.3;Description of Siemens LungCAD System;441
8.9.3.1;Lung Segmentation;441
8.9.3.2;Candidate Generation;441
8.9.3.3;Feature Extraction;442
8.9.4;Classification;443
8.9.4.1;Multiple Instance Learning;443
8.9.4.2;Exploiting Domain Knowledge in Data--Driven Training--Gated Classifiers;444
8.9.4.3;Ground Truth Creation: Learning from Multiple Experts;445
8.9.5;ImageCLEF Challenge;446
8.9.5.1;Materials and Methods;446
8.9.5.2;Results;447
8.9.6;Discussion and Conclusions;448
8.9.6.1;Clinical Impact;448
8.9.6.2;Future Extensions of CAD;450
8.9.7;References;451
8.10;Medical Image Classification at Tel Aviv and Bar Ilan Universities;453
8.10.1;Introduction;453
8.10.1.1;Visual Words in Medical Archives;454
8.10.2;The Proposed TAU--BIU Classification System Based on a Dictionary of Visual--Words;455
8.10.2.1;Patch Extraction;456
8.10.2.2;Feature Space Description;456
8.10.2.3;Quantization;457
8.10.2.4;From an Input Image to a Representative Histogram;458
8.10.2.5;Classification;459
8.10.3;Experiments and Results;460
8.10.3.1;Sensitivity Analysis;462
8.10.3.2;Optimizing the Classifier;464
8.10.3.3;Classification Results;467
8.10.4;Discussion;468
8.10.5;References;469
8.11;Idiap on Medical Image Classification;470
8.11.1;Introduction;470
8.11.2;Multiple Cues for Image Annotation;471
8.11.2.1;High--Level Integration;472
8.11.2.2;Mid--Level Integration;473
8.11.2.3;Low--Level Integration;473
8.11.3;Exploiting the Hierarchical Structure of Data: Confidence Based Opinion Fusion;474
8.11.4;Facing the Class Imbalance Problem: Virtual Examples;475
8.11.5;Experiments;475
8.11.5.1;Features;475
8.11.5.2;Classifier;478
8.11.5.3;Experimental Set--up and Results;479
8.11.6;Conclusions;480
8.11.7;References;481
9;External views;483
9.1;Press Association Images --- Image Retrieval Challenges;485
9.1.1;Press Association Images --- A Brief History;485
9.1.1.1;The Press Association;485
9.1.1.2;Images at the Press Association;487
9.1.2;User Search Behaviour;488
9.1.2.1;Types of Users;488
9.1.2.2;Types of Search;489
9.1.2.3;Challenges;490
9.1.3;Semantic Web for Multimedia Applications;491
9.1.3.1;Introduction to the Semantic Web;491
9.1.3.2;Success Stories and Research Areas;491
9.1.3.3;The Semantic Web Project at Press Association Images;493
9.1.4;Utilizing Semantic Web Technologies for Improving User Experience in Image Browsing;494
9.1.4.1;PA Data set: Linking to the Linked Data Cloud;494
9.1.4.2;Information Extraction and Semantic Annotation;496
9.1.5;Conclusions and Future Work;497
9.1.6;References;497
9.2;Image Retrieval in a Commercial Setting;499
9.2.1;Introduction;499
9.2.2;Evaluating Large Scale Image Search Systems;502
9.2.3;Query Logs and Click Data;503
9.2.4;Background Information on Image Search;506
9.2.5;Multilayer Perceptron;507
9.2.6;Click Data;509
9.2.7;Data Representation;511
9.2.7.1;Textual Features;511
9.2.7.2;Visual Features;513
9.2.8;Evaluation and Results;514
9.2.8.1;Analysis of Features;516
9.2.9;Discussion of Results;518
9.2.10;Looking Ahead;519
9.2.11;References;520
9.3;An Overview of Evaluation Campaigns in Multimedia Retrieval;522
9.3.1;Introduction;522
9.3.2;ImageCLEF in Multimedia IR (MIR);524
9.3.2.1;INEX XML Multimedia Track;525
9.3.2.2;MIREX;526
9.3.2.3;GeoCLEF;526
9.3.2.4;TRECVid;527
9.3.2.5;VideOlympics;529
9.3.2.6;PASCAL Visual Object Classes (VOC) Challenge;529
9.3.2.7;MediaEval and VideoCLEF;530
9.3.2.8;Past Benchmarking Evaluation Campaigns;531
9.3.2.9;Comparison with ImageCLEF;532
9.3.3;Utility of Evaluation Conferences;533
9.3.4;Impact and Evolution of Metrics;534
9.3.5;Conclusions;536
9.3.6;References;537
10;Glossary;541
11;Index;546