Integrated Image Analysis and Retrieval
This project is funded by the ETH.
This project investigates the analysis and retrieval of images based on their content in very-large image databases. The retrieval process requires index support, as well as feedback from users to effectively navigate through groups of images. Our activities are in two areas: image indexing/ and image retrieval.
This project investigates the analysis and retrieval of images based on their content in very-large image databases. The retrieval process requires index support, as well as feedback from users to effectively navigate through groups of images. Our activities are in two areas: image indexing/ and image retrieval.
Image indexing: Image indexing can be based on a number of similarity-based search techniques. Examples include:
- Colour or texture distributions within images,
- Pictorial content, and
- Metadata, that is, data about the images: where captured, by what method, with what precessing techniques.
These approaches each lead to different images features with different similarity measures:
- Feature vectors as points in high-dimensional space (e.g. 45 dimensions), similarity measured in Euclidean or Manhattan distance.
- Strings characterising image content, similarity measured in edit distance,
- Partial match over semistructured matadata, similarity based on goodness of match.
The first image-indexing problem we are addressing, therefore, is the that of providing similarity search over image databases based on each/ and combinations of these classes of features. For example, one search may be based purely in metadata. However, with relevance feedback, subsequent searches will be based on both metadata and colour distribution information. However, at least two difficulties arise: firstly, the access structures best-suited to each class of search are different; and secondly, that of combining ranked result lists from different structures.
An attractive approach overcoming these difficulties is to develop a single access structure for each class of similarity search based on signatures. A signature function is provided for each class of similarity search, and a single signature structure is used to search on colour, metadata and pictorial content of images. The advantage, therefore, is that a normalisation function can then be introduced to achieve an overall relevance measure for each image.
The task of searching over signature based access structures for such data is described in the related page on the HERMES project.
Image Retrieval: The image retrieval problem, similar to document retrieval, is not a one-step process but must be based rather on a combination of search and relevance feedback. A performance problem arising in relevance feedback for image search, however, is that transmission of images for relevance assessment is in itself a costly. Furthermore, high latency is unacceptable for interactive browsing.
In this project part, therefore, we are addressing the interaction between network-bandwidth availability and image delivery for relevance assessment. For example, image quality may be reduced in the event that high bandwidth is not available. This is reasonable since an expert user can often asses relevance without the need for the highest quality version of an image.
Development work in this project is being carried out primarily within the framework of our in-house prototype system CONCERT.
Term
36 month
Project Partners:
This project is closely-associated with image retrieval activities within the HERMES project. It is collaborative with image analysis and network specialists from the institutes of Communications Technology and Computer Systems.
Funding
SFr. 96'300.-
This project is funded by ETH.
References
H.-J. Schek, R. Weber: Higher Order Databases and Multimedia Information. In: Proc. of the Swiss/Japan Seminar Advances in Database and Multimedia, Kyoto, Japan, February 2000.
H.-J. Schek, K. Böhm, T. Grabs, U. Röhm, H. Schuldt, R. Weber: Hyperdatabases. In: Proc. of the 1st Int. Conf. on Web Information Systems Engineering (WISE'00), Hong Kong, China, June 2000.
R. Weber, K. Böhm: Trading Quality for Time with Nearest-Neighbor Search. In: Proc. of the 7th Conf. on Extending Database Technology (EDBT 2000), Konstanz, Germany, March 2000.
R. Weber, H.-J. Schek: A Distributed Image-Database Architecture for Efficient Insertion and Retrieval Tasks. In Fifth International Workshop on Multimedia Information Systems (MIS'99), Indian Wells, Palm Springs Desert, California, USA, October 1999.
R. Weber, H.-J. Schek, J. Bollinger, T. Gross: Architecture of a Networked Image Search and Retrieval System. In Eighth International Conference on Information and Knowledge Management, Kansas City, Missouri, USA, November 1999.
R. Weber, P. Zezula: Is similarity search useful for high dimensional spaces?. In International Workshop on Similarity Search (IWOSS'99), Florence, Italy, September 1999.
R. Weber, H.-J. Schek, S. Blott: A Quantitative Analysis and Performance Study for Similarity-Search Methods in High-Dimensional Spaces. In: 24th Int. Conf. on Very Large Databases (VLDB), New York, August 1998.
S. Blott, R. Weber: A Simple Vector Approximation File for Similarity Search in High-Dimensional Vector Spaces. In: Technical report, Esprit Project Hermes (no. 9141), March 1997.
