The Techniques Of Content Based Image Retrieval Computer Science Essay

Published: Last Edited:

This essay has been submitted by a student. This is not an example of the work written by our professional essay writers.

This paper gives us an idea of currently available relevance feedback techniques. Content based image retrieval is done by many relevance feedback techniques, here we compare the five techniques. Here we take the experimental data set and retrieve the result and these results says that QM is better than QR and QPM and Log-based relevance feedback technique is gives us reliable result and it take less time to process the given query. Navigation pattern relevance feedback is used for to achieve the effectiveness and efficiency of a Content based image retrieval for a large amount of image database.

Keywords- Query re-weighting, Query expansion, Query point movement, Log-based relevance feedback, Navigation pattern relevance feedback.


Content based image retrieval (CBIR) has received much attention in the last decade, which is motivated by the need to efficiently handle the rapidly growing amount of multimedia data. Content based image retrieval is the technologies that retrieve images from a very large data base by their low level visual features such as color, texture and shape. It covers versatile areas, such as image segmentation, image feature extraction, representation, mapping of features to semantics, storage and indexing, image similarity-distance measurement and retrieval making CBIR system development a challenging task.

Many CBIR systems have been developed, including QBIC [12], Photobook [27], MARS [26], NeTra [22], PicHunter [10], Blobworld [7], VisualSEEK [34], SIMPLIcity [39] and others [2], [6], [9], [14], [17], [24], [32],[38]. Many researchers in information-technology field and leading academic institutions try to develop content based image retrieval system for very large image database. Recently

Researcher focus in CBIR has moved to an interactive

mechanism called Relevance feedback that involves a human

as part of the retrieval process.[9],[4] In this approach, the

retrieval process is interactive. To search for desirable images, a user provides the query image, and the system returns a set of similar images based on the extracted features. In CBIR systems with relevance feedback (RF), a user can mark returned images, which are then fed back into the systems as a new refined query for the next round of retrieval. The process is repeated until the user is satisfied with the query result. In this paper we have discussed about CBIR with Relevance

Feedback model with different relevance feedback strategies

such as query re-weighting(QR), query point movement (QPM), query expansion(QEX) and how they are used in short term learning and long term learning with other techniques to improve performance. The paper is organized as follows: A

review of past studies is described in Section II. In Section III, we describe the general design of CBIR with relevance feedback. Comparisons of various techniques and their results are discussed in Section IV. Finally, we conclude the paper in Section V.

Related Work

Content-based multimedia retrieval does not adapt the query and retrieval model based on the user perceptions of the visual similarity. To overcome this problem a number of relevance feedback techniques are used which refers to a set of approaches learning from an assortment of users' browsing behaviors on image retrieval [3].

Query Re-weighting

In this technique, the obtained results of the system are refined by using one of the well known RF techniques which is QR. QR considers the positive examples as an initial query point at each feedback. After much iteration the contour of the query point becomes close to the region of the user interest [1]. work concentrates on what visual features are important for those images (positive examples) picked up by the users at each feedback. The idea of QR is, if the ith feature fi exists in positive examples frequently, the system assigns the higher degree to fi. QR-like approaches convert image feature vectors to weighted-term vectors in early version of Multimedia Analysis and Retrieval System (MARS). Another interactive approach that allows the user to submit a coarse initial query to refine her/his need via a set of relevance feedbacks. In this work, the feature weights are dynamically updated to connect low-level visual features and high-level human concepts. NNEW, developed by You et al. [18], learns the user's query from positive and negative examples by weighting the important features. In that example, image retrieval is extremely limited due to diverse visual features. Fig. 1 illustrates this limitation. Although the search area is continuously updated by re- weighting the features, some targets could be lost.

Fig. 1 Relevance feedback in generalized QR technique.

Query Point Movement

This technique tries to improve the estimate of the ideal query point by moving it towards positive examples. This concept of example-based query refinement is often found in the information retrieval field as relevance feedback techniques [11],[12]. For example, the Rocchio's formula [13], which is based on the vector space model, is given as follows:

where Qo is the vector for the initial query, Ri is the vector

for relevant document i, Si is the vector for non-relevant document i, n1 is the number of relevant documents, and n2

is the number of non-relevant documents. One of the QPM approaches is the modified version of MARS [9].MARS uses weighted euclidean distance as similarity measures whereas MindReader [6] make use of generalized euclidean distance to look for the targets in well ellipsoids. A single measuring function cannot able to cover all target groups with various visual contents. Also, the modified query point of each feedback probably moves toward the local optimal centroid. It is hard to reach global optimal results through QPM.

Query Expansion

Query expansion consists in adding some synonym or relative words into the query set of original keywords to improve the recall and the precision of information retrieval. Traditional methods of query expansion did not make adequate use of semantic relations between query keywords. They often give bad results on recall and precision. These methods can be improved by using knowledge resources such as ontology and conceptual graph.

As QR and QPM cannot improve the performance of RF, researchers move on to QEX in solution space of RF recently. The straight forward search methods QR and QPM cannot cover user's interest completely as the feature space is broad.

The main idea of this technique is to add to the query a set of keywords which are extracted from the representation of the relations between concepts. In the first step, a conceptual graph is constructed based on LSCOM 5 ontology. In the second step, keywords are extracted by the intersection of connected subgraphs representing each video from the result. Finally, all keywords extracted will be added to the initial query. The features extracted from images are saved in indexes.

Log-based Relevance Feedback

Before formally describing the problem of log-based relevance feedback, we need to systematically organize the log data of users' feedback. Assume a user labels N images in each round of regular relevance feedback, which is called a log session in this paper. Thus, each log session contains N evaluated images that are marked as either "relevant" or "irrelevant." For the convenience of representation, we construct a relevance matrix (R) that includes the relevance judgments from all log sessions.

Fig. 2 shows an example of such a matrix. In this figure, we see that each column of a relevance matrix represents an image example in the image database, and each row represents a log session from the log database. When an image is judged as "relevant" in a log session, the corresponding cell in matrix R is assigned to the value þ1. Similarly, _1 is assigned when an image is judged as "irrelevant." For images that are not judged in a log session, the corresponding cells in R are assigned to zero values.

Based on the above formulation, we now define the logbased

relevance feedback problem. Let us first introduce the following notation:

q: a user query.

Nl: the number of labeled images for every log session.

Nimg: the number of image samples in the image database.

Nlog: the number of log sessions in the log database.

To retrieve the desired images, a user must first present a query q, either by providing a query image or by drawing a sketch picture. Let Z = {z1, z2,…, zNimgg} denote the identity of images in the image database. Let X ={x1,x2,…, xNimg } denote the image database, where each xi is a vector that contains the low-level features of the image zi. Let R ={r1, r2,.., rNlog }T denote the log data in the log database, where each ri contains relevance judgements in the ith log session. Let L = {(z1, y1),(z2, y2), . .(zNl,yNl)} be the collection of labeled images acquired through the online feedback for a user.

Fig. 2 Structure of Log-based Relevance Feedback

Then, the definition of a log-based relevance feedback problem can be given as follows: A log-based relevance feedback problem for image retrieval is to look for a relevance function fq that maps each image sample zi to a real value of relevance degree within 0 and 1,

fq : Z [0,1],

based on the feature representation of images X, the log data of users' feedback R, and the labeled images L acquired from online feedback.According to the above definition, both the low-level features of the image content, i.e., X, and the log data of users' feedback, i.e., R, should be included to determine the relevance function fq. Meanwhile, to reduce the number of iterations of online relevance feedback, a good learning algorithm should require only a small number of labeled image examples from the online relevance feedback, i.e.,| L |.

Navigation Pattern Relevance Feedback

Extracting multimedia data from the large multimedia repository suffer from problem such as redundant browsing and exploration convergence. Whenever the user query the database the resultant data is irrelevant with the user's query and it takes long iterations of feedback to produce the result. The goal is to assist the search strategy in efficiently retrieve the desired images. NPRF integrates the discovered navigation patterns and three RF techniques to achieve

efficient and effective exploration of images as shown in Fig.3. The task is divided into two major operations, offline knowledge discovery and online image retrieval.

Online image retrieval: This part contains two phase. In the first, Initial Query Processing Phase Without considering the feature weight, this phase extracts the visual features from the original query image to find the similar images. Afterward, the good examples (also called positive examples)

picked up by the user are further analyzed at the first feedback (also called iteration 0).In Second, Image Search phase, Behind the search phase, extend the one search point to multiple search points by integrating the navigation patterns and the proposed search algorithm NPRFSearch. In this phase, a new query point at each feedback is generated by the preceding positive examples. Then, the k-nearest images to the new query point can be found by expanding the weighted query. The search procedure does not stop unless the user is satisfied with the retrieval results.

Fig. 3 Work Flow of NPRFSearch

2) Offline Knowledge Discovery: This part contains two phase. In the first, Knowledge Discovery Phase, Learning from users' behaviors in image retrieval can be viewed as one type of knowledge discovery. Consequently, this phase primarily concerns the construction of the navigation model by discovering the implicit navigation patterns from users' browsing behaviors. This navigation model can provide image search with a good support to predict optimal image browsing paths. In the second phase, Data Storage Phase, The databases in this phase can be regarded as the knowledge marts of a knowledge warehouse, which store integrated, time variant, and nonvolatile collection of useful data including images, navigation patterns, log files, and image features. The knowledge warehouse is very helpful to improve the quality of image retrieval.

Result and Discussion

The experimental data is taken from the collection of the Corel image database and the web images. Seven data sets are prepared from different kinds of categories, as shown in Table 1. Each category contains 200 images. Initially QPM is performed to collect the log on the queries. Then, through

knowledge discovery, the navigation patterns are obtained. Next, the knowledge discovered from the navigation patterns can be utilized once the new query is submitted to NPRF.

Indeed, it does need time to gather the usage logs. However, larger the log, longer the collection time and higher the retrieval quality. An alternative way to reduce the whole collection time is to increase the size of collected logs incrementally such that the precision will also be enhanced gradually. All the experiments were implemented in C++, running on a personal computer with Intel Dual Core Xeon 3050 2.13 GHz processor and 1 G MB RAM.


Experimental Data Set

Data set

#of Categories

Categories set



{archit, bus, car, city night, Classical Painting, crop cycle,

Deer Antelope, desert, dog, eagle, flower, grass, group, indoor, lion, model, plane, satellite image, sub Sea, sunset}



Data set 1 + {cartoon, feasts,

owls, surfs, waterfall}



Data set 2 + {beach, masks,

pumpkin, stalactite, tiger}



Data set 3 + {AncestorDinoArt,

bluesky, doors, mountain, sculpt}



Data set 4 + {baseball, basketball, Billiards ball, tennis, volleyball}



Data set 5 + {castle, Fl_Gp, penguin, ship, soccer}



Data set 6 + {BWimage, elephant, planet, Motor GP, sunflower}


The survey of the all relevance feedback technique we can conclude that QR, QPM and QEX are failed at some point when the image database is very large. But the effectiveness of QEX is better than those of QPM and QR. But QEX brings out higher computation cost and more feedbacks in RF. The Log-based Relevance Feedback is better than all above three techniques. NPRF technique integrating the navigation pattern mining and a navigation pattern-based search approach named NPSearch address the long iteration problem of CBIR with RF. NPRF is designed to efficiently optimize the retrieval quality of interactive CBIR.