Analyzed Different Image Retrieval Systems Computer Science Essay

Published: Last Edited:

This essay has been submitted by a student. This is not an example of the work written by our professional essay writers.

In this paper we have reviewed and analyzed different image retrieval systems. The purpose of this survey however, is to provide an overview of the functionality of temporary image retrieval systems in terms of technical aspects: querying, relevance feedback, features, matching measures, indexing data structures, and result presentation.We have reviewed different techniques like text based retrieval ,content based retrieval, image annotation to get images captured by digital camera. The classification techniques such as k-KNN,SVM,Decision stump, Manifold Ranking, Hash Encoding Algorithm followed by a suitable relevant feedback model via cross domain learning , GMI-SVM , Laplacian Regularized Least Squares(LapRLS), Search Result Clustering(SRC)Algorithm , Biased Discriminative Euclidean embedding (BDEE)to refine the image retrieval result of consumer photos. After thorough study, this review also claims that most systems uses low level features and only few uses high level semantically meaningful features and the image retrieval results affect due to this semantic gap . The semantic gap is often regarded as a major problem in the field of image retrieval research. The comparative chart presents the details of different image retrieval system and addresses the features to be considered for designing any image retrieval utility.

Keywords- Text based retrieval, Content based retrieval, Image annotation, Classifiers, Relevant feedback (Query refinement strategies)


There is lots of increasing interest in the world of digital photography. Now a day's every person has a digital camera or mobile. Every person has rights to capture photos of beautiful nature and surrounding. These captured images are not consisting of semantic concepts such as proper name like web images. Person may be put a proper name to the captured image or may not be and these images are organized in folders without providing indexing. So we remove the difficulty of retrieving images from personal collections different image retrieval techniques are present.

How to retrieve images?

And also with the ever-growing number of images on the Internet, retrieving relevant images from a large collection of database images has become an important research topic.

It is well known that the major problem in image retrieval is the semantic gap between the low-level features (color, texture, shape, etc.) and the high-level semantic concepts. So it is very important question , How to minimize "semantic gap" as shown in figure 1?

Fig 1: Semantic gap problem

Most of Proposed systems use web image database and classifiers to satisfy users demand for proper relevant image retrieval. Some of the systems used popular Wordnet onthology for automatically retrieved images which are relevant and irrelevant using inverted file method.

Image annotation as the application of computer vision for retrieving images is also known as automatic image tagging or linguistic indexing. Using tagging or annotation method computer system directly assigns captioning or keywords to a digital image [5]. It is commonly used to classify images according to the high-level semantic concepts. It is generally used as an intermediate stage for TBIR image retrieval and must be performed before it because the semantic concepts are analogous to the textual terms that describe document contents[9].When a input text given by user are not present in the current set of vocabularies then user needs to perform another annotation to consumer photos.


Let us consider the basic idea of image retrieval.

General goal of image retrieval system are:

- That is able to process natural language query.

- That is able to search among annoted and non-annoted images.

- That takes into account human visual perception.

- That processes various features (color, texture, shapes) .

Generally most of current image retrieval systems use low-level features such as color, texture and shape because they are extracted by a machine automatically as shown in Table 1.

Table 1: Overview of commonly used features in IR[7]


Histogram,Color co-occurrence histograms


Segmentation & Contour extraction followed by: counter matching, moments, template matching


Directionality,periodicity,randomness,Fourier domain characteristics, random fields


Wavelet coefficient,eigenimages,edge-map of user made sketch, image context vectors


1] Textual query-based consumer photo retrieval system:

In this paper Yiming Liu, Dong Xu, Member, IEEE, Ivor Wai-Hung Tsang, and Jiebo Luo, proposes a novel methodology for Textual query-based consumer photo retrieval system as shown in figure 3.

The process can be done as:

a) They introduce how to retrieve consumer photo considering millions of web images with their rich textual descriptions.

b) They perform integration of large database and Wordnet to get relevant and irrelevant images based on textual query. After that apply classification techniques such as kNN, SVM, Decision stumps.

c) To refine the retrieval result of personal photos consisting of feature distribution may differ in web images and personal photos; we propose cross-domain approach.


1) Images can be retrieved without using image annotation process.

2) Framework is efficiently used for large scale consumer photo retrieval.

2] Bag-based ranking:

In this paper, Lixin Duan,Wen Li,Ivor Wai-Hang Tsang and Dong Xu proposes a novel methodology for improving web image search by bag based reranking[2].The proposed methodology used text query to get relevant images and then performed reranking using visual features.

The process is done as follows:

a) Combining of both visual and textual features, they form cluster of relevant images. Each cluster can be considering as a "bag" and the images present in bag are treat as "instances," then apply multi-instance (MI) learning problem. b) They use mi-SVM as MI learning method so that can be readily incorporated into bag-based reranking framework. While observing we know that a positive bag contains certain portion of positive instances and negative bag may also consist of positive instances. c) Finally use the generalized MI learning method.

c) GMI-SVM was developed to enhance retrieval performance observing the ambiguities of instances present in positive and negative bags. A bag ranking method was proposed to acquire bag annotations for GMI learning so that they performed ranking to all the bags according to the defined bag ranking score.


1) The automatic bag annotation process achieves best performances as compared with existing methods of image reranking.

2) GMI-SVM can achieve better performances.


1) Labels of relevant training images are quite noisy so the constraints on positive bags may not always be satisfied in this application.

3] Manifold- Ranking Algorithm:

In this paper, J. He, M. Li, H. Zhang, H. Tong, and C. Zhang proposes a novel methodology Manifold-

Ranking Based Image Retrieval [3]. The proposed methodology is based on the following steps:

1) They propose a novel transductive learning framework for image retrieval based on a manifold ranking algorithm- here first weighted graph is formed using kNN approach and assign a positive ranking score to each query and zero to remaining points.

2. Then design and investigate different schemes for utilizing the positive and combination of positive ,negative relevance feedback to improve the retrieval result

3. Finally use active learning methods to speed up the convergence to the query concepts.


1) Processing time can be greatly reduced.

2) It reduced scale of weighted graph to form a small graph.


1) It degrades the performance of Relevance feedback.

4] Tag based Image Retrieval:

In this paper Lin Chen, Dong Xu, Ivor W. Tsang, Jiebo Luo Tag-based Image Retrieval Improved by Augmented Features and Group-based Refinement [4] In this paper, they propose a new tag-based image retrieval framework to improve the retrieval performance of a group of related personal images captured by the same user within a short period of an event by considering millions of training web images and their associated rich textual descriptions.

a) For any given query tag the inverted file method is employed to automatically determine the relevant and irrelevant training web images that are associated with the query tag.

b) Using these relevant and irrelevant web images as positive and negative training data respectively, they propose a new classification method called SVM with Augmented Features (AFSVM) to learn an adapted classifier by leveraging the pre-learned SVM classifiers of popular tags that are associated with a large number of relevant training web images.

c) For refinement process, they propose to use the Laplacian Regularized Least Squares (LapRLS) method to further refine the relevance scores of test photos by utilizing the visual similarity of the images within the group.



1) The technique captures the geometry of the data points in the high-dimensional space.


5] Biased Discriminative Euclidean Embedding (BDEE)

In this paper Wei Bian and Dacheng Tao presents Biased Discriminate Euclidean Embedding for

Content-Based Image Retrieval has represented images by low-level visual features. Now a day's very popular image retrieval technique is the Content-based image retrieval (CBIR) which used visual information. In this they must have to give query as an example instead of text query. It is also known as query by image content

[12]. The retrieval process consist of the contents of the image such as textures, shapes,colors and other information of image itself.[3].They have designed a mapping to select the effective subspace from for separating positive samples from negative samples based on a number of observations. They have proposed the Biased Discriminative Euclidean Embedding (BDEE) which parameterizes samples in the original high-dimensional ambient space to discover the intrinsic coordinate of image low-level visual features.


1) It preserves both the intraclass geometry and interclass discrimination

2) It is superior to the popular relevance feedback dimensionality reduction algorithms.

3) Its extension considers the unlabelled samples.

6]Support Vector Machine Active Learning for Image Retrieval:

In this paper S. Tong and E. Chang presents Support Vector Machine Active Learning for Image Retrieval Mostly CBIR systems returns semantically relevant images to the user's query image. So depending upon the application the number of techniques present in CBIR varies. But result images should all share common elements with the provided example. However as person's point of view, it is more convenient and natural for a user to retrieve images using a query as text. The early relevance feedback method directly adjusts weights of various features. SVM-based relevance feedback methods were proposed [3][6]. Relevance feedback can be proposed in CBIR systems to recover the semantic gap. In RF search results will be improved or refine the results based on whether the results are related not related or neutral to search query then repeating the search with the new information.SVM-based relevance feedback methods were proposed [3][6].


2) It degrades the retrieval performance of the techniques considering limited number of feedback images.

7] Auto-annotation:

In this paper Xin-Jing Wang, Lei Zhang, Feng Jing, Wei-Ying Ma presents AnnoSearch, a novel way to annotate images using search and data mining technologies.

a) In this approach at least one correct keyword is compulsory to enable text-based search for semantically similar images.

b) Then to retrieve visually similar images content-based search is performed. Finally annotations

performed from the descriptions (titles, URLs and surrounding texts) of these images.

c) For better efficiency and significantly speed up the content-based search process used high dimensional visual features which are mapped to hash codes.


1) Annotation can be performed with unlimited vocabulary which is impossible for all existing approaches





Classifiers and

Query Refinement strategies

Database used






Inverted Index Using Wordnet, kNN,SVM,Decision Stumps,Relevance Feedback with CDRR

Training Data-1.3million photos forum Photosig,Test data-Kodak Photo NUS-WIDE


Kodak dataset


NUS-WIDE datasets

Precision 13.5 %



Inverted File Method,

k-means Clustering method,GMI-SVM

NUs_WIDE dataset Flickr Images

Mean Average Precisions (MAPs )

GMI-SVM =62.4%



Manifold Ranking Algorithm, kNN, Relevance feedback with active learning methods

Corel Image Gallery

Precision and


Precision vs. Scope


Recall vs. Scope


were plotted



Inverted File Method AFSVM,Laplacian Regularized Least Squares(LapRLS)

Training Data-1.3million photos forum Photosig,

Test data-Kodak Photo, Flickr Photo

Mean Average Precisions (MAPs %)

AFSVM 11.7





Biased Discriminative Eclidean embedding(BDEE)


Image Gallery


And standard deviation

Average Precision

= 0.32

for 9 RF iterations



SVM Active Learning

Core1 Image CDs and the Internet.


Five rounds of querying it attains 99%,84% and 76% accuracy on the top-70 results





Hash Encoding Algorithm,Search Result Clustering(SRC) Algorithm

2.4 million photos forum Photosig







In this paper we have reviewed and analyzed different methods to retrieve images capture by digital camera or mobile which do not have high level semantics concept. We have reviewed different techniques like text based retrieval based on wordnet,,SVM,kNN, auto-annotation by hashmap,clustering , Augmented Features and Group-based Refinement approach, Manifold- Ranking Algorithm ,bag-based reranking method for efficient retrieve images, Biased Discriminate Euclidean Embedding for Content-Based Image Retrieval along with relevance feedback approach. As considering results of different methods, we conclude that for better retrieval performance we must used the techniques to increase values of Precision-Recall, Accuracy and Standard deviation.


[1] Textual Query of Personal Photos Facilitated by Large-Scale Web Data Yiming Liu, Dong Xu, Member, IEEE, Ivor Wai-Hung Tsang, and Jiebo Luo, Fellow, IEEE

[2] Improving Web Image Search by Bag-Based Reranking Lixin Duan, Wen Li, Ivor Wai-Hung Tsang, and Dong Xu, Member, IEEE

[3] J. He, M. Li, H. Zhang, H. Tong, and C. Zhang, "Manifold-Ranking Based Image Retrieval," Proc. ACM Conf. Multimedia, 2004

[4] L. Chen, D. Xu, I.W. Tsang, and J. Luo,"Tag-Based Web Photo Retrieval Improved by Batch Mode Re-

Tagging," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2010.


[6]Lin Chen, Dong Xu, Ivor W. Tsang, Jiebo Luo Tag-based Image Retrieval Improved by Augmented Features and Group-based Refinement

[7] Biased Discriminant Euclidean Embedding for Content-Based Image Retrieval Wei Bian and Dacheng Tao, Member, IEEE

[8] Y. Rui, T.S. Huang, and S. Mehrotra, "Content-Based Image Retrieval with Relevance Feedback in

Mars,"Proc. IEEE Int Conf. Image Processing, 1997.

[9] S. Tong and E. Chang, "Support Vector Machine Active Learning for Image Retrieval," Proc. ACM Conf.

Multimedia, 2001.

[10] AnnoSearch: Image Auto-Annotation by Search Xin-Jing Wang, Lei Zhang, Feng Jing, Wei-Ying Ma

[11] New Methods for Image Retrieval Zoran Peˇcenovi´c, Minh Do, Serge Ayer, Martin Vetterli Laboratory for Audio-Visual Communications, Swiss Federal Institute of Technology (EPFL) CH-1015 Lausanne, Switzerland