Supervised Image Search Re Ranking Computer Science Essay


This essay has been submitted by a student. This is not an example of the work written by our professional essay writers.

Image search methods usually fail to capture the user's intention when the query term is ambiguous. It gives unsatisfactory result. Therefore, reranking with user interactions is highly demanded to effectively improve the search performance. The essential problem is how to identify the user's intention effectively. To complete this goal, this paper presents a structural information based active sample selection strategy to reduce the user's labeling efforts. Furthermore, to localize the user's intention in the visual feature space, a novel local-global discriminative dimension reduction algorithm is proposed. In this algorithm, a submanifold is learned by transferring the local geometry and the discriminative information from the labeled images to the whole (global) image database.


semi supervised image search, structural information (SInfo) based active sample selection, local-global discriminative (LGD) dimension reduction.


Currently most of the image search image usually fails to capture user's satisfaction when the query term is ambiguous because they are built for "query by keyword" scenario, which is also known as text-based image search technique. That is search engine return the corresponding images by processing its file name, surrounding text, URL etc. Today's commercial Internet scale image search engines use only text information. Users type keywords in the hope of finding a certain type of images. The search engine returns thousands of images ranked by the text keywords extracted from the surrounding text. However, many of returned images are noisy, disorganized, or irrelevant. Reranking of image search is to refine the text based search result by using example image by user. It will produce much more relevant image in the result.

Image search engine can categorized into three based on how image are indexed, they are text-based index, image-based index and hybrid form of text-based and image based index. The text-based index representation of image includes filename, caption, surrounding text that displays the image. The second one is image-based index; the image is represented in visual features such as color, texture, and shape. The textual information is insufficient for semantic image retrieval; a natural recourse is the visual information. When the query term is ambiguous, the previous methods do not give better result. At that time the developer need the user interaction to identify his/her intention. Reranking the text-based search result with user's interaction is named as semi supervised image search reranking. This simplified

semi supervised image search reranking is in [2].In this paper, the system re-rank the test-based search result in an interactive

manner. After query by keyword, user can label image as query image then re-rank all returned images according to the query image. The query image is relevant image of query key word.

The semantic space is user-driven, according to their different intentions but with identical query keywords. That is a user type the query word e.g., "mouse" then the search engine returns corresponding images by processing the associated textual information. The same query word prefers an animal mouse image and computer peripheral.

Recently a dozen of image reranking method [2],[3],[4],[5],[6] have been proposed to exploit the usage of the visual similarity search. The paper intent search[2] propose a fast and effective online image search re-ranking algorithm based on one query image only without online training. Here visual similarity to re-rank the text-based results. The proposed Adaptive Similarity is motivated by the idea that a user always has a specific intention when submitting a query image. In this paper [3] a novel and generic video/image reranking algorithm IB reranking , which reorder result from text only searches by discovering the salient visual patterns of relevant and irrelevant shots from the approximate relevance provided by text result. Pseudo-relevance feedback (PRF) is one such tool which has been shown to improve upon simple text search results in both text and video retrieval. PRF is the top-ranking documents are used to rerank the retrieved documents assuming that a significant fraction of top-ranked documents will be relevant. In this paper [4], an intuitive graph-model based method for content-based image retrieval (CBIR). CBIR is one of the methods for searching image from data set. In which relevant images are selected based on the content of sample image. The image-ranking problem into the task of identifying "authority" nodes on an inferred visual similarity graph and propose an algorithm to analyze the visual link structure that can be created among a group of images. Through an iterative procedure based on the PageRank computation, a numerical weight is assigned to each image; this measures its relative importance to the other images being considered. The aim of this paper [5] is to improve web image retrieval by reordering the images retrieved from an image search engine. The re-ranking process is based on relevance model, which is probabilistic model that evaluate the relevance of the HTML (hyper text markup language) document linking to the image and assign a probability of relevance. Instead of accepting the result from a web image search engine, the image rank list as well as associated HTML document is fed to a re-ranking process. Lightweight reranking method [6] that compares each result not only to the other query results but also to an external, contrastive class of items.

In semi supervised reranking, the main problem is how to capture the user's intention, i.e., to distinguish relevant images from irrelevant images from irrelevant images. To complete this goal, this paper presents a structural information based sample selection strategy to reduce the user's labeling effort and to localize the user's intention in a visual feature space; a novel local global discriminative dimension reduction algorithm is proposed. In this algorithm a sub manifold is learned by transferring the local geometry and the discriminative information from labeled images to the whole image database.

We represent general frame work for semi supervised image reranking in section 2. Section 3 describes collecting labeling information from users to obtain the specified semantic space by using structural (SInfo).The local global dimension reduction (LGD) dimension reduction is described in section 4 followed by the Bayesian Reranking detail in section 5.


The proposed general work frame work for semi supervised image search reranking is taking the query term "panda" as an example. When "panda" is submitted to the image search engine, an initial text-based search result is returned to the user. This result is totally unsatisfactory because both person and animal images are retrieved as top results. This is caused by ambiguity of the query term. Without the user interactions, it is impossible to eliminate this ambiguity.

Re-ranked result

Dimension reduction

Submit the query word

User interaction

Text-based search result

Active sample selection

Image reranking

Fig 2.1 block diagram of semi supervised image search reranking

Some images are first selected according to an active sample selection strategy and display to the user. Then the user is required to label them. For that purpose the user select one image from text-based search result. In active sample selection, the system display most similar and dissimilar image based on the selected image from text-based search result to user. In user interaction step user has to select the required image from the active sample selection. A dimension reduction step is thus introduced to localize the effective visual features of image according to the user's intention. With the effective features of user's intention, the reranking process is conducted and different kinds of animal pandas are returned. The image is displayed by descending order of ranking score. The given example, the user select one animal panda image from displayed image to the user which is given from result of active sample selection. From that selected image, the system can understand the user's intention. Based on user's intention the system re-rank the images and display animal panda images to user. That result is satisfactory because the re-ranking process is conducted based on user's intention.

The Fig 2.1 shows the block diagram of system. There are two key steps in learning the user's intention effectively and completely, i.e., the active sample selection strategy and the dimension reduction. This paper implements these two steps via a new SInfo sample selection and a novel LGD dimension reduction algorithm.


SInfo active sample selection [1] strategy is presented to learn the user's intention efficiently which selects images by considering not only the ambiguity but also the representativeness in the whole image database. Ambiguity and representativeness are two important aspects in active sample selection. Labeling a sample which is more ambiguous will bring more information. On the other side, the information provided by individual sample can be played in dense area. Therefore, the more representative samples are preferred for labeling.

3.1 Ambiguity

The ambiguity denotes the uncertainty whether the image is relevant or not. In the proposed system reranking, it is direct and reasonable to measure the ambiguity with the ranking scores obtained in the reranking process. For an image Ii, 0≤ri≤ 1 is its ranking score, where ri=1 means Ii is definitely query relevant while ri=0 means Ii is totally query irrelevant. (1-ri) and ri can be regarded as the probability of Ii to be relevant and irrelevant respectively. Then the ambiguity can be measured via information entropy. The ambiguity of image Ii is

Hr(Ii)=-ri log ri-(1-ri) log (1-ri) (1)

Because reranking is conducted based on the initial text based search result, the ambiguity in the initial search result should also be taken into account, i.e.

Hr(Ii) =-ri log ri-(1-ri) log (1-ri) (2)

Where 0≤ r≤i 1 is the initial search ranking score for image Ii. By combining (1) and (2), the total ambiguity for Ii is

H (Ii) =α Hr (Ii) + (1-α) Hr (Ii) (3)

Where α (€ [0, 1]) is a trade - off parameter to control the influence of the two ambiguity terms. Its value is 0.85 give more better results

3.2. Representativeness

Once the web image search system gets the labeling information of image, it is very important to consider how many other images can share the labeling information of an image. Labeling an image in dense area will be more helpful than labeling an isolated one because the labeling information of the image can be shared with other surrounding images. In this paper representativeness of the image Ii via probability density p (Ii) which can be estimated by using the kernel density estimation (KDE) [8]

P (Ii) = (4)

Where Ni is set of neighbors of Ii, xi is the visual feature for image Ii, k(x) is a kernel function that satisfies both k(x) > 0 and =1. The Gaussian kernal is adopted in this paper.

3.3 Active Sample Selection

Most informative sample images should meet both representativeness and ambiguity simultaneously. The structural information of image Ii. SI (Ii) can be measured by the product of the two terms.

SI (Ii) = p (Ii) H (Ii) (5)

Then the most informative image I* is selected from the unlabelled image set U according to

I*=arg (6)

The angle diversity criterion [7] is a good choice to achieve for iteratively select images which are more informative and also diverse to the already selected images set S. For an unlabelled image Ii, the diversity between Ii and S is measured by minimum angle between Ii and each image Ii , then, the image are selected iteratively according to


Where β ( [0, 1]) is a trade off parameter which is used to balance the effect of two components: the structural information and the angle diversity.


To localize the effective visual features of user's intention, this paper presents a novel local - global discriminative dimension reduction algorithm. The LGD [1] considers both the local information contained in the labeled images and global information of the whole image database simultaneously. In detail, LGD transfer the both the local geometry of the labeled relevant images and discriminative information in the labeled image, to the global domain (the whole image database). This cross domain transfer process is completed by building different local and global patches for each image, and then aligning those patches together to learn a consistent coordinate. One patch is a local area formed by a set of neighboring images. I have three types of image: labeled relevant, labeled irrelevant and unlabelled .therefore we build three types of patches. Which are 1) local patches for labeled relevant images - to represent the local geometry of them and the discriminative information to separate relevant images from irrelevant ones, 2) local patches for labeled irrelevant images - To represent the discriminative information to separate irrelevant images from relevant ones, 3) global patches for both labeled and unlabelled image - for transferring both the local geometry and the discriminative information from all labeled images to the unlabelled ones.

For convenience, we use superscript "+" to denote the labeled images and "-" to denote the labeled irrelevant images. If refers to an arbitrary image which may be labeled relevant, labeled irrelevant or unlabelled images. If there is no superscript, it refers to an arbitrary image which may be labeled relevant, labeled irrelevant or unlabelled.

4.1 Local Patches for Labeled Relevant Images

This represents the local geometry of the labeled relevant and discriminative information to separate relevant images from irrelevant one. The query relevant samples may vary in appearance and corresponding visual features. For this reason, instead of requiring relevant images to be close to each other in the projected subspace. It is more proper to remain the local geometry of relevant images and the discriminative information between the relevant images and all irrelevant images. This paper models the local patches for the low - dimensional representation yi+ as

Min tr(Yi+ Li+(Yi+ )T) (8)

Where Yi + = [yi+,yi1,......................yi k1, yk1+1,.............yi(k1+k2)]

Li+ = ( LiA+ LiB)

LiA =

The local patch for a labeled relevant image should preserve both the local geometry of relevant images and the discriminative information between the relevant images and all irrelevant images. This paper models the local patch for the low-dimensional representation yi+ of the labeled relevant image Ii+.

4.2 Local Patches for Labeled Irrelevant Images

Discriminative information is also partially encoded in all irrelevant images, so we construct local patches for labeled irrelevant images by separating each irrelevant image from all relevant images. Because each irrelevant image is irrelevant in its own way, it could be unreasonable to keep the local geometry of the irrelevant images. Because every irrelevant image are different its own way [11]. In this paper, the system models the local patch for the low-dimensional representation of labeled irrelevant image. This paper models the local patches for the low - dimensional representation as

Min tr (Yi- Li-(Yi-) T) (9)

The {Ii1,......., In} is I- 's k nearest neighbors in the labeled relevant images set"+". The matrix Li- can be calculated in the similar way of computing Li- by setting k1=0 and K2=k.

4.3 Global Patches for both Labeled and Unlabelled Images

In semi supervised image reranking, users would like to label only a small number of images, so it is lavish and unreasonable to abandon a large number of unlabelled images. With only the labeled images, the learned subspace will bias to that spanned by these labeled images and cannot generalize well to the large amount of unlabelled data. Global patches transfer the local geometry and the discriminative information, which is exploited in the domain of labeled images to the domain unlabelled images. With the global patches, this paper aims to preserve the principle subspace to keep the sub manifold of relevant images. The noise information contained in the ambient space should be eliminated. The principle component analysis (PCA) is a suitable choice. The global patch for the low - dimensional representation yi of the image Ii as

Max tr ((yi-ym) (yi-ym)T (10) (3.10)

Where ym is the centroid of the projected low- dimension feature. This paper uses the variant version of the original definition of PCA to achieve a formula-level consistency for both local and global patches. So the above equation can rewrite as

Max tr (YiLiPCAYiT) (11)

Where = [yi,yi1,......yiN-1] with {Ii1,......., In} are the rest N-1 images beyond Ii

To make use of both the labeled and unlabelled images, the most important thing is to exploit the information contained in them. With the global patches, we aim to preserve the principal subspace to keep the submanifold of relevant images. The noise information contained in the ambient space should be eliminated.

4.4 Patch Coordinates Alignment

PCA based global patches, the subspace with maximum variance is preserved, and so manifold structure of relevant samples can also be preserved. By integrating global patches and local patches, this paper [1] can discover the intrinsic sub manifold of relevant samples, and separate relevant samples from irrelevant samples. This paper [1] can combine all the patches defined in (8) (9) and (10) together,

Max tr (YL) (12)

Where L=γ-- and γ(≥0) is a control parameter.Each patch has its own coordinate system. With calculated local and global patches, we can align them together into a consistent coordinate.


In this paper take the Bayesian reranking [9] as the basic reranking algorithm for illustration. The optimal reranked score r* list is obtained by minimizing the following energy function

E(r)= Reg (G,r)+c *Dist(r,) (13)

where =[ r1 , r2,....,rN ]T is the initial text search score list, c is a trade-off parameter and is a graph which is constructed with nodes being the images and the weights being their visual similarities, and Reg (G,r) is the regularizer. For the regularization term, the local kernel is adopted

Reg(G,r)= rTRLr (14)

Where RL is the local kernel matrix. A point-wise distance is adopted for the ranking distance

Dist(r,)= (ri- )T(ri- ) (15)

Images are represented by 428-D low-level visual features, including 225-D color moment in LAB color space, 128-D wavelet texture as well as 75-D edge distribution histogram. For the initial text search score list , because images are all downloaded from Web search engines According to [3], the normalized rank is adopted as the pseudo score ri= 1-i/N where N is the total no of image returned by the text based search result.


The paper has presented a novel active reranking framework for Web image search by using user interactions. To target the user's intention effectively and efficiently, we have proposed an active sample selection strategy and a dimension reduction algorithm, to reduce labeling efforts and to learn the visual characteristics of the intention respectively. To select the most informative query images, the structural information based active sample selection strategy takes both the ambiguity and the representativeness into consideration. To learn the visual characteristics, a new local-global discriminative dimension reduction algorithm transfers the local information in the domain of the labeled images domain to the whole image database.


We greatly appreciate the effort took by the experts in the area of content based search method We extend our sincere thanks to Mr. Xinmei Tian for his valuable advices to our queries also our thanks to to Dr. R.Elijah Blessing Vinoth, Director and The Head of the Department, School of Computer Science and Engineering for his encouragement and guidance.

Writing Services

Essay Writing

Find out how the very best essay writing service can help you accomplish more and achieve higher marks today.

Assignment Writing Service

From complicated assignments to tricky tasks, our experts can tackle virtually any question thrown at them.

Dissertation Writing Service

A dissertation (also known as a thesis or research project) is probably the most important piece of work for any student! From full dissertations to individual chapters, we’re on hand to support you.

Coursework Writing Service

Our expert qualified writers can help you get your coursework right first time, every time.

Dissertation Proposal Service

The first step to completing a dissertation is to create a proposal that talks about what you wish to do. Our experts can design suitable methodologies - perfect to help you get started with a dissertation.

Report Writing

Reports for any audience. Perfectly structured, professionally written, and tailored to suit your exact requirements.

Essay Skeleton Answer Service

If you’re just looking for some help to get started on an essay, our outline service provides you with a perfect essay plan.

Marking & Proofreading Service

Not sure if your work is hitting the mark? Struggling to get feedback from your lecturer? Our premium marking service was created just for you - get the feedback you deserve now.

Exam Revision

Exams can be one of the most stressful experiences you’ll ever have! Revision is key, and we’re here to help. With custom created revision notes and exam answers, you’ll never feel underprepared again.