Knowledge Discovery In Multimedia Databases Computer Science Essay

Published: Last Edited:

This essay has been submitted by a student. This is not an example of the work written by our professional essay writers.

Multimedia data can be introduced as a digital media which includes text, images, audio, video, animations and graphics. Due to the tremendous expansion in multimedia applications, a need of multimedia databases has emerged. Consequently in the world of research, finding and developing, an effective as well as efficient multimedia database has become a strong research interest. Many decision making applications in multimedia tend to find patterns and important knowledge from multimedia databases, as with the rapid growth of multimedia databases. As a result, "Knowledge Discovery in Multimedia Databases" has become an emerging topic in research world. This review paper will provide a broad overview of multimedia databases as well as data mining and knowledge discovery processes. And also this review paper will critically review current researches, applications and further research improvements under the topic "Knowledge Discovery in Multimedia Databases".

1. Introduction

During recent past years multimedia data expanded tremendously due to the high usage of personal computers, technical developments in high resolution devices and high density storage devices, availability of high speed data communication networks and due to many other reasons. Therefore multimedia data has become an essential component of most of the existing as well as future applications [23]. Based on this huge expansion a need of a proper storing, indexing and retrieving technologies for multimedia data has emerged [4]. As a solution to these burning requirements, as well as to develop multimedia applications in different fields and help multi-user operations, database systems that can handle multimedia data have been introduced. These introduced multimedia databases or multimedia database management systems (MMDBMS) not only support multimedia data types but also support general functionalities of a traditional DBMS [5]. After the growth of multimedia databases, most of the multimedia related decision making applications tend to find patterns and important knowledge from multimedia databases. And also this research area, knowledge discovery process in multimedia databases is more important and applicable in various fields like art, design, hypermedia, digital media, medical multimedia analysis and computational modeling, since these fields use variety of data sources and structures. Therefore the need of tools and techniques, which can discover patterns that can lead to a new knowledge, is an increasing requirement in today's context. Consequently, data mining and knowledge discovery tools and techniques have entered in to this subject area to find data patterns and extract hidden useful knowledge from multimedia databases. Therefore, the research topic "Knowledge Discovery in Multimedia Databases" has inherited a strong interest in the world of research currently.

As mentioned in the above, the subject matter "Knowledge Discovery in Multimedia Databases" is a topic that can be heard in the world of research these days. As a result I have motivated to study about this subject matter as my independent study. And also, most of the time databases are used to develop software applications. But normal databases are incapable of handling huge amount of data in multimedia. Therefore I decided to study this subject matter in order to gain a good knowledge on, how these multimedia databases are structured, how they are implemented and how they support storage, indexing, retrieval of wide range of data. Moreover I have motivated to study this subject area, in order to study about tools needed for discovering relationships between objects within images, classifying images based on the content, extracting patterns in sound, categorizing speech and music, tracking objects in video streams and etc.

The remainder of this review paper is organized as follows. Section 2 mainly provides a broad overview of the subject area "Knowledge Discovery in Multimedia Databases". And within the same section an overview for multimedia databases and an overview for knowledge discovery and data mining will be presented. Finally at the end of section two the applicability of knowledge discovery in multimedia databases will be discussed. In the section 3 this paper will be going to critically review some researches related to this subject matter. In the broader sense section 3 critically reviews the researches which were based on multimedia data mining methodologies. More over section 3 will discusses some researches which use techniques in data mining methods and data mining approach via multimedia databases. Section 4 is about the applications of knowledge discovery and data mining in multimedia databases. This section will be further divided into sub sections based on applications of image data mining, applications of video data mining, applications of audio data mining, applications of text data mining and applications of multimedia mining. Section 5 will be identified the future development in this research topic. And section 6 logically represents the overall identifications, classifications and integration achieved by studying this emerging subject in research world. Finally this paper will provide referred materials used in this literature review

2. Overview - Knowledge Discovery in Multimedia Databases

As mentioned before, "Knowledge Discovery in Multimedia Databases", is an emerging topic in research world, due to the rapid growth, demand and expansion occurred on multimedia data and due to the need of finding tools and techniques which can discover hidden patterns and useful knowledge among wide range of multimedia data.

2.1 Multimedia Databases

Multimedia means combination of different media or media types. These media can be categorized into two types like static and dynamic. Text, images and graphics are categorized as static media while audio, music, speech, video and animations are categorized as dynamic [18]. Even though these media data can be easily stored in file systems, having a database management system is better, since a DBMS is capable of preserving consistency of data, providing efficient data access, dealing with huge amount of data and etc. Therefore to deal with different media types, a multimedia database management system (MMDBMS) is essential [3]. Multimedia databases on the other hand provide special functionalities which cannot be seen in usual databases. Multimedia databases help the development of multimedia applications in different fields. And mainly it can be used to preserve decaying photographs, maps, films and to preserve historically and nationally important data. Not only that but also having a proper multimedia database help multi user operations [18].

2.2 Data Mining and Knowledge Discovery

Knowledge discovery is the process of extracting useful knowledge from large amount of data. Data mining is also very much similar to the process of knowledge discovery, where knowledge discovery has some additional features like preparation, selection and cleaning of data, as well as integration and interpretation of the results of mining [20].

As with the rapid development of databases, the need of extracting useful knowledge from those databases has emerged. Therefore identifying tools and techniques to extract knowledge from a given database can be introduced as the base for the subject area, knowledge discovery in databases (KDD) and data mining. Even though there are some manual methods to analyze and interpret knowledge, those methods are slow, expensive and highly subjective. And also it is impractical to use manual methods to extract useful knowledge from databases, since databases are usually increasing day by day [20]. Therefore this KDD subject area plays a major role in most of the fields which requires hidden useful knowledge to make correct decisions.

Figure 1 show the key steps of KDD process and reflect its iterative feature. Learning the application domain, select target data set, clean and preprocess data, reduce and transform data, choose data mining functions, choose mining algorithms, search for patterns through data mining, evaluate patterns and represent knowledge and use discovered knowledge are the main steps in KDD process[6] . In order to get more accurate results this processes are usually execute iteratively. On the other hand data mining process which involves in fitting models and determining patterns, use different mining algorithms. These algorithms are composed of three components known as, model, and preference criterion and search algorithm [20].

Interpretation/ evaluation


Target Data

Pre-processed Data

Transformed Data






Data Mining

Figure 1: Overview of the Steps of KDD Process

2.3 Applicability of Knowledge Discovery in Multimedia Databases

In today's context multimedia databases plays a key role. As with this huge expansion in multimedia and multimedia databases, most of the multimedia related applications tend to seek hidden useful knowledge from those multimedia data. Therefore finding tools and techniques to extract knowledge and patterns from multimedia databases has emerged. As a result KDD process has got a huge responsibility over multimedia databases as well.

3. Major Researches - Knowledge Discovery in Multimedia databases

Even though there are some algorithms and applications to mine and extract knowledge from different types of data, mining and extracting hidden useful knowledge from multimedia data is at an experimental stage. Classification, clustering, association, visualization, summarization, deviation detection, estimation, link analysis are some major data mining methods which uses some related techniques like neural networks, fuzzy sets, rough sets, time series analysis, Bayesian networks, decision trees etc [15].

3.1 Critical Review of Major Data Mining Methods

Classification method in data mining involves in classifying items into classes based on predictions. Decision tree based methods, rule based methods, and memory based reasoning, neural networks, Bayesian networks and support vector machines can be introduced as classification techniques. Clustering method identify objects which are similar in characteristics and cluster them accordingly. Association is the method which discovers association rules among a given set of data. In order to facilitate and enhance human understandability data mining uses a method known as visualization. Summarization methodology on the other hand describes a group in a systematic way. And also deviation detection, estimation, link analysis describe finding changes, predicting a continuous value and finding relationships respectively.

3.1.1. Comparison, Classification and Association Methodologies

Image and video repositoryMultiMediaMiner is a prototype which has been designed and developed as a multimedia data mining system. This MultiMediaMiner system involves in mining knowledge from image and video databases, using data mining methodologies like summarization, comparison, classification and association.

Image Excavator

C-BIRD Pre-Processor

C-BIRD Search Engine

M-MMiner User interface

M-MMiner Discovery Modules

C-BIRD database

Multimedia Data Cube

Figure 2: General Architecture of MultiMediaMiner

Figure 2 [17] shows the general architecture of MultiMediaMiner. Content Based Image Retrieval system from Digital libraries (C-BIRD) provides a web agent for extracting images and videos from multimedia repositories, a pre-processor for extract image features and compute data, in order to store in the database, a user interface and a search engine to match queries with stored features in the database. Multimedia data cube in this architecture facilitates the multidimensional analysis of multimedia data.

Due to the usage of the above mention mining methods MultiMediaMiner has inherited four major functionalities. They are MM-Characterizer, MM-Comparator, MM-Associator, and MM-Classifier. These four modules support MultiMediaMiner to improve its mining capabilities. The module MM-Characterizer in MultiMediaMiner, characterizes feature in a multimedia database at different abstraction levels. This helps mining process by providing roll-up, drill-down and two dimensional visualization capabilities. As a result it is easy to find characteristics on more concrete values or specialized concepts. This functionality can be represented by means of a histogram [17].

The module MM-Comparator in this system compares characteristics and contrasts features of different classes of data in a multimedia database. This module keeps a target set of data and compares it with other sets of data in order to distinguish the general features of the target set. Since this maintains a target data set it can be concluded as a base for KDD process [17].

MM-Associator is another special module in MultiMediaMiner system. This module discovers some association rules from image and video databases. These rules tend to represent patterns and relationships of data in the database. This also helps the process of knowledge discovery in multimedia databases. In addition to this rules discovering process MM-Associator provides some facilities to visualize these rules. For that it uses a grid to represent two attributes and uses a histograms to represent the validity of a given association rule and its confidence [17].

Multimedia data can be classified based on previously categorized class labels, with the help of MM-Classifier module. This module classifies large set of data into classes and represents detailed characteristics of those classes. The result of this module is represented by means of a decision tree. This decision tree can also be used to make some predictions about multimedia data. Therefore this module MM-Classifier also helps to the key process of KDD [17].

As mentioned before MultiMediaMiner has been designed and developed by using mining methodologies like classification, association and comparison. As a result MultiMediaMiner which is a multimedia data mining system prototype has got the capability of enhancing data mining process with the help of multidimensional cube and four inherited modules. Moreover, as a conclusion these four modules can be introduced as a base functions to the process of KDD.

3.1.2 Clustering Methodology

The clustering method plays a major role in data mining process. There are some researches which used this cluster methodology to work with large databases and find patterns. "A New Hierarchical Approach for Image Clustering" is one of them which have been done by Lei Wang and Latifur Khan [10].

Image Vectors

Object Groups and Corresponding Weights

Input Images

Image Segmentation

Group objects and assign weights

Construct Image Vectors


Detected Objects

Hierarchical tree

In this approach similar images are grouped into a hierarchy. When grouping images based on similarities they considered features of single objects or features of granularity level in addition to the features of entire image.

Figure 3: Major Steps of hierarchical Image Clustering

Figure 3 [10] shows the major steps in this hierarchical image clustering method. This method segment input images into objects and calculate similarities of those objects. These similarities are calculated on the basis of features like color, texture and shape. Objects can then be clustered into groups depending on these similarities. These groups will be assigned a value base on appropriate weights. Each image will be given a vector model in order to find similarities. Then a clustering algorithm will be used to cluster images based on similarities. Dynamic Growing Self-Organizing Tree (DGSOT) is the clustering algorithm introduced through this approach for image clustering.

DGSOT is a tree structure which has organized using neural network techniques, in order to discover an accurate hierarchical structure for a given data set. This clustering algorithm DGSOT decides sub clusters at each hierarchical level, while it is growing in horizontally and vertically [10]. This sub clusters also based on color similarities, texture similarities and shape similarities. Therefore this hierarchical clustered image can then identified based on color, shape and texture. As a result data mining approach has emerged from this finding of hierarchical image clustering.

3.1.3 Visualization Methodology

Visualization is also another important method or aspect in multimedia data mining. As a result from the world of research visualization systems has been emerged. A visualization system designed and developed by Chen Yu and et al can be extracted as an example.

This visualization system consists of three components. They are an interface between visualization and data mining, a tool to explore and query multimedia raw data and another interface between multimedia data and derived data. Video, audio and motion tracking data are the mostly used multimedia data types in this visualization system. The main tool provided by this system is visualization window which allows users to discover new patterns and relationships from derived data streams. This system supports two kinds of temporal data, they are continuous and event variables.

This visualization system supports the visualization of single data stream alone or visualization of multiple data streams parallel. And more over for data mining purposes they used two histograms like local and global. Therefore this system can be used by users who are seeking to gather hidden useful knowledge from large amount of unstructured multimedia data. Since this visualization tool provides data discovery and data analysis facilities [1].

As mentioned before visualization can help data mining process as well as knowledge discovery process, since visual representations are easy to capture. The above visualization system provides more feature and pattern discovery tools. It mainly supports data mining process in multimedia data types like audio, video and motion. Since data mining is the core of KDD, this approach can be critically reviewed as a pre requisite system for KDD.

3.1.4 Summarization Technology

Summarization is another major data mining technology. Therefore, there are some researches which have been used summarization technique with other relevant technologies, to mine multimedia databases and for multimedia knowledge discovery process. "Automatic Multimedia Knowledge Discovery, Summarization and Evaluation" is such kind of research topic which covers summarization mining technique as well as other useful techniques in multimedia mining.

In this research, a method has been introduced for discovering, summarizing and evaluating multimedia knowledge automatically. Even though there exist some techniques for discovery, summarization and evaluation, those techniques can only handle media data independently. As a result the method in this research has been introduced. This method includes some automatic techniques for constructing perceptual knowledge using image clustering, constructing semantic knowledge, reducing the size of multimedia knowledge and for evaluating the quality of multimedia knowledge.

As mentioned above clustering technique plays a major role in this automatic method. When clustering images, this research mainly focused on discovering similarities and statistical relationships between clusters. As a result semantic knowledge automatic technique has been developed. Under this automatic concept, similar concepts are clustered together in a way it will be able to reduce the size of multimedia knowledge.

When extracting perceptual knowledge, have to extract visual and text feature descriptors which discovers similarities and statistical relationships between clusters, from images and text as the initial step. Then as the second step have to find out perceptual concepts using those descriptors. As the final step, have to discover similarities and statistical relationships between clusters using descriptors. On the other hand when constructing semantic knowledge, as the first step, words are tagged and chunked into word phrases. Extracting semantic concepts and identifying semantic relationships will be done in later steps. After capturing perceptual and semantic knowledge, multimedia summarization will be done. Calculate distances among concepts, cluster the concepts and knowledge reduction will be done in this summarization stage. And finally, in this automatic concept has introduced a mechanism to multimedia knowledge evolution by introducing automatic ways of measuring consistency, completeness and conciseness of multimedia knowledge [27].

3.2 Tree Structures for Multimedia Data Mining

In the world of research, there exist tree structures to solve most of the critical problems. The Peano Count Tree (P-Tree) was designed and developed in order to overcome multimedia data mining issues. This is a special data structure which has been designed to provide efficient data mining capabilities. With the help of this data structure it is possible to mine large data sets of any formats. P-trees will be able to provide a lossless, compressed, data mining-ready representation

P-Tree data structure can be extended to work with lots of data mining technologies since its fast calculation capability. By comparing classical decision tree with a P-Tree based decision tree, we can come to a conclusion about P-Tree data structures' suitability towards multimedia data mining. Therefore P-Tree structure can be viewed as a data mining ready structure which supports efficient data mining process [22].

Not only P-Tree, but also there some researches which have been done using fuzzy decision tree. For an example applying fuzzy decision tree to mine video data can be presented. Fuzzy data mining technology can be applied for classical as well as for fuzzy data repositories. When extracting video news the main media type that has to be segmented is text, audio and images. In this video mining system they have used fuzzy decision tree to summarize and explain the news, since decisions made by fuzzy data structure is natural and understandable. As a result of using fuzzy decision tree for this video extraction people who are going to access this video news will get a more personalized browsing capabilities. [12].

As with this reviews it is clear that fuzzy decision tree can also be used in data mining as well as in knowledge discovery (KDD). Multimedia databases are generally in the form of unstructured way. Fuzzy queries are usually used to retrieve data from those multimedia databases. This also implies fuzzy decision tree itself can be used in KDD process and discover patterns and relationships in multimedia databases.

3.3 Multimedia Data Mining Approaches through Multimedia Databases

Multimedia databases support different media types by storing them and providing different retrieval methods. Primarily there are two types of multimedia databases. They are Linked Multimedia Databases and Embedded Multimedia Databases. Linked multimedia databases are organized as metadata where these metadata links actual data which are stored in online manner or offline manner. On the other hand embedded multimedia databases contain actual multimedia objects in binary form. When querying a multimedia database, it can be done in two ways. Using well defined queries and using fuzzy queries. When using fuzzy queries, there exist three approaches. They are keyword querying, semantic querying and visual querying. Need to use indexes and pattern matching mechanisms in semantic querying while visual querying need Query By Image Context (QBIC) mechanisms [18]. Using tree data structures for storing as well as querying from MMDBs, xml for matching similarities, metadata to store more details are some techniques involve in multimedia databases, in order to enhance data mining process.

3.3.1 Mining Multimedia Database Queries

Queries are used to access information from media objects resides in different databases. Therefore improving query performance through identifying structural equivalence relationships has emerged. As a result, a research named "Generalized Affinity-Based Association Rule Mining for Multimedia Database Queries" has evolved.

A relative affinity value has been used to measure the frequency of accessing given media objects together in a set of queries. Since these values contain the number of accesses of queries instead of containing number of queries, they are more informative. Then by using those calculated affinity values, data mining algorithm, which has the capability of discovering a set of quasi-equivalent media objects in a network of databases has been generated. This algorithm needs to make only one pass over the database and during this pass it reads and supports only one record. This reduces the processing overhead, since it scans database only once. Not only that but also the affinity-based association rule mining algorithms are naturally reasonable and understandable, because they consider the access frequency of queries rather than considering about number of queries. This makes affinity based association rule mining algorithms to work properly with multimedia databases, which contains large amount of data. From the experimental studies, it has been revealed that, this proposed method performs well than normal association rules in discovering quasi-equivalence relationships [28]. As a result, this method can be introduced as a base for mine multimedia databases as well as discover knowledge form multimedia databases.

3.3.2 XML - The Key Technique in Multimedia Databases

Figure 4 [14] shows the complete architecture of the proposed generic multimedia database. Feature extract module will be used to extract all possible features from a media file. The module semantic libraries store classes which describe some features, and then those extracted features will be compared with this module in order to classify a media. Metadata scheme contains a standard metadata schemes to support different media types. Ontology module is a knowledge representation component in this structure which can be useful in adding meanings to queries performed by user. On the other hand search logic module provides efficient search and retrieval of a media file. All these modules provide some base towards the data mining and knowledge discovery process in a multimedia database.

Feature Extractor


Metadata Schemas

Semantic Libraries

Search Logics

User Profile

Query Features

Query Results



Figure 4: Complete Architecture of a Generic Multimedia Database

The XML database is the key feature of this architecture. In this architecture, XML database consists with three sections. In first section there is a media file reference, second section has the extracted metadata using profiles and third section contains metadata extracted automatically using feature extractor [14].

This database system supports two types of queries. They are normal queries and Query by Example. Then through ontology and feature extraction modules, add meaning to the queries and extract feature from queries will be done. These results are then compared with XML database to classify media inputs given by the user. Since these whole process do some feature extraction and some knowledge representation, this database architecture can be improved in a way that it can be used in a proper data mining and KDD processes.

4. Applications of knowledge Discovery and Data Mining in Multimedia Databases

Knowledge discovery in multimedia has become a burning requirement in most of the fields. As with the development of multimedia database technologies people tend to find hidden useful knowledge from those databases As a result there are so many applications exist under this research topic. They can be classified based on the media type they required. Image mining, audio mining, video mining, text mining are some of the classification that can be done.

4.1 Applications Based on Image Mining

Low level feature representation like color histogram and texture, intermediate feature representation like visual codebook, high level feature representation like objects and events, small scale data set and large scale data set are the image mining trends that can be seen in different fields [2].

In the research world people has used image mining to discover various patterns and relationships. There are some researches like detecting deforestation patterns through satellites, automatic view selection, and etc which has used image mining to develop their research areas.

In the research detecting deforestation patterns through satellites has used image mining methodologies like image clustering, classification and segmentation to detect geographical land features [11]. In the research clustering shoe prints, has extracted the RGB values from their database and clustered data based on those RGB values. Then compared and analyzed those results to discover knowledge, through some patterns and relationships. For the clustering process they have used 'K- means' clustering technique [21].

4.2 Applications Based on Video Mining

Video mining also has some trends. They are intra video mining like clustering news, video weather, interview, stock exchange and etc; inter video mining like clustering you tube video clips and content level and concept level similarities [2].

Video mining can be used in many situations like sports highlights extraction, monitoring of surveillance cameras, web browsing using video search engines, detection of traffic patterns for traffic understanding management and etc. Not only that but also in research world, video mining plays a major role.





Mine knowledge from image and video databases

Used summarization, comparison, classification and association mining technologies

Has components like Content Based Image Retrieval system, pre-processor, search engine and a data cube

Has functionalities like MM-Characterizer, MM-Comparator, MM-Associator and MM-Classifier

Research 1

Research 2

Research 3

Use C-BIRD to extract images and videos from multimedia repositories

Use separate feature extraction module

Used a search engine for match queries

Use search semantic library module for match queries


Generic Multimedia Database Architecture

Use XML based database

Support two types of queries like normal queries and Query by Example

Has components like feature extraction module, semantic library module, metadata scheme, ontology module and search logic module

Has a multi-dimensional cube to analyze multimedia data

Metadata scheme to support different media types


Visual Mining of Multimedia Data for Social and Behavioral Studies

Support visualization of single data stream alone multiple data streams parallel

Support media data types like video, audio and motion tracking data

Has a visualization window

Mine knowledge from image and video databases

providing efficient search and retrieval of a media file

Supports video, audio and motion tracking


Automatic Multimedia Knowledge Discovery, Summarization and Evaluation

Includes automatic techniques for constructing perceptual and semantic knowledge

Use clustering technique

Summarize and evaluate knowledge

Research 4

Research 5

Handle multiple multimedia data at a time

Can only handle images


A New Hierarchical Approach for Image Clustering

Group images into a hierarchy

Considered features of single objects

Similarities are calculated and clustered on the basis of color, texture and shape like features

Used a clustering algorithm known as, Dynamic Growing Self-Organizing Tree

Entire hierarchy, based on color, texture and shape

Has a unique feature of constructing perceptual and semantic knowledge, and evaluating multimedia knowledge

Has a unique feature of identifying the entire hierarchy, based on color, texture and shape.

Use clustering technique

Use clustering technique


Multimedia Data Mining Using P-Trees

Provides efficient data mining capabilities

Provide a lossless, compressed, data mining-ready representation

Has the multimedia mining capability than other classical decision trees

Research 6

Research 7

Provides a lossless, compressed and data mining-ready representation.

Can be applied in both classical and fuzzy data repositories

Can make more natural and understandable decisions


Fuzzy multimedia Mining Applied to Video News

Can be applied for classical as well as fuzzy data repositories

Make more natural and understandable decisions


Generalized Affinity-Based Association Rule Mining for Multimedia Database Queries

Used a relative affinity value

Keep track of number of accesses of queries

Capable of discovering a set of quasi-equivalent media objects

Needs to make only one pass over the database

Supports only one record during a pass

Naturally reasonable and understandable

Performs well than normal association rules in discovering quasi-equivalence relationships

Used fuzzy logic which has been derived with the help of image mining techniques.

Table 1: Comparison of Major Researches on "Knowledge Discovery in Multimedia Databases"

Video mining for creative rendering is a one of the researches that has used video mining technologies. In this research video input has been analyzed and clustered based on their motions. For that they have used a motion mining algorithm and motion is quantized and clustered based on hierarchical methodology [13].

4.3 Applications Based on Audio Mining

Audio is on the other hand has a huge demand in many fields. It can be introduced as a technology that has the capability of retrieve information contained in recorded video footage, radio and television broadcasts, telephone conversations, call center dialogues, and help desk recordings. Audio mining provides not only these facilities but also facilities like music mining, language translation and many more applications [8].

4.4 Applications Based on Text Mining

Text also handles a huge responsibility in multimedia since it is the most common way people used to communicate. As a result text mining can be seen in many fields. A text mining system has been used to discover knowledge from bio medical documents [16], detecting rumors with web based text mining [24] can be presented as text mining applications.

4.5 Applications Based on Multimedia Mining

Manual rule based mining, automatic ontology rule based mining, multimodal fusion, multimodal synchronization and cross modal correlation are the trends in multimodal data mining [2].

Multimedia data mining, which is a combination of all media types like image, video, audio and text, has the top level demand in world of research. Multimedia data mining framework has been used to capture raw video sequences [9]. In this research they have introduced a new mining framework which grouped video frames into segments and extracted features from those frames, then indexed and clustered those frames based on extracted features. After that mining process appeared and discovered various patterns and relationships. Multimedia data mining has also been used for traffic video sequences [19].

5. Future Developments under Knowledge Discovery in Multimedia Databases

Up to this section, this review paper critically reviewed major researches and the technologies used in those researches. And more over, this paper has described applicable areas of multimedia mining and knowledge discovery in multimedia. As every good thing has some problems, this research topic "knowledge Discovery in Multimedia Databases" also has some problems or burning issues. As a result there are lots of future researches areas can be introduced under this topic. Not only because of the issues, but because of this research area "Knowledge Discovery in Multimedia Databases" is a currently experimenting subject matter, this research are has inherited lots of future developments.

5.1 Identified Future Developments

One of these researches is distributed or collective data mining. At the moment most of the data mining has been done to the data which physically locates in one place like database or data warehouse. Anyhow the problem arises where people need to classify data which physically locates in two different places. Mining data which are distributed in two or more locations is known as distributed data mining (DDM). The challenge is to effectively mine data which are distributed over heterogeneous sites [7]. Therefore this idea can be effectively apply to multimedia data as well. As with the development of mining technologies for mine multimedia databases, the closest next research area would be mining multimedia data which are physically distributed in many locations. This will lead to do researches and make improvements in distributed multimedia databases mining or knowledge discovery in distributed multimedia databases.

Another multimedia data mining research area is audio mining or mining music. Audio mining will be useful than video mining, because in video mining graphical patterns will be revealed and users have to concentrate on those patterns. Therefore video mining is not always useful. But in audio mining it is possible to transform patterns into sound and music. Therefore users will get the privilege of identifying patterns by listening to pitches, rhythms, tune, and melody. Anyhow properly doing audio music mining is still a challenge [7].

Real-time multimedia data mining systems is another research topic that has and that will be emerged in the world of research. For this kind of a multimedia data mining system, queries and transactions have to meet timing constraints. The problem arises, because timing constraints are hard, soft or firm. In order to handle these constrains, there should be time constraints on queries. At the moment there are techniques for processing real time queries, but the challenge is on impose time constraints on those queries. In real-time multimedia data mining systems, there should be a mechanism to guarantee and show the accuracy of results, the confidence and other corrective mechanisms [25]. This research area is yet to be done and emerged more and more while developing data mining technologies for multimedia databases.

For the success of multimedia databases, content-based multimedia information retrieval contributed in a great manner. As with this development the challenge of maintaining a video database has emerged. Video data compression, user-video interaction, segmentation, object extraction, video data clustering and indexing are some of the challenges faced in developing video databases. Even though some of these challenges were overcome, still there are problems in mining video databases. Shots and scene detection is not easy. As a result video summarization has become a challenge in the world of research. Another problem in video summarization is the level and granularity of summarization. And more over this summarization process will be difficult due to the degree of semantics involved. Therefore, video segmentation and summarization is an emerging research topic which related to multimedia mining [26].

In the research conference "International Workshop on Multimedia Data Mining" has proposed some research topic that will cover multimedia mining and knowledge discovery research area. "Man-machine interfaces for multimedia data mining", "Multimodal techniques for interactive multimedia data mining and exploration", "Complexity, efficiency and scalability of multimedia data mining algorithms", "Knowledge representation and integration for multimedia data mining", "Multimedia data mining techniques for specific domains and applications", "Theoretical frameworks for multimedia data mining", 'Multimedia data-specific sampling and preprocessing" and "Topic and event discovery in large multimedia repositories" are some of those research areas introduced via the research "Knowledge Discovery in Multimedia Databases". These are some of the future research area which is yet to be achieved.

5.2 Possible Suggestions for Future Developments

Future developments stated in the previous section have been identified and exposed to the world by the researchers who involve in this subject matter. As well, with the knowledge gained by doing this independent study, I have come up with some future possible developments or research areas in Multimedia data mining and knowledge discovery. Most of these suggestions have been generated, based on the especial features inherited to multimedia and its high as well as competitive demand in the modern computerized world.

One of them is applying all these multimedia mining techniques in mobile phones. Mobile device is something that is used by most of the people in the world. Since people carry their mobile phone, where ever they go, applying multimedia mining and knowledge discovery techniques in mobile phones will be useful. If a person takes a photograph of a car from his mobile phone, that photograph will contain background images as well. In order to make the output of that process more feasible, multimedia mining techniques can be used. That means extracting only the car from that image, by eliminating background images will be useful and fit for the purpose of the person who has taken the photo. Not only image mining techniques but also audio and video mining techniques will be helpful for mobile users. People can record voice signals using their mobile phones. With this facility they will be able to listen to those voice signals in later time. But it is better if they can track only the needed information from that voice signal based on the topic they wish to study. Text mining is also a possible suggestion in mobile phones. Sending short messages (SMS) is a major functionality of a mobile phone. In order to support the busy life of people, multimedia mining can be used in mobile phones. That means using multimedia text mining capabilities; it is possible to add a functionality to simplify long messages in to small meaningful separate sentences. Therefore, from the above possibilities, it can be concluded that having multimedia mining capabilities in mobile phones is a feasible approach that can be achieved under this research area.

Another possible suggestion in the development of multimedia mining and knowledge discovery is, making computers to visualize knowledgeable outputs based on mining results. As mentioned before multimedia image mining can be developed in a way to extract only the intended object from a photograph by eliminating background objects. Once computers are knowledgeable enough to achieve this objective, it is possible to add functionality to integrate those outputs into a meaningful visualization. That means, by mining computers can identify a car in a photograph. Therefore by considering the shape of the car that has been mined, computers will be able to visualize new, but possible shapes for cars. As well this can be done for video and audio media types. After mining these multimedia data types, innovative integrations can be predicted or done.

From these suggestions that are possible under the research topic "Knowledge Discovery in Multimedia Databases" show the competitive and challengeable demands inherited to multimedia and multimedia databases. As a result of these factors it is possible to introduce the research area; multimedia mining and knowledge discovery as an experimenting research area which has lots of future developments paths.

6. Conclusion - Overall Identifications of the Research Area

Through the research topic "Knowledge Discovery in Multimedia Databases", I have discovered and got to know about multimedia databases, knowledge discovery, data mining and integrations of all these three areas, including the evolvement of this subject matter.

As I have identified multimedia databases support all kinds of media types like images, video, audio, text, graphics, animations and etc. In order to support these different media types, there exist two multimedia databases, known as Linked Multimedia Databases and Embedded Multimedia Databases. And also through this research I got to aware that multimedia databases support two querying types like well define queries and fuzzy queries.

The other related area of under the topic "Knowledge Discovery in Multimedia Databases" is data mining and KDD. Knowledge discovery is finding patterns from large amount of data and it differs from data mining with some features like selection, cleaning and integration. Under this subject area I have found data mining supervised learning methods like classification, characterization and many more, as well as unsupervised learning methods like association, clustering and summarization. More over I got to know about techniques used by these methods as well.

As a result of identifying and critically reviewing these two subject areas I have found some useful information regarding how the data mining methods have been used in researches in order to support and extract knowledge as well as patterns and relationships from multimedia data or multimedia databases. And more over how multimedia databases can be developed to support data mining and knowledge discovery processes by itself. I have identified not only research areas in multimedia related data mining and KDD, but also research areas as well as applications which relates to this research topic in anyway has been identified.

Through these identifications I can come to a conclusion on knowledge discovery in multimedia databases. That is multimedia databases can even handle and support data mining and KDD process by itself and we can apply data mining methodologies on multimedia databases. Since this research topic is a mostly discussing and experimenting one in the world of research, there can be more improvements in the future, than what has been mentioned in chapter 5.


I would like to thank Mr. Saminda Premarathne, senior lecturer, faculty of Information Technology, university of Moratuwa, for guiding me through this research area, and also for his valuable support and advices, which made me to identify this research area properly.