This essay has been submitted by a student. This is not an example of the work written by our professional essay writers.
Abstract-Internet users post queries online to retrieve specific knowledge. Non-Arab Muslims who speak various languages find it hard to retrieve verses related to the posted queries related to Islam. Islamic scholars (Ulama) will map the queries and their referenced verses. The aim of this paper is to investigate the effectiveness of the state of the art IR techniques for verse retrieval problem. This has been done by developing a test collection manually. The results show that traditional IR measurements, precision and recall, are not suitable for verse retrieval while MAP and precision 1, 5 and 10 are applicable for Quranic text retrieval.
Keywords-verse; semantic; information retrieval; Al-Quran; muslims
The holy Quran  is a universal source of knowledge for mankind in general and for Muslims in particular. Al-Quran is the exact words of Allah (swt), delivered in Arabic. It has the answers to all queries about living in this world and beyond in which references can be found in the verse(s) of the Al-Quran. It is clearly noted in Surah 6 (Al-An'aam), verse 38, translated as follows:
"There is not an animal in the earth, nor a flying creature flying on two wings, but they are peoples like unto you. We have neglected nothing in the Book (of Our decrees). Then unto their Lord they will be gathered."
The Prophet Muhammad (saw) is designated to disseminate and teach all human about the knowledge in the holy Al-Quran since it is the core source of knowledge in Islam. For instance, the Al-Quran contains the knowledge about performing prayer and Prophet Muhammad (saw) teaches us on how to perform the prayer. The knowledge from the teaching of Prophet Muhammad (saw) is spread to the rest of the world by mean of Al-hadith ; a collection of words and deeds of Prophet Muhammad (saw). The process of teaching and disseminating the knowledge continues for centuries. Such practices have brought Islam to its glorious age sometimes ago.
Unfortunately, many Muslims cannot speak Arabic and cannot obtain such knowledge directly from Al-Quran and without going through a formal education of Islamic studies. The emergence of web and technology makes the knowledge easily accessible for them. It is possible to send inquiries to the existing Web search engine (e.g. Google) and to get a reasonable answer to their questions by referencing related verses in Al-Quran.
The online translators are a form of Information Retrieval (IR) to get the equivalent meaning of topics from one language to another. Using IR, the problem of verse retrieval can be solved by matching the topic to verses e.g.  which serves both scholars and amateur users. However, the Quran text retrieval has a different nature. The translated verses are usually concise and consist of unique words that not usually used in day-to-day life. Verses are usually further elaborated by the experts to relate to the context of the query in order to maintain the meanings for audiences (users).
The scope of this paper is to investigate the effectiveness of the state of the art IR techniques for verse retrieval problem. The test collections were built based on manually indexed topics of Quran. Relatively, we have discussed the retrieval evaluation measurements and compared between them to choose the most suitable ones. We have run the experiments on a state of the art IR system, Terrier  to discover the effectiveness of the verse retrieval problem.
The rest of this paper is organized as follows. Section 2 presents a brief summary of the related work. Section 3 introduces the test collection for Quran verse retrieval for the conducted experiments. In section 4, the evaluation of IR retrieval measurements is presented. The section 5 discusses the results of experiments. Finally, section 6 summarizes the paper along with the future research goals.
Unfortunately, the main focus of current researches is on Arabic documents as done (e.g. KISS project at Sheffield University, AIR at Syracuse University , and QARAB at DePaul University ). These systems support both ad-hoc retrieval and question answering. They range from monolingual, multilingual, and cross-language. The implemented retrieval techniques include stemming, class menus, and topical and keywords (concepts/facets).
In , terms selection based on class menus was proposed in order to facilitate a structural relationships that touched on hierarchical structure of broad and narrower. Unluckily, the degree of terms' significance is missing. It is applicable for Quran text retrieval (Quranic verse retrieval).
A citation analysis was proposed in  to determine the inclusion of items in the database/storage by providing block-level link, stemming, sentence completion, and other common retrieval techniques like phrase searching for Quranic text. None of the proposed methods has been applied to Quranic texts, even though they have been applied in other systems. Block-level link has been applied in stemming for Arabic documents  and Yahoo News . Sentence completion was applied in the work of Gbaski and Scheffer , but for e-mail's reply template.
A dialogue based visualization system had been proposed in  for Quranic text learning in order to help retrieving knowledge from large corpus (Al-Quran) as a result of user's query. It allows multiple references to specific verses because verse may appear frequently in the same surah or different surah(s).
In , another visualization system had been proposed to help non -Arabic speakers for comprehending Arabic documents. This can help to learn Arabic language by showing Arabic text with its translations and audio recitation of Quran verses.
Relatively, a visualization web based system of similarities between root words in Malay translated Quran documents was proposed in . It can be used to realize new resources from the selected domain. The new resources in turn can be used in the process of analyzing and understanding the specific domain or other related domains.
TEST COLLECTION FOR QURANIC VERSE RETRIEVAL
Existing Malay test collection based on the translated Quranic text (, ) does not design for verse retrieval experimental setup. The collection is design to facilitate typical retrieval experiment for Malay texts. We have to retrieve accurate and relevant verses from Al-Quran in respond to a given query from the user. So, the test collection was built from manually indexed topic of Quran.
A test collection used in our evaluation is suitable for IR. It consists of four components, the collection of documents (or verses of Al-Quran in this case), the collection of queries, the relevance judgments and the distinct words . In this research, the Arabic verses of Al-Quran and the translated Malay and English text of the verses are used. Only Malay and English translated texts are used in indexing and retrieval anticipating that the query from the users will be based on these two languages. The Arabic verses are only used for presentation of the search result.
The test collection was originally Indonesian translated verses. By using DBP, we translated the topics back to Malay. After that, these Malay terms will be translated back to English by using Google translate. The English translation is the Shakir . This test collection was converted to Text Retrieval Conference (TREC) format in order to run on Terrier system.
There are a number of assumptions related to this test collection. First, the topics and relevant verses are correctly identified by authors whom familiar with Quran as well as Malay language. Also, we assume that Google Translate will provide reliable translations from Malay to English.
The objective of IR is to retrieve only the relevant documents (verses in our case). Unfortunately, state of the art retrieval techniques will not do that. They retrieve many documents that may be irrelevant.
In this section, we will study the measurements that are used to evaluate the suitability of using in Quranic verse retrieval and how they are interpreted.
Precision and Recall
Precision and recall are the most common measurements to evaluate the effectiveness of IR systems. According to , relevancy of retrieved verses will be assumed to have its broader meaning of 'aboutness' and 'appropriateness'.
Precision is the fraction of retrieved verses that are relevant to the search while recall is the fraction of the verses that are relevant to the query that are successfully retrieved.
A: relevant verses, B: retrieved verses
Precision and recall are not suitable in evaluating Quranic verses retrieval since they are unaware of the retrieved verse rank. Users commonly target the first document (verse) since they believe it is the most relevant to query. When the users click on the first retrieved verse and found it not really relevant, they may ignore other results. Therefore, we will discard precision and recall from our evaluation even though they got high scores for our test collection.
Precision at 1, 5 and 10
Precision at 1 is similar to the traditional precision with only one difference. Precision at 1 measures the precision only for the verse only at the top rank. In other words, it measures the percentage of verses' relevancy found in the first rank. That means these verses should be the most relevant to user's query. Other verses will not be included in the calculation and considered as irrelevant.
Precision at 5 is a similar measure while the precision will be calculated for the first 5 retrieved verses rather than the first verse only. It measures the percentage of verses' relevancy found among the top 5 verses. Indeed, if the most relevant verses exist within the first 5 retrieved verses the value will get higher.
On the other hand, precision at 10 has another dimension. The focus of precision at 10 will be on pages rather than single verses. Precision 10 measures the precision for the verses within the first 10 pages. It describes the percentage of verses' relevancy found among the top 10 pages.
All these measurements are applicable for our problem of Quranic verse retrieval since the paid attention to the rank of results (retrieved verses).
Mean Average Precision (MAP)
MAP is one of the most standard measures among TREC community. It is the mean of the average precision scores for each query. According to , MAP can be calculated by taking the arithmetic mean of average precision values for individual information needs.
Where Q denotes the number of queries.
Since it is based on average precision, it is assumed that the user is interested in finding many relevant verses for each query. Also, MAP provides a succinct summary of ranking effectiveness used by the IR system.
MAP will be part of our evaluation since it provides an effectiveness evaluation of the retrieval and their displaying (ranking).
Mean Reciprocal Rank (MRP)
MRP is a statistic for evaluating any process that produces a list of possible responses (verses) to a query, ordered by probability of correctness . The reciprocal rank of a query response is the multiplicative inverse of the rank of the first correct answer. The mean reciprocal rank is the average of the reciprocal ranks of results for a sample of queries Q.
We will use the MRP in our evaluation since it evaluates the retrieval while consider the rank of retrieved verses.
RESULTS AND DISCUSSION
In this section we will present the results of experiments conducted on Malay and English translated versions of the Holy Quran.
Comparing against retrieval models in Terrier
We had classified the queries based on language into English and Malay. All queries had been evaluated using Terrier 3.0 search engine.
Table 1 shows the results of retrieving English translated verses by using MRP.
Percentage of answers found in the first rank
Percentage of answers found among the top 5 documents
Percentage of answers found among the top 10 documents
Percentage of answers found among the top 20 documents
Percentage of answers found among the top 50 documents
Percentage of documents not found in top 50 documents
Comparing Few Relevant Verses against Many Relevant Verses
Headings, or heads, are organizational devices that guide the reader through your paper. There are two types: component heads and text heads.
Automatic Query Expansion using Pseudo Relevance Feedback
Positioning Figures and Tables: Place figures and tables at the top and bottom of columns. Avoid placing them in the middle of columns. Large figures and tables may span across both columns. Figure captions should be below the figures; table heads should appear above the tables. Insert figures and tables after they are cited in the text. Use the abbreviation "Fig. 1", even at the beginning of a sentence.
Conclusion and future work
Use the abbreviation "Fig. 1", even at the beginning of a sentence.
This research is funded by Ministry of Higher Education (MOHE), Malaysia under the Fundamental Research Grant Scheme (FRGS) (FRGS/1/10/TK/UPM/02/46).