Evaluating the effect of preprocessing in Arabic documents clustering

Other Title(s)

تقييم تأثير المعالجة المسبقة في عنقدة المستندات العربية

Dissertant

Ghanim, Usamah Abd al-Fattah

Thesis advisor

al-Hanjuri, Muhammad Ahmad Muhammad

Comitee Members

Abu Haybah, Ibrahim Sulayman
Zaqqut, Ihab Salah al-Din

University

Islamic University

Faculty

Faculty of Engineering

Department

Department of Computer Engineering

University Country

Palestine (Gaza Strip)

Degree

Master

Degree Date

2014

English Abstract

Clustering of text documents is an important technique for documents retrieval.

It aims to organize documents into meaningful groups or clusters.

Preprocessing text plays a main role in enhancing clustering process of Arabic documents.

This research examines and compares text preprocessing techniques in Arabic document clustering.

It also studies effectiveness of text preprocessing techniques: term pruning, term weighting using (TF-IDF), morphological analysis techniques using (root-based stemming, light stemming, and raw text), and normalization.

Experimental work examined the effect of clustering algorithms using a most widely used partitional algorithm, K-means, compared with other clustering partitional algorithm, Expectation Maximization (EM) algorithm.

Comparison between the effect of both Euclidean Distance and Manhattan similarity measurement function was attempted in order to produce best results in document clustering.

Results were investigated by measuring evaluation of clustered documents in many cases of preprocessing techniques.

The most frequent and basic measures for text mining evaluation, precision and recall, were used for evaluation measurements.

In addition to F-Measure, which used as a combination of precision and recall.

Experimental results show that evaluation of document clustering can be enhanced by implementing term weighting (TF-IDF) and term pruning with small value for minimum term frequency.

In morphological analysis, light stemming, is found more appropriate than root-based stemming and raw text.

Normalization, also improved clustering process of Arabic documents, and evaluation is enhanced.

Finally, K-means in document clustering was found more efficient than EM algorithm, and Euclidean distance similarity measurement function is superior.

Keywords: Arabic Text Mining, Arabic document clustering, Arabic text preprocessing, Term weighting, Arabic morphological analysis (Arabic stemming / light stemming), Vector Space Mode (VSM), TF-IDF, K-means, EM

Main Subjects

Mathematics

Topics

• Word processing

No. of Pages

93

Table of Contents

• APA
• MLA
• AMA

Language

English

Data Type

Arab Theses

Record ID

BIM-530232