Using wordnet for text categorization

Abstract EN

This paper explores a method that use Word Net concept to categorize text documents.

The bag of words representation used for text representation is unsatisfactory as it ignores possible relations between terms.

The proposed method extracts generic concepts from Word Net for all the terms in the text then combines them with the terms in different ways to form a new representative vector.

The effects of this method are examined in several experiments using the multivariate chi-square to reduce the dimensionality, the cosine distance and two benchmark corpus the reuters-21578 newswire articles and the 20 newsgroups data for evaluation.

The proposed method is especially effective in raising the macro-averaged F1 value, which increased to 0.714 for the Reuters from 0.649 and to 0.719 for the 20 newsgroups from 0.667.