Combining statistical method with ordinal association rules for effective outliers identification in data sets

Abstract EN

Data preparation is an important and critical step in the data mining process that has a huge impact on the success of data mining projects.

Association rules, one of the most important data mining techniques, has been widely utilized in market basket analysis (Boolean association rules) and extensive studies exist to find efficient association rules mining algorithms.

A recent extension of the Boolean association rules, ordinal association rules, that incorporates ordinal relationships among data items has been used to solve the problem of detecting outliers in data sets.

Also, the problem of detecting outliers has a long history in statistics, but has largely focused on data that is univariate (i.

e.

one attribute / field).

In this paper, we propose an effective algorithm that integrates the statistical method with ordinal association rules method to the problem of detecting outliers in data sets.

A prototype tool of the proposed algorithm is introduced.

A numerical large data set is used to execute the algorithm as a case study to compare the results of applying the statistical, ordinal association rules and the proposed algorithms on a real world data set.