Particle swarm based feature selection for improving random forest classification accuracy

Dissertant

al-Wahidi, Bara Khalid Abd al-Razzaq

Thesis advisor

al-Rusan, Thamir
Abu Hamidah, Mahir Abd al-Qadir Abd al-Munim

University

Isra University

Faculty

Faculty of Information Technology

Department

Department Software Engineering

University Country

Jordan

Degree

Master

Degree Date

2019

English Abstract

Iterating over every possible combination of features and building each combination as a decision tree takes massive processing power especially when there aremany features to select from.

The main drawback with using decision tree classifiers is the tendency of the tree to be over fitted to a specific scenario.

The random forest classifier resolves this issue by using randomly selected features as nodes.

The problem with this approach is that it requires more time and computational power to construct the trees.

Researchers have identified this issue and worked on multiple variations of random forest to reduce the number of decision trees to be grown.

Some of the successful variations use Symmetrical Uncertainty, and other methods to select a feature combination that will yield the highest accuracy achieving trees and generate a random forest for these features rather than the entire dataset.

Others have employed the genetic algorithm in accordance with random forests to optimize the order and appearance of the features in making the random forest.

In this research we employed an optimization algorithm called Binary Particle Swarm.

The binary particle swarm optimization algorithm is a powerful algorithm in the field of optimization.

We used this algorithm to pick the best features that represent a dataset as input for a random forest classifier.

We have achieved impeccable results in terms of accuracy and precision while maintaining minimum user interaction.

We used the Wisconsin breast cancer dataset which can be obtained from the UCI machine learning repository.

The objective in this dataset is to predict whether the patient has a benign or malignant tumor based on the attributes provided.

The other dataset we used was the Titanic disaster dataset which can also be obtained from the UCI machine learning repository.

In this dataset, the objective is to predict whether the passenger has survived or not based on the provided attributes.

We obtained a 97% on average and a best 98% classification accuracy on the Wisconsin breast cancer dataset.

Using the same technique, we obtained 97% classification accuracy on the Titanic datase

Main Subjects

Information Technology and Computer Science

No. of Pages

54

Table of Contents

• APA
• MLA
• AMA

Language

English

Data Type

Arab Theses

Record ID

BIM-988682