Performance evaluation of supervised machine learning classifiers for mapping natural language text to entity relationship models

Abstract EN

Transforming natural language requirements into entities involves a thorough study of natural language text.

Sometimes mistakes are made by designers when manually performing this transformation.

Often, the process is time-consuming and inaccurate.

Hence, multiple research studies have been performed to assist inexperienced designers in mapping a natural language text into entities and reducing the time and error that such a method entails.

This work is part of those studies.

Human intervention is a significant constraint for prior studies.

In this paper, machine learning classifiers are used to eliminate human intervention.

The system performs well in predicting entities and has achieved 85%, 75% and 80% for recall, precision and the F-score, respectively.

The system also performs well in predicting nouns which do not represent entities and has achieved 68%, 79% and 76% for recall, precision and the F-score, respectively.

The performance level of the system is the same as other model generation tools found in the literature.

The system is distinguished from these tools in using machine learning classifiers as a technique for establishing entities with no human intervention.

Furthermore, the study finds that when distinguishing entities from other nouns, logic-based classifiers, perceptron-based classifiers and SVM classifiers perform better than statistical learning classifiers.

The decision tree classifier, neural network classifier and SVM classifier all work well.

The decision tree is the better because it can provide a decision tree that defines when a noun is an entity and when it is not based on given features; this is not the case with the neural network classifier and SVM classifier.