Towards automated testing of multi-agent systems using Prometheus design models

Abstract EN

Multi-Agent Systems (MAS) are used for a wide range of applications.

Goals and plans are the key premise to achieve MAS targets.

Correct and proper execution and coverage of plans and achievement of goals ensures confidence in MAS.

Proper identification of all possible faults in MAS working plays its role towards gaining such confidence.

In this paper, we devise a model based approach which ensures goals and plans coverage.

A Fault model has been defined covering faults in MAS related to goal and plan execution and interactions.

We have created a test model using Prometheus design artifacts, i.e., Goal overview diagram, Scenario overview, Agent and Capability overview diagrams.

New coverage criteria have been defined for fault identification.

Test Paths have been identified from test model.

Test cases have been generated from test paths.

Our technique is then evaluated on actual implementation of MAS in JACK Intelligent Agents is a framework in Java for multi-agent system development (JACK) by executing more than 100 different test cases.

Code has been instrumented for coverage analysis and faults have been injected in MAS.

This approach successfully finds the injected faults by applying test cases for coverage criteria paths on MAS execution.

‘Goal plan coverage’ criterion has been more effective with respect to fault detection while scenario, capability and agent coverage criteria have relatively less scope in fault identification.