Improving natural language queries search and retrieval through semantic image annotation understanding

Joint Authors

Samih, Haytham
Radi, Shirin
Ismail, Manal A.
Gharib, Tariq F.

Source

International Journal of Intelligent Computing and Information Sciences

Issue

Vol. 20, Issue 2 (31 Dec. 2020), pp.67-78, 12 p.

Publisher

Ain Shams University Faculty of Computer and Information Sciences

Publication Date

2020-12-31

Country of Publication

Egypt

No. of Pages

12

Main Subjects

Information Technology and Computer Science

Topics

Abstract EN

Retrieving images using detailed natural language queries remains a difficult challenge.

Traditional annotation-based image retrieval systems using word matching techniques cannot efficiently support such query types.

Significant improvements for this problem can be achieved with a semantic understanding for those query sentences and image annotations.

This paper presents a two-stage semantic understanding approach for natural language query sentences.

At the first stage, the Stanford parser and a designed rule-based relation extraction tool are used in triple extraction process to efficiently extract the objects attributes, instances and natural language annotations relationships involving these objects.

The second stage integrates the extracted relations with external commonsense knowledge source in a mapping process to provide high-level sematic meanings to the extracted triples.

Experiments are conducted for evaluating the benefit of the proposed semantic understanding against a testing set of natural language sentences from the Flickr8k dataset.

The results show that the proposed approach succeeds to extract relational triples with average accuracy value of 97 % for the different types of annotations relationships : attributes and instance relations, multiword dependence relations, and semantic relations.

American Psychological Association (APA)

Samih, Haytham& Radi, Shirin& Ismail, Manal A.& Gharib, Tariq F.. 2020. Improving natural language queries search and retrieval through semantic image annotation understanding. International Journal of Intelligent Computing and Information Sciences،Vol. 20, no. 2, pp.67-78.
https://search.emarefa.net/detail/BIM-1086194

Modern Language Association (MLA)

Samih, Haytham…[et al.]. Improving natural language queries search and retrieval through semantic image annotation understanding. International Journal of Intelligent Computing and Information Sciences Vol. 20, no. 2 (2020), pp.67-78.
https://search.emarefa.net/detail/BIM-1086194

American Medical Association (AMA)

Samih, Haytham& Radi, Shirin& Ismail, Manal A.& Gharib, Tariq F.. Improving natural language queries search and retrieval through semantic image annotation understanding. International Journal of Intelligent Computing and Information Sciences. 2020. Vol. 20, no. 2, pp.67-78.
https://search.emarefa.net/detail/BIM-1086194

Data Type

Journal Articles

Language

English

Notes

Includes bibliographical references : p. 77-78

Record ID

BIM-1086194