Image-Text Joint Learning for Social Images with Spatial Relation Model

المؤلفون المشاركون

Feng, Jiangfan
Fu, Xuejun
Zhou, Yao
Zhu, Yuling
Luo, Xiaobo

المصدر

Complexity

العدد

المجلد 2020، العدد 2020 (31 ديسمبر/كانون الأول 2020)، ص ص. 1-11، 11ص.

الناشر

Hindawi Publishing Corporation

تاريخ النشر

2020-03-28

دولة النشر

مصر

عدد الصفحات

11

التخصصات الرئيسية

الفلسفة

الملخص EN

The rapid developments in sensor technology and mobile devices bring a flourish of social images, and large-scale social images have attracted increasing attention to researchers.

Existing approaches generally rely on recognizing object instances individually with geo-tags, visual patterns, etc.

However, the social image represents a web of interconnected relations; these relations between entities carry semantic meaning and help a viewer differentiate between instances of a substance.

This article forms the perspective of the spatial relationship to exploring the joint learning of social images.

Precisely, the model consists of three parts: (a) a module for deep semantic understanding of images based on residual network (ResNet); (b) a deep semantic analysis module of text beyond traditional word bag methods; (c) a joint reasoning module from which the text weights obtained using image features on self-attention and a novel tree-based clustering algorithm.

The experimental results demonstrate the effectiveness of using Flickr30k and Microsoft COCO datasets.

Meanwhile, our method considers spatial relations while matching.

نمط استشهاد جمعية علماء النفس الأمريكية (APA)

Feng, Jiangfan& Fu, Xuejun& Zhou, Yao& Zhu, Yuling& Luo, Xiaobo. 2020. Image-Text Joint Learning for Social Images with Spatial Relation Model. Complexity،Vol. 2020, no. 2020, pp.1-11.
https://search.emarefa.net/detail/BIM-1139917

نمط استشهاد الجمعية الأمريكية للغات الحديثة (MLA)

Feng, Jiangfan…[et al.]. Image-Text Joint Learning for Social Images with Spatial Relation Model. Complexity No. 2020 (2020), pp.1-11.
https://search.emarefa.net/detail/BIM-1139917

نمط استشهاد الجمعية الطبية الأمريكية (AMA)

Feng, Jiangfan& Fu, Xuejun& Zhou, Yao& Zhu, Yuling& Luo, Xiaobo. Image-Text Joint Learning for Social Images with Spatial Relation Model. Complexity. 2020. Vol. 2020, no. 2020, pp.1-11.
https://search.emarefa.net/detail/BIM-1139917

نوع البيانات

مقالات

لغة النص

الإنجليزية

الملاحظات

Includes bibliographical references

رقم السجل

BIM-1139917