Visual Question Answering Using Semantic Information from Image Descriptions

Authors

  • Tasmia Tasmia University of Kentucky
  • Md Sultan Al Nahian University of Kentucky
  • Brent Harrison University of Kentucky

DOI:

https://doi.org/10.32473/flairs.v34i1.128460

Keywords:

Deep Learning, Natural Language Processing, Computer Vision

Abstract

In this work, we propose a deep neural architecture that uses an attention mechanism which utilizes region based image features, the natural language question asked, and semantic knowledge extracted from the regions of an image to produce open-ended answers for questions asked in a visual question answering (VQA) task. The combination of both region based features and region based textual information about the image bolsters a model to more accurately respond to questions and potentially do so with less required training data. We evaluate our proposed architecture on a VQA task against a strong baseline and show that our method achieves excellent results on this task.

Downloads

Published

2021-04-18

How to Cite

Tasmia, T., Nahian, M. S. A., & Harrison, B. (2021). Visual Question Answering Using Semantic Information from Image Descriptions. The International FLAIRS Conference Proceedings, 34. https://doi.org/10.32473/flairs.v34i1.128460

Issue

Section

Main Track Proceedings