Files in this item



application/pdfDAVE-THESIS-2018.pdf (11MB)
(no description provided)PDF


Title:Multimodal machine translation
Author(s):Dave, Mihika
Advisor(s):Hockenmaier, Julia
Department / Program:Computer Science
Discipline:Computer Science
Degree Granting Institution:University of Illinois at Urbana-Champaign
Subject(s):multimodal machine translation
neural machine translation
multi-task learning
image captioning
Abstract:Over the past few years, there has been a lot of progress being made in machine translation through deep learning networks. But there is relatively lesser progress made in using images to catalyze the translation tasks. In this study, we explore various models to incorporate the image features in the machine translation models. We start with a monomodal translation model which uses only textual features. We extend this model to develop the multimodal system which incorporates the visual features related to the source sentence. We also propose a multitask system which uses image captioning task to aid the translation task. Our models are tested on multiple datasets using the automatic evaluation metrics like METEOR and BLEU. The experiments show that the proposed models outperform the text-only baseline model.
Issue Date:2018-04-25
Rights Information:Copyright 2018 Mihika Dave
Date Available in IDEALS:2018-09-04
Date Deposited:2018-05

This item appears in the following Collection(s)

Item Statistics