VGTU talpykla > Fundamentinių mokslų fakultetas / Faculty of Fundamental Sciences > Moksliniai straipsniai / Research articles >

Lietuvių   English
Please use this identifier to cite or link to this item: http://dspace.vgtu.lt/handle/1/4092

Title: Text Augmentation Using BERT for Image Captioning
Authors: Atliha, Viktar
Šešok, Dmitrij
Keywords: image captioning
augmentation
BERT
Issue Date: 2020
Publisher: MDPI
Citation: Atliha, V.; Šešok, D. Text Augmentation Using BERT for Image Captioning. Appl. Sci. 2020, 10, 5978.
Series/Report no.: 10;17
Abstract: Image captioning is an important task for improving human-computer interaction as well as for a deeper understanding of the mechanisms underlying the image description by human. In recent years, this research field has rapidly developed and a number of impressive results have been achieved. The typical models are based on a neural networks, including convolutional ones for encoding images and recurrent ones for decoding them into text. More than that, attention mechanism and transformers are actively used for boosting performance. However, even the best models have a limit in their quality with a lack of data. In order to generate a variety of descriptions of objects in different situations you need a large training set. The current commonly used datasets although rather large in terms of number of images are quite small in terms of the number of different captions per one image. We expanded the training dataset using text augmentation methods. Methods include augmentation with synonyms as a baseline and the state-of-the-art language model called Bidirectional Encoder Representations from Transformers (BERT). As a result, models that were trained on a datasets augmented show better results than that models trained on a dataset without augmentation.
Description: This article belongs to the Section Computing and Artificial Intelligence
URI: http://dspace.vgtu.lt/handle/1/4092
ISSN: 2076-3417
Appears in Collections:Moksliniai straipsniai / Research articles

Files in This Item:

File Description SizeFormat
Text Augmentation Using BERT for Image Captioning.pdf1.81 MBAdobe PDFView/Open

Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.

 

Valid XHTML 1.0! DSpace Software Copyright © 2002-2010  Duraspace - Feedback