Automatic question generation from Indonesian texts using text-to-text transformers

Fuadi, Mukhlish ORCID: https://orcid.org/0000-0002-7595-7646 and Wibawa, Adhi Dharma (2022) Automatic question generation from Indonesian texts using text-to-text transformers. Presented at 2022 International Conference on Electrical and Information Technology (IEIT), 15-16 September 2022, Semarang, Indonesia.

Text 14572.pdf - Published Version Restricted to Repository staff only Download (308kB) | Request a copy

Abstract

Answering questions is one method to increase or measure understanding. However, creating relevant and answerable questions from the given context is not easy. Automatic Question Generation (AQG) is a part of Natural Language Processing (NLP) which can generate questions automatically from text input. Many studies related to AQG have been carried out but are still very limited in Indonesian texts, especially those that use the latest Transformer variations. This study proposes an AQG system that utilizes the latest power Transformer, the multilingual Text-to-Text Transfer Transformer (mT5). We fine-tune the mT5 model to extract answers from context and generate questions based on those answers. We use the Indonesian dataset extracted from the TyDiQA dataset and evaluate this model against the TyDiQA validation set using BLEU (BiLingual Evaluation Understudy) and ROUGE (Recall-Oriented Understudy for Gisting Evaluation) metrics. This model achieved BLEU-1, BLEU-2, BLEU-3, BLEU-4, and ROUGE-L scores of 36.54, 28.24, 22.61, 18.44, and 39.57, respectively. Our model performs well and generates questions in understandable Indonesian with good word choice and grammar based on manual validation.

Item Type:	Conference (Paper)
Keywords:	AQG; question generation; Transformer; mT5
Subjects:	08 INFORMATION AND COMPUTING SCIENCES > 0801 Artificial Intelligence and Image Processing > 080107 Natural Language Processing
Depositing User:	Mukhlish Fuadi
Date Deposited:	19 May 2023 15:20

Downloads

Origin of downloads

Actions (login required)

View Item