Generative Models for Multiple-Choice Question Answering in Portuguese: A Monolingual and Multilingual Experimental Study

Authors

  • Guilherme Dallmann Lima University Federal of Pelotas
  • Emerson Lopes Federal University of Pelotas
  • Henry Pereira Federal University of Pelotas
  • Marilia Silveira Federal University of Pelotas
  • Larissa Freitas Federal University of Pelotas
  • Ulisses Corrêa Federal University of Pelotas

DOI:

https://doi.org/10.32473/flairs.38.1.138969

Keywords:

Transformers, Question Answering, Generative AI

Abstract

Multiple-choice questions are commonly used to assess knowledge through a set of possible answers to a given question. Determining the correct answer relies on the balance between understanding the question’s content and the associated logic. Generative models are widely applied in Multiple-Choice Question Answering (MCQA) tasks, as they can process the context and predict the correct answer based on the provided input. In this regard, the language used in the question is a critical factor, as comprehension may require understanding linguistic nuances. This work investigates the performance of transformer-based generative models in the MCQA task for Portuguese, under both zero-shot and one-shot scenarios. We compare monolingual (Sabiá-7B and Tucano-2B4) and multilingual (LLaMA-8B and LLaMA-3B) models on MCQA datasets focused on college entrance exams, aiming to evaluate the influence of prior knowledge and the model's adaptation to complex languages. Our results demonstrate that, although LLaMA-8B was not specifically trained for Portuguese, it outperforms the Sabiá-7B model on the ENEM-Challenge and BLUEX datasets. Finally, we show that multilingual models with more recent architectures outperform monolingual models.

Downloads

Published

14-05-2025

How to Cite

Dallmann Lima, G., Lopes, E., Pereira, H., Silveira, M., Freitas, L., & Corrêa, U. (2025). Generative Models for Multiple-Choice Question Answering in Portuguese: A Monolingual and Multilingual Experimental Study. The International FLAIRS Conference Proceedings, 38(1). https://doi.org/10.32473/flairs.38.1.138969

Issue

Section

Special Track: Applied Natural Language Processing