Modeling Age of Acquisition Norms Using Transformer Networks

摘要

The age at which children acquire words is an important psycholinguistic property for modeling the growth of children's semantic networks. Much work over the years has explored how to effectively exploit statistical models to predict the age at which a word will be acquired, ranging from simple linear regression to LSA and skip-gram. However, thus far no work has explored whether transformers are any better at modeling word acquisition, despite the superior performance they have achieved on a wide variety of natural language processing (NLP) benchmarks. In this paper, we explore using several transformer models to predict the age of acquisition norms for several datasets. We evaluate the quality of our models using various experiments based on prior work and compare the transformers against two baseline models. We obtain promising results overall, as the transformers can outperform the baselines in most cases.