Next Sentence Prediction with BERT as a Dynamic Chunking Mechanism for Retrieval-Augmented Generation Systems

Abstract

Retrieval-Augmented Generation systems enhance the generative capabilities of large language models by grounding their responses in external knowledge bases, addressing some of their major limitations and improving their reliability for tasks requiring factual accuracy or domain-specific information. Chunking is a critical step in Retrieval-Augmented Generation pipelines, where text is divided into smaller segments to facilitate efficient retrieval and optimize the use of model context. This paper introduces a method that uses BERT's Next Sentence Prediction to adaptively merge related sentences into context-aware chunks. We evaluate the approach on the SQuAD v2 dataset, comparing it to standard chunking methods using Recall@k, Precision@k, Contextual-Precision@k, and processing time as metrics. Results indicate that the proposed method achieves competitive retrieval performance while reducing computational time by roughly 60%, demonstrating its potential to improve Retrieval-Augmented Generation systems.