SeniorSafeAI: LLM-based Chatbot to Assist Senior Citizen Victims of Cybercrime

Abstract

This study introduces SeniorSafeAI, an open-source chatbot trained on a curated dataset of cybersecurity Q&As to assist senior citizens in identifying and responding to cybercrimes. We trained and evaluated eight large language models using both quantitative metrics (F1, BertScore, n-gram overlap) and qualitative assessments (clarity, accuracy, relevance, and usefulness). While quantitative results indicate modest performance, qualitative evaluations of top models, including ChatGPT-4o and Qwen2.5 variants, reveal a notable discrepancy between numerical precision/recall and perceived response quality. Future work will focus on further finetuning with an expanded evaluation dataset and conducting user testing to improve usability and interface accessibility.