In Search of a Lightweight "Good Enough" Offline Generative AI for Mobile Robots: Performance Benchmarking

Abstract

We present a novel benchmarking methodology for evaluating offline Large Language Models (LLMs) in resource-constrained mobile robotics applications. Using an Nvidia Jetson Nano platform with 4GB RAM limitation, we demonstrate the feasibility of deploying tuned ChatGPT4All for robotic control tasks. The model, trained on 22,000+ ICRA proceedings papers, achieves 82% similarity to ChatGPT responses while maintaining sub-second inference time. Our evaluation framework combines TF-IDF similarity scoring and LDA topic coherence analysis across several thousand test cases. Results show consistent performance within hardware constraints, with 92% of responses exceeding 0.80 similarity threshold and 98% completing within one second. This study establishes viability of lightweight LLMs for offline mobile robotics applications, providing a foundation for future resource-aware AI deployments.