How the Architectural Design of the Detection Model Can Enhance the Effect of Adversarial Patches

Abstract

Object detection is a central task in computer vision, with wide adoption in real-world applications such as surveillance systems, autonomous driving, healthcare monitoring, and smart devices. Models such as YOLOv8, SSD, Faster R-CNN, and DETR are widely used to identify and locate objects in images or video streams, making them essential building blocks of modern artificial intelligence technologies.\cite{ultralytics2023}\cite{zou2023} This research explores the vulnerability of object detection models to adversarial patches beyond surface-level performance evaluation, emphasizing a deeper mechanistic understanding of architectural behavior. Initial experiments with YOLOv8 revealed substantial performance degradation under adversarial influence, consistent with the results shown in Figure 1, which motivated a broader comparative study incorporating three additional detection models: DETR, SSD, and Faster R-CNN. Comparative, structural, and probing approaches are used to examine how model architectures respond to adversarial patches and to identify factors affecting model robustness and reliability. These investigations reveal not only which models fail under attack, but also why they fail, demonstrating how hidden visual signals can disrupt detection pipelines and how architectural design choices contribute to both resilience and fragility. Overall, the findings provide a comparative assessment of detection model vulnerability while offering deeper insight into the internal mechanisms of modern object detection systems, thereby establishing a foundation for the development of more reliable and secure computer vision architectures.