A Modular Multimodal Multi-Object Tracking-by-Detection Approach, with Applications in Outdoor and Indoor Environments

Object detection and tracking are integral components of numerous modern robotics systems, playing an essential role in applications like autonomous driving and industrial Autonomous Mobile Robots (AMRs). In this paper, we propose a modular multimodal multi-object detection and tracking system tailored for AMRs in complex industrial environments. The proposed system employs a tracking-by-detection approach, utilizing both 3D point cloud and RGB data to detect and track multiple objects simultaneously. To develop it, a baseline unimodal framework was created using a PointPillars detector and the AB3DMOT tracker, operating exclusively on point cloud data. To enhance detection and tracking accuracy, a 2D object detector (YOLOv8) was integrated, enabling multimodal detection. The system’s performance was evaluated on the KITTI dataset, demonstrating notable improvements in detection accuracy and tracking consistency. This enhancement strengthens the system’s robustness and reliability, which are critical factors for real-time perception in AMRs.

Recommended citation: Borges, E., Garrote, L., Nunes, U. J. (2024). "A Modular Multimodal Multi-Object Tracking-by-Detection Approach, with Applications in Outdoor and Indoor Environments." In Proceedings of the 21st International Conference on Informatics in Control, Automation and Robotics (ICINCO 2024) - Volume 2
Download Paper | Download Bibtex