Real-time Point Cloud Transmission for Immersive Teleoperation of Autonomous Mobile Robots

Autonomous mobile robots (AMRs) are increasingly deployed for remote inspection in various scenarios. While AMRs can perform missions autonomously, human intervention becomes necessary in cases where safety risks or potential damage to the robot arise. Since these platforms are often deployed in remote or potentially dangerous areas, physical access is not always available. In such situations, teleoperation provides an effective solution to resolve issues. The navigation stack of AMRs include obstacle avoidance and mapping functionalities that rely on the environmental feedback, typically acquired with stereo cameras which represent depth and color information using Point Clouds (PCs). In addition to obstacles mapping, PCs provide operators and algorithms with real-time environmental data improving situational awareness and supporting AI-based downstream tasks. However, the large volumes of data inherent to PCs pose challenges for real-time transmission. For this reason, dedicated transmission pipelines and data compression algorithms compatible with the limited computational capabilities of the on-board computer are required. This paper demonstrates such a scenario, equipping a differential drive ground robot with a stereo camera that produces PCs that are streamed to a remote operator wearing a standalone VR Headset. To accomodate the limited computational capabilities of the on-board computer and reduce bandwidth requirements, a distance-based filtering and quantization encoding is applied. Users will step in to assist the robot, with remote real-time teleoperation, when the AMR issues a "cry for help" signal during an autonomous mission.