System and Method for A Modular and Continually Learning Remote Guidance System for Autonomous Vehicles

Systems, methods, and autonomous vehicles may obtain sensor data associated with an environment surrounding an autonomous vehicle; provide the sensor data to a plurality of plugins; independently determine, with each plugin, based on the sensor data, whether to request a remote guidance session for the autonomous vehicle, each plugin of the plurality of plugins including a different model that is applied by that plugin to the sensor data to determine whether to request the remote guidance session; receive, from at least one plugin, a request to initiate the remote guidance session; and in response to receiving the request to initiate the remote guidance session, communicate with a computing device external to the autonomous vehicle to establish the remote guidance session.