METHODS AND SYSTEM FOR CONTROLLING A VEHICLE USING FUSION OF MULTI_MODALITY PERCEPTION DATA

Systems and methods for controlling a vehicle are provided. The systems and methods receive a frame of visible image data from at least one camera of the vehicle and a frame of invisible perception data from a perception sensor. The systems and methods fuse the frame of invisible perception data and the frame of visible image data to provide a fused frame of perception data. Object detection, classification and tracking are performed using a neural network based on the fused frame of perception data to provide object detection data. Steering, propulsion and/or braking of the vehicle are controlled based on the object detection data.