TRACK CONFIDENCE MODEL

Techniques for determining an output from a plurality of sensor modalities are discussed herein. Features from a radar sensor, a lidar sensor, and an image sensor may be input into respective models to determine respective intermediate outputs associated with a tracks associated with an object and associated confidence levels. The Intermediate outputs from a radar model, a lidar model, and an vision model may be input into a fused model to determine a fused confidence level and fused output associated with the track. The fused confidence level and the individual confidence levels are compared to a threshold to generate the track to transmit to a planning system or prediction system of an autonomous vehicle. Additionally, a vehicle controller can control the autonomous vehicle based on the track and/or on the confidence level(s).