Generating Training Datasets for Training Machine Learning Based Models for Predicting Behavior of Traffic Entities for Navigating Autonomous Vehicles

A vehicle collects video data of an environment surrounding the vehicle including traffic entities, e.g., pedestrians, bicyclists, or other vehicles. The captured video data is sampled and the sampled video frames are presented to users to provide input on a traffic entity's state of mind. The system determines an attribute value that describes a statistical distribution of user responses for the traffic entity. If the attribute for a sampled video frame is within a threshold of the attribute of another video frame, the system interpolates attribute for a third video frame between the two sampled video frames. Otherwise, the system requests further user input for a video frame captured between the two sampled video frames. The interpolated and/or user based attributes are used to train a machine learning based model that predicts a hidden context of the traffic entity. The trained model is used for navigation of autonomous vehicles.