SYSTEM AND METHOD FOR PROVIDING LONG TERM AND KEY INTENTIONS FOR TRAJECTORY PREDICTION

A system and method for providing long term and key intentions for trajectory prediction that include receiving image data and LiDAR data associated with RGB images and LiDAR point clouds that are associated with a surrounding environment of an ego agent and processing a long term and key intentions for trajectory prediction dataset (LOKI dataset) that is utilized to complete joint trajectory and intention prediction for heterogeneous traffic agents. The system and method also include encoding a past observation history of each of the heterogeneous traffic agents and sampling a respective goal. The system and method further include decoding and predicting future trajectories associated with each of the heterogeneous traffic agents based on data included within the LOKI dataset, the encoded past observation history, and the respective goal.