SYSTEM AND METHOD FOR PROVIDING AN AGENT ACTION ANTICIPATIVE TRANSFORMER

A system and method for providing an agent action anticipative transformer that include receiving image data associated with a video of a surrounding environment of an ego agent. The system and method additionally include analyzing the image data and extracting short range clips from the image data. The system and method also include analyzing the short range clips and extracting clip-level features associated with each of the short range clips. The system and method further include executing self-supervision using causal masking with respect to the extracted clip-level features to output action predictions and feature predictions to enable ego-centric action anticipation with respect to at least one target agent to autonomously control the ego agent.