BOUNDARY AWARE TOP-DOWN PREDICTION

Techniques for determining unified futures of objects in an environment are discussed herein. Techniques may include determining a first feature associated with an object in an environment and a second feature associated with the environment and based on a position of the object in the environment, updating a graph neural network (GNN) to encode the first feature and second feature into a graph node representing the object and encode relative positions of additional objects in the environment into one or more edges attached to the node. The GNN may be decoded to determine a first predicted position of the object. The first predicted position may be determined to be outside of a bounded area of the environment. Based on this determination, a second predicted position of the object may be determined using map data associated with the object.