END-TO-END MONOCULAR 2D SEMANTIC KEYPOINT DETECTOR AND TRACKER LEARNING

A method for semantic keypoint detection is described. The method includes linking, using a keypoint graph neural network (KGNN), semantic keypoints of an object within a first image of a video stream into a 2D graph structure corresponding to a category of the object. The method also includes embedding descriptors within the semantic keypoints of the 2D graph structure corresponding to the category of the object. The method further includes tracking the object within subsequent images of the video stream using the embedded descriptors within the semantic keypoints of the 2D graph structure corresponding to the category of the object.