SYSTEM AND METHODS FOR TRAINING AND VALIDATION OF AN END-TO-END ARTIFICIALLY INTELLIGENT NEURAL NETWORK FOR AUTONOMOUS DRIVING AT SCALE

The technology disclosed comprises systems and methods for the training and validation for an end-to-end neural-network learning model configured for autonomous driving. The end-to-end neural-network learning model is trained using human-operated driving demonstration data to curate training data examples of driving tasks and driving routes, as well as curation of particularly difficult driving tasks. The determination of difficulty of driving tasks uses a combination of entropy measurements in training, evaluation of model performance, and manual labeling. The conditional imitation learning model can be configured as a memory-augmented transformer model that leverages a memory-cached frame buffer to access previous states in a driving trajectory. The disclosed technology can be applied to passenger vehicles or autonomous robots for delivery tasks.