SYSTEMS AND METHODS FOR ASSISTING A DRIVER USING A FOUNDATION MODEL IN A SHARED-AUTONOMY DRIVING MODE OF A VEHICLE

Systems and methods for assisting a driver using a foundation model in a shared-autonomy driving mode of a vehicle are disclosed herein. One embodiment of a shared-autonomy assistance subsystem processes, in a vehicle operating in a shared-autonomy driving mode, inputs including vehicle state information, external-road-agent state information, vehicle environmental sensor data, and map data using one or more encoder neural networks that have been trained to extract features for a large language model (LLM). The subsystem inputs the extracted features to the LLM. The subsystem predicts, using the LLM, an objective of a driver of the vehicle. The subsystem then executes, based on an output from the LLM, one or more actions to assist the driver in meeting the predicted objective. The one or more actions include controlling, at least in part, operation of the vehicle.