HF Sagemaker Setting LLM Parameters

After a lot of scouring on google for this exact problem I found an example that may be the solution you are looking for on a sagemaker-examples.

You need to nest the parameters into another dictionary

hyper_params = {"max_length": 30, "top_k": 50, "top_p": 0.95, "do_sample": True}
hyper_params_dict = {"HYPER_PARAMS": str(hyper_params)}

sagemaker-examples.readthedocs.io

max_length - SageMaker JumpStart Foundation Models - HuggingFace Text2Text...

This notebook’s CI test result for us-west-2 is as follows. CI test results in other regions can be found at the end of the notebook.