Working my way up to build a AI Model from scratch

Hello ALL I know some AI basics like how to use StableDiffusion, but now want to build a LLM of my own from scratch (say a prompt to video LLM!) Any ideas would be appreciated! Thank you.