Fine-tuning Gemma-4-E2B on MacBook M3

Hi. I’m trying to fine-tune Gemma-4-E2B on MacBook M3 but I haven’t been able to do so. I had previously fine-tuned Llama and Qwen models with no issues. Gemme-4 is presenting real challenges. Having resolved the linear layers selection issue and the tokenizer chat template issue, now I’m stuck with high Training Loss rates > 40 that refuse to decrease. Some info I found blame bfloat16 support on MacBooks. I tried using float16 and even float32 but the model exploded after few epochs. I wonder if there’s any Gemma-4-E2B/E4B best settings info or gotchas list to watch out for or maybe a guide that helps overcome these issues? Any guidance would be truly appreciated!