Hey,
I am finishing a project on evaluating code language models on "creative" programming (shadercode). Just yesterday I finished fine-tuning sanatacoder on three different datasets to evaluate on my metric.
Now this new project popped up but it's vastly larger. Prohibitively so. I was unable to run 6B models on the RTX A5000 I have access to.
For finetuning santacoder (no_fp16, batch_size 2 and sequence length of 2048) 97% of the 24GB VRAM was used using a slightly adapted version of the provided script.
Do you have any numbers on what requirements there are for PEFT on this model? And is there a set way to run inference with the hardware I listed?
Thanks for releasing yet another interesting model!
Hey,
I am finishing a project on evaluating code language models on "creative" programming (shadercode). Just yesterday I finished fine-tuning sanatacoder on three different datasets to evaluate on my metric.
Now this new project popped up but it's vastly larger. Prohibitively so. I was unable to run 6B models on the RTX A5000 I have access to.
For finetuning santacoder (no_fp16, batch_size 2 and sequence length of 2048) 97% of the 24GB VRAM was used using a slightly adapted version of the provided script.
Do you have any numbers on what requirements there are for PEFT on this model? And is there a set way to run inference with the hardware I listed?
Thanks for releasing yet another interesting model!