I successfully installed oobavooga on a PC, and by using only the CPU I can run a vicuna and a wizardlm model. I can't run any llama, opt, gpt-j which looks required when trying to train the model through Lora. Do you have any suggestion?

you are viewing a single comment's thread
view the rest of the comments

[–] abhibeckert@lemmy.world 2 points 2 years ago* (last edited 2 years ago)

While it won't help OP, there are GPUs that have plenty of VRAM but are lacking CUDA.

Apple sells some with 192GB of VRAM for example... and as a Mac user I'm pretty frustrated by all the dependencies on CUDA. The few tools that don't require it run great on my Mac (Stable Diffusion for example, runs in 30 seconds on my laptop with typical settings).