When to Use CPU vs. GPU
| Task | CPU or GPU |
|---|---|
| Tokenization | CPU |
| LLM Training/Fine-tuning | GPU |
| LLM Inference | Either, but GPU recommended |
When you request memory for HPC, that is CPU memory.
If you request a GPU, you will receive all of that GPU’s memory.
| Task | CPU or GPU |
|---|---|
| Tokenization | CPU |
| LLM Training/Fine-tuning | GPU |
| LLM Inference | Either, but GPU recommended |
When you request memory for HPC, that is CPU memory.
If you request a GPU, you will receive all of that GPU’s memory.