Skip to main content

Search

Type to search the site. Results will appear below as you type.

Search

RC Learning Portal

RC Learning Portal

Home
Programming Courses
Tutorials
People
Contact

Light Dark Automatic

Large Language Models (LLMs) on HPC

Overview of LLMs

NLP Problems Solved by LLMs Terminology Theory Behind LLMs

Setup

Software Modules and Containers Open OnDemand – JupyterLab Exercise 1 Install transformers and datasets packages Downloading LLMs on UVA HPC Exercise 2

HPC Resources for LLMs

GPUs for LLMs When to Use CPU vs. GPU General GPU workflow GPUs on UVA HPC UVA HPC - NVIDIA DGX BasePOD Wait Time in the Queue GPU Access on Rivanna GPU Dashboard GPU Memory Usage

Model Selection

Hugging Face Choosing a Hugging Face Model Finding Model Size on Hugging Face Benchmarking LLMs

Inference

Using an LLM Transformers Pipeline Exercise 3a Pipeline Debugging Tips Exercise 3b Pipeline Batching with GPU Exercise 3c CPU Allocation -Inference Selecting a GPU for Inference Exercise 4

Optimizing LLMs with Fine-Tuning Fine-Tuning Classification Hugging Face Trainer Class Exercise 5 CPU Allocation - Fine-Tuning Selecting a GPU for Fine-Tuning General Advice

Example Slurm Script - PyTorch More Slurm Options

Wrap-up

Need more help?

What is Inference?

Inference is the process in which a trained (or fine-tuned) LLM makes a prediction for a given input.

Inference diagram showing a prompt phase followed by decode iterations that generate tokens one at a time until EOS.

EOS: end of sequence

Previous

Benchmarking LLMs

Next

Last updated on Feb 23, 2025

© 2026 The Rector and Visitors of the University of Virginia

Cite