Search

RC Learning Portal

RC Learning Portal

Home
Short Courses
Tutorials
People
Contact

Large Language Models (LLMs) on HPC

Overview of LLMs

NLP Problems Solved by LLMs
Terminology
Theory Behind LLMs

Setup

Software Modules and Containers
Open OnDemand – JupyterLab
Exercise 1
Install transformers and datasets packages
Downloading LLMs on UVA HPC
Exercise 2

HPC Resources for LLMs

GPUs for LLMs
When to Use CPU vs. GPU
General GPU workflow
GPUs on UVA HPC
UVA HPC - NVIDIA DGX BasePOD
Wait Time in the Queue
GPU Access on Rivanna
GPU Dashboard
GPU Memory Usage

Model Selection

Hugging Face
Choosing a Hugging Face Model
Finding Model Size on Hugging Face
Benchmarking LLMs

Inference

Using an LLM
Transformers Pipeline
Exercise 3a
Pipeline Debugging Tips
Exercise 3b
Pipeline Batching with GPU
Exercise 3c
CPU Allocation -Inference
Selecting a GPU for Inference
Exercise 4

Optimizing LLMs with Fine-Tuning
Fine-Tuning Classification
Hugging Face Trainer Class
Exercise 5
CPU Allocation - Fine-Tuning
Selecting a GPU for Fine-Tuning
General Advice

Example Slurm Script - PyTorch
More Slurm Options

Wrap-up

Need more help?

Open OnDemand – JupyterLab

Click on the kernel to open a Jupyter Notebook.

Packages from the selected kernel will be available for use in the notebook.

Previous

Software Modules and Containers

Next

Last updated on Feb 23, 2025

© 2025 The Rector and Visitors of the University of Virginia

Cite