Skip to main content

Search

Type to search the site. Results will appear below as you type.

Search

RC Learning Portal

RC Learning Portal

Home
Programming Courses
Tutorials
People
Contact

Light Dark Automatic

Multi-GPU LLM Inference

GPU Memory Requirements

UVA HPC

GPUs on UVA HPC GPU access on UVA HPC UVA BasePOD POD Access Wait Time in the Queue Viewing Available GPUs GPU Dashboard Calculating GPU Memory Requirements Gemma 3 Memory Requirements CPU Resource Allocation

Multi-GPU Strategies

Data Parallelism Pipeline Parallelism (Inter-Layer) Tensor Parallelism (Intra-Layer) Key Tools and Frameworks

Accelerate

Advantages of Accelerate Accelerate Code Demo

DeepSpeed

Transformer-specific Latency Advantages Advantages of DeepSpeed

vLLM

Advantages of vLLM

Identifying Bottlenecks

Wrap Up

UVA HPC

UVA RC now offers access to NVIDIA H200 GPUs on Afton, the newest HPC cluster at UVA.

Previous

GPU Memory Requirements

Next

Multi-GPU LLM Inference

Last updated on Jul 8, 2025

© 2026 The Rector and Visitors of the University of Virginia

Cite