Exercise 3c

Batch Size and GPU Memory

As batch_size increases, so does GPU memory usage. If you get an OOM (out of memory) error while using the GPU, try decreasing the LLM batch size.

Source and more information

Exercise 3c: Batch Size and Num Workers - Text Summarization

1. Open the ex3c.ipynb file from the workshop folder.

2. Run each cell of this notebook and complete the EXERCISES as you go.

3. Watch the GPU memory using GPU Dashboard as you run the cells.