GPU Memory
Noun · AI & Machine Learning
Definitions
The memory available on a GPU for storing model weights, activations, batches, and intermediate computation during training or inference. GPU memory is often the main bottleneck when serving or training larger models.
In plain English: The memory on a GPU used to hold model data and computations.
Example: "They had to shrink the batch size because GPU memory ran out once they increased sequence length."