GPU Hardware & CUDA

The cards and Multi-Instance GPU (MIG) slices listed here cover the many configurations that can back a GPU class, and not every configuration is in service in every quarter. A session requests a class, not a card; see GPU Class Sizes.

Specifications¶

The tables group cards by architecture, oldest first, and list the MIG slices cut from a card after it. Workloads by GPU Class lists the cards and slices behind each class.

Bandwidth is peak memory bandwidth. A slice's bandwidth is its share of the card's and is nominal; see MIG Slices. The chip column gives the GPU chip and its compute capability, written as the sm_ target that CUDA builds and error messages use.

Turing¶

Turing cards have second-generation tensor cores, which run FP16 and INT8 but not BF16. FlashAttention 2 does not support Turing.

Ampere¶

Ampere cards have third-generation tensor cores, which add BF16 and TF32. FlashAttention 2 supports Ampere.

Ada Lovelace¶

Ada Lovelace cards have fourth-generation tensor cores, which add FP8.

Hopper¶

Hopper cards have fourth-generation tensor cores and the Transformer Engine, which runs FP8. The H100s on PARASOL (formerly DSMLP) are the PCIe variant, with HBM2e memory at 2,000 GB/s. The SXM variant has HBM3 at 3,350 GB/s, so a PCIe H100 has about 60 percent of an SXM H100's bandwidth.

Blackwell¶

Blackwell cards have fifth-generation tensor cores, which add FP4 (NVFP4) to FP8.

MIG Slices¶

Multi-Instance GPU (MIG) divides one card into isolated slices. Each slice has its own memory, cache and compute, and a session on a slice can use only that slice. Sessions on different slices of one card do not share memory or bandwidth.

A slice's name gives its compute and its memory: 2g.20gb is two compute units and 20 GB. A compute unit is a seventh of an H100's compute, or a quarter of an A30's or an RTX PRO 6000's. A slice's bandwidth follows its share of the card's memory rather than its compute. MIG divides an H100's memory into eighths, so the 2g.20gb slice has two-eighths of the card's bandwidth.

Inside a session, nvidia-smi -L lists the card and, on a slice, the slice's profile.

CUDA Versions¶

As of Fall 2026, PARASOL supports CUDA 12 and CUDA 13. The standard GPU image, scipy-ml-notebook, includes CUDA 12. The image's Dockerfile records its exact toolkit version; see Finding the Package List. For a custom image that carries its own toolkit, see Custom CUDA Toolkits.

CUDA Profiling¶

Three profilers work on PARASOL. GPU performance counters are open to users inside the container, so each can collect hardware metrics.

The standard images do not include Nsight Systems or Nsight Compute. NVIDIA's container images at nvcr.io are fully supported on PARASOL and include both. The PyTorch profiler is part of PyTorch, which scipy-ml-notebook includes.

Nsight Systems and Nsight Compute¶

Launch an NVIDIA image, such as nvcr.io/nvidia/pytorch, with -i. -s starts a shell and no notebook server, because these images are not built on the standard images:

launch-scipy-ml.sh -W <WORKSPACE> -g 1 -l gpu-class=medium -i nvcr.io/nvidia/pytorch:<tag> -s

These images are large, so the first launch on a node is slow. See Pulling a Large Image Ahead of a Session.

Profile the whole run with Nsight Systems to find where the time goes, then profile the kernels that dominate with Nsight Compute:

nsys profile -o timeline python train.py
ncu --launch-count 20 -o kernels python train.py

Nsight Compute replays each kernel many times to read its counters, so a run under ncu is far slower than a normal one. --launch-count limits it to the first kernels launched. Both reports open in NVIDIA's desktop applications of the same names.

PyTorch Profiler¶

In a PyTorch script, wrap a few training steps:

import itertools

from torch.profiler import ProfilerActivity, profile

with profile(activities=[ProfilerActivity.CPU, ProfilerActivity.CUDA]) as prof:
    for batch in itertools.islice(loader, 10):
        train_step(batch)

print(prof.key_averages().table(sort_by="cuda_time_total", row_limit=10))