The standard images are the software environments IT Services maintains for Datahub and PARASOL (formerly DSMLP), and most courses and projects run on one of them. Find out which image a session runs, what it contains, and how a course can pin its workspace to one version of an image for a term.
The canonical list of the images IT Services (ITS) maintains, with their
current tags, is the
ucsd-ets/datahub-docker-stack
repository. Three standard images cover most courses and projects, receive
priority support, and serve as the starting point for a custom image. See
Building & Publishing a Custom Image.
| Image | What it adds | GPU |
|---|---|---|
datascience-notebook |
Widely used data analysis libraries from the Python, R and Julia communities. The default for most courses. | No |
scipy-ml-notebook |
Everything in datascience-notebook, plus CUDA/GPU tooling, TensorFlow and PyTorch. |
Yes |
rstudio-notebook |
Everything in datascience-notebook, plus the RStudio development environment. |
No |
rstudio-notebook is not GPU-enabled. It extends the CPU image,
datascience-notebook, not the machine-learning image, scipy-ml-notebook. A
course that needs both RStudio and GPUs cannot meet that need by choosing
rstudio-notebook. It requires a custom image or a second environment.
Instructors of such a course contact ITS through
Getting Help.
RStudio opens from the JupyterLab launcher and does not require a separate
sign-in. The first RStudio session in a new account requires a personal package
library before install.packages() works. The procedure is in
Installing R Packages.
quay.io/jupyter/datascience-notebook Jupyter's Python, R and Julia image
└── datascience-notebook + tools, packages and nbgrader
├── scipy-ml-notebook + CUDA, TensorFlow, PyTorch
└── rstudio-notebook + RStudioA child image contains everything in its parent. A package in
datascience-notebook is also in scipy-ml-notebook and rstudio-notebook.
The reverse does not hold.
datascience-notebook is built on the Jupyter project's
quay.io/jupyter/datascience-notebook image, and is the base of the other two
standard images. It is the smallest standard image, and the base to derive
from when build time or image size matters. A custom image built on
scipy-ml-notebook inherits the whole CUDA stack whether or not the course uses
it.
scipy-ml-notebook already carries CUDA with a matching PyTorch and TensorFlow.
A custom CUDA image is rarely needed. A custom CUDA stack must stay compatible
with the driver on the node.
Building & Publishing a Custom Image covers custom
image builds.
CUDA Versions gives the CUDA
versions PARASOL supports.
Images are published to the GitHub Container Registry and are named
ghcr.io/ucsd-ets/<image>:<tag>:
ghcr.io/ucsd-ets/datascience-notebook:stable
ghcr.io/ucsd-ets/scipy-ml-notebook:2024.4-stableThe :stable tag follows each update. A dated tag such as :2024.4-stable
does not. A workspace that must not change partway through a term uses the
dated tag. See Pinning a Workspace.
A dated tag names the year and the quarter of the build: 2024.4-stable is the
Fall 2024 image, and 2023.2-stable the Spring 2023 image. The
datahub-docker-stack
repository lists the tags currently published.
The older ucsdets/<image> naming still appears in published articles and in
course repositories. Where a launch command needs a full image name, prefer the
ghcr.io/ucsd-ets/... form.
datahub-base-notebook, which some published articles and course repositories
name as a base image, is no longer built or updated. Derive from
datascience-notebook instead.
The image a session runs depends on how the session is launched.
On Datahub, the workspace determines the image. Its spawn menu offers one or more configurations, each an image together with CPU, RAM and GPU quantities, from which students and instructors choose. A course commonly offers a CPU-only environment for most of the term and a GPU environment for its project. What a Workspace Is and What It Controls describes workspace settings.
From the shell, the wrapper script determines the image unless -i overrides
it. Bare launch.sh defaults to ghcr.io/ucsd-ets/scipy-ml-notebook:stable.
launch.sh Reference describes
wrapper scripts and flags.
Package lists change with every image build. The current contents are published from the image repository:
datahub-docker-stack/images.
This is also where the CUDA toolkit version in scipy-ml-notebook is
recorded.For a package or CUDA version that no published image carries, write to parasol@ucsd.edu. Customizing an Environment covers adding a package that a standard image lacks.
A cuda.md file in the ucsd-ets/dsc200-notebook repository publishes a table
of GPU models, counts and node names dated Fall 2020. It is not current. Do not
size work from it. GPUs are requested by class label. See
GPU Classes.
GPU Hardware & CUDA gives the specifications of
the cards and slices that can back a class.
ITS generally updates the standard images ahead of Fall quarter, and again as needed during the academic year. New package versions are added, old ones are removed, and behavior occasionally changes with them. An assignment validated in week 2 against one version of a library can fail against the next.
A workspace may pin its image so that an update does not move a class to
different software partway through a term. Members stay on the build the
course tested until a move is requested. A course that needs a fixed environment
for the duration of a term requests a pinned tag rather than relying on
:stable. See Pinning a Workspace.
What a Workspace Is and What It Controls
describes workspace settings.
| Tag form | Behavior |
|---|---|
:stable |
Follows each update. Always the current build. |
A dated tag, e.g. :2024.4-stable |
Fixed. That build, until a different one is named. |
A course image branch tag, e.g. :wi24 |
Rebuilt on every push to that branch. Not fixed. |
A course image git tag, e.g. :fa24 |
Fixed at the tagged commit. |
Warning
A branch tag does not pin an image. The next push overwrites the branch tag,
so an image set to wi24 still changes whenever a commit is pushed to wi24.
To freeze a course image build, create a git tag and ask that the course use it. Course Images covers course image branches and tags.
Container tags are a system-side course setting, adjusted by ticket in the same way as resource limits and disk quotas. ITS staff make the change. Email to parasol@ucsd.edu opens a ticket. To pin a workspace:
Administrative Requests describes the ticket process. Instructors, TAs & Course Staff describes course setup timing. A pin applied after students have started is possible, but it changes their environment a second time.
A command-line launch names the image directly, so a fixed tag pins that launch:
launch.sh -W DSC40_FA26_001 -i ghcr.io/ucsd-ets/datascience-notebook:2024.4-stableThis affects only that account's own launches. It does not change the image the workspace serves to anyone else, and it does not replace a course pin.
A pinned image stops receiving package updates, including security fixes to the software inside it. A pin applies for a term, not indefinitely.
A pin fixes the image only. The node, its drivers and the cluster around it continue to be maintained. For GPU work, the driver on the node is not part of the image and is not fixed by a pin.
A move to a newer base image requires assignments to be re-validated against it. Assistance with maintenance following an image update is within the scope of a 1:1 Consultation. See Support & Technical Consultation.