Projects & Independent Study

This page is for work on Datahub and PARASOL (formerly DSMLP) outside a scheduled course. It assumes familiarity with a shell. Access through an enrolled course is separate and follows different rules. See Using Datahub in a Course and Working from the Command Line.

Eligibility & Requesting Access¶

Project access covers for-credit and campus-sponsored work outside a scheduled course:

Designated independent study courses: 198/199, 293/298/299
Independent thesis or dissertation research
State-supported capstone projects
Campus-sponsored co-curricular activities, such as projects, workshops, clubs, and teams

Non-credit personal projects may be approved on a case-by-case basis as resources are available. The programs in Self-Supporting Programs are eligible, subject to the cost recovery set out there.

For non-students, and for research that is not for credit, Research IT can help connect faculty, staff, and student researchers with compute platforms. Research on PARASOL covers individual research on the cluster.

Submitting a Request¶

Request access through the Independent Study Request form. A request states who is sponsoring the work, what will be run, the resources required, and for how long. A request that names a faculty sponsor and a concrete workload is provisioned considerably faster than one that describes the work only in general terms. Setting up access takes up to 5 business days from the request. See Independent Study, Capstones & Special Projects.

How Project Access Differs from Course Access¶

Most of what may be obtained for a course may be obtained for a project, and the mechanics are the same. Project access differs in priority, support, duration, and environment configuration.

Priority Relative to Coursework¶

Instruction has priority. When the cluster is busy, during 10th and Finals Weeks and at major assignment deadlines, coursework is served ahead of project work. Demand is generally lower during the day than in the evening.

Support Tier¶

Project work has no instructor or TA as a first tier of support. Support questions go directly to the ITS Service Desk, whose staff have no prior knowledge of the project. Project issues may take longer to answer than issues in a course. See Individual User Issues. Support lists the contacts.

Duration of Access¶

Project access is granted for a defined period rather than tied to an instructional term. Submit an extension request before access lapses rather than afterwards. When Access Starts & Ends covers access periods.

Environment Configuration¶

No environment is configured on the student's behalf, and no course menu of environments is offered. A course receives an image selected and tested by its instructor. A project receives the standard images and whatever is built upon them, and the project decides what runs and how. Environments covers images.

Getting Set Up¶

Sign in at datahub.ucsd.edu for the browser route, or ssh to the login node for a terminal. Duo applies, and the VPN is not required for either route. See Connecting over SSH. Access describes the routes.
Locate the project workspace. Project access is provisioned as its own workspace, with its own home directory, separate from any course the student is enrolled in. Where both exist, select the workspace deliberately at launch. See Belonging to Several Workspaces.
Launch a container with launch-scipy-ml.sh for the GPU-capable image, or launch-datascience.sh for the CPU image. Running Jobs covers launching.
Place large inputs in shared or external storage rather than in a per-user copy. A project home directory is not large. Workspaces & Storage covers storage.

The login node is for launching jobs and moving files, not for running work. See What the Login Node Is For.

Doing the Work¶

Project work uses the same procedures as course work, documented on these pages.

Task	Documented in
Request CPU, memory, or a GPU	`launch.sh` Reference
Run work that outlives the terminal	Job Modes
Use `sbatch` and related commands	Slurm Compatibility Wrappers
Edit in Visual Studio Code	Remote Editor Setup
Install packages, or build an image	Environments
Move data in and out	Moving & Sharing Data
Share results with collaborators	Inside the Workspace

Service Unit Budget¶

Warning

Launching an eligible GPU session draws on the project's Service Unit budget, whether or not the reservation calendar is used. See On-Demand Lease Charges.

No TA observes a project's consumption or remarks when the budget is nearly exhausted. Remaining Balance covers the balance.

Idle GPU Sessions¶

A GPU session that stops using its GPU is reclaimed. See What Counts as Idle.

Missed Reservations¶

A booked window that is missed is charged. See The Claim Window and The Cancellation Penalty. A waiver is a workspace-manager action, and no TA watches a project calendar to offer one. Canceling at least 24 hours ahead costs nothing; a later cancellation keeps part of the cost.

Reservation Events¶

The reservation system reports on a GPU session with Kubernetes events, shown by kubectl describe pod, by kubectl get events after the pod is gone, and by Datahub while a session starts. Reservation Events defines each one:

While a session waits: WaitingForReservation, ReservationFull, ReservationTooSmall, OnDemandLeaseDenied, OnDemandLeaseRejected, OnDemandAdmissionPaused, UnknownGpuClass, NoReservation, NoGpuRequest, AnnotationIgnored, NoMatchingNode, WaitingForNode.
When it is admitted: RuntimeGuaranteed, ReservationRelinked, OverstayRelinked, BestEffortAdmitted.
When it is stopped: Preempted, ReservationCancelled, ReservationReassigned.

Checkpointing¶

Checkpoint long-running work. See Checkpointing & Logging Long Runs.

Troubleshooting¶

See Error Messages. Common symptoms in project work are documented on these pages.

Symptom	Documented in
`OOMKilled`	Resource Requests and Limits
`DeadlineExceeded`	The Runtime Limit
`Pending`, with a `FailedScheduling` event	Normal while the reservation system decides; the reason is in the reservation event beside it. See Missing or Misspelled Class Label
The session ended unexpectedly	What Counts as Idle, or End of a Reservation Window
Reservation-related events in a pod	Reservation Events

Outgrowing Project Access¶

Work has outgrown project access when it needs capacity beyond what a project allocation carries, has become a lab's work rather than one person's, needs multi-day reservations, or needs storage measured in terabytes.

Situation	Route
Independent work on an ongoing basis	Research on PARASOL
Provisioning for a group	Provisioning belongs with a PI. Refer the PI to Setting Up a Research Lab.
Work beyond this cluster	Research IT can discuss other platforms.

Policy¶

Highly sensitive data (P4), such as clinical records or export-controlled information, must not be used on PARASOL, and legally or contractually protected information (P3) may be permitted after review. See Data Classification. Policy sets out appropriate use, the sharing of compute resources, and the other conditions of use.

Support¶

Questions and tickets go to parasol@ucsd.edu or the ITS Service Desk. A ticket includes what was run, the full error, and the workspace name. Contents of a Support Ticket describes ticket contents further. Getting Help describes other routes.

Report a problem with this page