Using the VSC Jupyterhub¶

The VSc has a jupyterhub on jupyterhub.vsc and offers a few nodes to all users with accounts.

You can run interactive Notebook sessions on VSC4 and VSC5.

You can get access to A100 and A40 (VSC4) GPU nodes if you need. That might be a good way to test your code, before requesting more resources via the batch system.

Spawning a new notebook server¶

One has to make a decision from the available options:

spawn-server

Select a profile (VSC4 or VSC5)

There are basic options:

VSC4 Singularity
VSC5 Singularity
VSC4 python venv (virtual environment)

select-profile

Select the singularity/apptainer image to use:

notebook-image

Select the necessary resources:
- Memory
- Cores
- Runtime

If you are ready, then press start and wait until your Notebook server is served and you are redirected.

Opening this page in a new tab allows you to get a feeling for the usage of that service.

Using the VSC Jupyterhub¶

The Jupyterhub is a bit different from the Jupyterhub on JET. Some of the profiles (singularity) are containerized runtimes, which have only limited access to the host (no VSC modules), but profiles like python venv do.

There are images for VSC4 and VSC5, remember that /gpfs/jetfs is only available on VSC5.

It is possible to use a customized jupyter image and specifying the path:

Available Images:

/gpfs/data/fs71386/imgw/containers/Jupyter.imgw.202409.sif
/gpfs/data/fs71386/imgw/containers/Jupyter.PyMagic.202506.sif

This will then launch the Jupyterhub with these defined environments. If you have any specific paths that are not available inside of the container then, please set the SINGULARITY_BIND variable in your .bashrc and restart the jupyterlab server.

The Recipe for building a container for the Jupyterhub on VSC is available here and looks like this:

# Singularity Recipe for VSC Jupyterhub
# By MB
# build:
# apptainer build --ignore-fakeroot-command --ignore-subuid Jupyter.vsc.img.sif Singularity.jupyter.img

# version 3.1.1 is required by VSC (2024.09)
BootStrap: docker
From: jupyter/minimal-notebook:hub-3.1.1

%post
# install some required packages:
# - jupyterlab, widgets, monitor
# - additional packages
mamba install --quiet --yes jupyterlab-system-monitor \
    jupyterlab-git \
    jupyterlab_widgets \
    xarray \
    netcdf4 \
    zarr \
    cartopy \
    cartopy_offlinedata \
    eccodes \
    scipy \
    nco \
    h5netcdf
# clean caches
mamba clean --all -f -y
# need to install batchspawner for hub integration
/opt/conda/bin/pip install --no-cache-dir jupyterhub==3.1.1 git+https://github.com/katringoogoo/batchspawner.git@1.2.0+auth_fix ecmwf-data ecmwf-opendata 
# add modules to default
cp /opt/conda/init/profile.sh /etc/profile.d/modules.sh
# You can simply use the solutions provided in the warning. Setting the env variable in the definition file is probably sufficient;
%environment
TINI_SUBREAPER=true
# fin