JupyterHub

What it is for

JupyterHub gives each analyst or admin their own JupyterLab in the browser, signed in with the same Keycloak account. They use Python (pandas, charts) and query the lake through Trino. Vendors and logins without a role are refused (the realm role arrives as the roles claim; the hub’s admins are the realm admins).

How it is built

Piece	Detail
Hub	`jupyterhub` service, runs as root (it must start other users’ units); venv `/opt/jupyterhub/venv`; version 5.5.x (6.x is not yet compatible with the spawner and authenticator used)
Proxy	`configurable-http-proxy` (npm, `/opt/jupyterhub/chp`, version `CHP_VERSION`)
Config	`/etc/lake/jupyterhub/jupyterhub_config.py`, `lakenames.py`; secrets in `jupyterhub.env` (root only, mode 600)
State	`/var/lib/jupyterhub`
Ports	8000 (open this), 8001 proxy API, 8091 hub API (8081 is Spark)
Notebook environment	one shared venv `/opt/lake-notebook/venv`, root-owned and read-only (list: `requirements-notebook.txt`)
Each user	a transient systemd unit `jupyter-nb-<name>-<hash>` with `DynamicUser` (a throwaway Linux user), read-only system, private `/tmp`, no new privileges, no sudo
Home	`/var/lib/nb-<name>-<hash>`, kept between sessions
Keycloak client	`chdc-notebook` (confidential), created by `keycloak_client.py` on every run of step 18

Linux names come from lakenames.py: e-mail addresses are not valid user names, and the nb- prefix keeps notebook homes away from service folders.

Limits (per user)

Setting	Default	Variable
Memory	4G (the kernel is stopped, not the box)	`NOTEBOOK_MEM_LIMIT`
CPU	2 cores (`CPUQuota` 200 %)	`NOTEBOOK_CPU_LIMIT`
Idle time	60 minutes, then the server stops, files stay	`NOTEBOOK_IDLE_MINUTES`

Check it

systemctl is-active jupyterhub
curl -s http://127.0.0.1:8000/hub/health && echo ok
systemctl list-units 'jupyter-nb-*' --no-legend           # running notebook servers
journalctl -u jupyterhub -n 50 --no-pager

How to use it

As an analyst: open http://localhost:18000 (development tunnel) or the web app’s Notebooks link. Sign in with your lake account. The first start copies start.ipynb to your home. See the Notebooks guide.

As an admin:

Task	How
See who has a running server	`systemctl list-units 'jupyter-nb-'` or the hub’s Admin* page
Stop one user’s server	Hub Admin page > Stop, or `sudo systemctl stop jupyter-nb-<name>-<hash>`
Check a unit’s limits	`systemctl show jupyter-nb-<name>-<hash> -p MemoryMax -p CPUQuotaPerSecUSec -p DynamicUser`
Add a Python package	Add it to `infra/jupyterhub/requirements-notebook.txt`, deploy, re-run step 18. `installed.txt` in each venv records every version actually installed
Free a user’s disk	Their home is `/var/lib/nb-<name>-<hash>`

Configure it

Settings come from cluster.env via step 18: JUPYTERHUB_CLIENT_SECRET (required, openssl rand -hex 32), CHP_VERSION, optional JUPYTERHUB_PUBLIC_URL and KEYCLOAK_PUBLIC_URL (the addresses people’s browsers use; the defaults are the tunnel addresses), and the three limits above. For the web app’s menu link, set NOTEBOOK_URL for the API.

A re-run of step 18 restarts only the hub; running notebooks keep going.

Security notes

A notebook can reach every table through Trino and every port on the box. This is a demo-level setup. Before real analysts use it: access control in front of Trino (or notebooks that go through the API), a read-only S3 key, a real hostname with TLS, and PySpark with guard rails.
cluster.env must not be readable by notebook users: step 19 locks it down and step 18 warns if it is open.
Notebook users have no S3 keys, so raw files are not reachable from a notebook.

Things that go wrong

Symptom	Likely cause
The hub cannot read the user (401 from Keycloak)	Fixed: the hub reads the user from the ID token; the roles mapper puts `roles` in the ID token
`Invalid redirect` on login	Open the hub at the address configured in `JUPYTERHUB_PUBLIC_URL` (localhost in dev)
`apt` pulled in hundreds of packages / restarted NiFi	Fixed in step 18 (`--no-install-recommends`, `NEEDRESTART_MODE=l`)
Not yet checked	The memory cap stopping a runaway kernel, the idle culler after 60 minutes, a hub restart with servers running