Skip to main content Skip to navigation

Department of Physics

Connecting to Jupyter

Setting up packages and Security for your Notebook Server

If you have a "freshly-created" SCRTP account, the first time you try to get a notebook server up and running, you will not have appropriate python packages installed. First we need to run an installer script, so please follow steps 1 to 4 of the instructions below. Replace "phrxyz" with your SCRTP username (notyour university number)

Part 1: Install packages for Jupyter and ASE

  1. Log on to the node via ssh: Load a terminal window on your local machine and connect to a remote machine such as hetmathsys1
     ssh phrxyz@hetmathsys1.scrtp.warwick.ac.uk
    You will need your SCRTP username and your SCRTP password. If this succeeds, you will be at the Linux command line on hetmathsys1.
  2. On hetmathsys1, run this script for setting up various packages to allow you to runJupyter Notebooks(cut and paste the command):
    /storage/physics/apxpcw/QAM/install_jupyter
  3. When prompted, choose a password for your Jupyter Notebook - this can be the same as your SCRTP password.
  4. Ask for help if you get any error messages in red. An exception is any that begin "ERROR: pip's dependency resolver does not currently take into account all the packages that are installed" which is (as far as I know) harmless.
  5. At the end of the setup process, you will need to source your .bashrc file again as instructed.

The current modules this loads (subject to change!) are:
module purge; module load GCC/11.3.0 OpenMPI IPython jupyter-server JupyterLab SciPy-bundle

There is no need to repeat any of the above steps in future, as long as they worked without errors (shown in red) the first time.

Part 2: Enabling a Direct Connection

To enable a secure, https-based connection to your jupyter notebook server (so that it is safe to enter your password to connect to other machines), we create and add a ssl key and add it to your jupyter notebook configuration. I have created the following script which does this (read through it first if you are interested):

/storage/physics/apxpcw/QAM/add_key_to_jupyter_config
You will be prompted for information about yourself for the private key you will use. You should put some basic details in here so the key is distinguishable from others, but there is no need to fill it all in accurately as no one else sees this data.
Please note this is a "self-signed" key, not lodged with any certificate authority, so it will not be recognised by your browser as sufficient for what it thinks is a secure connection. You will have to tell the browser to connect anyway: the exact way to do this differs between different browsers but it is usually under "Advanced" or similar on the warning page (yellow borders in firefox).

Part 3: Launching a Notebook Server

When you have all the setup tasks launch the notebook by running the command:
nohup jupyter notebook &
(the & puts it in the background so you can still use the terminal window, and the "nohup" makes it run a persistent notebook that does not get cancelled if you close the terminal window).
Now type
cat nohup.out
If the process ended immediately, there may be an error message in "nohup.out" so ask a demonstrator about that if the solution is not obvious.
Otherwise, the output of this command should tell you what port (eg 8888) it ran on - the last 4-digit number in the output after the address. Make a note of this number.
How you now connect to your notebook depends on whether you are on a machine on the Warwick network or outside it. If you are outside the Warwick network you will need to skip the next section and follow the instructions below under "Remote Access".

Part 4a: Connecting from within the Warwick Network

You can now use your browser to navigate to the jupyter notebook. If you are connecting from within the Warwick network or are on a remote desktop connection to an SCRTP machine, you will need a link that is similar to one of these (modify as appropriate):

You should replace "hostname" with the machine you are running on (eg "stan1") and replace "8888" with the right port number as reported by nohup.out. If you don't know the hostname of the machine, then "uname -n" will tell you.
You should now be presented with the "Home" page of your instance of Jupyter.

Part 4b: Connecting from Outside the Warwick Network (Remote Access)

From outside the Warwick network, the range of ports on which we run these notebooks is blocked for external access by the university firewall. Therefore we need to establish an "ssh tunnel" via an allowed, secure method, ie ssh. The purpose of this tunnel is to map a TCP/IP port on your local machine to a port on a remote machine. For simplicity we will use the same port number on both machines.

Let's say you want to get access to a Jupyter Notebook on stan1, running on port 8889. To map port localhost:8889 on stan1 to localhost:8889 on your own machine you would run the following command in a terminal window on your own computer(the one you are at which you are physically sitting - not the one where you are running the server!):

  ssh -N -f -L localhost:8889:localhost:8889 phrxyz@stan1.csc.warwick.ac.uk

Replace "stan1" and "phrxyz" and both copies of the port number "8889" with the appropriate values for the machine you have run on, your SCRTP username and the port you got assigned.

Then you would point your browser at this

 https://localhost:8889Link opens in a new window

(again, replace the port number) to launch the notebook. You will have to say yes or "continue anyway" to various dire warnings about self-signed certificates (these are OK as you made them).

If you try to connect again later on, but get no response, it maybe that you need to restart the notebook (eg after a reboot of the server).

Part 5: Fetching and Downloading a Workshop Notebook

To experiment with the functionality, launch a new notebook: Choose "New" then "Python3" under "Notebooks". This will launch in a new browser tab. It's helpful to leave the "Home" tab open ... if you close it you can get it back by following the same link (which you could bookmark for convenience). Once you have familiarised yourself with the interface, you are ready to begin the Workshop.
If you have successfully followed the instructions in the previous section, you should have a tab labelled "Assignments" in your Jupyter Notebook Home tab. Ask for help if this tab is blank or if the dropdown box labelled "Released, downloaded, and submitted assignments for course:" cannot be set to "QAM".
When you are ready, click "Assignments" and then click the "Fetch" button next to the appropriate notebook for this workshop. It will fetch all files associated with the notebook. These should then appear under "Downloaded Assignments". Clicking the name to open either part will launch a new tab. From then on, follow the instructions inside the Notebook.

Part 6: Shutting down a Jupyter Notebook Server

You can shut your notebook server down from inside the notebook (there is a "Quit" button) or from the command line by killing the process ("fg" to bring it back to foreground, the Ctrl-C).

Updating Packages

Those with an older SCRTP account found some of their Python packages are out of date. The following worked to update them but may no longer be compatible with the new OS:

pip3 install --user --upgrade pygments prompt-toolkit ipython nbclient nbconvert jupyter-core jupyter-console

If you have trouble with nglview, this command may help (you'll have to restart jupyter after running this)

pip3 install --upgrade ipywidgets==7.6.0