NeCTAR¶
As stated on the Nectar website:
[Nectar] provides an online infrastructure that supports researchers to connect with colleagues in Australia and around the world, allowing them to collaborate and share ideas and research outcomes, which will ultimately contribute to our collective knowledge and make a significant impact on our society
but note the emphasis on research. So should anyone require a VM for another purpose (automation, web server, etc.) you cannot use the platform. But this doesn't necessarily mean that door is 100% closed: the NeCTAR national server program is housed at various locations around the country, and there is a node here at UTAS, the [[https://www.tpac.org.au/|Tasmanian Partnership for Advanced Computing]] (TPAC). Each node of the server program can allocate resources on a discretionary basis, meaning provided TPAC is happy for the project to run, it is in principle possible. There are a number of downsides to this, notably that the project must be renewed every 12 months; however, it is a pathway forward. To set out on this journey, use the [[https://dashboard.rc.nectar.org.au/|NeCTAR cloud login]], and once inside, select //Allocations// and create a //New Request//. The team has stated that the more details that are provided about a given project, the faster things will move.
Jove2021¶
This section is definitely a case of activity-based learning: what appears below is the documentation of the deployment of JupyterHub on the Jove2021 server, which is hopefully illuminating for the deployment your own NeCTAR instance.
Summary¶
Getting the NeCTAR allocation approved was both straightforward and not, with bugs in the application process making things a bit sticky, but once these were removed, it all moved pretty smoothly.
Getting started I have never administered virtual machines, only used them, so this will be a learning process!
I started by creating an instance jove with an Ubuntu 21.04 image. This was a fiarly straightforward process, with the only requirement that I select the flavour of the installation, which in my case was m3.xlarge, chosen such that I could actually use the resources that I have been allocated.
I generated an SSH key pair (jove-base) but this is where I started to run out of gas, as it was not clear if I could talk to the instance. With not much searching, I came across the NeCTAR tutorials, which fortunately had a discussion of exactly this. The main thing to do was create a security group which allowed for SSH access to the VM.
First access The IP address of the VM is given in the Instances display under Compute, and in this case, it is 131.217.173.123. From the tutorial, a default user has been configured on all images, and in the case of Ubuntu images, the default user is ubuntu.
ssh -i 'C:\Users\ajm32\OneDrive - University of Tasmania\Physics\Networking\Credentials\jove-base.pem' ubuntu@131.217.173.123
Success! The first thing to do is to set the user password for user ubuntu:
$ sudo passwd ubuntu
which set to the "standard" physics labs password.
VNC I followed this guide (although actually found this guide to be better) to get the VNC server up an running, which uses the Xfce desktop environment (a fast and lightweight alternative to Gnome) and installation is performed via:
$ sudo apt install xfce4 xfce4-goodies
The suggested VNC server is TigerVNC, which is installed using
$ sudo apt install tigervnc-standalone-server
Repository locations
In the process of trying to install the above software, I lost connection to the Australian mirror for ubuntu repository, making it difficult to get new software. Consequently, I wanted to change it to <anything else>, which I had never done previously. The location of the apt servers is kept in the /etc/apt/sources.list.d/ directory, and I simply changed http://au.archive.ubuntu.com/ubuntu/ to http://archive.ubuntu.com/ubuntu/ and my problem was solved. I will change it back once I have finished with the setup.
To set the VNC password, one can use
$ vncpasswd
which was once again set to the "normal" lab password. TigerVNC can be configured to use Xfce by creating the file:
$ nano ~/.vnc/
and adding the lines
#!/bin/sh
unset SESSION_MANAGER
unset DBUS_SESSION_BUS_ADDRESS
exec startxfce4
The xstartup file requires executable permissions, which can be granted via
$ chmod u+x ~/.vnc/xstartup
and the VNC server can be tested using
$ vncservice
Permission issues
Despite altering the permissions above, the vncservice could not execute due to permission denied to execute xstartup, but executing
$ sudo chmod 755 ~/.vnc/xstartup
fixed the issue.
To start the server on boot, one can make a systemd unit file:
$ sudo nano /etc/systemd/system/vncserver@.service
and add the lines:
[Unit]
Description=Remote desktop service (VNC)
After=syslog.target network.target
[Service]
Type=simple
User=ubuntu
PAMName=login
PIDFile=/home/%u/.vnc/%H%i.pid
ExecStartPre=/bin/sh -c '/usr/bin/vncserver -kill :%i > /dev/null 2>&1 || :'
ExecStart=/usr/bin/vncserver :%i -depth 24 -geometry 1440x900 -localhost -fg
ExecStop=/usr/bin/vncserver -kill :%i
[Install]
WantedBy=multi-user.target
Note here that the -localhost option means that the vncserver only allows connections which originate from the instance to be enacted. It is then necessary to reload the daemon process, enable to service on boot and then start the process, which can be done sequentially as follows:
sudo systemctl enable vncserver@1.service $ sudo systemctl start vncserver@1.service
The process can be verified by checking the status:
$ sudo systemctl status vncserver@1.service
In order to securely connect to the VNC server, the best practice is the SSH into the VM and then connect to the VNC. Given I am on a windows machine, I cannot natively establish a tunnel, rather I need to use an SSH client, which means using PuTTY. Making matters worse, the private key encryption used by the VM for SSH is not compatible with PuTTY, so one needs to use PuTTYgen to covert the .pem file with the key to something that putty can handle. The process is something that is described in an AWS tutorial.
Then a tunnel is created with a source port of 59000 and a destination of localhost:5901, meaning that a VNC client can be pointed to port 59000 and this will be routed to port 5901 on the VM. The session configuration can be saved under the Session tab of PuTTY, meaning that when one wants to access via VNC, one need only select the relevant session.
The real question is does it work? Well, using VNC connect and connecting to localhost:59000, we have:
success!
JupyterHub
Initialisation
Given the point of the service is to run a JupyterHub, I am following the instructions on the github page, which first require that I install python, which I do using conda:
$ curl -O https://repo.anaconda.com/archive/Anaconda3-2021.05-Linux-x86_64.sh
Upon verification that that the hash is good:
$ sha256sum Anaconda3-2021.05-Linux-x86_64.sh
I installed conda:
$ bash Anaconda3-2021.05-Linux-x86_64.sh
Changing to the shell environment to the base conda environment:
$ source ~/.bashrc
one can proceed with installing the installation of jupyterhub:
(base) $ conda install -c conda-forge jupyterhub
For the actual installation of JupyterHub, I went with The Littlest Jupyter Hub (TLJH), which has a very easy install procedure:
$ curl -L https://tljh.jupyter.org/bootstrap.py | sudo -E python3 - --admin jove
which ran without error. In principle, one can then directly access the service based on IP address alone (communication is over port 80) but we must take an aside into the NeCTAR DNS service. In short, NeCTAR provides a DNS zone in which records can be set. The zone for this project is jove2021.cloud.edu.au, which already as the NeCTAR name servers, and one need only provide a record to point to the VM instance. In order to get allow access to the VM, one must update the Security Groups to allow HTTP access. It was at this point that I learned that unless specified, the VM will boot with the default group, meaning that if appropriate, one should simply add SSH, HTTP, and HTTPS access to the default group, rather than creating new groups. With this done, one can access the JupyterHub. As it stands, there is a single admin user with the standard physics labs password
Customisation
The JupyterHub is highly configurable, but the basic steps are to set up how people get access to the server. This simplest method for authentication is to distribute a single shared password, but this is obviously not ideal. The best would be to allow people to sign in with their UTAS credentials, but that would be torture to implement. It is possible with a GitHub username, so that might be something to do into the future. In the short term, one can just insist that users create a username and password, and request authorisation, and just auto approve usage, unless this turns out to be a bad idea. To do this:
sudo tljh-config set auth.NativeAuthenticator.open_signup true $ sudo tljh-config reload
The next thing to do is to limit the allocated resources to a given user, which is done via:
sudo tljh-config set limits.cpu 2 $ sudo tljh-config reload
where the allocation is for 4 GB of memory and 2 CPUs, which is somewhat generous given the total allocation of CPUs on the instance is 16 with 32 GB of RAM, but I certainly do no want people to be limited be resources.
It is also worthwhile to create an admin account for educators that will use the service, which is done through:
sudo tljh-config reload
where the username has been set to pater with password UTASphysics
A useful command to know is:
$ sudo tljh-config show
which shows all the current configurations to the hub.
HTTPS
If the service is to accessed on the web, it is critical that the service be trusted. Following the guide on the littlest jupyter hub, one can convert the http service to a https service using letsencypt as the certificate authority. To do this, one need only execute the following:
sudo tljh-config set https.enabled true sudo tljh-config set https.letsencrypt.email physics.labs@utas.edu.au sudo tljh-config add-item https.letsencrypt.domains jove2021.cloud.edu.au
Package installation
How to install packages to the hub
Distribution of material
As described in the installation guide for distributing materials, a common goal is to have students be directed to the jupyter hub with a single click and have access to the relevant code (and likewise access to data). However, following the guide, I ran into a roadblock early on when the link to mybinder.org did not send where to anything useful.
Using it as a start point, I then followed the trail to the nbgitpuller page, which has installation instructions, but in short it is to install nbgitpuller (which was already installed) and then link to a git(hub) repository, which will be cloned locally for all students, and updated whenever the link is clicked.
As a refresher (for when I make notes)
git init git commit -m "Initial commit"
git push origin master
Although the branch main should be used into the future.
With an active git repository, one then can use nbgitpuller link generator, which requires that one input JupyterHub URL: https://jove2021.cloud.edu.au/ Git Repository URL and branch File to open Application to open: Classic Jupyter Notebook
For Jove, one can autofill the hub link with the addition of 'link?hub=https://jove2021.cloud.edu.au' which yields the "master" link. One can also add the 'repo' and 'branch' tags, to create a mostly populated page
Troubleshooting¶
End of life¶
If as part of ones routine server maintenance you are treated to a suite of errors along the lines of
The repository 'http://au.archive.ubuntu.com/ubuntu <UBUNTU RELEASE>' no longer has a Release file
when attempting to update packages, there can be a few causes, but perhaps the most common is being end-of-lifed whereby the OS is removed from the list of supported distributions. You can continue to use the system, although for security reasons it should be upgraded; you can check the status of ubuntu distributions on the ubuntu wiki. Should you wish to continue using your chosen distribution, you will need to alter the location of the repositories used to source packages as described here, but again it is worth emphasising that end of life versions no longer receive security updates, and so if choose to do this, don't think you're getting the latest software.