Connecting/Logging in

All the connections to the Hoffman2 Cluster are based on a secure protocol that requires authentication. Currently we support the following ways to connect to the cluster:

it is also possible to perform remote direct rendering on GPU nodes.

As all connections are based on a secure protocol, when logging in for the first time, for security reasons, you will be asked to confirm the authenticity of the host you are connecting to by double checking the hostkey fingerprint.

Connecting via terminal and SSH

One of the most powerful ways to use the cluster is via a Unix shell, such as Bash, tcsh, or others. Unix shells are programs that provide an interface between the user and the operating system via the command-line shell prompt onto which the user types commands. A Unix shell is executed by a terminal program. Depending on the operating system of your computer, you have several choices for which terminal to use when connecting with the cluster.

On any Linux distribution, you can use the standard SSH-client generally installed with the OS and available via any terminal application. Open a terminal window and type:

$ ssh login_id@hoffman2.idre.ucla.edu

where login_id is replaced by your cluster user name.

To prevent SSH connections dropping, we recommend setting the following entries in your ssh_config file or your $HOME/.ssh/config:

$ ServerAliveInterval=30
$ ServerAliveCountMax=5

On macOS (the operating system running on Mac computers) an SSH-client is available and can be used via the Terminal application (although other terminal programs are also available for macOS). The Terminal application can be located via Spotlight Search or Finder. Once you have opened a Terminal window, type the following:

$ ssh login_id@hoffman2.idre.ucla.edu

where login_id is replaced by your cluster user name.

To prevent SSH connections dropping, we recommend setting the following entries in your ssh_config file or your $HOME/.ssh/config:

$ ServerAliveInterval=30
$ ServerAliveCountMax=5

On Windows, install any one of the following free ssh client programs, which offer a terminal emulator for connecting to remote computers. Starting from Windows 10, Windows Subsystem for Linux can also be used.

Note

Only one of the following software is needed to connect to the cluster via SSH. The order in which options are ordered from our most to least recommended applications (users may still opt for different solutions).

  • MobaXterm an SSH client that includes an X11 server (for running GUI applications).

  • GitBash an emulation shell provided by Git for Windows <https://gitforwindows.org/>`_

  • Putty an open source SSH and telnet client.

  • Cygwin a collection of tools that provide Linux functionalities in Windows systems.

  • Windows Subsystem for Linux: A Windows native compatibility layer for running Linux binaries on Windows (available from Windows 10).

Tip

MobaXterm will also supply the ability to open GUI application without the need to install any other software (see: Opening GUI applications).

Problems with the instructions on this section? Please send comments here.

Opening GUI applications

Some interactive applications (e.g., Matlab Desktop, Mathematica Notebooks, gnuplot, etc.) require the ability to open a graphical user interface (GUI) window.

This section, and the Connecting via remote desktop section, cover cases in which no direct rendering is required (and software rendering is performed instead). The Remote direct rendering section covers instances when applications need to perform direct rendering on the Hoffman2 cluster graphical nodes and display the results on your local computer.

On UNIX-like systems, an X Window System server is generally available and running with the default installation (or can be readily installed via the OS package manager). You can enable X11 Forwarding (that is the ability to visualize, on your local computer, GUI applications running remotely on the cluster) by adding the -X flag to the ssh command you use to connect to the cluster. To do so, type into your terminal window:

$ ssh -X login_id@hoffman2.idre.ucla.edu

where login_id is replaced by your cluster user name.

Alternatively, you can override the ssh default for all your sessions on the Hoffman2 cluster by creating (or adding to) a file named config in the $HOME/.ssh local directory on your computer and adding the options:

$ Host hoffman2
$ Hostname=hoffman2.idre.ucla.edu
$ User=login_id
$ ForwardX11 yes

where login_id is replaced by your cluster user name. You can then login on the cluster as user login_id simply issuing:

$ ssh hoffman2

On macOS (i.e., the operating system running in Mac computers) the X windows system is called XQuartz which can be downloaded from the XQuartz page. As indirect GLX was disabled by default in Xorg 1.17, starting from XQuartz version 2.7.9, to enable indirect GLX and to allow remote visualization on the cluster, you will have to open the macOS Terminal application and (before connecting to the cluster) issue the command:

$ defaults write org.macosforge.xquartz.X11 enable_iglx -bool true

Note

You will need to reboot your machine before being able to open GUI applications on Hoffman2.

The macOS Terminal application can be used to connect to the cluster (although other terminal programs are also available for macOS). To enable X11 Forwarding (that is the ability to visualize, on your local computer, GUI applications running remotely on the cluster) type into your terminal window:

$ ssh -Y login_id@hoffman2.idre.ucla.edu

where login_id is replaced by your cluster user name.

Note

By default, a mac assumes that a ssh -X connection is untrusted and you therefore need to use ssh -Y`.

Alternatively, you can override the ssh default for all your sessions on the Hoffman2 cluster by creating (or adding to) a file named config in the $HOME/.ssh local directory on your computer and adding the options:

$ Host hoffman2
$ Hostname=hoffman2.idre.ucla.edu
$ User=login_id
$ ForwardX11 yes

where login_id is replaced by your cluster user name. You can then login on the cluster as user login_id simply issuing:

$ ssh hoffman2

On Windows, third-party software needs to be installed. Starting from Windows 10, Windows Subsystem for Linux can also be used. Here is a list of possible choices:

Note

Only one of the following software is needed to connect to the cluster via SSH. The order in which options are ordered from our most to least recommended applications (users may still opt for different solutions).

  • MobaXterm: MobaXterm is a free, enhanced terminal for Windows with X11 server, SSH client, network tools, and much more.

  • Xming and Xming fonts: Xming and Xming fonts are a no-cost X Window System Server for Microsoft.

  • Cygwin: Cygwin is a free Linux-like environment for Windows. To add Cygwin/X server, select the xinit package from the X11 category.

  • Xshell: Xshell is a commercial option.

  • Windows Subsystem for Linux: A Windows native compatibility layer for running Linux binaries on Windows (available from Windows 10).

Problems with the instructions on this section? Please send comments here

Setting up SSH clients for Windows

Instructions for MobaXterm

Download and install the MobaXterm Home Edition, either the Portable or Installer edition will work. To log into the Hoffman2 Cluster open MobaXterm and click on the Session button (the Sessions button is circled in green in MobaXterm application with the Sessions button circled in green.).

Image showing the MobaXterm application open with the Sessions button in the upper left corner circled in green

MobaXterm application with the Sessions button circled in green.

A new window will pop-up to allow you to choose a type of session. Choose SSH by clicking on the SSH button (upper left corner) and fill the Remote host field with the Hoffman2 Cluster address, that is:

hoffman2.idre.ucla.edu

save the session by pressing the OK button.

Image showing the MobaXterm application open with the SSH Sessions in which the Remote host field has been filled with the Hoffman2 Cluster address (hoffman2.idre.ucla.edu). The OK button in the center bottom is circled in green

MobaXterm application with the SSH session set for Hoffman2.

You can also create a new session type which you will use to transfer files back and from the cluster to your local machine, by selecting the Sessions button again and choosing SFTP as the session type and fill the Remote host field with the Hoffman2 Cluster data transfer node address, that is:

dtn1.hoffman2.idre.ucla.edu

save the session by pressing the OK button.

To access the cluster double click on the save SSH session.

Problems with the instructions on this section? Please send comments here.

Instructions for XMing and GitBash

Download and install XMing and Xming fonts following the installer instructions. Download and install GitBash following the installer instructions.

Start Xming from the programs, open GibBAsh from the Start Menu, at its shell prompt issue:

export DISPLAY=localhost:0

Connect to the Hoffman2 Cluster by typing:

ssh -XY login_id@hoffman2.idre.ucla.edu

where login_id is replaced by your cluster user name.

Instructions for Cygwin users

  • Install Cygwin (including the X11 packages) on your Windows system.

  • Open a Cygwin terminal.

  • Enter the command: startxwin. This starts the X11 server on your Windows computer.

  • In the (graphical) window that pops up, use the command ssh -X login_id@hoffman2.idre.ucla.edu where login_id is replaced by your cluster user name.

Problems with the instructions on this section? Please send comments here.

Instructions for Windows Subsystem for Linux

Starting from Windows 10, Windows offers a compatibility layer for running Linux binaries on Windows. The Windows Subsystems for Linux (WSL) needs to be enabled before it can work. You can enable WSL via Windows Settings or typing commands on a Windows PowerShell.

From the Start Menu:

  • open Settings

  • click on Apps

  • scroll down to Related Settings

  • click on Programs and Features

  • click on Turn on Windows features on or off

  • scroll down and select Windows Subsystems for Linux and click on the OK button

  • click on Restart Now button

At this point you can access the Microsoft store to install a Linux distribution or start using the WSL shell provided by Microsoft.

See WSL Microsoft documentation to learn how to enable WSL via Windows PowerShell.

Problems with the instructions on this section? Please send comments here.

Connecting via remote desktop

NX is a secure, compressed protocol for remote X Window System connections for Windows, Linux, Mac OSX, and Solaris that allows users to open a virtual desktop on the cluster. We currently support connecting to the Hoffman2 cluster via the NoMachine client as well as the X2Go client. Advantages of connecting to the cluster via the NX protocol are: speed and the ability to suspend and reconnect to a virtual desktop session.

NoMachine client

Download NoMachine Client for your operating system

A free NoMachine client is available from NoMachine for Windows, Mac OSX, and most Linux distributions.

  • Download NoMachine from the website

  • Use the download link for your operating system (Windows, Mac OSX, or Linux Distributions)

Install

  • Once downloaded, double click on the downloaded package file.

  • Click the Run/Install button.

  • When the NoMachine Client Setup Wizard starts, accept all the setup defaults.

Configure

  • Open the NoMachine client and continue to the connection page

  • Click on the New link to create a connection to the Hoffman2 Cluster

  • Specify the protocol as SSH and press Continue

  • Provide the host name nx.hoffman2.idre.ucla.edu*

  • Leave the port field set to 22 and press the Continue button

  • Select Use the system login and press the Continue button

  • Select Password and press the Continue button

  • Select Don’t use a proxy and press the Continue button

  • Name your connection and press the Done button

Please notice that if you are planning to suspend and reconnect to an NX session, you will need to repeat the configuration steps described above for the hosts:

nx1.hoffman2.idre.ucla.edu
nx2.hoffman2.idre.ucla.edu

Run

  • Open the NoMachine client and continue to the connection page

  • Click on the saved connection that you previously configured

  • Enter your username and password and your NX session with a virtual desktop on the cluster will start

How to get a terminal window

After you have logged into the Hoffman2 remote desktop, click on the Applications menu. Then select System Tools and Terminal.

Note

To pin the terminal to the panel, right click on the Terminal menu item and select Add this launcher to panel.

How to terminate your NX session

From the gnome desktop, select:

System -> Log Out login_id …

A small window will open. Click the Log Out button.

How to suspend an NX session

Make note of the actual NX node where your session is running (typically NX1 or NX2) to allow for future reconnections. To terminate the NX client on your computer, close the application or close the virtual desktop window.

How to reconnect to a suspended NX session

If you are planning to reconnect to a session, make note of the actual NX node where your session is running (typically NX1 or NX2). Make sure to configure your NX client for the NX node where your session is running. You will need to create a configuration using either nx1.hoffman2.idre.ucla.edu or nx2.hoffman2.idre.ucla.edu. See the Configuration section for more detailed instructions.

Note

Reconnecting to a suspended NX session is not always possible. Lengthy computations should be run as batch jobs. We recommend terminating NX sessions (logout) instead of suspending them.

On fast networks, a preferred alternative to NX is to run an X server on your platform (e.g., X11 packages for linux platforms, XQuartz for mac OSX, XMing and Xming fonts for Windows), and connect to the cluster via a terminal (or via PuTTY on Windows platforms) by setting up X11 forwarding.

X2Go client

Install X2Go client

Follow the installation guide at X2Go site. Download the X2Go client, select the appropriate client for your operating system (Windows, Mac OS, Linux) and the correct version onto your computer.

Double click on the downloaded package, accept the licensing terms, and follow the instruction on the screen.

Note

macOS users will need to install the XQuartz dependency prior to installing X2Go. It may be obtained on the XQuartz website. You may encounter a security warning prompting you that x2goclient.app is from an unidentified developer. Simply right click on the app again and select Open. See also the Mac portion of Opening GUI applications.

Configure the X2Go client to connect to Hoffman2

Launch the application by double clicking on the X2Go Client desktop icon, if you have chosen to create one during installation, or by searching for the application in your Applications folder and double clicking on the x2goclient icon. If prompted by the Windows Firewall, select Allow access so that sshd.exe and other X2Go related programs will be able to run.

Create a new session

From the Session tab in the X2Go client select New session. You can name your session with any name (e.g., x2go@hoffman2). Leave the Path field unchanged and in the Host field enter:

x2go.hoffman2.idre.ucla.edu

Leave the other fields unchanged and choose the Session type to be GNOME. Save the new session by pressing the OK button.

You can also create sessions for the specific nodes x2go1 and x2go2 if you will need to reconnect to a suspended session. However, we strongly suggest you start new sessions by using the session associated with the x2go.hoffman2.idre.ucla.edu address. Sessions on the x2go1 and x2go2 nodes can be created by using in the “Host” field respectively:

x2go1.hoffman2.idre.ucla.edu
x2go2.hoffman2.idre.ucla.edu

Name the sessions accordingly (for example: x2go1@hoffman2, x2go2@hoffman2).

Start the X2Go client

To connect to the Hoffman2 cluster open the X2Go client, double click on an already created session, and enter your Hoffman2 username and password. The first time you connect, you will be prompted to trust the host key. Do so by pressing the Yes button. On Windows, a few more firewalls will pop up and you will have to allow access to the X2Go Client and associated programs.

You should now be connected to a remote desktop on the Hoffman2 cluster.

Problems with the instructions on this section? Please send comments here.

Connecting via Jupyter Notebook/Lab

It is possible to connect to the Hoffman2 cluster by opening a Jupyter Notebook or a Jupyter Lab user interface.

Tip

While you can connect to Hoffman2 with X11-Forwarding and run a Jupyter Notebook on a web-browser instance opened on the Hoffman2 cluster, running Jupyter Notebook or Jupyter Lab on your local browser is a preferred strategy that can eliminate substantial latency.

The following are directions on how to run a Jupyter Notebook, or Jupyter Lab, on the Hoffman2 cluster while displaying it on browser local to your computer.

Jupyter NB

On your local computer, make sure to have the following software:

  • A working python installation (version 2.6 or greater)

  • A web browser

  • An SSH client.

Download the script h2jupynb.

Issue at the terminal/GitBash command prompt:

$ wget https://gitlab.idre.ucla.edu/dauria/jupyter-notebook/raw/master/h2jupynb

or:

$ curl -O https://gitlab.idre.ucla.edu/dauria/jupyter-notebook/raw/master/h2jupynb

Note

You will run the script h2jupynb from either a terminal (Linux/Mac) or Git BASH (Windows) shell open on your client machine (not connected to the Hoffman2 cluster).

To execute the h2jupynb script, issue at the terminal/GitBash prompt:

$ python h2jupynb -u username

where: username is your Hoffman2 username.

Make sure that the script is executable by issuing:

$ chmod u+x h2jupynb

and invoke the notebook with:

$ ./h2jupynb -u username

As soon as the interactive session is allocated on the Hoffman2 cluster, a Jupyter Notebook will open on your default web browser running on your local computer.

Note

Please note: Unless you have set up passwordless authentication with the Hoffman2 cluster, you will be prompted to enter your Hoffman2 password twice. Also, the default settings are so that the remote Jupyter Notebook will be allowed to use up to 1GB of memory, one core, and will run for up to 2 hours.

To learn how to change the default options, run:

$ ./h2jupynb --help

which will give you:

Usage:

h2jupynb [-u <Hoffman2 user name>] [-t <time, integer number of hours>]
[-m <memory, integer number of GB per core>]
[-e <parallel environment: 1 for shared, 2 for distributed>]
[-s <number of slots>] [-v <python-version>]
[-o <run on group owned nodes: yes/no>]
[-x <run on an exclusively reserved node: yes/no>] [-a <CPU-type>]
[-d <path to directory from which the jupyter NB/lab will start>]
[-p <port number>] [-g <run on a gpu node: yes/no>] [-c <gpu-card-type>] [-l <cuda-version>]
If no arguments are given to this script, it is assumed that
        your Hoffman2 user name is the same as on your client machine
        the time duration for your session is of 2 hours
        the parallel environment is shared
        the memory per slot for your session is of 1GB
        the number of slots for your session is of 1
        the python version for your notebook is 2.7.13
        the port on which the server is started is 8789
        the starting directory on Hoffman2 is your $HOME
        use GPU? default is no
        GPU type default is P4 (if -g yes)
        CUDA version 9.1 (if -g yes)
        not running on owned nodes
        not running in exclusive mode
        no specific CPU selected (see ARCH in qhost output)

        python versions currently available are: 2.7.13, 2.7.16, 3.6.1, 3.7.0, 3.7.2
        or: anaconda2, anaconda3 and anaconda/python3-4.2
        python versions: 2.7.3 or 3.4 are available but deprecated.

        cuda versions currently available are: 7.0, 7.5, 8.0, 9.1 and 10.0

        GPU cards currently publicly available: P4, RTX2080Ti and V100

Jupyter LAB

Please download the latest version of the h2jupynb script and start the Jupyter Notebook requesting python version 3.7.0:

$ ./h2jupynb -v 3.7.0 ...

(where … is any other option that you need to pass).

When you get to the jupyter session on your browser, go to the address bar and modify it so that instead of localhost:PORT/tree it will have localhost:PORT/lab. For example, should your address will look like:

http://localhost:9394/tree

replace tree with lab as in:

http://localhost:9394/lab

A jupyter lab session should start.

Problems with the instructions on this section? Please send comments here.

Remote direct rendering

It if possible to harness the computing power of GPUs on the Hoffman2 cluster to render images while displaying the result on your local client. You will need to install a VNC viewer and, if using a Windows computer, Git BASH. Here are the steps to follow:

  1. Download and install VNC viewer.

  2. Log on to the Hoffman2 cluster.

    • If you are using a Mac or Linux computer, use the terminal application to log on.

    • If you are using a Windows computer, use Git Bash.

  3. At the Hoffman2 prompt, issue the command:

    qrsh -l h_rt=2:00:00,gpu,P4,exclusive
    

    Tip

    You should change the time and other settings (e.g., the type of GPU card) in your qrsh command as needed.

  4. When the interactive session is awarded, issue the following commands:

    xauth add `cat /usr/local/etc/auth-for-gdm`
    x11vnc -display :0
    

Note

Make note of the GPU node where it started.

  1. Open a new terminal on your local machine (on Windows, please use Git Bash) and assuming that your interactive session started on g1, issue:

    ssh -f -N -L 5900:g1:5900 -L 5909:g1:5909 username@hoffman2.idre.ucla.edu
    

    Note

    Substitute g1 with the actual GPU node on which your qrsh session is running (and where you have just started x11vnc) and username with your Hoffman2 user name.

  2. On your local computer, start VNC viewer and set up a session with the following settings:

    VNC Server localhost:0
    
  3. Double click on the newly created VNC session and a VNC viewer window should appear.

  4. Look for your account name and log onto the cluster.

  5. You should be now logged on the GPU node, which qrsh had originally assigned to you and you can go ahead and perform your rendering.

Problems with the instructions on this section? Please send comments here.

Problems with the instructions on this section? Please send comments here.