Computing overview¶

A compute cluster is made of individual nodes which are interconnected between themselves and which aggregate power can be harnessed to address problems which nature requires either distributed or capacity computing.

A node, or host, is a physical server comprised of multiple CPU cores (which are referred to as slots in the scheduler lingo) rack-mounted in one of our two data centers and interconnected to other nodes (as well as to the storage) to form the cluster.

Upon connecting/logging into the cluster (unless Connecting via Jupyter Notebook/Lab) users access the cluster via its login nodes. Login nodes are special hosts which sole purpose is to provide a gateway to the compute nodes and their computational resources. For more information, see Role of login nodes.

Computational resources (such as memory, cores, runtime, CPU-type, GPU-type, etc.) on board of compute nodes are managed by a job scheduler. Any CPU, GPU or memory intensive computing task should be performed on compute nodes through the scheduler that applies to the environment where you are working.

The Computing section has three main pages:

this overview page, which explains node types, group-owned nodes, and the meaning of highp, shared and campus jobs
Computing with Altair Grid Engine on the production cluster, for the production Hoffman2 Cluster environment that uses Altair Grid Engine commands such as qrsh, qsub, qstat and qdel
Computing on the Slurm preview cluster, for the Slurm preview cluster reached through login.hoffman2.oarc.ucla.edu and using Slurm commands such as srun, salloc, sbatch, squeue and scancel

Do not mix Altair Grid Engine and Slurm commands in the same job workflow. Choose the page that matches the login host, operating system environment and scheduler where you are working.

Node types¶

A summary of the types of nodes that you will encounter while using the Hoffman2 Cluster and a description of their intended use is given in the Types of nodes on the Hoffman2 Cluster table:

Types of nodes on the Hoffman2 Cluster¶
login nodes	you access a login node upon connecting to the Hoffman2 Cluster via: terminal and SSH at the fully qualified domain name: hoffman2.idre.ucla.edu remote desktop at the NoMachine or X2Go services running on: nx.hoffman2.idre.ucla.edu or x2go.hoffman2.idre.ucla.edu respectively Note Login nodes are meant for light-weight tasks such as editing your code and submitting jobs to the scheduler. Login nodes are a resource shared by many concurrent users and are not intended for CPU or memory intensive tasks. Please see Role of login nodes. Important All CPU and/or memory intensive (as well as GPU) computations need to run on compute nodes accessed via the scheduler. For production Altair Grid Engine jobs, see Computing with Altair Grid Engine on the production cluster. For jobs in the Slurm preview cluster reached through `login.hoffman2.oarc.ucla.edu`, see Computing on the Slurm preview cluster.
Data transfer nodes	The Hoffman2 Cluster has two dedicated and performance-tuned data transfer nodes with advanced parallel transfer tools to support your research workflows.
CPU-based compute nodes	Most of the nodes on the Hoffman2 Cluster are CPU-based compute nodes. These are where your jobs execute after they are scheduled through the production Altair Grid Engine environment or the Slurm preview cluster.
GPU-based compute nodes	A portion of compute nodes on the Hoffman2 Cluster is equipped with one or more GPU cards available on the Hoffman2 Cluster of various types. Please refer to: Role of GPU nodes to see what workload is best suited to run on these nodes. Production GPU access is documented in Computing with Altair Grid Engine on the production cluster; Slurm preview GPU requests are documented in Computing on the Slurm preview cluster.

The Hoffman2 Cluster has a number of compute nodes available to the entire UCLA community. Additionally, research groups can purchase dedicated compute nodes. Users in groups who have contributed nodes to the cluster can access their nodes in a preferential fashion and for extended runtimes or access unused cores across the wider cluster (see: Highp vs shared vs campus jobs).

Group-owned nodes¶

Group-owned nodes, allow users to run jobs (interactive or batch) on their computational resources for an extended runtime (up to fourteen days). Moreover, the portion of the jobs submitted to owned-resources, that can be concurrently allocated on them, are guaranteed to start within twenty-four hours from their submission (wait time is typically less). Node ownership also allow users in that group to access any currently unused resource owned by a different group for up to a runtime of 24 hours.

If your group is interested in purchasing nodes, please visit: Purchasing additional resources.

Highp vs shared vs campus jobs¶

In the Hoffman2 Cluster jargon jobs submitted to owned resources are referred to as highp jobs while jobs submitted to other groups’ currently unused resources as shared jobs. Jobs submitted by users in groups that have not purchased nodes are limited to run on IDRE-owned resources for up to 24 hours; jobs from these users are referred to as campus jobs and the users as campus users.