News

Hoffman2 Fall 2021 maintenance outage

Attention

The Hoffman2 Cluster will be unavailable from 5:00AM on Monday, September 20, 2021 through 6:00PM on Wednesday, September 22, 2021 for scheduled maintenance.

What is changing

  • The operating system distribution on the cluster will be upgraded from CentOS 6 to CentOS 7.

  • Singularity will be available on the entire cluster.

  • Elliptical curve (ed25519) ssh keys will be available.

  • Home directory quotas will be doubled in size to 40GB and the maximum file count will be increased to 500,000 per user.

What to expect

  • Starting Monday, September 6, we will hourly reduce the maximum job run time in order to completely drain the cluster’s running jobs by September 20. If your job requests more than this maximum time, it will not start. All jobs (queued or running) at the time of this outage will be killed; affected users will have to resubmit their jobs after the outage. Please plan your runs accordingly.

  • During the maintenance window, you will not be able to log in, run jobs, check out licenses, or transfer files.

Warning

During the maintenance the contents of $SCRATCH will be completely erased. If you have files on $SCRATCH from a recently completed job, please copy them before the maintenace outage to persistent storage (your home or project directory).

  • After the maintenance, you may need to recompile your code (including R and Python libraries, etc.) so it can run on CentOS 7. Please refer to the section Code and libraries that might need recompilation.

  • Before the maintenance to reduce work disruption you may consider testing your jobs on the small number of nodes that are currently already running CentOS 7 and can be used to recompile your code and test its compatibility with the new distribution. Details can be found on the recompilation section.

Code and libraries that might need recompilation

In going from CentOS 6 to CentOS 7 the version of the GNU C Library (glibc) and of the default GCC compiler are changing from: 2.12 to 2.17 and 4.4.7 to 4.8.5 respectively. Accordingly you can expect that any component of your code compiled on CentOS 6 may need to be recompiled to take full advantage of the new system libraries. This applies to any C, C++, FORTRAN code that you have compiled on the cluster or it may apply to any python and R libraries you may have installed in your $HOME or project directory.

We encourage our users to test their applications by requesting interactive sessions on the nodes already on the next version of the OS.

Note

When requesting interactive sessions on the nodes already on the next version of the OS any R library will be automatically installed in a new location in your $HOME. This will avoid any conflict between running R on the nodes on the current version of the OS and on nodes on the next version of the OS.

After the transition you might consider archiving the directory: $HOME/R/x86_64-pc-linux-gnu-library/ as all the needed installed R packages will be in your: $HOME/R/x86_64-pc-linux-gnu-library-RH7/.

Note

To prevent possible conflicts with python libraries, we encourage our users to set the $PYTHONUSERBASE environmental variable appropriately when installing libraries on nodes on the next version of the OS to minimize conflict with existing libraries installed in the user $HOME directory. A possible choice would be:

$ export PYTHONUSERBASE=$HOME/.local_rh7

You can set such environmental variable in your initialization files, by appending the following lines in your $HOME/.bashrc or $HOME/.cshrc:

test=$(grep "release 7" /etc/redhat-release)
if [ $? == 0 ]; then
    export PYTHONUSERBASE=$HOME/.local_rh7
fi

After the transition you might consider archiving the directory: $HOME/.local as all the needed installed python libraries will be in your: $HOME/.local_rh7.

Note

Software available in /u/local/apps has been recompiled or re-installed, should a code you might need not be available let us know by submitting a ticket at our support site.

What we are doing

We will be upgrading the cluster operating system from CentOS 6 to Centos 7, performing data center maintenance, upgrading the research network, migrating data to allow for expanded storage capacity, and more.

Where to get help

If you have any questions or problems regarding this maintenance, please submit a ticket at our support site.

Nodes available on the next version of the OS

A number of nodes are now available on the next version of the Operating System. Building and installing applications on these nodes will be generally easier and more expedite as the nodes have newer default system libraries than those present on the nodes running the current default version of the Operating System (e.g., the login nodes and a large part of the compute nodes).

We encourage our users to test this newer version of the Operating Systems as we will be transitioning to it soon. Because the default version of the GCC compiler, the system libraries and the version of the Intel compiler are newer on these nodes, your applications, python or R libraries may need to be recompiled/installed.

To access these nodes interactively please use:

$ qrsh -l rh7

you can customize the qrsh command by requesting resources such as runtime, memory, etc. To learn more please see the sections: Examples of how to request resources and: Examples of how to request multiple cores.

To access these nodes in batch jobs, please add the following line to the preamble of your job submission script:

#$ -l rh7

to learn how to write your batch job submission script please see the section: Use qsub with a submission script.

Note

The -l highp option cannot be used to submit jobs to nodes on the next version of the OS unless your group has transitioned part, or all of its nodes, to this version of the OS.

If you have any questions or problems, please submit a ticket at our support site.

GPU nodes available

A number of nodes with various types of GPU cards are available on the Hoffman2 Cluster. To learn how to access these nodes please see the sections: How to access GPU nodes and: How to specify GPU types.

If you have any questions or problems, please submit a ticket at our support site.

Mathematica version 12.1

Mathematica Version 12.1 is now available on the Hoffman2 Cluster.

Mathematica is a modern technical computing system spanning most areas of technical computing.

To learn how to run Mathematica on Hoffman2 please refer to:

https://www.hoffman2.idre.ucla.edu/Using-H2/Software/Software.html#mathematica

If you have any questions or problems, please submit a new ticket at our support site https://support.idre.ucla.edu/


MATLAB version 9.8 (R2020a) Total-head-count-license

MATLAB Version 9.8 (R2020a) is now available on the Hoffman2 Cluster.

This installation of MATLAB under the Total Academic Headcount License (https://www.mathworks.com/academia/tah-portal/ucla-31454052.html) supports access to the full suite of packages and virtually unlimited licenses.

MATLAB is a proprietary multi-paradigm programming language and numerical computing environment developed by MathWorks.

To run MATLAB on Hoffman2 please refer to:

https://www.hoffman2.idre.ucla.edu/Using-H2/Software/Software.html#matlab

If you have any questions or problems, please submit a new ticket at our support site https://support.idre.ucla.edu/


Q-Chem version 5.3.0

Q-Chem Version 5.3.0 is now available on the Hoffman2 Cluster.

Q-Chem is a comprehensive ab initio quantum chemistry software for accurate predictions of molecular structures, reactivities, and vibrational, electronic and NMR spectra. The new release of Q-Chem 5 represents the state-of-the-art of methodology from the highest performance DFT/HF calculations to high level post-HF correlation methods (https://www.q-chem.com/).

To run Q-Chem on Hoffman2:

To run the shared memory version (from a bash script):

. /u/local/Modules/default/init/modules.sh module load qchem/5.3.0_sm qchem -nt $NSLOTS sample.in sample.out_$JOB_ID

To run the shared memory version (from a bash script):

. /u/local/Modules/default/init/modules.sh module load qchem/5.3.0_sm qchem -mpi -nt 1 -np $NSLOTS sample.in sample.out_$JOB_ID

where you will substitute sample.in and sample.out with the actual input and output file names.

If you have any questions or problems, please submit a new ticket at our support site https://support.idre.ucla.edu/