Main navigation

News Archive


May 2017

March 2017

February 2017

January 2017

December 2016

November 2016

September 2016

June 2016

May 2016

March 2016

February 2016

January 2016

December 2015

November 2015

October 2015

September 2015

July 2015

June 2015

May 2015

April 2015

March 2015

February 2015

January 2015

November 2014

October 2014

September 2014

June 2014

June 2014

April 2014

March 2014

February 2014

January 2014

December 2013

November 2013

October 2013

August 2013

May 2013

April 2013

March 2013

February 2013

January 2013

December 2012

October 2012

September 2012

August 2012

June 2012

May 2012

April 2012

March 2012

February 2012

January 2012

December 2011

November 2011

October 2011

September 2011

August 2011

July 2011

June 2011

May 2011

April 2011

March 2011

February 2011

January 2011

December 2010

November 2010

October 2010

September 2010

August 2010

June 2010

May 2010

March 2010

February 2010

January 2010

November 2009

October 2009

September 2009

August 2009

July 2009

May 2009

March 2009

February 2009

December 2008

November 2008

September 2008

August 2008

July 2008

June 2008

May 2008

April 2008

January 2008


News Items

Using Hoffman2 Cluster Seminar

Date: May 11, 2017
Time: 1-3 p.m.
Location: 5628 Math Science (IDRE Portal)
RSVP: https://idre.ucla.edu/calendar-event/using-hoffman2-cluster-2017-05-11
Free of charge.

This session is a general introduction to how to use UCLA Hoffman2 Cluster. The objective is to familiarize current and potential cluster users with the Hoffman2 Cluster, so they can make the best use of UCLA computational resources. The following topics will be covered:

1. How to access and login to the Hoffman2 Cluster, primarily in terminal/command-line mode.

2. Exploration of how to run various computational tasks on or from the Hoffman2 Cluster, illustrated by hands-on example transcripts.

3. A brief introduction to Univa Grid Engine (UGE) commands and how to achieve more control over resources.

This session is appropriate for entry-level and mid-level cluster users. No prerequisite knowledge is required, but some working experience with basic Unix system commands and Fortran or C programming languages will be helpful.

The registrants are recommended to do a quick survey in the following link before the class: https://www.surveymonkey.com/r/LGBD3KK

TOP

Scratch filesystem upgrade


Note: May 15, 2017 $OLDSCRATCH becomes read-only
June 1, 2017 $OLDSCRATCH is removed.

We have upgraded the Hoffman2 scratch filesystem to a new, higher performing disk storage system.

In order to minimize downtime and disruption to your workflow, on Saturday, March 25, at midnight (12:00AM), we will be changing the value of the $SCRATCH environment variable to point at the new filesystem.

After midnight on Saturday, March 25, 2017:

  • $SCRATCH will point to the new scratch filesystem
  • $OLDSCRATCH will point to the old scratch filesystem

Any compute jobs starting after that time will point to the NEW scratch filesystem location.

Any currently running jobs will continue to write data to the OLD scratch filesystem location.

Any of your existing active or suspended login sessions, nx sessions, or screen sessions will still point to the OLD scratch location.


We recommend that after midnight on Saturday, March 25, 2017, you log out or terminate any active or suspended login, nx, or screen sessions, and log back in to get the new value of $SCRATCH.

Your Old Data on $SCRATCH

In order to access your old data on the old scratch filesystem, we are providing the environment variable $OLDSCRATCH. Any new login, nx login, interactive sessions, or jobs starting after 3/25/17 at midnight can reference $SCRATCH, pointing to the new scratch filesystem (which will start out empty) and can reference $OLDSCRATCH to access any files you may have stored on the old scratch filesystem.

TOP

New g09.q.parallel script

We are pleased to have a new version of the g09.q.parallel script for creating a UGE command file to run Gaussian 09 in parallel. If you have any old cmd files created with g09.q.parallel, you should rebuild them.

Gaussian is licensed computational chemistry software. If you are interested in using Gaussian software, please contact Brian Levandowski in the Department of Chemistry and request authorization.

Gaussian can be run in one of three different modes:

serial
uses one cpu
multithread
uses multiple cpus on a single node
parallel
uses multiple cpus on multiple whole nodes

See How to Run Gaussian

Documentation is available from http://www.gaussian.com
The g09 Users Reference is available as g09ur.tgz

TOP

Hoffman2 Security Update on Sunday, Feb. 26, 2017

At noon (12:00PM) on Sunday, 2/26/2017, we will be restarting the Hoffman2 login, nx, and dtn nodes to apply an important security-related vendor patch for our CentOS distribution.

For uninterrupted access to the cluster, please log out of any active ssh sessions you have to Hoffman2 login nodes and log back in. When you log back in to hoffman2.idre.ucla.edu you will connect to an updated login node (login5).

The nx and dtn nodes will be unavailable from noon (12:00PM) to 12:30PM on Sunday while the updates are applied. Compute nodes reboots will be scheduled by the job scheduler on an ongoing basis and will not interrupt or terminate any existing running jobs.

TOP

IDRE Winter 2017 HPC Classes

Using Hoffman2 Cluster
Thursday, February 9
10:00 am – 12:00 pm
5628 Math Science — The Portal

This session is a general introduction to how to use UCLA Hoffman2 Cluster. The objective is to familiarize current and potential cluster users with the Hoffman2 Cluster, so they can make the best use of UCLA computational resources. The following topics will be covered:

1. How to access and login to the Hoffman2 Cluster, primarily in terminal/command-line mode.

2. Exploration of how to run various computational tasks on or from the Hoffman2 Cluster, illustrated by hands-on example transcripts.

3. A brief introduction to Univa Grid Engine (UGE) commands and how to achieve more control over resources.

The session is appropriate for entry-level and mid-level cluster users. No prerequisite knowledge is required, but some working experience with basic Unix system commands and Fortran or C programming languages will be helpful. The registrants are recommended to do a quick survey in the following link before the class: https://www.surveymonkey.com/r/5KW8LQ2. Register here
https://idre.ucla.edu/calendar-event/using-hoffman2-cluster-2017-winter

Python for High Performance Computing I
Friday, February 10
10:00 am – 12:00 pm
5628 Math Science — The Portal

Python, originally developed as a general purpose programming language, has gained its popularity in the scientific community in recent years owning to its elegant and easy-to-understand syntax and powerful libraries. Python fully supports both functional and object-oriented programming styles. This class will present a number of useful features of Python in the context of scientific computing, and introduce a number of supporting packages, including numpy for array-based computations and h5py to access HDF5 files, and the use of Python in parallel computing (MPI style).

This is part 1 of a 2-part presentation. Register here
https://idre.ucla.edu/calendar-event/python-high-performance-computing

Python for High Performance Computing II
Friday, February 17Python for High Performance Computing II
10:00 am – 12:00 pm
5628 Math Science — The Portal

This is part 2 of a 2-part presentation. Register here
https://idre.ucla.edu/calendar-event/python-high-performance-computing-ii

TOP

New NX server

A new NX server running on two new nodes was installed during the Winter Maintenance, December 12-14, 2016. The old NX server node, login6, has been decommissioned.

NX is a secure, compressed protocol for remote X Window System connections. You can use NX if you need to run graphical applications on Hoffman2 Cluster from a local Windows machine. NX is also suitable to run graphical applications on the Hoffman2 Cluster if connecting from any platform over a slow network.

Previously supported NX connections to login6.hoffman2.idre.ucla.edu no longer work. To connect via NX to the Hoffman2 Cluster, follow the configuration steps described in NX Client

TOP

How to change the time request on pending jobs

If your pending job requests more time than is available before the scheduled maintenance window begins, it will not start. If your job can finish in less time, you could use the qalter command to change its time limit to less than the remaining time. Here’s how to do it.

1. Find out all the -l arguments that your job uses. At the shell prompt, enter:

qstat -j jobnumber | grep 'hard resource_list'

where jobnumber is your job number. It returns something like:

hard resource_list: h_data=4000M,h_rt=1209600,highp=true

2. Use the qalter command to fix the job. At the shell prompt, enter:

qalter -l h_data=4000M,highp=true,h_rt=288:00:00 jobnumber

where 288:00:00 is less than the current maximum h_rt value and jobnumber is your job number.

* You can find the current maximum h_rt value with:

cat /etc/motd | grep h_rt=

or, qconf -se global which shows remaining time in seconds.

Note that the current maximum h_rt value changes hourly.

* You can find all of your pending (qw) jobs with:

myjobs -s p

or, qstat -s p

If you have a lot of jobs that need qalter, you can save a list of the job numbers, one per line, in a file named $USER.joblist and use the create-qalter-commands script to make those qalter commands. The script does not run the qalter commands, it just creates them. You may want to use it like:

qstat -s p > $USER.joblist
create-qalter-commands 288 > my.qalter.cmds

Where 288 is your new time request in hours.

Check the script output:

cat my.qalter.cmds

Run the qalter commands:

sh -x my.qalter.cmds

If your job doesn’t start before your new time, you may need to do the same thing again later — if your job can complete in the remaining time before maintenance begins.

TOP

bedtools updated to 2.26.0

Bedtools has been updated to version 2.26.0

bedtools are a set of utilities for a wide-range of genomics analysis tasks.

See How to Run BEDtools

Info on bedtools can be found at:
http://bedtools.readthedocs.org/en/latest/index.html

TOP

Intel 17.0.1 available

Intel Cluster Studio suite version 2017 is now available on the Hoffman2 Cluster. The suite, along with the Intel compilers (version 17.0.1), MKL(version 2017.0) and other related tools (see https://software.intel.com/en-us/intel-parallel-studio-xe/details) also comes with the Intel MPI library (version 2017.1).

To load this version of the Intel compiler and related libraries use:

module load intel/17.0.1

To permanently load this version of the compiler, add the above line to your $HOME/.bashrc (or for csh/tcsh shells, $HOME/.cshrc) file.

Please note that Intel MPI supports both gnu and Intel compilers. Use:

mpicc/mpicxx/mpif90 for gnu compiler version of Intel MPI

mpiicc/mpiicpc/mpiifort for Intel version of Intel MPI

Parallel applications compiled and linked against the Intel MPI compiler and library can use the queue script intelmpi.q You should save the SGE command file that intelmpi.q produces with its -ns (no submit) option:

intelmpi.q -ns [other queue options] your_executable [arguments]

then edit the resulting cmd file, replacing

module load intel/13.cs

with

module load intel/17.0.1

In case you would like to create your own submission script, include the lines:

# for sh shell scripts:

. /u/local/Modules/default/init/modules.sh
module load intel/17.0.1

# for csh shell scripts:

source /u/local/Modules/default/init/modules.csh
module load intel/17.0.1

before invoking mpirun in your submission script.

For Hoffman2 documentation on the Intel Cluster Studio please see:

Intel Cluster studio

Hoffman2 Cluster and Data Center Winter Maintenance, December 12-14, 2016

The Hoffman2 Cluster will be unavailable from 6:00AM on Monday, December 12, 2016 through 12:00PM on Wednesday, December 14, 2016 for scheduled maintenance. 

What to expect…

You will not be able to log in, run jobs or transfer files during the maintenance window.  Starting Monday, November 28, we will hourly reduce the maximum run time in order to completely drain the cluster’s running jobs by December 12.  If your job requests more than this maximum time, it will not start. All jobs (queued or running) at the time of this outage will be killed; affected users will have to resubmit their jobs after the outage. Please plan your runs accordingly.

What we are doing…

We are laying the foundations for major upgrades to our core research network infrastructure and physically relocating our high performance storage systems to new NEBS Seismic Zone 4 network cabinets mounted on seismic isolation bases.  These high strength racks and isolation bases will provide dramatically increased protection for valuable research data located on the UCLA campus in the case of a seismic event (earthquake).

We will also be integrating minor security updates for the running compute node image.  These updates should not affect compatibility of your existing code.

Other announcements…

We would also like to remind you of the upcoming Winter Holiday Closure. UCLA will close for 11 days between Friday, December 23, 2016, and Monday, January 2, 2017, with plans to reopen on Tuesday, January 3, 2017.  OIT/IDRE will be operating with reduced staffing during this time and will keep the Hoffman2 cluster operating on a best-effort basis.  Non-urgent support tickets will be answered after the campus reopens on January 3, 2017.  We wish you a relaxing winter break.

TOP

Data Transfer Nodes fingerprints changed

The host keys for dtn1 and dtn2 data transfer nodes have changed. You should see a warning message the next time you transfer files or ssh to one of the dtn nodes.

Here are the new fingerprints for both dtn nodes. As always, the fingerprint your ssh client shows you should match one of the six strings, or you should refuse the connection:

RSA key:
2048 xotov-buget-sigyv-gozac-derel-dabaz-ryfep-tohyl-pacoc-pukot-zoxex
2048 3c:9c:67:d8:c5:a4:ae:77:07:5f:10:2f:20:4a:75:0f

RSA1 key:
2048 xocen-pufiz-folal-haruf-bahid-fusyg-cihon-merul-mecyn-laton-poxix
2048 e5:75:a2:e8:07:a4:52:c4:c1:10:40:bd:bc:55:ed:3a

DSA key:
1024 xuher-gizaf-kedap-labif-mihiz-silys-hosih-hefez-pagiz-baguk-zexux
1024 09:43:5d:54:01:67:f0:ee:6e:be:20:e3:d4:ec:45:ac

Your ssh client may tell you the old offending key is in your .ssh/known_hosts file. You can use a text editor to remove any lines in known_hosts that begin with dtn dtn1 or dtn2. They will be re-generated with the new information the next time you access a dtn nodes.

IDRE Fall 2016 High Performance Computing Classes

We are pleased to announce the following free classes during the
upcoming Fall quarter 2016. For more information, please go to the
respective links:

Using Hoffman2 Cluster:
October 6, 2016 @ 2:00 pm – 4:00 pm
https://idre.ucla.edu/calendar-event/using-hoffman2-cluster-2016-fall

Job scheduling on Hoffman2 cluster:
October 12, 2016 @ 2:00 pm – 4:00 pm
https://idre.ucla.edu/calendar-event/job-scheduling-on-hoffman2-cluster-2016-fall

Introduction to MPI (Message Passing Interface):
October 13, 2016 @ 2:00 pm – 4:00 pm
https://idre.ucla.edu/calendar-event/introduction-to-mpi-2016-fall

Computing Environment on Linux — I:
October 18, 2016 @ 2:00 pm – 4:00 pm
https://idre.ucla.edu/calendar-event/computing-linux-i

Computing Environment on Linux — II:
October 20, 2016 @ 2:00 pm – 4:00 pm
https://idre.ucla.edu/calendar-event/computing-linux-ii

TOP

Mathematica 11

Mathematica has been upgraded to version 11 It is available for both job scheduler batch (math.q) and interactive use. It is available as both serial and parallel applications.

Mathematica is a scientific program which features mathematical computation, symbolic manipulation, a large collection of numeric functions, graphics and a high-level programming language.

Please see How to Run Mathematica
for more information.

Mathematica documentation is available online from the vendor Wolfram
Research, Inc. at http://reference.wolfram.com/language/ref/program/mathematica-unix.html

TOP

Mac XQuartz update

On Mac OS X, the X windows system is called XQuartz. XQuartz versions 2.7.9 and 2.7.10 are generating problems when running some remote GUI applications on the cluster.

The source of the problem in XQuartz 2.7.9 and 2.7.10 is support for the GLX protocol. Indirect GLX was disabled by default in Xorg 1.17. To enable indirect GLX and to allow remote visualization on the cluster you will have to issue at the terminal command prompt of your mac:

defaults write org.macosforge.xquartz.X11 enable_iglx -bool true

See About X11 and OS X Mountain Lion. For Mac OS X 10.4 (Tiger): X11 should be installed from the OS media and manually started.

If you have updated to 2.7.9 or 2.7.10, please consider reinstalling XQuartz version 2.7.8 which is proven to allow remote visualization on the cluster. You can obtain version 2.7.8 from https://www.xquartz.org/releases/XQuartz-2.7.8.html

This was not fixed in XQuartz version 2.7.10 released Oct 22, 2016.

TOP

Hoffman2 Cluster and Data Center Summer Maintenance, June 13-14, 2016

The Hoffman2 Cluster will be unavailable from 6:00AM on Monday, June 13, 2016 through 6:00PM on Tuesday, June 14, 2016 for scheduled upgrades and maintenance.

During this time, we will be installing a new OS image with the latest Centos 6.7 release, upgrading firmware on storage equipment, performing important network updates, and performing data center maintenance.

You will not be able to log in, run jobs or transfer files during the maintenance window. Starting Monday, May 30, we will hourly reduce the maximum run time in order to completely drain the cluster’s running jobs by June 13. If your job requests more than this maximum time, it will not start. All jobs (queued or running) at the time of this outage will be killed; affected users will have to resubmit their jobs after the outage. Please plan your runs accordingly.

The Hoffman2 license manager will be unavailable during this maintenance window.

CASS archival storage may be intermittently unavailable from 05:00 – 08:00 on Tuesday, June 14 during upgrades on the core OIT/IDRE network.

TOP

Quantum-Espresso-5.3.0-available

Quantum ESPRESSO v5.3.0 is now available on the Hoffman2 Cluster.

Quantum ESPRESSO is an integrated suite of computer codes for electronic-structure calculations and materials modeling at the nanoscale. It is based on density-functional theory, plane waves, and pseudopotentials (both norm-conserving and ultrasoft).

To run parallel Quantum ESPRESSO in batch, enter: espresso.q

espresso501.q and espresso503.q are also available for earlier versions.

The Quantum ESPRESSO documentation is available on-line at http://www.quantum-espresso.org/wp-content/uploads/Doc/user_guide/ and
http://www.quantum-espresso.org/users-manual/specific-documentation/

TOP

Tecplot 360 version 16.1.1

Tecplot 360 has been upgraded to version 16.1.1 Tecplot 360 is a program for the visualization and animation of scientific and engineering data. It includes support for CFD-type data. Please see How to Run Tecplot

Tecplot documentation in pdf format is available from the vendor at http://www.tecplot.com/support/documentation/

TOP

R version 3.2.3 is production

Version 3.2.3 of the stat package R has been promoted to production. To use it, at the command prompt, enter:

module load R

See all available versions of R with

module avail R

See How to Run R

To see a list of all R extension packages available in this version, at the R command prompt issue the command:

library()

R is a free software environment for statistical computing and graphics. It provides a wide variety of statistical and graphical techniques: linear and nonlinear modelling, statistical tests, time series analysis, classification, clustering, etc. Please consult the The R Project for Statistical Computing home page http://www.r-project.org for further information and documentation.

TOP

Globus access changes February 13, 2016

Globus is changing the way you access the Globus service, allowing you to log into Globus with your Hoffman2 username and login password. If you wish, you can continue to use your existing Globus username and password.

The Globus service will be unavailable for a few hours February 13, 2016 during this major upgrade.

Globus is a software tool to transfer files across the web in a reliable, high-performance and secure way. It provides fault-tolerant, fire-and-forget data transfer using simple web or command line interfaces. It is recommended for transferring very large files between your desktop machine and a remote machine like Hoffman2.

More information about the access change is available at www.globus.org/blog/enhanced-login-mechanism-streamlines-access-globus

TOP

Mathematica upgraded to 10.3

Mathematica has been upgraded to version 10.3 It is available for both job scheduler batch (math.q) and interactive use. It is available as both serial and parallel applications.

Mathematica is a scientific program which features mathematical computation, symbolic manipulation, a large collection of numeric functions, graphics and a high-level programming language.

Please see How to Run Mathematica for more information.

Mathematica documentation is available online from the vendor Wolfram Research, Inc. at http://reference.wolfram.com/language/ref/program/mathematica-unix.html

TOP

IDRE Winter 2016 HPC Classes

We are pleased to announce the following free classes during UCLA Winter Quarter, 2016. For more information please go to the IDRE Calendar page at https://idre.ucla.edu/calendar

Using Hoffman2 Cluster
* Feb. 2, 2016, 2pm-4pm in 5628 Math Sciences (IDRE Portal)
* RSVP: http://cfapps.ats.ucla.edu/cfapps/events/rsvp/RSVPNow.cfm?EveID=3489&SecID=3477

Computing Environment on Linux – I
* Feb. 4, 2016, 1pm-3pm in 5628 Math Sciences (IDRE Portal)
* RSVP: http://cfapps.ats.ucla.edu/cfapps/events/rsvp/RSVPNow.cfm?EveID=3490&SecID=3478

Computing Environment on Linux – II
* Feb 16, 2016, 1pm-3pm in 5628 Math Sciences (IDRE Portal)
* RSVP: http://cfapps.ats.ucla.edu/cfapps/events/rsvp/RSVPNow.cfm?EveID=3491&SecID=3479

Profiling- Code Clinic
* Feb 24, 2016, 1pm-3pm in 5907 Math Sciences
* RSVP: http://cfapps.ats.ucla.edu/cfapps/events/rsvp/RSVPNow.cfm?EveID=3493&SecID=3481

National Leadership Class Computing Resources and Opportunities for UCLA Researchers
* Mar. 1, 2016, 1pm-3pm in 5628 Math Sciences (IDRE Portal)
* RSVP: http://cfapps.ats.ucla.edu/cfapps/events/rsvp/RSVPNow.cfm?EveID=3492&SecID=3480

TOP

MATLAB 8.6 R2015b Upgrade

MATLAB has been upgraded to version 8.6 (R2015b). If you have compiled matlab executables made with the mcc command or mcc.q you should rebuild them before running them.

MATLAB is a language for technical computing that combines numeric computation, advanced graphics and visualization, and a high-level programming language.

Please see MATLAB for information on how to run MATLAB on the Hoffman2 Cluster.

MATLAB documentation is available from the vendor at http://www.mathworks.com/help/

TOP

Connecting to Hoffman2 with NX Client and login6

Update: December 16, 2015. Use login6.hoffman2.idre.ucla.edu with your nx client to connect to the Hoffman2 nxserver. The new CentOS 6.7 distribution does not include an NX server. Until we can create and install a new image on the login nodes that does include an NX server, login6 has been brought up with the old CentOS 6.6 image.

You can use an NX client on your local machine to connect to the Hoffman2 Cluster and run Hoffman2 Cluster graphical applications. NX provides near-local speed application responsiveness over high-latency, low bandwidth links.

NX is a secure, compressed protocol for remote X Window System connections. It is an alternative to running an X Server on your local machine. An NX server runs on Hoffman2 Cluster login nodes; you run an NX client on your local machine.

There are free NX clients available from NoMachine (www.nomachine.com) for Windows, Linux, Mac OSX, and Solaris.

See NX Client for more information about using NX to access the Hoffman2 Cluster.

Documentation available at https://www.nomachine.com/documents

TOP

Hoffman2 Cluster Maintenance Complete

Winter maintenance has completed successfully ahead of schedule. The Hoffman2 Cluster is now available for login and is accepting jobs.

Initially, jobs requiring 2 hours or less will start. After we have validated cluster stability, all jobs may start, including long-running jobs.

During the outage we upgraded the Panasas firmware and drivers, updated the running system image to the latest Centos 6.7 release, upgraded the job scheduler to a new version, and performed other maintenance tasks.

The major focus of the outage was completing a major upgrade of the Infiniband fabric in the POD data center.

We retired our old and increasingly unreliable QDR core switch and installed a new Mellanox FDR core switch. We replaced the majority of our QDR leaf switches with new FDR leaf switches to support nodes that have FDR infiniband cards onboard.

During testing, the new fabric has shown to be dramatically more resilient and much faster than the old.

We invite you to test your parallel code in the POD and let us know about any performance improvements or problems that you may encounter.

*** Important campus closure information relating to the Hoffman2 cluster ***

Please be aware that the IDRE Research Technology Group will have limited staffing during the winter Campus Closure from 12/24/2015 through 1/3/2016.

We will make every effort to keep the Hoffman2 cluster operational during the campus closure, within the constraints imposed by limited service, delivery, and staffing schedules.

If you have any questions regarding this maintenance or encounter any problems that may be due to upgrades, please submit a ticket to our support site at:

https://support.idre.ucla.edu/helpdesk

TOP

Stata 14 available

The Stata 14 is now available on the Hoffman2 Cluster for both SGE batch and interactive applications.

Stata is a complete, integrated statistical package that provides a broad suite of statistical capabilities, complete data-management facilities, and publication-quality graphics.

Please see Stata for information about how to run Stata on the Hoffman2 Cluster.

Stata documentation is available at http://www.stata.com/features/documentation/ or in pdf format in /u/local/apps/stata/14/docs/.

For statistical consulting help, send email to IDRE Statistical Consulting idrestat@ucla.edu

TOP

Winter Maintenance Dec 14-15, 2015

The Hoffman2 Cluster will be unavailable from 6:00AM on Monday, December
14, 2015 through 6:00PM on Tuesday, December 15, 2015 for scheduled
upgrades and maintenance.

During this time, we will be installing a new OS image with the latest
Centos 6.6 release, performing upgrades on storage equipment, and doing
a major upgrade of the Infiniband fabric.

The primary focus of the maintenance window will be the upgrade of the
Infiniband infrastructure in the POD data center.

We will be replacing the existing QDR Infiniband core switch in the POD
with a new, modern FDR core switch. We will also be replacing many of
the QDR leaf switches with FDR leaf switches and replacing and upgrading
cabling. These upgrades should greatly increase the throughput and
reliability of our IB fabric.

Due to the major changes involved in this upgrade, we will be closely
monitoring the performance of parallel jobs running on the Infiniband
fabric following the upgrade.

You will not be able to log in, run jobs or transfer files during the
maintenance window. Starting Monday, December 1st, we will hourly
reduce the maximum run time in order to completely drain the cluster’s
running jobs by December 14. If your job requests more than this
maximum time, it will not start. All jobs (queued or running) at the
time of this outage will be killed; affected users will have to resubmit
their jobs after the outage. Please plan your runs accordingly.

IDRE Fall 2015 Classes

The UCLA IDRE High Performance Computing Group is happy to announce the
following free classes during the upcoming Fall Quarter, 2015. To see
class descriptions and sign up, please visit https://idre.ucla.edu/calendar

Computing Environment on Linux:
Oct.1, 2015 @2 PM – 4 PM in 5628 Math Sciences, Visualization Portal
RSVP and details:
https://idre.ucla.edu/calendar-event/computing-environment-on-linux-2

Using Hoffman2 Cluster:
Oct. 5, 2015 @2 PM – 4 PM in 5628 Math Sciences, Visualization Portal
RSVP and details:
https://idre.ucla.edu/calendar-event/using-hoffman2-cluster-2

Profiling- Code clinic session:
Oct. 6, 2015 @2 PM – 4 PM in 5907 Math Sciences
RSVP and details:
https://idre.ucla.edu/calendar-event/profiling-code-clinic-session-3

Parallel Computing using MPI:
Oct. 8 @2 PM – 4 PM in 5628 Math Sciences, Visualization Portal
RSVP and details:
https://idre.ucla.edu/calendar-event/parallel-computing-using-mpi

Introduction to Stata:
Oct. 13 @9 AM – 12 noon in CLICC Classroom C, 320 Powell Library
RSVP and details:
https://idre.ucla.edu/calendar-event/introduction-to-stata

Scientific Computing using PETSc:
Oct. 15, 2015 @ 2 PM – 4 PM in 5907 Math Sciences
RSVP and details:
https://idre.ucla.edu/calendar-event/scientific-computing-using-petsc-2

Python for High Performance Computing -Part 1:
Oct. 19, 2015 @2:30 PM – 5 PM in 5628 Math Sciences
RSVP and details:
https://idre.ucla.edu/calendar-event/python-for-high-performance-computing-part-1

Introduction to CUDA:
Oct. 20, 2015 @ 2 PM – 4 PM in 5628 Math Sciences, Visualization Portal
RSVP and details:
https://idre.ucla.edu/calendar-event/introduction-to-cuda

Python for High Performance Computing – Part 2:
Oct. 26 @ 2PM – 4 PM in 5628 Math Sciences
RSVP and details:
https://idre.ucla.edu/calendar-event/python-for-high-performance-computing-part-2

TOP

Abaqus-updated-to-version-6.14

Abaqus version 6.14 is now available on the Hoffman2 cluster.

ABAQUS is a suite of general-purpose, nonlinear finite element analysis (FEA) programs for stress, heat transfer, and other types of analysis. ABAQUS runs a wide range of linear and nonlinear engineering simulations.

Please see How to Run ABAQUS

ABAQUS manuals are not available on the web. ABAQUS documentation is available on Hoffman2 for X sessions with the abaqus doc command.

TOP

Q-Chem version 4.3

Q-Chem version 4.3 is now available on the Hoffman2 Cluster for both parallel and serial jobs.

Q-Chem is a comprehensive ab initio quantum chemistry package for accurate predictions of molecular structures, reactivities, and vibrational, electronic and NMR spectra. The new release of Q-Chem 4 represents the state-of-the-art of methodology from the highest performance DFT/HF calculations to high level post-HF correlation methods.

For information on how to run Q-Chem on the Hoffman2 Cluster, see How to Run Q-Chem

Other versions of Q-Chem are available with the module command. Enter module available qchem at a shell prompt.

For more information about Q-Chem, see http://www.q-chem.com/

Q-Chem 4.3 User’s Manual

TOP

Purchased storage migration of /u/home/groupname directories

As part of our migration to a new storage system, all sponsored group directories located in /u/home/groupname have been migrated to /u/project/groupnameor /u/nobackup/groupname depending on whether your group sponsor has purchased backups.

This does not affect your personal home directory.

If you have a project or project-groupname symbolic link in your home directory, we have updated it to point to the new location of your purchased storage. If your sponsor decided to no longer puchase backups, your link may now be nobackup-groupname.

If you have made your own links to /u/home/groupname you need to fix them.

For more information on Hoffman2 filesystem classifications, please see this Knowledge Base link:

https://support.idre.ucla.edu/helpdesk/KB/View/2721181-hoffman-cluster-filesystem-classifications

TOP

NWChem version 6.5

NWChem version 6.5 is now available for both parallel and serial jobs with a new queue script nwchem65.q   The nwchem.q script still references version 6.1

NWChem is a computational chemistry package that is designed to run on high-performance parallel supercomputers as well as conventional workstation clusters. It aims to be scalable both in its ability to treat large problems efficiently, and in its usage of available parallel computing resources.

For more information about NWChem, see http://www.nwchem-sw.org/

For information on how to run NWChem on the Hoffman2 Cluster, see How to Run NWChem

TOP

LAMMPS version 10Feb2015 available

LAMMPS 10 Feb 2015 stable version is now installed on the Hoffman2 Cluster. It is built with Intel compiler v14.0 and Intel mpi v5.0 This software runs as a parallel application.

LAMMPS, Large-scale Atomic/Molecular Massively Parallel Simulator, is a classical molecular dynamics simulation code designed to run efficiently on parallel computers. It was developed at Sandia National Laboratories.

Note: Bi-annual LAMMPS Users Workshop, August 6 – 7, 2015 in Albuquerque, NM. See http://lammps.sandia.gov/workshops.html

See How to Run LAMMPS

LAMMPS documentation is available at http://lammps.sandia.gov/doc/Manual.html

TOP

Portland Group Compilers Available

Portland Group Compilers version 15.3 are now available on Hoffman2.

PGI compilers deliver world-class performance across a wide spectrum of applications and benchmarks. PGI is the compiler-of-choice among many popular performance-critical applications used in the fields of geophysical modeling, mechanical engineering, computational chemistry, weather forecasting, and high-energy physics.

PGI offers a complete and integrated suite of compilers and tools for developing high performance parallel applications including the fully-integrated graphical PGDBG® and PGPROF® parallel MPI/OpenMP application debugging and profiling tools.

PGI gives individual developers the power to incrementally accelerate applications on x64+accelerator platforms simply by adding portable compiler directives to existing standard-compliant Fortran and C programs. The PGI Accelerator supports CUDA.

To access the Portland Group compilers, enter:

 module load pgi_compiler

The module command makes these and other compiler commands available to you:

 pgcc (ISO/ANSI C11 and K&R C)
 pgc++ (ISO/ANSI C++11 with GNU compatibility)
 pgf77 (ANSI FORTRAN 77)
 pgf95
 pgfortran (ISO/ANSI FORTRAN 2033)
 pgdbg (Source code debugger)
 pgprof (Performance profiler)

Note: The pgcpp C++ compiler has been replaced by the pgc++ C++ compiler.

PGI 2015 C++ object code is incompatible with prior releases. All C++ source files and libraries that were built with prior releases must be recompiled to link with PGI 2015 or higher object files.

For more information, see:

TOP

Hoffman2 Cluster and Data Center Major Summer Maintenance, June 15-17, 2015

The Hoffman2 Cluster and the Research Data Center facilities will be undergoing major system maintenance on Monday, June 15, 2015 at 4:00AM through Wednesday, June 17, 2015 at 11:00AM.

During this time, we will be upgrading the cluster operating system, adding requested software packages, upgrading our storage system, upgrading our VMWare Enterprise Cluster, and performing hardware and network maintenance.

We will also be having our vendors perform major data center facility maintenance, requiring a full power outage.

Core switches on the Research Network will be upgraded to the latest code.

CASS Archival Storage will be unavailable from 4AM – 12PM (noon) on Monday, June 15, 2015 for software upgrades and network maintenance.

You will not be able to log in, run jobs or transfer files during the maintenance window. Starting Monday, June 1st, we will hourly reduce the maximum run time in order to completely drain the cluster’s running jobs by June 15. If your job requests more than this maximum time, it will not start. All jobs (queued or running) at the time of this outage will be killed; affected users will have to resubmit their jobs after the outage. Please plan your runs accordingly.

The Hoffman2 license manager will be updated at this time and software licenses will be unavailable for checkout.

If you have any questions regarding this maintenance, please submit a ticket to our support site at:

https://support.idre.ucla.edu/helpdesk

TOP

IDRE Spring 2015 Classes

IDRE is pleased to offer the following classes in the area of scientific computing. All classes are free. All classes are held in 5628 Math Science, Visualization Portal. To see class descriptions and sign up, please visit https://idre.ucla.edu/calendar

    • Revision Control Using git sign up here
      Web Feb 18 @ 10:00 am – 12:00 pm

 

    • Computing Environment on Linux sign up here
      Wed Feb 18 1:00 pm – 3:00 pm

 

    • Python for Scientific Computing: an Overview sign up here
      Mon Feb 23 @ 2:00 pm – 4:00 pm

 

    • Using the Hoffman2 Cluster sign up here
      Tue Feb 24 @ 2:00 pm – 4:00 pm

 

 

    • How to write a Makefile sign up here
      Mon Mar 2 @ 2:00 pm – 4:00 pm

 

    • Scientific Computing using PETSc (I) sign up here
      Tue Mar 10 @ 10:00 am – 12:00 pm

 

    • Scientific Computing using PETSc (II) sign up here
      Thu Mar 12 @ 2:00 pm – 4:00 pm

 

TOP

SIMULIA Abaqus Update Seminar Feb 19, 2015

Introducing the SIMULIA Academic Suite: New Research and Teaching packages with dramatically increased capabilities by Dick Rotelli of SIMULIA Academia.

Please RSVP to attend this seminar:
http://cfapps.ats.ucla.edu/cfapps/events/rsvp/RSVPNow.cfm?EveID=3400&SecID=3389

  • Date: Feb. 19, 2015
  • Time: 9AM-11AM
  • Location: 5628 Math Science (Visualization Portal)

Dick Rotelli is also available to meet individually or in small groups with UCLA faculty and students to learn how they are using Abaqus or other SIMULIA software, and get their feedback. These meetings can be scheduled anytime during the day, before or after the Update Seminar presentation. To reserve a time slot, please contact him at Dick.Rotelli@3ds.com or 401-531-5362.

TOP

Using the Hoffman2 Cluster Class Feb 24, 2015

This free 2-hour session is an introduction on how to use Hoffman2 Cluster. It is is appropriate for entry- to mid-level cluster users. Topics include:

  • Overview of computational resources accessible to the UCLA campus community
  • How to access and log into the Hoffman2 Cluster
  • How to complete various computational tasks on the Hoffman2 Cluster
  • How to efficiently submit and run jobs on the Hoffman2 Cluster

February 24, 2015
2:00 – 4:00 pm
OIT Portal (5628 Math Sciences).

RSVP at http://cfapps.ats.ucla.edu/cfapps/events/rsvp/RSVPNow.cfm?EveID=3343&SecID=3332

No prerequisite knowledge is required, but some working experience with basic Linux system commands and C programming languages will be helpful.

TOP

Tecplot 360 new version 2015R1EX

Tecplot 360 has been upgraded to version 2015 R1 EX. Tecplot 360 is a program for the visualization and animation of scientific and engineering data. It includes support for CFD-type data.

Please see How to Run Tecplot

Tecplot documentation in pdf format is available from the vendor at http://www.tecplot.com/support/documentation/

TOP

COMSOL upgraded to 5.0

COMSOL has been upgraded to version 5.0  It is available for both job scheduler batch (comsol.q) and interactive use.

COMSOL Multiphysics is a simulation software environment to facilitate physics-based modeling process.

COMSOL documentation is available online from the vendor at http://www.comsol.com

Please see How to Run COMSOL for more information.

TOP

Hoffman2 Winter Maintenance December 22, 2014

The Hoffman2 Cluster will be undergoing system maintenance on Monday December 22, 2014, from 4:30AM – 4:30PM.

During this time, we will be upgrading the cluster operating system, adding requested software packages, upgrading our storage system, and performing hardware and network maintenance.

You will not be able to log in, run jobs or transfer files during the 12-hour maintenance window. Starting Monday, December 8th, we will hourly reduce the maximum run time in order to completely drain the cluster’s running jobs by December 22nd. If your job requests more than this maximum time, it will not start before the down time. Pending jobs will start after maintenance completes.

The Hoffman2 license manager will be updated at this time and software licenses will be unavailable for checkout.

TOP

MATLAB upgraded to R2014b

MATLAB has been upgraded to version 8.4 (R2014b). If you have compiled matlab executables made with the mcc command or mcc.q you should rebuild them before running them.

MATLAB is a language for technical computing that combines numeric computation, advanced graphics and visualization, and a high-level programming language.

Please see How to Run MATLAB

MATLAB documentation is available from the vendor at http://www.mathworks.com/access/helpdesk/help/helpdesk.html

TOP

Opensees updated to version-2.4.4

Opensees default version on Hoffman2 Cluster has been updated to 2.4.4

Opensees, Open System for Earthquake Engineering Simulation, is an object-oriented, open source software framework. It allows creation of finite element computer applications for simulating the response of structural and geotechnical systems subjected to earthquakes and other hazards. The development and application of OpenSees is sponsored by the Pacific Earthquake Engineering Research Center through the National Science Foundation engineering and education centers program.

See How to Run opensees

Documentation is available at http://opensees.berkeley.edu/wiki/index.php/OpenSees_User

TOP

Mathematica upgraded to 10

Mathematica has been upgraded to version 10 It is available for both job scheduler batch (math.q) and interactive use. It is available as both serial and parallel applications.

Mathematica is a scientific program which features mathematical computation, symbolic manipulation, a large collection of numeric functions, graphics and a high-level programming language.

Mathematica documentation is available online from the vendor Wolfram Research, Inc. at http://reference.wolfram.com/language/ref/program/mathematica-unix.html

Please see How to Run Mathematica for more information.

TOP

Abaqus version 6.13 now available on Hoffman2

Abaqus version 6.13 is now available on Hoffman2.

ABAQUS is a suite of general-purpose, nonlinear finite element analysis (FEA) programs for stress, heat transfer, and other types of analysis. ABAQUS runs a wide range of linear and nonlinear engineering simulations.

Please see How to Run ABAQUS

ABAQUS manuals are not available on the web. ABAQUS documentation is available on Hoffman2 for X sessions with the “abaqus doc” command.

Hoffman2 Summer Maintenance Monday, June 16, 2014

The Hoffman2 Cluster will be undergoing system maintenance on Monday June 16, 2014, 4:30AM – 12:30PM to install some important software updates to our storage, management, and network infrastructure.

You will not be able to log in, run jobs or transfer files during the 8-hour maintenance window. Starting Monday, June 2nd, we will hourly reduce the maximum run time in order to completely drain the cluster by June 16th. If your job requests more than this maximum time, it will not start before the down time. Pending jobs will start after maintenance completes.

We will be applying security and software updates to our storage system, with the added benefit of being able to do routine future upgrades non-disruptively, reducing the need for future downtime.

We will also be upgrading management infrastructure to new, high performance, redundant hardware for added reliability and uptime.

Finally, important network upgrades and maintenance will take place.

TOP

Spring 2014 HPC Classes

We are pleased to announce to announce the following high performance computing classes at no cost. All classes will be held in 5628 Math Science, the Visualization Portal. The class description is contained in each link below:

Seats are limited; please RSVP. We hope to see you there.

TOP

TreeMix 1.12

TreeMix 1.12 is now available. See How to Run TreeMix

TreeMix is a method for inferring the patterns of population splits and mixtures in the history of a set of populations. In the underlying model, the modern-day populations in a species are related to a common ancestor via a graph of ancestral populations. We use the allele frequencies in the modern populations to infer the structure of this graph. See https://code.google.com/p/treemix for more information.

Documetation:

TOP

MOPAC2012

MOPAC version 2012 is now available. See How to Run MOPAC

MOPAC (Molecular Orbital PACkage) is a semiempirical quantum chemistry program based on Dewar and Thiel’s NDDO approximation. Seehttp://openmopac.net for more information.

Documentation:

TOP

Gaussian 09 D.01 available

Gaussian 09, Revision D.01 is now available on the the Hoffman2 Cluster. It supports parallel distributed memory execution (Linda-parallelism) and shared memory execution (SMP-parallelism). GaussView 5 is also available.

Gaussian provides state-of-the-art capabilities for electronic structure modeling. GaussView is a graphical interface available for Gaussian. With GaussView, you can import or build the molecular structures that interest you, set up, launch, monitor and control Gaussian calculations, retrieve, view and visualize the results, GaussView 5 provides comprehensive support for importing and working with structures from PDB (protein data base) files.

See How to Run Gaussian There are queue scripts to invoke Gaussian 09.

See also How to Run GaussView There is information about GaussView and how to run Gaussian from within GaussView.

Gaussian 09 and GaussView 5 documentation is available at http://www.gaussian.com Gaussian 09 Release Notes are available athttp://www.gaussian.com/g_tech/rel_notes.pdf

TOP

GROMACS 4.6.5

GROMACS version 4.6.5 is now installed on the Hoffman2 Cluster. This version of GROMACS can run only multithreaded or parallel.

GROMACS is an engine to perform molecular dynamics simulations and energy minimization.

See

TOP

NXserver weekly restart

Due to some inherent instability of the nxserver, which often does not truly terminate a closed session and so prevents a new nx session from starting, we have implemented a weekly restart of the nxserver on the login nodes. The nxserver restart schedule will begin Sunday February 9, 2014.

For more information about NX, point your browser at: NX Client

TOP

Seminar on private cloud-compute resource using OpenStack-Havana release

DATE: Jan. 29, 2014 @ 10 a.m. to 12 p.m.
Location: 5628 Math Science, the Visualization Portal
Speaker: Prakashan Korambath, IDRE, UCLA

View the presentation (pdf): Private Cloud Computing as in Infrastructure as a Service (IaaS) model using OpenStack. There is also a video of the presentation on youtube: Deploying OpenStack Havana Release at UCLA.

The Virtual Computing Laboratory of IDRE invites you to a presentation of private cloud-compute deployment in Infrastructure as a Service model using OpenStack-Havana release open source software package. In this presentation we will discuss the OpenStack IaaS architecture as well as the actual deployment of Havana release on CentOS 6.5 and Ubuntu 12.04 operating systems.

This talk will go over the implementation of software defined networking and use of namespaces in OpenStack network API called neutron in Havana release. The purpose of this presentation is to share our expertise in cloud computing among campus researchers, students and system administrators and assist them to deploy their own private cloud-compute infrastructure.

The Virtual computing Laboratory has deployed and operated cloud-compute infrastructure using OpenStack Essex and Grizzly releases hitherto and is building a Havana release based cloud-compute infrastructure currently. Some details of our deployment can be viewed here.https://sites.google.com/site/hpccloud/

OpenStack software package is released under Apache 2.0 license, which makes it possible for academia and industry to contribute, distribute and use the software for both academic and commercial use free of charge.

To attend the seminar please RSVP

TOP

R version 3.0.1

The default version of R on the cluster is R v 3.0.1 To use it, type at the command line:

module load R

R extension packages have also been upgraded to version 3.0.1. To see all packages which are installed, at the R command prompt issue the command:

> library()

R is a free software environment for statistical computing and graphics. It provides a wide variety of statistical and graphical techniques: linear and nonlinear modelling, statistical tests, time series analysis, classification, clustering, etc. Please consult the R Project homepage (http://www.r-project.org/) for further information and documentation.

See How to Run R

TOP

Winter 2014 Parallel Computing Classes

We are pleased to announce the following free high performance computing classes. The class outline is contained in the RSVP link.

  • Using Hoffman2 Cluster

    Location: 5628 Math Sciences
    Time: Monday, January 13, 2014, 10:00-12:00
    RSVP

  • Introduction to Parallel Computing and OpenMP

    Location: 5628 Math Sciences
    Time: Monday, January 13, 2014, 14:00-16:00
    RSVP

  • Introduction to MPI

    Location: 5628 Math Sciences
    Time: Tuesday, January 14, 2014, 10:00-12:00
    RSVP

We hope to see you there.

TOP

Hoffman2 Cluster up during UCLA Winter Closure

The Hoffman2 Cluster will be up and operational during the UCLA winter closure December 23rd through January 1, 2014. We will do our best to insure the cluster remains in operation but, depending on the types of issues that arise, we may not be able to address them until January 2nd.

If you encounter any problems during the UCLA winter closure period, please let us know as soon as possible. Send email to hpc@ucla.edu

TOP

New hoffman2#ucla Globus file transfer endpoint

There are several recent changes to Globus Online. On November 14th its name was changed to simply Globus and its web site was renamedhttp://www.globus.org It is still accessible with its former name. Its support email address is now support@globus.org

The Hoffman2 Cluster has a new Globus endpoint: hoffman2#ucla.edu

Use your regular Hoffman2 username and password with the new endpoint. (Formerly people needed to use their UCLA Grid credential). The old Hoffman2 endpoints will be retired in the near future.

Globus is a software tool to transfer files across the web in a reliable, high-performance and secure way. It provides fault-tolerant, fire-and-forget data transfer using simple web or command line interfaces. It is appropriate for transferring very large files either between your desktop machine and a remote machine like the Hoffman2 Cluster, or between two remote machines on which you have accounts.

For more information, please see High-speed file transfer with Globus

TOP

Winter Maintenance Dec. 16 thru 18, 2013

Hoffman2 Cluster winter maintenance will start 9AM Monday December 16th and continue through 5PM Wednesday, December 18. This is the week before the University closes for winter break.

You will not be able to login or run jobs during this period. Starting Sunday December 1st, we will dynamically reduce the maximum time for long-running (highp) jobs. After 9AM Sunday December 15th only interactive sessions and express jobs will run.

During this maintenance window, no pending jobs will be able to be carried forward and all pending jobs that do not start before the maintenance window begins will be deleted.

During maintenance we will install firmware upgrades and system image updates. Our storage systems will be upgraded, and some storage reconstruction will take place. There will be significant upgrades to both the Ethernet and Infiniband fabrics. The POD datacenter will undergo maintenance as well.

TOP

Hoffman2 Cluster is open and running jobs

The Hoffman2 cluster experienced problems with system storage October 14th, causing many compute nodes to drop off-line. These issues have been resolved, and the cluster is once again open for business. All file systems are up. No data were lost. No user-owned data were at risk.

The Hoffman2 Cluster is accepting jobs and new jobs are running. Some compute nodes are still off-line to allow running jobs to complete. As those jobs finish, the nodes will be returned to service.

TOP

Fall 2013 Parallel Computing Classes

We are pleased to announce the following free high performance computing classes. The class outline is contained in the RSVP link.

We hope to see you there.

TOP

Hoffman2 September 20th, 2013 maintenance canceled

The Hoffman2 Cluster September 20th maintenance has been canceled.

Our storage vendor, Panasas, has informed us that they intend to release an operating system update with a critical bug fix on or around October 1, 2013. Because of the importance of this update, we will need to take the Hoffman2 Cluster down to apply it as soon as it is received.

In order to avoid inflicting multiple down-times on the Hoffman2 user community, we are opting to cancel our previously scheduled September 20th downtime, and will schedule a new downtime when the update is released.

Because of the importance of this fix, it is possible that the next downtime could be on shorter notice than usual.

As always, thank you very much for your understanding and patience.

The Hoffman2 Systems Team
IDRE Research Technology Group

Hoffman2 Fall Maintenance Sept 20th, 2013

The Hoffman2 Cluster will undergo quarterly maintenance on September 20, 2013 between 9AM and 5PM. At this time, the cluster and its related file systems will be unavailable.

In the two weeks leading up to the maintenance period, we will dynamically reduce the maximum time for long-running (highp) jobs. After 9AM Thursday September 19th only interactive sessions and express jobs will run.

During this outage, updates will be made to the Hoffman2 compute nodes, login nodes, and Panasas storage subsystem. Jobs submitted prior to the outage will be queued for execution at the conclusion of maintenance.

If you have any questions or problems, please send email to High Performance Computing hpc@ucla.edu

SCRATCH file system upgraded

A recently purchased Panasas AS14 shelf was put into production the last week in May. This has increased the number of spindles serving /u/scratch by 50% and added an additional director blade. The additional disks will help the resiliency and performance of /u/scratch under heavy client load.

TOP

Hoffman2 Summer maintenance June 17-19, 2013

Hoffman2 Cluster summer maintenance will start 9AM Monday June 17th and continue through 5PM Wednesday, June 19th. This is during the week after spring quarter ends.

You will not be able to login or run jobs during this period. Starting Monday June 3rd, we will dynamically reduce the maximum time for long-running (highp) jobs. After 9AM Sunday June 16th only interactive sessions and express jobs will run. No pending jobs will be able to be carried forward and all pending jobs that do not start before the maintenance window begins will be deleted.

During the outage HP will perform semi-annual maintenance on POD hardware. We will install new Panasas storage system firmware and perform storage reconstruction tasks and other maintenance.

TOP

Spring 2013 Parallel Computing Classes

We are pleased to announce the parallel computing workshop series for this quarter. The schedule and locations of these free classes are:

Class details are available in the links above. Seats are limited.

TOP

Hoffman2 changes job scheduler Monday April 1st

During the preview window, we have determined that the new version of job scheduler (UGE 8.1.4) is unstable under load (confirmed by the vendor). In order to return the cluster to a stable operational condition, we have decided to revert back to an older stable scheduler. Doing so requires a number of steps:

  • Sunday, March 31 (9AM) we will start draining jobs running under the unstable UGE 8.1.4. Already running jobs will run to completion. All pending jobs that cannot start by 9AM will be deleted; users will have to resubmit deleted jobs once the cluster re-opens.
  • Monday, April 1 (9AM) all login nodes will be disabled; all user logins will be terminated. The cluster will be unavailable as we reboot compute nodes and switch the scheduler. As soon as we verify the scheduler is operational, we will return the cluster to its normal operation.

We expect this procedure to go smoothly. In the unlikely event of unforeseen complications, we may need to take further action which could include draining the cluster of long-running jobs and installing another version of the UNIVA job scheduler.

TOP

Hoffman2 Cluster is online

Spring maintenance has completed successfully. The Hoffman2 Cluster is now available for login and is accepting jobs on a “preview” basis, while we observe the new job scheduler under load. Only jobs requesting maximum 24 hours can be submitted now.

Long-running jobs, which request more than 24 hours wall-clock time, will be available to resource groups which have contributed nodes to Hoffman2 starting Monday, April 1st at 9AM. The new job scheduler will have shown itself to be stable under load by then, and its “preview” status upgraded to “production”.

During the Hoffman2 Spring maintenance period we upgraded the operating system to CentOS 6.4, installed a new version of the job scheduler Univa Grid Engine 8.1.4, upgraded $SCRATCH temporary storage, migrated campus home directories, upgraded node firmware and performed other maintenance. If you encounter any problems that might be due to the upgrades, please let us know as soon as possible.

TOP

Maple 17

Maple has been upgraded to version 17.

Maple is a symbolic and numeric mathematical program. Major areas include: algebra, calculus, differential equations, linear algebra, and statistics. Integrated visualization.

Please see How to Run Maple

Maple documentation is available from the vendor at http://www.maplesoft.com/documentation_center/

TOP

Hoffman2 Spring maintenance March 25-27, 2013

Hoffman2 Cluster spring maintenance will start 9AM Monday March 25th and continue through 5PM Wednesday, March 27th. This is spring break.

You will not be able to login or run jobs during this period. Starting Monday March 11th, we will dynamically reduce the maximum time for long-running (highp) jobs. After 9AM Sunday March 24th only interactive sessions and express jobs will run.

If early testing is successful, a new version of the job scheduler will be made production. In this case, no pending jobs will be able to be carried forward and all pending jobs that do not start before the maintenance window begins will be deleted. If we do not upgrade the job scheduler, pending jobs will start when the system comes back online.

During the outage we will install firmware upgrades and system image updates. We will retire the old scratch storage. Temporary SCRATCH storage has been replaced with a new, higher performance Panasas 14 system. We will also perform storage reconstruction tasks and other maintenance.

TOP

Hoffman2 new SCRATCH storage coming March 4th

New, faster temporary storage will be available starting Monday March 4th. At that time we will change the value of everyones SCRATCH environment variable to point to this new resource.

You do not have to make any changes in your code or UGE command files. If you have running jobs, they will continue using the temporary storage that SCRATCH pointed to at the time your job started. Jobs which start on or after March 4th will automatically use the new scratch storage. Your scratch storage quota remains 2 terabytes.

When SCRATCH is changed, we will also make an OLDSCRATCH environment variable that will point to your old scratch storage. You can use this variable to retrieve data written to old scratch by your jobs. For example:

cp $OLDSCRATCH/myfile $HOME/myscratchfile

The old scratch file system will be retired during the Hoffman2 Spring maintenance window the last week in March.

TOP

LAMMPS upgraded

LAMMPS 22 Feb 2013 version is now installed on the Hoffman2 Cluster in the /u/local/apps/lammps/current directory. The software can be run as a parallel application. It is available as a UCLA Grid application.

LAMMPS, Large-scale Atomic/Molecular Massively Parallel Simulator, is a classical molecular dynamics simulation code designed to run efficiently on parallel computers. It was developed at Sandia National Laboratories. The LAMMPS WWW Site at http://lammps.sandia.gov has more information about the code and its uses.

See How to Run LAMMPS LAMMPS documentation is available at http://lammps.sandia.gov/doc/Manual.html

TOP

Mathematica 9 in Education and Research class

Register for this free class at http://cfapps.ats.ucla.edu/cfapps/events/rsvp/RSVPNow.cfm?EveID=3238&SecID=3227

When, where:

Thursday February 28, 2013
10:00 am – 11:30 am
Visualization Portal (5628 Math Sciences)

Event Description:
Speaker: Kelvin Mischo, Certified Mathematica Instructor
http://www.wolfram.com/training/instructors/mischo.html

What:

  • See the newest ways to get students started with Mathematica 9 for course projects.
  • Learn how to author slideshows with formatted text/formulas along with live calculations in the same environment.
  • Review the basics of the programming language and useful ways to work with a users data or the load-on-demand data in Mathematica.
  • Discover new free-form input capabilities, and how to create: applications that show varying parameters in any type of calculation, publication-quality graphics, symbolic and numeric calculations, and new statistical capabilities in Mathematica 9.

Each attendee will receive an electronic copy of the examples to use as a guide for projects or course examples. No prior knowledge of Mathematica is required.

TOP

CPMD upgraded

The current CPMD version on Hoffman2 Cluster has been upgraded 3.15.3-1 and is available in the /u/local/apps/cpmd/current/parallel directory. The location of its pseudo-potential library is /u/local/apps/cpmd/3.15.3-1/contrib/PP_LIBRARY

CPMD (Car-Parrinello Molecular Dynamics) is an ab initio Electronic Structure and Molecular Dynamics Program. The CPMD code is a parallelized plane wave/pseudopotential implementation of Density Functional Theory, particularly designed for ab-initio Molecular Dynamics simulation.

See How to Run CPMD

Point your browser at http://www.cpmd.org/ for further information about CPMD and the CPMD Consortium.

TOP

BIG data tools Jan 29 1PM

Please join us Tuesday January 29th 1pm in the Visualization Portal, 5628 Math Sciences for a presentation on Resource aware protocols and hosted services for research data management by Rajkumar Kettimuthu, Fellow in the Computation Institute at Argonne National Laboratory and the University of Chicago.

Abstract: Big data is becoming a critical component of science. Researchers need tools that address their end-to-end distributed data management needs. But the small and medium research labs often lack the resources to deploy and maintain sophisticated software infrastructures. So, there is a strong need to do data management with modest effort. End users do not have the time and expertise to learn complex software. So the data management tools must be exceedingly easy to use. At the same time, these tools must be highly efficient and utilize the capabilities of advanced compute and network infrastructure. In this talk, I will present our work on hosted services that enables researchers to move, sync and share data in a simple, yet efficient way. I will also briefly discuss our work on resource and application aware protocols to effectively utilize 100 Gigabit networks and beyond.

Bio: Rajkumar Kettimuthu is a Fellow in the Computation Institute at Argonne National Laboratory and the University of Chicago. He is the technology coordinator for Globus GridFTP, widely used bulk data movement tool. Currently, he leads multiple projects to develop tools for research data management, and data movement protocols for next generation networks. He has published more than 50 articles in the areas of parallel, distributed and Grid computing. He is a senior member of both IEEE and ACM.

To RSVP for this free event, please go to:

http://cfapps.ats.ucla.edu/cfapps/events/rsvp/RSVPNow.cfm?EveID=3231&SecID=3220

TOP

Winter 2013 Parallel Computing Classes

IDRE is pleased to announce the high performance computing workshop series for this quarter. The topics and schedules are:

  • Feb  6, 2-4pm, Using Hoffman2 Cluster
  • Feb 11, 2-4pm, Intro to Parallel Computing and OpenMP
  • Feb 13, 2-4pm, Introduction to MPI
  • Feb 20, 2-4pm, Introduction to CUDA

The class descriptions and sign-up links are in this web page:

https://idre.ucla.edu/calendar

TOP

Mathematica upgraded to 9

Mathematica has been upgraded to version 9 It is available on both Hoffman2 and the UCLA Grid Portal http://grid.ucla.edu for both job scheduler batch (math.q) and interactive use. It is available as both serial and parallel applications.

Mathematica is a scientific program which features mathematical computation, symbolic manipulation, a large collection of numeric functions, graphics and a high-level programming language.

Software Central invites you to attend a no cost seminar Mathematica 9 in Education and Research on February 28, 2013, 10 to 11:30am in the Visualization Portal, 5628 Math Sciences Building. RSVP for this free event at http://cfapps.ats.ucla.edu/cfapps/events/rsvp/RSVPNow.cfm?EveID=3238&SecID=3227

Mathematica documentation is available online from the vendor Wolfram Research, Inc. athttp://reference.wolfram.com/mathematica/guide/Mathematica.html

Please see How to Run Mathematica for more information.

TOP

Hoffman2 Scheduled Maintenance January 31

Campus electricians will be on site at the POD data center the morning of Thursday, January 31 to correct problems with the high voltage power supply. We will necessarily have to shut down POD operations during this visit. It will not affect operations in the MSA data center or the IDRE data center.

We have begun dynamically reducing the maximum time for long-running jobs on POD nodes. Resource groups with purchased nodes that are located in the POD data center will still be able to submit jobs that request more time than is available before the outage begins. But those pending jobs will not start until after the outage concludes.

Thu, 31 Jan 2013 07:23:39 Power has been restored to the POD following the scheduled outage to fix an overvoltage problem.

TOP

Tecplot 360 upgraded

Tecplot 360 has been upgraded to version 2012.

Tecplot 360 is a program for the visualization and animation of scientific and engineering data. It includes support for CFD-type data.

Please see How to Run Tecplot.

Tecplot documentation is available from the vendor at http://www.tecplot.com/Support/Documentation.aspx

TOP

How to change the time request on pending long running jobs

If your pending job requests more time than is available before the scheduled maintenance window begins December 17th, it will not start. All pending jobs that do not start before the maintenance window begins will be deleted.

See Hoffman2 Winter Maintenance December 17 through 19, 2012 for information about the scheduled maintenance.

If your job can complete in less time, you may want to use the qalter command to change its h_rt value to something less than the remaining time to shutdown. Here’s how to do it.

  • Find all the -l arguments that your job uses. At the shell prompt, enter:
       qstat -j jobnumber | grep 'hard resource_list'
    

    where jobnumber is your job number. It returns something like:

       hard resource_list: h_data=4000M,h_rt=1209600,highp=true
    
  • Use the qalter command to fix the job. At the shell prompt, enter:
       qalter jobnumber -l h_data=4000M,highp=true,h_rt=288:00:00
    

    where 288:00:00 is today’s maximum highp h_rt value.

You can find today’s maximum highp h_rt value with:

   qconf -sq highp-queue-name

For example:

   qconf -sq ats_pod.q | grep h_rt

You can find all of your pending (qw) jobs with:

    myjobs -s p

or,

    qstat -s p -u $USER

If you have a lot of jobs that need qalter, you can save a list of the job numbers, one per line, in a file named $USER.joblist and use the create-qalter-commands script to make those qalter commands. The script does not run the qalter commands, it just creates them. You may want to use it like:

   qstat -s p | grep $USER > $USER.joblist
   create-qalter-commands 288 > my.qalter.cmds

where 288 is your new time request.

Check the script output:

   cat my.qalter.cmds

Run the qalter commands:

   sh -x my.qalter.cmds

If your job doesn’t start today, you will need to do the same thing tomorrow — if your job can complete in the remaining time to shutdown.

TOP

Mathematica upgraded to 8.0.4

Mathematica has been upgraded to version 8.0.4 It is available on both Hoffman2 and the UCLA Grid Portal for both job scheduler batch (math.q) and interactive use. It is available as both serial and parallel applications.

Mathematica is a scientific program which features mathematical computation, symbolic manipulation, a large collection of numeric functions, graphics and a high-level programming language.

Mathematica documentation is available online from the vendor Wolfram at http://reference.wolfram.com/mathematica/guide/Mathematica.html

Please see How to Run Mathematica for detailed information.

TOP

Hoffman2 Winter Maintenance December 17 through 19, 2012

There will be a scheduled outage starting Monday December 17th 9AM through Wednesday December 19th 6PM of the Hoffman2 Cluster. During this time we will upgrade data storage systems and electrical power, and perform other maintenance.

This maintenance window will occur after the end of Fall quarter.

We expect to dynamically reduce the time for long-running jobs starting December 3rd. Because we will have a new version of UGE, no pending jobs will be able to be carried forward. All pending jobs that do not start before the maintenance window begins will be killed.

TOP

Fall 2012 Parallel Computing Classes

IDRE High Performance Computing Group is pleased to announce the parallel computing workshop series for this quarter. The topics and schedules are:

  • October 29, 2-4pm, Using Hoffman2 Cluster
    5907 Math Sciences
  • October 30, 2-4pm, Introduction to Parallel Computing and OpenMP
    5907 Math Sciences
  • October 31, 2-4pm, Introduction to MPI
    5907 Math Sciences

The class descriptions and RSVP links are in this page: https://idre.ucla.edu/calendar

TOP

OpenACC GPU Programming Workshop

Location: UCLA Engineering IV Tesla Conference Room 53-125

The Pittsburgh Supercomputing Center, the National Institute for Computational Sciences and Georgia Tech will be presenting an OpenACC GPU programming workshop on October 16 and 17. Based on demand, they will be presenting this at 10 satellite sites. The hands-on platform will be Keeneland, the NSF’s premier GPU resource. The agenda is below and it is designed as two short days to be time-zone friendly. OpenACC is the accepted standard using compiler directives to allow quick development of GPU capable codes using standard languages and compilers. It has been used with great success to accelerate real applications within very short development periods. This workshop assumes knowledge of either C or Fortran programming. It will have a hands-on component using the NSF large scale GPU computing platform presently being deployed at the National Institute for Computational Sciences.

TOP

New Bio-Informatics tools

New versions of several bioinformatics programs have been installed:

bowtie 0.12.8
BWA 0.6.2
cufflinks 2.0.2
Picard tools 1.77
SAMtools 0.1.18 and bcftools
tophat 2.0.4

TOP

Picard tools 1.77

Picard comprises Java-based command-line utilities that manipulate SAM files, and a Java API (SAM-JDK) for creating new programs that read and write SAM files. Both SAM text format and SAM binary (BAM) format are supported. SAM (Sequence Alignment/Map) format is a generic format for storing large nucleotide sequence alignments. The Picard command-line tools are packaged as executable jar files.

The picard tools are installed in /u/local/apps/picard-tools/current/ To use them interactively, enter qrsh to start a session on a compute node. At the compute node shell prompt, enter:

java jvm-args -jar PicardCommand.jar option=value ...

where jvm-args are java arguments, PicardCommand.jar is one of the 40 available jar files, and option=value… are PicardCommand option value pairs.

For more information, see picard manual page: https://broadinstitute.github.io/picard/command-line-overview.html

TOP

tophat 2.0.4

TopHat is a fast splice junction mapper for RNA-Seq reads. It aligns RNA-Seq reads to mammalian-sized genomes using the ultra high-throughput short read aligner Bowtie, and then analyzes the mapping results to identify splice junctions between exons.

To use tophat, enter:

module load tophat
tophat

For more information, see tophat manual page: http://ccb.jhu.edu/software/tophat/manual.shtml

TOP

cufflinks 2.0.2

Cufflinks is a tool for transcriptome assembly and isoform quantitiation from RNA-seq reads. Cufflinks assembles transcripts, estimates their abundances, and tests for differential expression and regulation in RNA-Seq samples. It accepts aligned RNA-Seq reads and assembles the alignments into a parsimonious set of transcripts. Cufflinks then estimates the relative abundances of these transcripts based on how many reads support each one, taking into account biases in library preparation protocols

To use cufflinks, enter:

module load cufflinks
cufflinks

For more information, see cufflinks manual page: http://cole-trapnell-lab.github.io/cufflinks/manual/

TOP

bowtie 0.12.8

Bowtie is an ultrafast, memory-efficient short read aligner geared toward quickly aligning large sets of short DNA sequences (reads) to large genomes.

To use bowtie, enter:

module load bowtie
bowtie

For more information, see bowtie manual page: http://bowtie-bio.sourceforge.net/manual.shtml

TOP

BWA 0.6.2

Burrows-Wheeler Aligner (BWA) is an efficient program that aligns relatively short nucleotide sequences against a long reference sequence such as the human genome. It implements two algorithms, bwa-short and BWA-SW. The former works for query sequences shorter than 200bp and the latter for longer sequences up to around 100kbp. Both algorithms do gapped alignment. They are usually more accurate and faster on queries with low error rates. [BWA home page: http://bio-bwa.sourceforge.net]

To use bwa, enter:

module load bwa
bwa

bwa perl scripts qualfa2fq.pl solid2fastq.pl xa2multi.pl are located in /u/local/apps/bwa/current/

For more information, see BWA manual page: http://bio-bwa.sourceforge.net/bwa.shtml

TOP

SAMtools 0.1.18 and bcftools

SAMtools – Utilities for the Sequence Alignment/Map (SAM) format

SAM (Sequence Alignment/Map) is a flexible generic format for storing nucleotide sequence alignment. SAM Tools provide various utilities for manipulating alignments in the SAM format, including sorting, merging, indexing and generating alignments in a per-position format. bcftools are included with the samtools installation. [SAMtools home page: http://samtools.sourceforge.net]

To use samtools, at a shell prompt, enter:

module load samtools
samtools

bcftools – Utilities for the Binary Call Format (BCF) and VC

The view command of bcftools calls variants, tests Hardy-Weinberg equilibrium (HWE), tests allele balances and estimates allele frequency. See /u/local/apps/samtools/current/bcftools/README_bcftools for more description.

For more information, see SAMtools manual page: http://samtools.sourceforge.net/samtools.shtml

TOP

NAG Fortran compiler available

The Numerical Algorithms Group (NAG) Fortran Compiler is robust, highly tested, and valued by developers all over the globe for its checking capabilities and detailed error reporting. Release 5.3 has extensive support for both legacy and modern Fortran features. It also supports parallel programming with OpenMP.

To access nagfor or its debugger dbx90, enter:

module load nag

For more information, see Compilers and the Documentation links on that page.

TOP

ABAQUS 6.12 available

ABAQUS has been upgraded to version 6.12

ABAQUS is a suite of general-purpose, nonlinear finite element analysis (FEA) programs for stress, heat transfer, and other types of analysis. ABAQUS runs a wide range of linear and nonlinear engineering simulations.

Please see How to Run ABAQUS.

ABAQUS manuals are not available on the web. ABAQUS documentation is available on Hoffman2 for X sessions with the abaqus doc command.

TOP

Quantum ESPRESSO 5.0.1 available

Quantum ESPRESSO v5.0.1 is now available on the Hoffman2 Cluster.

Quantum ESPRESSO is an integrated suite of computer codes for electronic-structure calculations and materials modeling at the nanoscale. It is based on density-functional theory, plane waves, and pseudopotentials (both norm-conserving and ultrasoft). More information can be found at http://www.quantum-espresso.org/

To run parallel Quantum ESPRESSO in batch, enter: espresso.q

You may want to run Quantum ESPRESSO interactively to, for example, test your input deck:

  • Use qrsh to obtain an interactive session
  • Enter: source /u/local/bin/set_qrsh_env.sh
    (or source /u/local/bin/set_qrsh_env.csh for tcsh shell users)
  • Enter: module load quantum-espresso
  • Enter: mpiexec -n $NSLOTS QE executable -inp input file

where QE executable is a Quantum ESPRESSO executable, and input file is the name of your input file.

The Quantum ESPRESSO documentation is available on-line at http://www.quantum-espresso.org/wp-content/uploads/Doc/user_guide.pdf

TOP

Amber12 available

Amber12 and Amber Tools12 are now available on the Hoffman2 Cluster for both job scheduler batch (amber.q) and interactive use. Amber12 is available as both serial and parallel applications through the UCLA Grid Portal

Amber Molecular Dynamics Package is a set of molecular mechanical force fields for the simulation of biomolecules and a package of molecular simulation programs.

The Amber12 Reference Manual is available from http://ambermd.org/doc12/Amber12.pdf AmberTools12 Reference Manual is available fromhttp://ambermd.org/doc12/AmberTools12.pdf

Please see How to Run Amber for detailed information.

TOP

Virtual School of Computational Science and Engineering (VSCSE)

UCLA’s Institute for Digital Research and Education is pleased to announce its participation in the 2012 Virtual School of Computational Science and Engineering (VSCSE). Students and recent graduates from all disciplines can gain valuable skills in computational science–including programming for many-core processors and heterogeneous systems and leveraging cloud resources–during hands-on summer school courses offered by the VSCSE. The VSCSE helps students use emerging petascale computing resources to address real problems in a wide range of science and engineering disciplines

IDRE is offering scholarship awards to UCLA students to cover the registration fee costs ($100 per course) for these classes. To register, go tohttp://www.vscse.org/index.html

TOP

Using the SCRATCH file system

On July 1st any of your files in /u/scratch outside of your personal $SCRATCH directory will be deleted.

We need to enforce the use of $SCRATCH in order for /u/scratch to be usefully searchable and files to be findable. Currently /u/scratch contains over 40 terabytes of temporary files; there are over 1,000 active users on Hoffman2. You have a 2TB quota on this file system.

At the completion of the Hoffman2 Spring quarter maintenance window in March we announced:

A new environment variable $SCRATCH has been defined for each person’s account. You should use the $SCRATCH environment variable in your job command files and scripts instead of hard-coding a specific directory name. Use it like:

$SCRATCH/myfilename

If you have any problems or questions, please send email to High Performance Computing.

TOP

Hoffman2 Cluster is online

Summer maintenance has completed successfully ahead of schedule. The Hoffman2 Cluster is now available for login and is accepting jobs.

During the outage we performed hardware maintenance and upgrades to the Panasas storage system, did re-wiring in the POD Data Center, updated the operating system and accomplished several network tasks.

If you encounter any problems that might be due to the upgrades, please let us know as soon as possible.

TOP

Hoffman2 Summer Scheduled Maintenance, June 4-6, 2012

There will be a scheduled outage starting Monday June 4th 9AM through Wednesday June 6th 5PM of the Hoffman2 Cluster. You will not be able to login or run jobs during the outage. We expect to dynamically reduce the time for long-running jobs starting May 21st.

During outage we will perform hardware maintenance and upgrades to the Panasas storage systems, re-wiring in the POD Data Center, an update to the operating system and several network tasks.

TOP

MATLAB 7.14 R2012a

MATLAB has been upgraded to version 7.14 (R2012a). If you have compiled matlab executables made with the mcc command or mcc.q you should rebuild them before running them.

MATLAB is a language for technical computing that combines numeric computation, advanced graphics and visualization, and a high-level programming language.

Please see How to Run MATLAB.

MATLAB documentation is available from the vendor at

TOP

Data Transfer Node dtn2

dtn2.hoffman2.idre.ucla.edu has been configured to expedite file transfers which use scp, sftp, or rsync commands on your local machine to copy data to or from the Hoffman2 Cluster.

The fastest, easiest method of file transfer is still Globus Online.

For applications which are not suited to Globus Online, use dtn2 for large data files transfer instead of using a login node.

Upload example using dtn2:

scp mybigfile login_id@dtn2.hoffman2.idre.ucla.edu:mydir/

where:

  • scp command on your local desktop or laptop
  • mybigfile is the name of a local file to upload
  • login_id is your Hoffman2 Cluster user name
  • mydir/ is an existing Hoffman2 directory

Download example using dtn2:

scp login_id@dtn2.hoffman2.idre.ucla.edu:mydir/mybigfile .

where:

    • scp command on your local desktop or laptop
    • login_id is your Hoffman2 Cluster user name
    • mydir/mybigfile is the name of a Hoffman2 file to download

.

    means the same file name in your current local directory

For more information on scp, please see Using scp or sftp

TOP

Maple 16

Maple has been upgraded to version 16.

Maple is a symbolic and numeric mathematical program. Major areas include: algebra, calculus, differential equations, linear algebra, and statistics. Integrated visualization.

Please see How to Run Maple.

Maple documentation is available from the vendor at http://www.maplesoft.com/documentation_center/

TOP

Spring 2012 Parallel Computing Classes

We are pleased to announce the parallel computing workshop series for this quarter. The topics include how to use the Hoffman2 Cluster, how to port a serial code to multi-core platforms using OpenMP, how to write distributed memory programs using MPI, and how to write GPU computational code.

The schedules and locations of these free classes are:

  • April 18 Using Hoffman2 Cluster, 5907 Math Sciences
  • April 23 Intro to Parallel Computing and OpenMP, 5907 Math Sciences
  • April 25 Introduction to MPI, 5907 Math Sciences
  • April 30 Introduction to CUDA, 5907 Math Sciences
  • May 2 Using Hoffman2 Cluster, 5907 Math Sciences

The class descriptions are within the Event Details links at

http://www.idre.ucla.edu/

under Upcoming Events.

TOP

Scratch file system upgraded

Spring quarter downtime completed March 29th. The Hoffman2 Cluster is available for login and is accepting jobs.

    • The operating system has been upgraded to CentOS 6.2

 

  • The new, high-performance Panasas Storage System is now hosting the scratch file system. There is 3 terabytes per user available for short term storage.

A new environment variable $SCRATCH has been defined for each person’s account. You should use the $SCRATCH environment variable in your job command files and scripts instead of hard-coding a specific directory name. Use it like:

$SCRATCH/myfilename

Note for people with files in the old scratch file systems:

    • /u/scratch/. If you have written files to /u/scratch before downtime started March 26th, you can find them in /u/scratch3 for the next 7 days. Then /u/scratch3 file system will be deleted.

 

  • /u/scratch2/. If you have written files to /u/scratch2 before downtime started March 26th, you can find them in /u/scratch/.

Please see Data Storage for more information.

If you encounter any problems that might be due to the upgrades, please let us know as soon as possible.

TOP

Hoffman2 Spring Maintenance March 26 through 28, 2012

There will be a scheduled outage starting Monday March 26th 9AM through Wednesday March 28th 5PM of the Hoffman2 Cluster. During this time we will upgrade the cluster operating system to CentOS 6.2 and perform other maintenance.

We have delayed this maintenance until after the end of the current quarter so as to minimize inconvenience to our users. We expect to dynamically reduce the time for long-running jobs starting March 12. More information on the outage will be available soon.

TOP

Winter 2012 Parallel Computing Classes

We are pleased to announce the parallel computing workshop series for this quarter. The topics include how to use the Hoffman2 Cluster, how to port a serial code to multi-core platforms using OpenMP, how to write distributed memory programs using MPI, and how to write GPU computational code.

The schedules and locations of these free classes are:

  • January 31 Using Hoffman2 Cluster, 11348 Young Research Library
  • February 2 Intro to Parallel Computing and OpenMP, 5907 Math Sciences
  • February 3 Introduction to MPI, 5907 Math Sciences
  • February 8 Introduction to CUDA, 5907 Math Sciences

The class descriptions are within the RSVP links at

http://www.idre.ucla.edu/

under “Upcoming Events”.

TOP

UCLA Winter Closure December 22, 2011 – January 2, 2012

Hoffman2 will be up and fully operational during the UCLA winter closure, however there will not be any IDRE personnel onsite during this time. We will do our best to insure the cluster remains in operation but, depending on the issues that arise, we will not be addressing them until January 3rd.

TOP

Hoffman2 upgrades

Winter quarter downtime has completed successfully ahead of schedule. The Hoffman2 Cluster is available for login and is now accepting jobs.

IMPORTANT: There are new versions of compiler libraries. If you have programs written in gcc, python, etc., they will need to be re-compiled. Please see FAQ Re-compiling for CentOS 6? for more information.

    • The operating system has been upgraded to CentOS 6.

 

    • A new job scheduler, Univa Grid Engine 8.0.1 which is an upgrade to the Sun Grid Engine, was installed.

 

    • Network supervisor modules in the network core were upgraded. This effectively doubles the throughput and capacity of UCLA’s research computing networks. In addition, additional port capacity on newly installed high-speed line cards will not only increase the performance and efficiency of existing high-performance storage and compute systems, but will also make it possible to integrate new equipment into the research computing environment.

 

  • Scratch space firmware has been upgraded. We expect increased performance and thoughput.

If you encounter any problems that might be due to the upgrades, please let us know as soon as possible.

TOP

How to change the time request on pending long-running jobs

If your pending job requests more time than is available before the scheduled shutdown (see Hoffman2 Cluster Downtime Starts November 27th) it will will be deleted November 27th.

If your job can complete in less time, you may want to use the qalter command to change its h_rt value to something less than the remaining time to shutdown. Here’s how to do it.

  1. Find out all the “-l” arguments that your job uses. At the shell prompt, enter:

    qstat -j jobnumber | grep ‘hard resource_list’

    which returns something like:

    hard resource_list: h_data=4000M,h_rt=1209600,highp=true

  2. Use the qalter command to fix the job. At the shell prompt, enter:

    qalter jobnumber -l h_data=4000M,highp=true,h_rt=288:00:00

    where jobnumber is the job number, and
    where 288:00:00 is today’s maximum highp h_rt value.

You can find today’s maximum highp h_rt value with:

qconf -sq highp-queue-name

For example:

qconf -sq ats_msa.q | grep h_rt

You can find all of your pending (qw) jobs with:

myjobs -s p

or,

qstat -s p -u $USER

If you have a lot of jobs that need qalter, you can save a list of the job numbers, one per line, in a file named $USER.joblist and use the create-qalter-commands script to make those qalter commands. The script does not run the qalter commands, it just creates them. You may want to use it like:

qstat -s p | grep $USER > $USER.joblist
create-qalter-commands > my.qalter.cmds

Check the script output:

more my.qalter.cmds

Run the qalter commands:

sh -x my.qalter.cmds

Note that the create-qalter-commands script uses today’s maximum h_rt value. If your job doesn’t start today, you will need to do the same thing tomorrow — if your job can complete in the remaining time to shutdown.

TOP

Hoffman2 Cluster Downtime Starts November 27th

There will be a scheduled outage Sunday November 27, 2011 1PM through Wednesday November 30th 5PM of the Hoffman2 Cluster.

During this time we will install a new version of the operating system, CentOS 6, and a new job scheduler, Univa 8, which is an upgrade to the current Sun Grid Engine. We will migrate and expand /u/scratch to our newest, fastest Panasas storage. We will also perform extensive network maintenance and other maintenance.

Long-running job time will be dynamically reduced beginning 5PM November 13th. After that date, the job scheduler will not accept jobs that cannot complete before the outage begins. After a job is accepted, SGE will let it start only if it can complete before 1PM November 27th. Only express jobs will be accepted after 1PM Saturday November 26th.

The express queue will accept jobs until 11AM November 27th:

qsub -l express [other qsub options] your-executable-or-script

The interactive queues will be available until 11AM November 27th:

qrsh -l i [other qrsh options]

Jobs which have not completed before the outage will not be carried over to the new Univa job scheduler. Any job that did not start before the outage begins will be deleted.

We will make every effort to keep the downtime as short as possible. In all probability we will not need the entire time scheduled, but we want to insure we get all this work completed.

The next quarterly downtime, if necessary, is tentatively scheduled for the week of March 19-23, 2012.

TOP

Fall 2011 Parallel Computing Classes

We are pleased to announce the parallel computing workshop series for this quarter. The topics include how to use the Hoffman2 Cluster, how to port a serial code to multi-core platforms using OpenMP, how to write distributed memory programs using MPI, and how to write GPU computational code.

The schedules and locations of these free classes are:

  • October 26, 2-4pm, Using the Hoffman2 Cluster, 5907 Math Sciences
  • October 27, 2-4pm, Introduction to Parallel Computing and OpenMP, 5907 Math Sciences
  • November 1, 2-4pm, Introduction to MPI, 5907 Math Sciences
  • November 9, 2-4pm, Introduction to CUDA, 5907 Math Sciences

The class descriptions are within the RSVP links at

http://www.idre.ucla.edu/

TOP

Live web tutorial: Moving Data with Globus Online

Globus Online is hosting a webcast for UCLA cluster users on Wednesday, September 28, at 2 p.m. Pacific.

Register at https://www.globus.org/case-studies/semel-ucla

The topic is high-performance data movement — if your work requires transferring data to or from a UC Grid site, then you will want to know how to use Globus Online, the fast, reliable service for file transfer. Globus Online makes it easy for researchers to move data wherever they need it, not only among UC Grid sites but to any location. In this live webcast, attendees will learn how to move files between any two locations, including a UC Grid resource, campus server, or their own laptop. The session also will include a demo of the most frequently used features, tips and tricks, and Q&A with the Globus Online project lead. For more information or to register, visit https://www.globus.org/case-studies/semel-ucla

TOP

SAMtools 0.1.17 and BWA 0.5.9

SAMtools SAM (Sequence Alignment/Map) is a flexible generic format for storing nucleotide sequence alignment. SAM Tools provide various utilities for manipulating alignments in the SAM format, including sorting, merging, indexing and generating alignments in a per-position format. bcftools are included with the samtools installation. [SAMtools home page: http://samtools.sourceforge.net]

SAMtools version 0.1.17 is located in /u/local/apps/samtools/current/bin/ directory. Perl scripts are in /u/local/apps/samtools/current/misc/

To use samtools, at a shell prompt, enter:

module load samtools
samtools

bcftools The view command of bcftools calls variants, tests Hardy-Weinberg equilibrium (HWE), tests allele balances and estimates allele frequency. bcftools are located in /u/local/apps/samtools/current/bcftools/ See the README file there for more description.

For more information, see SAMtools manual page: http://samtools.sourceforge.net/samtools.shtml

BWA Burrows-Wheeler Aligner (BWA) is an efficient program that aligns relatively short nucleotide sequences against a long reference sequence such as the human genome. It implements two algorithms, bwa-short and BWA-SW. The former works for query sequences shorter than 200bp and the latter for longer sequences up to around 100kbp. Both algorithms do gapped alignment. They are usually more accurate and faster on queries with low error rates. [BWA home page: http://bio-bwa.sourceforge.net]

bwa version 0.5.9 and perl scripts qualfa2fq.pl and solid2fastq.pl are located in:

/u/local/apps/bwa/current/

For more information, see BWA manual page: http://bio-bwa.sourceforge.net/bwa.shtml

TOP

LONI Pipeline

The Pipeline Environment is a free workflow application for computational sciences. Pipeline users can navigate and discover existing end-to-end workflow solutions, develop new module descriptions and processing graphical protocols, execute complex heterogeneous analyses, take advantage of distributed hardware infrastructure and databases, validate and openly disseminate the provenance of their data and processing protocols. See:

http://pipeline.loni.ucla.edu/

Available Imaging and NGS Analysis Pipeline Workflows

There is a wide array of informatics and genomics modules and workflows that are already developed (e.g., BLAST, EMBOSS, mrFAST, GWASS, PLINK, R, MAQ, SAMtools, Bowtie, CNVer, QC, GATK, etc.) Additional workflows are constantly added with help from colleagues across the globe. See:

http://pipeline.bmap.ucla.edu/learn/pipeline-events/

Try the Pipeline Environment online

You can try the Pipeline workflows and computational environment via your java-enabled browser without any special software installation, hardware requirements or account privileges:

URLs
References

TOP

10Gb File Transfer with Globus Online and dtn1

The Hoffman2 Cluster production Endpoint ucla#dtn1 with its 10 gigabit ethernet connection is now available for Globus Online file transfer. Please see:

High-speed file transfer with Globus Online

The Globus Online organization will be hosting a webinar for us in the near future. We will announce it when the date has been set.

Here are a few sites that are part of Globus Online. If you have an account at one of these sites, you can use Globus Online to transfer data there.

  • Biomedical Informatics Research Network (BIRN) at UCI, UCLA, UCSD, UCSF, and many others across the country.
  • Earth System Grid (ESG)
  • Energy Sciences Network (ESNET) at Argonne, Lawrence Berkeley, and Brookhaven National Laboratories
  • National Energy Research Scientific Computing Center (NERSC)
  • National Grid Service (UK Academic Computing Grid) at many locations
  • Open Science Grid (OSG) at many locations
  • Structural Biology Grid (SBGRID) at many locations
  • TeraGrid (TG) resources
  • UCLA Hoffman2 Cluster (UCLA)

TOP

High-speed file transfer with Globus Online

The Hoffman2 Cluster is pleased to offer high-speed file transfer service through Globus Online. Globus Online is a software tool to transfer files across the web in a reliable, high-performance and secure way. It provides fault-tolerant data transfer for large files. It has a simple, user-friendly web interface. It is appropriate for transferring large files.

You can use Globus Online to transfer files either between your desktop machine and a remote machine like the Hoffman2 Cluster, or between two remote machines on which you have accounts. For example if you also have an account at Argonne National Lab, you can use Globus Online to transfer files between your Hoffman2 and Argonne accounts. The remote site must also be a Globus Online participant and have grid-enabled your account there. File transfer between private desktop machines is not available at this time.

You need to use your UCLA/UC Grid account in order to transfer files to or from the Hoffman2 Cluster.

We have 10Gb hardware on order for the production ucla#dtn1 service and will announce it in the near future [announced August 4, 2011].

See High-speed file transfer with Globus Online for information on how to setup and use Globus Online with the Hoffman2 Cluster.

TOP

Connecting to Hoffman2 with NX Client

You can use an NX client on your local machine to connect to the Hoffman2 Cluster and run Hoffman2 Cluster graphical applications. NX provides near-local speed application responsiveness over high-latency, low bandwidth links.

NX is a secure, compressed protocol for remote X Window System connections. It is an alternative to running an X Server on your local machine. An NX server runs on Hoffman2 Cluster login nodes; you run an NX client on your local machine.

There are free NX clients available from NoMachine (www.nomachine.com) for Windows, Linux, Mac OSX, and Solaris.

See NX Client for more information about using NX to access the Hoffman2 Cluster.

Documentation available at http://www.nomachine.com/documents

TOP

CERN ROOT 5.30.00

CERN ROOT program has been updated to version 5.30.00. Version 5.26.00 is also available.

ROOT is an object-oriented program and library developed by CERN. It is a framework for data processing. “It was originally designed for particle physics data analysis and contains several features specific to this field, but it is also used in other applications such as astronomy and data mining.” [http://en.wikipedia.org/wiki/ROOT (27 June 2011)]

To use the CERN ROOT C++ interpreter:

  1. Use qrsh to obtain a session on a compute node
  2. At the compute node shell prompt, enter: module load cern_root
  3. At the compute node shell prompt, enter: root

See How to Run CERN ROOT.

For more information, please refer to http://root.cern.ch An extensive Users Guide is available from that site.

TOP

Hoffman2 Cluster is back online!

The Hoffman2 Cluster is now open to login and jobs, and the queues for long-running jobs have been restored.

Some of the major tasks accomplished:

  • Temporary local storage on compute nodes (SGE $TMPDIR space) has been increased to at least 10GB per processor.
  • The /u/scratch filesystem has been moved to our newest and fastest storage and increased in size to 50TB.
  • Final replication of the BlueArc storage has completed. The BlueArc’s firmware has been upgraded and it has been reconfigured.
  • The operating system on the compute nodes has been upgraded to Centos 5.6. Firmware and BIOS updates on the compute nodes.
  • Internal network extensions and hardware upgrades.

These changes should ensure the continued high performance and reliability of the Hoffman2 Cluster into the future. If you notice anything that doesn’t seem correct please let us know. Thank you for your patience during this upgrade.

TOP

Hoffman2 Cluster Downtime Starts June 13 9PM

There will be a scheduled outage Monday June 13th 9PM 2011 through Wednesday June 15th 9PM of the Hoffman2 Cluster. We have delayed this maintenance until after the end of the current quarter so as to minimize inconvenience to our users.

During these 48 hours we will complete BlueArc incremental copying to permanent storage, upgrade the cluster operating system to Centos 5.6, enlarge the local storage space on compute nodes and perform other necessary maintenance.

The time for long-running jobs will be dynamically reduced beginning May 30th. After that date, SGE will not accept jobs that cannot complete before the outage begins. Once a job is accepted, SGE will let it start only if it can complete before 9PM June 13th.

The express queue will be available until 7PM June 13th for jobs:

qsub -l expresss [other qsub options] your-executable-or-script

The interactive queues will be available until 7PM June 13th for sessions:

qrsh -l i [other qrsh options]

TOP

Spring 2011 Parallel Computing Classes

We are pleased to announce the parallel computing workshop series for this quarter. The topics include: how to efficiently use the Hoffman2 Cluster, how to port a serial code to multicore platforms using OpenMP, how to write distributed-memory programs using MPI, and how to use CUDA to run programs on graphics processors. These are free classes.

  • Using the Hoffman2 Cluster, April 27, 2 p.m.
  • Introduction to Parallel Computing and OpenMP, May 3, 2 p.m.
  • Introduction to MPI, May 5, 2 p.m.
  • Parallel Computing Lab, May 6, 2 p.m.
  • Introduction to CUDA, May 12, 2 p.m.

The class descriptions are within the Event Details links at http://www.idre.ucla.edu/ (under Upcoming Events).

TOP

Hoffman2 File System Performance Issue, April 27th outage

Over the last few weeks the Hoffman2 Cluster has experienced intermittent storage performance dropouts caused by our BlueArc file server. We believe these issues were caused by an error in the BlueArc storage server firmware.

BlueArc Corp. has provided us with new firmware to fix this problem, and it has been installed. However we find that in order to restore system performance, we must reformat all of our storage. This is similar to “defragging” a hard drive on a laptop, but orders of magnitude larger.

BlueArc Corp. will be lending us additional hardware so that we can accomplish the reformat, and we expect to receive that some time next week. As soon as it is installed, we will begin duplicating files to it. This will not affect your use of the Cluster.

Wednesday April 27th we will close the Cluster to logins and jobs, so that no files will change while final incremental copying is done. The long-running (highp) queues will be shortened gradually so that by April 27th all but long-running jobs started today (April 15th) will have completed.

After completion of the April 27th outage, we expect to see an immediate improvement in file system performance. At this time we will also migrate the /u/scratch filesystem to our newest and fasted storage. We expect scratch performance to be much improved as a result. In addition, the size of the scratch filesystem will increase by 2.5 times.

Files will then be copied back to permanent storage on our BlueArc. We expect this to take several days, but again this will not affect your use of the Cluster. We will then close the Cluster for a second time to complete the second round of incremental copying.

We would like to reassure everyone that no files or data have been lost. The issue is file system performance, which has affected our high performance system as a whole.

TOP

Mathematica 8 available

Mathematica version 8 is now available on the Hoffman2 Cluster and the UCLA Grid Portal (http://grid.ucla.edu) for both job scheduler batch (math.q) and interactive use. It is available as both serial and parallel applications.

Mathematica is a scientific program which features mathematical computation, symbolic manipulation, a large collection of numeric functions, graphics and a high-level programming language.

Mathematica documentation is available online from the vendor Wolfram at http://reference.wolfram.com/mathematica/guide/Mathematica.html

Please see How to Run Mathematica for detailed information.

TOP

New MATLAB v7.11

MATLAB has been upgraded to version 7.11 (R2010b). You should use matlab.q to rebuild any job scheduler command files (jobname.cmd) that you may have from previous versions.

MATLAB is a language for technical computing that combines numeric computation, advanced graphics and visualization, and a high-level programming language.

Please see How to Run Matlab for more information. MATLAB documentation is available from the vendor athttp://www.mathworks.com/access/helpdesk/help/helpdesk.html

TOP

Amber11 available

Amber11 and Amber Tools v.1.4 are now available on the Hoffman2 Cluster for both job scheduler batch (amber.q) and interactive use. Amber11 is available as both serial and parallel applications through the UCLA Grid Portal

Amber Molecular Dynamics Package is a set of molecular mechanical force fields for the simulation of biomolecules and a package of molecular simulation programs.

The Amber 11 User’s Guide is available from http://ambermd.org/doc11/Amber11.pdf Amber Tools User’s Guide is available fromhttp://ambermd.org/doc11/AmberTools.pdf

Please see How to Run Amber for detailed information.

TOP

New Default Job Time Limit

If you do not specify a time limit for your job or interactive session, the job scheduler will end it after two hours. The reason for this new limit is to protect the job scheduler and allow it to back-fill and schedule jobs more efficiently.

You should always specify a time limit for your jobs and interactive sessions. If you are using a queue script to create your job scheduler command file (job.q, matlab.q, mpi.q, etc.), it will prompt you for a time limit value. If you are using a native job scheduler qsub or qrsh command, add h_rt (or time) to your other parameters.

Example of submitting a job with an 8-hour time limit:

qsub -l h_rt=8:00:00,other-parameters

Example of starting an interactive session on the interactive nodes with a 4 hour time limit:

qrsh -l i,h_rt=4:00:00,other-parameters

TOP

Winter 2011 Free Parallel Computing Classes

We are pleased to announce the parallel computing workshop series for this quarter. The topics include how to efficiently use the Hoffman2 Cluster, how to port a serial code to multicore platforms using OpenMP, writing distributed-memory programs using MPI, and GPU computing using CUDA. The schedules and locations of these free classes are:

  • Feb 2, 2 to 4 p.m., Using the Hoffman2 Cluster, 5628 Math Sciences
  • Feb 8, 2 to 4 p.m., Introduction to Parallel Computing and OpenMP, 5628 Math Sciences
  • Feb 10, 2 to 4 p.m., Introduction to MPI, 5628 Math Sciences
  • Feb 11, 2 to 4 p.m., Parallel Computing Hands-on Lab, CLICC Classroom B (320B Powell)
  • Feb 16, 2 to 4 p.m., Introduction to CUDA, 5628 Math Sciences

To sign up for these free classes, please go to http://www.idre.ucla.edu and see the Upcoming Events column.

TOP

Express queue is production

The express queue, which was announced November 9th, has shed it experimental status. It has proved tremendously popular, accounting for over 90% of jobs run in December 2010 on the Hoffman2 Cluster. When you use express.q your jobs may start within a few minutes of submission.

It is no longer limited to evenings and weekends, so you do not have to tell qsub -w n when you submit a job to this queue.

The express queue is limited to serial jobs, or array jobs, or shared memory parallel jobs that run on a single node (shared parallel environment). You cannot request more than 2 hours of wall-clock time. Parallel multi-node jobs will not be able to use this queue. You cannot use this queue to run on your resource group’s own nodes (highp).

How to request the express queue
  1. Make sure that each of your jobs will finish in 2 hours. Jobs submitted to this queue will be unconditionally terminated after 2 hours.
  2. To submit jobs to this queue, add “-l express” in addition to other parameters you typically use. Request time of no more than 2 hours.
  3. To direct an existing pending job to this queue, use

    qalter -l express JOB_ID

    where JOB_ID is its job id as shown by qstat -u $USER or myjobs script.

Sample job submission scenarios

For a simple serial job requesting one hour

qsub -l express,h_rt=1:00:00 myjob.cmd

For a simple array job requesting one hour per jobtask

qsub -l express,h_rt=1:00:00 -t 1:1000 myjobarray.cmd

If you do not request time, your job will run for maximum 2 hours set by the queue and will then be terminated.

TOP

Queue scripts enhanced

The queue scripts that help you build a job scheduler command file (job.q, mpi.q, matlab.q, etc.) have been enhanced to support the recent job scheduler queue reconfiguration.

  • If you belong to more than one resource group which has contributed nodes to the Hoffman2 Cluster, you can direct your job to use a particular group’s nodes. (-u and -rg resource-group command line options).
  • If your resource group owns nodes in both the IDRE and MSA data centers, you can choose where your job will run. (-u and -dc data-centercommand line options).

Additional support has been added to the job scheduler for parallel jobs that request a large number of cores. (-u and -n parallel-tasks command line options).

For more information, select the new Info function from an interactive queue script’s menu, or enter man queue at the shell prompt, or point your browser atQueue Scripts.

TOP

Hoffman2 Scheduled Downtime

There will be a scheduled downtime for Hoffman2 from Wednesday, December 15th, 8:00am through Thursday, December 16th, 12 noon. During this period we will be doing the following upgrades and fixes:

  1. Replace two failing electrical breakers that power portions of Hoffman2 in the MSA Data Center.
  2. Reconfigure the BlueArc storage system to maximize performance and allow for future storage service offerings.
  3. Add a new management capability to our Infiniband fabric.
  4. Apply patches to the Hoffman2 OS.
  5. Reconfigure some equipment to allow for future growth of the cluster.

We will make every effort to keep the downtime as short as possible. In all probability we will not need the entire time scheduled but we want to insure we get all this work completed.

In the future we will be posting a schedule for quarterly maintenance downtime so it will be easier for our users to plan for these outages.

Holiday Shutdown Information

Hoffman2 will be up and fully operational during the holiday shutdown however there will not be any ATS personnel onsite during this time. We will do our best to insure the cluster remains in operation but, depending on the issues that arise, we will not be addressing them until after January 3rd.

TOP

TeXLive version 2010 available

The 2010 version of TeXLive has been installed. TeXLive is a comprehensive TeX/LaTeX system. TeX is a document-typesetting system commonly used to create scientific and mathematical literature. LaTex is a document markup language and document preparation system for the TeX typesetting program.

To load texlive into your environment, at the shell prompt, enter:

module load texlive

This will load TeXLive version 2010 for your current login session. To load a specific version, use:

module load texlive/2009

or,

module load texlive/2010

To see help information, at the shell prompt, enter:

module whatis texlive

or,

module whatis texlive/2010

Please see http://www.tug.org/texlive/ for more information about TeXLive. www.tug.org is the authoritative site for all TeX documentation.

This faq may be a useful place to start to learn about TeX:

http://www.tex.ac.uk/

TOP

R version 2.12.0 available

R and its extension packages haves been upgraded to version 2.12.0 To see all packages which are installed, at the R command prompt issue the command:

> library()

R is GNU S, a freely available language and environment for statistical computing and graphics which provides a wide variety of statistical and graphical techniques: linear and nonlinear modelling, statistical tests, time series analysis, classification, clustering, etc. Please consult the R project homepage (http://www.r-project.org/) for further information and documentation.

See How to Run R.

TOP

Experimental express queue

Over the past few months we have observed cluster utilization going down during weekends and nights. In order to offset that and boost the cluster’s utilization rate, we have set up an experimental queue that will allow lots of short-time jobs to go through more quickly with minimum or no effects on the existing shared cluster queues. The experiment may be modified or terminated in the future depending on the cluster utilization statistics. Currently, this queue is scheduled to receive jobs between 6:00 PM and 6:00 AM and weekends.

At this time, the experiment is limited only to serial jobs or array jobs with wall-clock time limit of 2 hours. Parallel (multi-node) jobs will not be able to use this queue. If your jobs belong this category, you are encouraged to try it out. You are likely to observe substantially higher throughput.

    1. Make sure that each of your jobs will finish in 2 hours. [Jobs submitted to this queue will be unconditionally terminated after 2 hours.]
    2. To submit jobs to this queue, add -l express in addition to other parameters you typically use with the exception of highp complex. Also request for time less than 2 hours.Example Job submission scenarios:For a simple serial job requesting one hour

      qsub -l express,h_rt=1:00:00 -w n job.cmd

      For a simple array job requesting one hour per jobtask

      qsub -l express,h_rt=1:00:00 -t 1:1000 -w n jobarray.cmd

      If you don’t request the hours, by default job will run for maximum 2 hours set by the queue and will be terminated. The -w n option asks SGE not to verify whether the calendar queue is on or off when you are submitting the jobs as it is on only during certain hours, but you should be able to submit jobs at any time to this queue.

    3. To direct your existing pending jobs to this queue, use

      qalter -l express JOB_ID

where JOB_ID is the job id shown by qstat -u $USER or run myjobs

TOP

Upcoming Free Parallel Computing Classes

ATS is offering the following free 2-hour long classes during the Fall Semester. Individual registration is required for each class. To sign up for any of these classes, point your browser at http://www.idre.ucla.edu

October 27, 2-4 p.m. Using the Hoffman2 Cluster 5628 Math Science
November 1, 2-4 p.m. Introduction to Parallel Computing and OpenMP 5628 Math Science
November 3, 2-4 p.m. Introduction to MPI 5628 Math Science
November 8, 1-3 p.m. Parallel Computing Lab CLICC Classroom C (320 Powell)
November 9, 2-4 p.m. Introduction to CUDA 5628 Math Science

 

TOP

Quantum Espresso version 4.2.1 available

Quntum ESPRESSO v 4.2.1 is now available on the Hoffman2 Cluster for SGE batch execution. Quantum ESPRESSO is an integrated suite of computer codes for electronic-structure calculations and materials modeling at the nanoscale. It is based on density-functional theory, plane waves, and pseudopotentials (both norm-conserving and ultrasoft). More information can be found at http://www.quantum-espresso.org/

Please see How to Run Quantum ESPRESSO for detailed information.

The Quantum Espresso documentaion is available on-line at http://www.quantum-espresso.org/wp-content/uploads/Doc/user_guide.pdf

TOP

Home Directory 20GB Quota and Job Scratch Space

The quota on campus users’ home directories has been increased to 20GB (gigabytes). It was formerly 10GB. To see your quota usage, at the shell prompt, enter:

myquota

Job Scratch Space

Please consider running your high I/O jobs from the /u/scratch directory. There is over 7 terabytes of global scratch space which is NFS-mounted over all compute and login nodes. Because /u/scratch resides on the faster fiber-channel-attached disks, for performance reasons writing to /u/scratch is a much better idea than writing to your home directory.

To use /u/scratch make a directory there named with your userid and place your files in it. Change into your /u/scratch/[userid] directory before running job.q, mpi.q or other queue scripts so that all file paths are set correctly in your SGE command file.

Here are some guidelines:

  • All files that need to be saved at the end of the calculation, including checkpoint files, should be written to /u/scratch.
  • Temporary files that are generated during the run and deleted at the end of calculation should be written to SGE’s local $TMPDIR directory. Please see How to use SGE scratch directory for file I/O

There is a 2TB per user limit on the /u/scratch filesystem. Under normal circumstances, files stored in /u/scratch are allowed to remain there for 7 days. Any files older than 7 days may be deleted automatically by system cleanup routines.

TOP

Hoffman2 is online

We are pleased to announce that the Hoffman2 is now back online and available for use.

As we announced several weeks ago, we have upgraded, replaced and/or moved a large number of nodes as well as upgraded the cluster’s operating system and installed a new version of the Sun Grid Engine. You should not experience any problems with the system, but if you do, please let us know immediately.

TOP

OpenMPI updated

OpenMPI has been upgraded to 1.4.2 and is available in the /u/local/intel/11.1/openmpi/1.4.2 directory.

You will need to recompile your code if you build your SGE command file with mpi.q

TOP

ABAQUS upgraded

ABAQUS has been upgraded to version 6.10

ABAQUS is a suite of general-purpose, nonlinear finite element analysis (FEA) programs for stress, heat transfer, and other types of analysis. ABAQUS runs a wide range of linear and nonlinear engineering simulations.

Please see How to Run ABAQUS.

Abaqus manuals are not available online.

TOP

Maple upgraded

Maple has been upgraded to version 14.

Maple is a symbolic and numeric mathematical program. Major areas include: algebra, calculus, differential equations, linear algebra, and statistics. Integrated visualization.

Please see How to Run Maple.

Maple documentation is available from the vendor at http://www.maplesoft.com/documentation_center/

TOP

MATLAB upgraded

MATLAB has been upgraded to version 7.9b

MATLAB is a language for technical computing that combines numeric computation, advanced graphics and visualization, and a high-level programming language.

Please see How to Run Matlab.

MATLAB documentation is available from the vendor at http://www.mathworks.com/access/helpdesk/help/helpdesk.html

TOP

Tecplot 360 upgraded

Tecplot 360 has been upgraded to version 2009.2.

Tecplot 360 is a program for the visualization and animation of scientific and engineering data. It includes support for CFD-type data.

Please see How to Run Tecplot.

Tecplot documentation is available from the vendor at http://www.tecplot.com/Support/Documentation.aspx

TOP

Hoffman2 Upgrade June 22-24, 2010

Hoffman2 will be taken offline for for maintenance and hardware upgrades from 12:01am June 22 through 11:59pm June 24, 2010. We understand that this outage affects your productivity and take that very seriously. However, we have come to a point where we have done everything we can and the remaining work that needs to be performed cannot be accomplished without taking the system offline. There are four aspects to the work we will be performing during the outage.

  1. We will be replacing and upgrading a large number of nodes in our IDRE data center. This will bring all of the nodes in the IDRE DC to at least 16GB and almost a third to 32GB. We will also be increasing the number of cores in the IDRE DC by almost 400.
  2. We will be bringing a newly installed row of racks for Hoffman2 expansion online in our Math Science data center. This new row will provide up to an additional 96 permanent nodes in the MSA DC and up to another 96 temporary nodes while we await the installation of a containerized data center in the fall. Additionally, we will be installing new shared cluster contributor nodes in the MSA DC at this time.
  3. We will complete the move of our BlueArc storage system from the IDRE DC to the MSA DC. This work has been ongoing behind the scenes for some time and included moving our tape robot as well. The final steps require us to have everyone off of the system to perform the final reconfiguration.
  4. We will be upgrading the operating system on the cluster to CentOS 5.5, adding an additional login node and making some other minor hardware adjustments.

Thank you for your patience and understanding,
UCLA Academic Technology Services

TOP

IDRE Offers Summer Computational Science Courses

IDRE will host three onsite computational science courses offered by the Virtual School of Computational Science and Engineering (VSCSE). Graduate students, post-docs and professionals from academia, government and industry can gain the skills they need to leverage the power of cutting-edge computational resources at these courses, which are being offered for a $100 per-course fee. Each class is five full days and will take place on the UCLA campus. Snacks and an evening reception will be provided; participants are responsible for travel and lodging costs. Scholarships are available. Register athttp://lists.stat.ucla.edu/pipermail/computing/attachments/20100625/5cca01ba/attachment-0001.pdf

TOP

Queue scripts enhanced

The master queue script has been enhanced to better support people who are in more than one Sun Grid Engine access list. When you use a queue script like job.q to build an SGE cmd file, now it will ask you something like:

Enter the name of the resource group you want to use in this job
(campus cnsi, default cnsi)
<or quit>:

You can press Enter to accept the default.

The resource group that you choose lets the queue script provide appropriate values for default and maximum memory. It does not guarantee that your job will use those resources. For example if your job requests 1024M memory and 1 hour, it will run where those resources are available soonest.

There is a new command-line option -rg for use in non-interactive mode. For example:

job.q -rg resource_group [other queue options] myprogram-or-script

where resource_group is the name of an SGE access list.

TOP

job.q and jobarray.q scripts enhanced

The two queue scripts, job.q and jobarray.q, have been enhanced to allow you to use more memory or more cores for your serial jobs.

job.queue

When the job.q script asks you for a memory request for your job, it will show a larger maximum memory value. For campus users, the new maximum memory size is 8,192 megabytes. If you belong to a group which has purchased high memory nodes, it will be larger.

The job.q script can reserve more memory for your serial job by telling the Sun Grid Engine to reserve additional slots on a single node. If you request more than the default amount of memory (1024 megabytes for campus users), your job may wait longer before starting while SGE waits for additional slots to become available.

jobarray.queue

Array jobs are serial jobs or multi-threaded jobs that use the same executable but different input variables or input files, as in parametric studies. The jobarray.q script now supports multi-threaded shared memory. It now will ask you:

Enter the number of tasks for your job (1 ≤ n ≤ 8, default 1)
<or quit>:

If your program is not multi-threaded, this new option will not change anything in the Sun Grid Engine command file that the jobarray.q builds for you. You can accept the default value of 1 task per job.

There is a new command-line option -mt for use in non-interactive mode. For example:

jobarray.q -mt n [other queue options] myprogram-or-script

where n is an integer less than or equal to 8.

Please see Running a Batch Job

TOP

Package PLINK updated to version 1.08

The package PLINK has been updated to version 1.08. The new version is available in the /u/local/apps/plink/current directory. The older version is still available in the /u/local/apps/plink/1.06 directory.

PLINK is a free, open-source whole genome association analysis toolset, designed to perform a range of basic, large-scale analyses in a computationally efficient manner.

See How to Run PLINK.

PLINK PDF documentation is available from http://pngu.mgh.harvard.edu/~purcell/plink/pdf.shtml

TOP

Interactive sessions and nodes

Interactive nodes that you can access from the login nodes with ssh are no longer available on the Hoffman2 Cluster. If you need to run your program interactively, you can request an interactive session from the Sun Grid Engine. Please see How to Get an Interactive Session through SGE.

You do not need to request an interactive session in order to run commercial programs like Matlab or Mathematica interactively, because they already have qrsh built into their startup scripts.

You can use maximum 8 slots (cores) at the same time in SGE interactive queue sessions. This could be 8 different sessions, or one or more parallel sessions.

TOP

Power Outage February 11, 2010

To the Hoffman2 user community,

At approximately 7am on Thursday February 11, 2010 the Hoffman2 cluster suffered a power incident which affected all systems that reside in the IDRE Data Center which is located in the CNSI building. The result of this incident was all jobs running on nodes (n1-n269) were killed. Hoffman2’s IDRE component was brought back online by 8 am.

This message is to share what we have learned as to the cause of the incident. After speaking to Facilities Management it appears the power incident affected much of the south end of campus and Westwood. One of the four power feeds that supply the campus from DWP suffered a transient fault which resulted in a voltage drop. A typical explanation for such an event is a palm frond briefly shorting a power line. The reason, apparently, that UCLA would suffer from an apparent DWP problem is the our co-generation facility is unable to provide sufficient power generation to cover the campus’ needs during the peak daytime hours leaving the campus vulnerable to these types of events.

Systems that were protected by UPS were unaffected. However, the compute nodes of Hoffman2 are not powered from UPS protected circuits, unfortunately, as the cost would be prohibitive to cover them all.

For these reasons we continue to urge our users to check-point their code, especially for long running jobs. Check-pointing will allow you to re-start your job after an unexpected event like a power fault or even a node hardware failure without losing all of your work. If you need help adding check-pointing to your code, contact High Performance Computing.

If you have any other questions or concerns regarding this incident or any other aspect of the Hoffman2 system please feel free to contact me directly.

Thank you,
Scott Friedman
Manager, Research Computing Technologies

TOP

New myquota command reports on space, file use

There is a new command myquota which reports on the amount of space used, number of files saved and quota for your userid. At the shell prompt, enter:

myquota -pm

The myquota command has several options, to see them, enter:

myquota –help

We are using the BlueArc Storage Server to provide home directory and scratch directory space. Currently, there is a 10 gigabyte quota limit on your home directory, unless your research group has purchased additional storage. [quota limit raised to 20GB August 25, 2010.]

You are encouraged to run jobs from the /u/scratch directory where there is 6 terabytes of scratch space mounted over all compute nodes. To place files in /u/scratch, make a directory there named with your login id and place your files in it.

Because /u/scratch resides on the faster fiber channel attached disks, it is recommended that for performance reasons you tell your parallel jobs, especially those with high I/O requirements, write to /u/scratch instead of to your home directory.

Under normal circumstances, files stored in /u/scratch are allowed to remain there for 7 days. Any files older than 7 days may be deleted automatically by system cleanup routines.

Point your browser at Data Storage for more information about data storage.

TOP

Hoffman2 Cluster Partial Outage Feb 9-10 2010

There will be a partial outage February 9th and 10th to upgrade the Math Science Data Center facility. The outage will affect only the Math Science Data Center. It will not affect the IDRE Data Center.

All users will be able to login and submit jobs during the outage. Jobs submitted to the IDRE Data Center nodes will run as usual. This includes all regular campus group accounts.

Jobs which need to run on the Math Science Data Center nodes — those belonging to the bern, chemeng, cnsi, hongzhou, margulis, mori, rosenzwe and sfurlane groups — may not run until after the upgrade has completed. Those jobs may run before the outage starts if there is sufficient time for them to complete before the outage starts. Any jobs running on the Math Science Data Center nodes at the time the outage begins will be terminated.

Interactive qrsh sessions on Math Science Data Center nodes will not be available during the outage. However all users will still be able to start interactive qrsh sessions of maximum 1GB/core on the IDRE Data Center interactive nodes.

The purpose of this outage is to provide for future growth in the portion of the Hoffman2 cluster located in the Math Sciences Data Center. The n2xxx nodes will be taken offline in order to perform relocation and electrical work. The outage of these nodes is scheduled for Tuesday, February 9, 2010 beginning at 12AM and continues through Wednesday, February 10, 2010. We will, however, bring the affected nodes back online as soon as the work completes.

TOP

SGE Upgrade

We are in the process of installing a new version of Sun Grid Engine on the Hoffman2 Cluster. You need to logout of any session you started before noon on Monday January 11th. When you login again you will automatically get the new SGE.

Jobs which are currently running will continue to run. However you will not be able to see their status with the SGE qstat command. The myjobs script will show both your old SGE jobs, if any, and your new SGE jobs.

You will need to resubmit any jobs which are waiting to run. Newly submitted jobs will be dispatched by the new SGE as soon as the new queues are ready; you may experience some delays today while we are in transition to the new system.

Interactive scripts which use qrsh, and the qrsh command itself, also will not work until the new queues are ready.

TOP

Gaussian 09 available

Gaussian 09, Revision A.02 is now available on the the Hoffman2 Cluster. It supports parallel distributed memory execution (Linda-parallelism) and shared memory execution (SMP-parallelism). GaussView 5 is also available.

Gaussian provides state-of-the-art capabilities for electronic structure modeling. GaussView is a graphical interface available for Gaussian. With GaussView, you can import or build the molecular structures that interest you, set up, launch, monitor and control Gaussian calculations, retrieve, view and visualize the results, GaussView 5 provides comprehensive support for importing and working with structures from PDB (protein data base) files.

See How to Run Gaussian. There are new queue scripts to invoke Gaussian 09.

See also How to Run GaussView. There is new information about GaussView and how to run Gaussian from within GaussView.

Gaussian 09 and GaussView 5 documentation is available at http://www.gaussian.com/ Gaussian 09 Release Notes are available athttp://www.gaussian.com/g_tech/rel_notes.pdf

TOP

Quantum ESPRESSO v 4.1.1 available

Quntum ESPRESSO v 4.1.1 is now available on the Hoffman2 Cluster for SGE batch execution. Quantum ESPRESSO is an integrated suite of computer codes for electronic-structure calculations and materials modeling at the nanoscale. It is based on density-functional theory, plane waves, and pseudopotentials (both norm-conserving and ultrasoft).

Please see How to Run Quantum ESPRESSO for detailed information.

The Quantum ESPRESSO documentaion is available on-line at http://www.quantum-espresso.org/wp-content/uploads/Doc/user_guide.pdf

TOP

Stata 11 available

The Stata 11 is now available on the Hoffman2 Cluster for both SGE batch and interactive applications. Stata is a complete, integrated statistical package that provides a broad suite of statistical capabilities, complete data-management facilities, and publication-quality graphics.

Please see How to Run Stata for detailed information.

Stata manuals are not available online. They may be borrowed from ATS Statistical Consulting http://www.ats.ucla.edu/stat/books/#Stata

TOP

LAMMPS version 7Sep09 is now available

LAMMPS version 7Sep09 is now installed on the Hoffman2 cluster in the /u/local/apps/lammps/current directory. The software can be run as a parallel application.

LAMMPS, Large-scale Atomic/Molecular Massively Parallel Simulator, is a classical molecular dynamics simulation code designed to run efficiently on parallel computers. It was developed at Sandia National Laboratories, a US Department of Energy facility, with funding from the DOE. The developers of LAMMPS are Steve Plimpton, Paul Crozier, and Aidan Thompson who can be contacted at sjplimp, pscrozi, athomps at sandia.gov.

LAMMPS is an open-source code, distributed freely under the terms of the GNU Public License (GPL).

See How to Run LAMMPS.

LAMMPS documentation is available at http://lammps.sandia.gov/doc/Manual.html

TOP

Intel compiler upgrade

The default Intel c and fortran compilers on Hoffman2 are now version 11.1

For more information, see Languages and Compilers and the documentation links on that page.

TOP

Priority for your jobs

Note: This new job scheduling feature is available to people who are in a sponsored group which has contributed cores to the Hoffman2 Cluster. If you are not in a sponsored group, the new highp complex will not work for you; using it will cause qsub to reject your job.

Now you can tell the Sun Grid Engine to run your job on your own group’s allocated cores. This modification, which adds “urgency” to the queue definitions, is designed to ensure that your group’s allocated cores are given back to your group within 24 hours.

To use your group’s own allocated cores, specify the new SGE complex highp either in your SGE command file or as an argument to qsub. Examples:

#$ -l highp,h_data=1024M,h_rt=24:00:00

or,

qsub -l highp [other arguments]

Currently both the new highp queues and the former queues are available to your jobs. After the old queues are disabled, if you don’t specify the highp complex your job will be sent to the shared cluster nodes which have a maximum time limit of 24 hours.

How to decide whether to run your job on your group’s own nodes or the shared cluster?
    1. If your job requires more than 24 hours, you must specify the highp complex.

 

    1. The qquota command will tell you how many cores are currently in use by your group. Example of qquota output:
      resource quota  rule limit   filter
      -----------------------------------------------------------------
      queue_limits/3  slots=67/72   users @groupname queues *d_4g_b.q
      rulset1/3       slots=85/144  users @groupname hosts @msa-amd_04g
      

      In the above example, there are 5 slots available in the group’s own allocation and 59 available in the group’s shared cluster allocation — it does not mean there are any cores free. So in case your job specifies highp and requests 72 or fewer cores, the job will be released to run on your group’s allocated nodes once your priority becomes first within your group. In case you do not specify highp, your job will go to the shared queue (max. 144 cores in this example) and may run or wait for other nodes to be free before it starts.

    2. If you rebuild your SGE command files with a queue script (e.g., job.q, mpi.q) then if highp is required, the queue script will supply it. The queue scripts will ask you if you want to use your own group’s cores for jobs requesting 24 hours or less. For people who use the queue scripts in non-interactive mode, there is a new queue script command line option -u which you can use to request urgency (highp).

 

Back-filling

There is a back-filling feature in SGE which is enabled. In order for back-filling to work, you should specify your best guess of the time required for your job and not take the default of 24 hours or 14 days.

TOP

GROMACS v4.0.5 installed

GROMACS version 4.0.5 is now installed on the Hoffman2 cluster in the /u/local/apps/gromacs/4.0.5 directory. The software can be run as a serial and/or parallel application.

GROMACS is an engine to perform molecular dynamics simulations and energy minimization. GROMACS is free software. The entire GROMACS package is available under the GNU General Public License.

See How to Run GROMACS.

GROMACS documentation is available at http://www.gromacs.org/Documentation/Manual

TOP

R version 2.9.1 is now available

R has been upgraded to version 2.9.1 and the following packages have been installed: MCMCpack, Rserve, abind, ape, coda, degreenet, ergm, gee, igraph, latentnet, network, networksis, rgl, scatterplot3d, shapes, sna, and statnet.

To see all packages which are installed, at the R command prompt, issue the command

> library()

R is GNU S, a freely available language and environment for statistical computing and graphics which provides a wide variety of statistical and graphical techniques: linear and nonlinear modelling, statistical tests, time series analysis, classification, clustering, etc. Please consult the R project homepagehttp://www.r-project.org/ for further information and documentation.

See How to Run R.

TOP

MATLAB 7.7 available

MATLAB has been upgraded to version 7.7 (Release R2008b).

MATLAB is a high-level language and interactive environment that enables you to perform computationally intensive tasks faster than with traditional programming languages such as C, C++, and Fortran.

See How to Run MATLAB.

The Hoffman2 Cluster has licenses for some MATLAB toolboxes. Please see Matlab Toolboxes Available on the Hoffman2 Cluster for a list of available toolboxes.

More MATLAB licenses and more toolboxes are available from the San Diego Super Computer OnDemand service. This is available to you through the UCLA Grid. It is also accessible with the new gapp command at the Hoffman2 Cluster shell prompt.

MATLAB documentation is available on the web at: http://www.mathworks.com/access/helpdesk/help/helpdesk.html

TOP

Power Outage Saturday Sept. 12th

All power was lost to the IDRE and engineering buildings Saturday, September 12th, at about 10:30 AM. All nodes in the IDRE Data Center that were on utility power immediately went down. Power was restored 3 hours later, and by 2:30 PM all nodes and filesystems were back online.

TOP

Hoffman2 Cluster major queue reconfiguration

All users must rebuild their current SGE command files using the appropriate queue scripts (for example: mpi.q, job.q, gaussian.q). This is a major reconfiguration of the queue structure due to increased compute resources.

The most important reasons are:

  1. Campus group users can only request 1GB per core. If your serial job needs more than 1GB, you must add the following SGE active comment to your SGE command file:

    #$ -pe shared 2

  2. There are new parallel environments. We are deprecating some of the old parallel environments and most of the current queues.
  3. We have a new interactive queue with a forced complex to make more interactive sessions available to you. In order to get this queue, campus group users must say:

    qrsh -l interactive
    or
    qrsh -l i

    Sponsored group users do not need to use “-l i” on their qrsh commands because it will use their sponsored group queues which do not have the forced complex.

  4. New nodes are available for CNSI group and some sponsored group users. If you are eligible to use the new nodes, you and your sponsor will receive more information.

We also have enabled qsub checking which will tell you immediately when you submit your job if it will never run, and not accept your job. This may happen because the resources it requests no longer exist. If you do not rebuild your SGE command file, you can expect to see this message:

Unable to run job: error: no suitable queues.

TOP

OpenMPI update

OpenMPI has been upgraded to 1.3.3 and is available in the /u/local/mpi/openmpi/current directory.

The Open MPI Project is an open source MPI-2 implementation that is developed and maintained by a consortium of academic, research, and industry partners. Open MPI is therefore able to combine the expertise, technologies, and resources from all across the High Performance Computing community in order to build the best MPI library available. Open MPI offers advantages for system and software vendors, application developers and computer science researchers.

For further details please refer to: http://www.open-mpi.org/

As per release documentation, users should not be required to recompile their codes.

TOP

PLINK installation

PLINK version 1.06 is now installed on the Hoffman2 cluster in the /u/local/apps/plink/1.06 directory.

PLINK is a free, open-source whole genome association analysis toolset, designed to perform a range of basic, large-scale analyses in a computationally efficient manner.

See How to Run PLINK.

PLINK PDF documentation is available from http://pngu.mgh.harvard.edu/~purcell/plink/pdf.shtml

TOP

Free Abaqus Student Edition download

Abaqus is finite analysis modeling, visualization, and process automation software. Abaqus SE (Student Edition) is available to academic students, professors and researchers from the vendor as a free download to your Windows XP Professional SP2, Windows XP Home Edition SP2, or Windows Vista personal computer.

You must register with the vendor, Dassault Systems, in order to be eligible for the free download (a “DS passport”).

free download at http://campus.3ds.com/simulia/freese

Abaqus SE includes the core Abaqus products: Abaqus/Standard, Abaqus/Explicit, and Abaqus/CAE. The maximum model size is set to 1000 nodes for both analysis and postprocessing. Access to features requiring compilers (user subroutines, Abaqus Make, C++ ODB API), parallel execution, or add-on products has not been included. Replay and journal files are not available for Abaqus/CAE.

More information at http://www.simulia.com/academics/student.html

TOP

Hoffman2 System Upgrade

During last couple of days the cluster went through a major software as well as key hardware upgrade. The login systems and SGE server have been upgraded by brand new and powerful servers. The modifications were also done in order to allow this cluster to expend beyond the physical limits of current IDRE data center. The cluster is now fully operational and back in service.

Although all of the abovementioned upgrades are transparent to user, yet software stacks like MPI, compiler and OS/run time libraries are new. Therefore you are advised to recompile user codes in order to avoid any unforeseen problems during run time. The codes provided from system side are already been recompiled with the new software stack.

TOP

Hoffman2 Cluster Outage May 27 2009

The Hoffman2 cluster will be down for system and storage upgrades from 8:00AM – 8:00PM on Wednesday, May 27, 2009. You will be unable to login or access the cluster during this period. The cluster will be upgraded to Centos 5.3 with many additional application updates, a faster Grid Engine scheduler, and more.

The maximum time allotted to cluster queues will decrease daily as the outage date nears to prevent unexpected job termination. All jobs still running under the Sun Grid Engine, for example jobs you submitted through UCLA Grid or with a command-line queue script, will be terminated when the cluster is taken down. You will need to resubmit your jobs when the cluster comes back.

TOP

Spring 2009 Parallel Computing Classes

We are pleased to announce the IDRE high performance computing workshops for this quarter. There are three classes:

  • Introduction to Parallel ComputingMonday, Apr 27, 10am-noon
    5628 Math Sciences (Visualization Portal)
  • Parallel Programming using MPIMonday, May 4, 10-noon
    5628 Math Sciences (Visualization Portal)
  • Parallel Computing LabMonday, May 11, 1-3pm
    320B Powell (CLICC Classroom B)

To sign up, use: http://idre.ucla.edu

Description of class content is available at

TOP

GAMESS application Update

The current GAMESS version on Hoffman2 Cluster is “12 JAN 2009 (R1)” and is available in the /u/local/apps/gamess/current directory.

GAMESS is a program for ab initio molecular quantum chemistry. A wide range of quantum chemical computations are possible using GAMESS.

See How to Run GAMESS.

GAMESS Documentation is available from http://www.msg.chem.iastate.edu/gamess/documentation.html

TOP

CPMD Application Update

The current CPMD version on Hoffman2 Cluster is 3.13.2 and is available in the /u/local/apps/cpmd/current/parallel/ib directory.

CPMD (Car-Parrinello Molecular Dynamics) is an ab initio Electronic Structure and Molecular Dynamics Program. The CPMD code is a parallelized plane wave/pseudopotential implementation of Density Functional Theory, particularly designed for ab-initio Molecular Dynamics simulation.

See How to Run CPMD.

Point your browser at http://www.cpmd.org/ for further information about CPMD and the CPMD consortium.

TOP

FFTW upgrade

The FFTW library has been upgraded to version 3.2. It is installed on the Hoffman2 cluster under the following directory:

/u/local/apps/fftw3/current

FFTW is a C subroutine library for computing the Discrete Fourier Transform (DFT) in one or more dimensions, of both real and complex data of arbitrary size. There are three versions of the fftw3 library depending on precision.

See How to Use the FFTW Library. For FFTW documentation, see http://www.fftw.org/fftw3_doc/

TOP

HDF5 upgrade

HDF5 has been upgraded to version 1.8.2. HDF5 (Hierarchical Data Format 5) Software Library and Utilities are a suite of data model, file format, API, library, and tools that makes possible the management of extremely large and complex data collections. HDF5 is installed at:

/u/local/apps/hdf5/current

See How to Use the HDF and HDF5 Libraries. For more information about HDF5, see http://www.hdfgroup.org/HDF5

TOP

Trilinos upgrade

Trilinos has been upgraded to version 9.0.1 It is installed on the Hoffman2 cluster under the following directory:

/u/local/apps/trilinos/current

Trilinos is a set of sophisticated software tools, containing more than 20 library packages. The Trilinos Project is an effort to develop algorithms and enabling technologies within an object-oriented software framework for the solution of large-scale, complex multi-physics engineering and scientific problems.

For more information about Trilinos, see http://trilinos.sandia.gov/ See How to Use Trilinos

TOP

Paraview upgrade

Paraview has been upgraded to version 3.4.0. See How to Run ParaView.

ParaView is an open-source, multi-platform data analysis and visualization application. ParaView was developed to analyze extremely large datasets using distributed memory computing resources. See http://www.paraview.org for more information.

TOP

ATLAS upgrade

ATLAS has been upgraded to version 3.8.2. The ATLAS library is installed in: /u/local/apps/atlas/current

See How to Use the ATLAS Library

ATLAS (Automatically Tuned Linear Algebra Software) provides C and Fortran77 interfaces to a portably efficient BLAS implementation, as well as a few routines from LAPACK. When the ATLAS library is built and installed, it automatically optomizes its performance for whatever system it was built on. In this case, these routines are optimized for the cluster compute nodes.

TOP

Intel compiler upgrade

The default Intel compiler on Hoffman2 is now version 11.0. For more information, see Compilers and the Documentation links on that page.

TOP

October Downtime for Hoffman2 Cluster

The Hoffman2 cluster will be down for an upgrde of the BlueArc file server and maintenance on Thursday, Oct 9, 2008 between 10:00 AM and 3:00 PM. All queues will be turned off 24 hours in advance. Any running jobs at that time will be terminated. We are sorry for any inconvenience it may cause you.

TOP

Fall 2008 Parallel Computing Classes

The Institute for Digital Research and Education (IDRE) is offering the following (free) high performance computing workshops this quarter:

  • Sep. 30, 2-4pm: Introduction to parallel computing
  • Oct. 7, 2-4pm: Parallel computing using MPI
  • Oct. 14, 2-4pm: Parallel computing hands-on session
  • Oct. 21, 2-4pm: Advanced topics of MPI (*)
  • Oct. 28, 2-4pm: HPC tools and libraries (*)

(*)’s are new ones introduced this quarter.

For more information or to sign up, please see
http://www.idre.ucla.edu/ (under “Upcoming Events”)

TOP

ABAQUS upgrade

ABAQUS has been upgraded to version 6.8. ABAQUS is finite element modeling, visualization, and process automation software. For a description of ABAQUS, download the Abaqus 6.8 brochure pdf file from http://www.simulia.com/products/abaqus_cae.html

The Hoffman2 Cluster has licenses for the following ABAQUS features: abaqus aqua cae design euler_lagrange explicit foundation parallel standard

See How to Run ABAQUS. ABAQUS documentation is not available on the web. To view ABAQUS documentation, at the shell prompt, enter:

abaqus doc

TOP

Tecplot upgrade

Tecplot 360 has been upgraded to version 2008 release 2. Tecplot 360 is numerical simulation and computational fluid dynamics (CFD) visualization software that combines engineering plotting with advanced data visualization. It has a graphical user interface. It is not available in batch mode. It is available from UCLA Grid interactive tab.

To use Teplot 360, at the shell prompt, enter:

tec360

See How to Run Tecplot. For more information, see www.tecplot.com/support/360/docs.aspx

TOP

PETSc available

ATS is pleased to announce that PETSc version 2.3.3-p13 has been installed on the Hoffman2 cluster under the following directory:

/u/local/apps/petsc/current

PETSc is a suite of data structures and routines for the scalable, parallel solution of scientific applications modeled by partial differential equations. It employs the MPI standard for all message-passing communication.

See How to Run PETSc. For more information see http://www.mcs.anl.gov/petsc/

TOP

Trilinos available

ATS is pleased to announce that Trilinos version 8.0.8 has been installed on the Hoffman2 cluster under the following directory:

/u/local/apps/trilinos/current

Trilinos is a set of sophisticated software tools, containing more than 20 library packages. The Trilinos Project is an effort to develop algorithms and enabling technologies within an object-oriented software framework for the solution of large-scale, complex multi-physics engineering and scientific problems.

See How to Run Trilinos. For more information, see trilinos.sandia.gov

TOP

HDF5 upgrade

HDF5 has been upgraded to version 1.8.1. HDF5 (Hierarchical Data Format 5) Software Library and Utilities are a suite of data model, file format, API, library, and tools that makes possible the management of extremely large and complex data collections.

HDF5 is installed at:

/u/local/apps/hdf5/current

See How to Use the HDF and HDF5 Libraries. For more information about HDF5, see https://www.hdfgroup.org/HDF5/

TOP

NetCDF upgrade

NetCDF has been upgraded to version 4.0. NetCDF (network Common Data Form) is a set of software libraries and machine-independent data formats that support the creation, access, and sharing of array-oriented scientific data.

NetCDF is installed at:

/u/local/apps/netcdf/current

See How to Use the netCDF Library. For more information about NetCDF, see http://www.unidata.ucar.edu/software/netcdf/

TOP

Application Cluster

Over the past few weeks, you may have noticed an increase in the number of slots available in the serial queue. This increase was temporary, while the Infiniband cards were on backorder for the 114 nodes being added to the cluster.

Adding these nodes to the serial queue, allowed Hoffman2 users to get some use out of them and allowed us to do testing and ensure the nodes were running properly prior to adding the Infiniband cards.

These nodes will be shutdown on Tuesday, September 2, 2008 to have their Infiniband cards installed and will be added to the parallel queue that afternoon.

TOP

Hoffman2 Cluster Shutdown and Upgrade August 20, 2008

The Hoffman2 Cluster will be shut down for a short time between 7:00AM-7:00PM on Wednesday, August 20th for a major hardware upgrade to the BlueArc file server.

All campus queues will be disabled on August 18th. Shared-cluster queues will only accept requests that will be finished before August 20th. All jobs running on the cluster will be terminated August 20th.

TOP

Hoffman 2 Cluster Shutdown and Upgrade July 9-11, 2008

The Hoffman2 Cluster will be shut down between Wednesday, July 9 and Friday, July 11 to have additional electrical power installed. The BlueArc file server will also be reconfigured during this period.

This outage is extended to 11AM Friday, July 11th due to unforeseen problems encounted by UCLA Facilities during the power upgrade.

The power upgrade will allow us to more than double the size of the Hoffman2 cluster. We will be adding 115 nodes or 920 cores to the cluster. These nodes were purchased by 5 separate research groups for inclusion in the Shared Cluster portion of Hoffman 2.

TOP

Stata10 available

The multi-processor version of Stata10 is now available on the Hoffman2 Cluster for both SGE batch and interactive applications. Stata10 and Xstata are available as serial applications through the UCLA Grid Portal

Stata is a complete, integrated statistical package that provides a broad suite of statistical capabilities, complete data-management facilities, and publication-quality graphics.

Please see How to Run Stata for detailed information.

Stata manuals are not available online. They may be borrowed from ATS Statistical Consulting.

TOP

GNU Scientific Library upgraded to version 1.11

The GNU Scientific Library has been upgraded to version 1.11 and is now the default version of GSL on the Hoffman2 Cluster. Point your browser at: GNU Scientific Library for information on how to run GSL on the Hoffman2 Cluster.

The old version 1.9 will remain available for a brief period.

TOP

OpenMPI 1.2.5 becomes the default MPI library

OpenMPI version 1.2.5 is now the default MPI library on the Hoffman2 Cluster, installed at: /u/local/mpi/openmpi/current

Like before, the default MPI compilers (mpiCC, mpic++, mpicc, mpicxx, mpif77, mpif90) are based on the corresponding Intel compilers (icc, icpc, ifort). The mpi.q script will use OpenMPI. The mvapich2.q script will access the former default.

TOP

Hoffman2 Cluster Outage – Monday Apr 14, 2008

Hoffman2 Cluster will be down for several hours on Monday April 14th. You will be unable to login or access the cluster during this period. This outage is to upgrade the firmware on the BlueArc file server.

All jobs running under the Sun Grid Engine, for example jobs you submitted through UCLA Grid, or with a command-line queue script, will be terminated at that time. You will need to resubmit your jobs when the cluster comes back.

TOP

Gnuplot 4.2.3 available

A new version of gnuplot 4.2.3 is now available. For information on how to run gnuplot, see How to Run gnuplot.

Version 4.2 contains a ton of new features, support for several new output devices, and improved performance when plotting large data sets.

Major additions include:

  • Text strings can be read and manipulated as normal data
  • New interactive terminal based on wxWidgets, pango and cairo
  • New 2D plot styles: histogram labels image rgbimage
  • New 3D plot styles: vectors labels image rgbimage
  • User control over color definitions and color use in plots
  • Improved font handling and text formatting
  • New syntax to handle string variables and string functions
  • Creation of animated gif sequences
  • Support for UTF-8 and other multi-byte font encodings
  • Japanese language documentation and internal help

Demo plots illustrating these and other features are online at:

http://gnuplot.sourceforge.net/demo_4.2/

(description from: http://www.gnuplot.info)

TOP

Hoffman2 Cluster Now Available

The new Hoffman2 Cluster is now installed and available for use.

TOP

Report Typos and Errors
UCLA OIT

© 2016 UC REGENTS TERMS OF USE & PRIVACY POLICY