***************** Unix command line ***************** As with any platforms running a unix-like operating system, one of the most powerful ways to interact with the Hoffman2 Cluster is via a :term:`unix shell` (or simply :term:`shell`) which is a program that interprets users' inputs entered at the :term:`command line` of a :term:`terminal emulator` (or :term:`terminal` for short). .. _unix-terminal-open-on-H2: .. figure:: ../../_static/unix-terminal-open-on-H2.png :width: 800 :alt: Image showing a unix :term:`terminal` open on the Hoffman2 Cluster Unix :term:`terminal` application showing user ``joebruin`` connected to the Hoffman2 Cluster. The cursor at the end of the ``[joebruin@login3 ~]$`` line indicates that the :term:`unix shell`, or :term:`command line interpreter`, is ready to accept user input. The default :term:`prompt` on a :term:`terminal` connected to the Hoffman2 Cluster is typically comprised of a field containing information on the user running the :term:`unix shell`, the :term:`node` on which the unix shell is running and the location within the filesystem from which the :term:`shell` is running. This field is followed by a ``$`` which we refer to as the :term:`shell prompt` and which indicates that the :term:`shell` is ready to accept user input. For example, upon logging onto the cluster user ``joebruin`` would see the following :term:`prompt`: .. code-block:: bash [joebruin@login3 ~]$ You can access the Hoffman2 Cluster :term:`command line` by logging into the system either via a terminal application running on your local computer (see: :ref:`Connecting via terminal and SSH`), or by opening a :term:`terminal` into a Hoffman2 remote desktop (see: :ref:`Connecting via remote desktop`) or a Jupyter Notebook/Lab (see: :ref:`Connecting via Jupyter Notebook/Lab`). This page presents a review of common tasks and their associated unix commands that you may need to execute or script in your shell. Unix environment & environmental variables ========================================== On a unix-like system strings preceded by a ``$`` symbol are variables, which means that the operating system will interpret the string according to what it has been set to. Environmental variables determine the behavior of the command-line or shell interpreter in use (typically `Bash `_) and they constitute the *unix environment* for the user. When new users first log into a unix-like system they inherit the default environment that the OS or the system administrators have set for them. Users can alter their unix environment by setting, or altering, the value of environmental variables. This can be done on the fly at the shell prompt or by setting their value permanently in the shell initialization files (read upon logging in or sourced at the command line). Examples of environmental variables that are set for you are: .. code-block:: console $HOME $SCRATCH $PATH $LD_LIBRARY_PATH To check the content of a variable, for example ``$HOME``, issue: .. code-block:: console $ echo $HOME The content of your ``$PATH`` and ``$LD_LIBRARAY_PATH`` influences the kind of executables and libraries that you can access at the command line (without specifying their full path). Navigation ========== Listing files ------------- To list what files are present in the current directory, type at the Hoffman2 command prompt: .. code-block:: console $ ls To list hidden files or folders, type: .. code-block:: console $ ls -a To get extra information on the files or folders in the current working directory, use: .. code-block:: console $ ls -l A typical line of output from ``ls -l`` could look, for example, like: .. code-block:: console -rwxr-xr-- 1 joebruin gobruins 3974052 Apr 17 12:53 myexecutable Adding ``-l`` to the ``ls`` command produces a long listing in which :ref:`file permissions ` (e.g., ``-rwxr-xr--``), number of links (e.g., ``1``), owner name (e.g., ``joebruin``), owner group (e.g., ``gobruins``), file size in bytes (e.g., ``3974052``), time of last modification (e.g., ``Apr 17 12:53``) and file or directory name (e.g., myexecutable) are shown. To display the size of the files in units of bytes, MB or GB: .. code-block:: console $ ls -lah The result will look something like: .. code-block:: console -rwxr-xr-- 1 joebruin gobruins 3.8M Apr 17 12:53 myexecutable To order the list of files by time (newest on top): .. code-block:: console $ ls -laht To order the list of files by time oldest on top): .. code-block:: console $ ls -latr Files and directories management ================================ Copying, moving, removing files ------------------------------- To copy a file from a location to another, issue the command: .. code-block:: console $ cp myfile.txt myfile1.txt To move a file from a location to another, type: .. code-block:: console $ mv myfile.txt myfile1.txt To remove a file, issue: .. code-block:: console $ rm myfile.txt Changing directory ------------------ Upon logging on to the Hoffman2 cluster, you land in your :ref:` ``$HOME`` directory `. To navigate to your temporary storage directory issue: .. code-block:: console $ cd $SCRATCH To check in which directory you currently are, issue: .. code-block:: console $ pwd user ``joebruin`` would get back a result that looks like: .. code-block:: console /u/scratch/j/joebruin To move up two directories, type: .. code-block:: console $ cd ../.. To move back to the ``$HOME`` directory, type: .. code-block:: console $ cd ~ or: .. code-block:: console $ cd $HOME Creating/removing directories ----------------------------- To create the directory ``test`` in your ``$HOME`` directory, issue: .. code-block:: console $ mkdir $HOME/test To navigate to this newly created directory, issue: .. code-block:: console $ cd $HOME/test to check the location of this new directory, issue: .. code-block:: console $ pwd user ``joebruin`` would get back a result like: .. code-block:: console /u/home/j/joebruin/test To remove an empty directory, issue: .. code-block:: console $ rmdir test To force removal of a file or a directory, type: .. code-block:: console $ rm -rf test Unix file permissions --------------------- When listing the files in a directory with the long listing format, i.e.: .. code-block:: console $ ls -la we get several information on the file(s) present in the current directory. Of these the first field o the output corresponds to the file permissions, or access rights, of specific users and groups, which tell us who owns the file and who can see it, modify it, and, if the file is executable, who can execute it or, if the file is a directory, who can access it. Permissions on a file or directory can also be determined with the ``stat`` command, for example: .. code-block:: console $ stat -c "%A" $HOME returns the access rights on the ``$HOME`` directory which typically is: .. code-block:: console drwx------ The file permissions are organized in a string of 10 characters of which the first indicates the file type (``-``, for a regular file, ``d`` for a directory, ``l``, for a link, etc.). The remaining 9 characters are subdivided in three groups, corresponding to the permissions on the file relative to the user who owns the file, the owner's unix group, and other users on the system (not part of the same unix group). Each of these three groups is then composed of three characters of which the first is relative to read permission (``r`` if enabled ``-`` if not), the second to the write permission (``w`` if enabled ``-`` if not) and the third to the execution/access permission (``x`` if enabled ``-`` if not). .. code-block:: console r w x | r w x | r w x user | group | others The execute/access character of each of the three permission fields cab also assume other values. A value ``s`` in the user execute/access field indicates that the file can be executed with the same privileges of the user who owns the file; a value ``s`` in the group execute/access field indicates that, if the file is a directory, any file/directory created therein will inherit the group of the parent directory despite the primary group of the user who created it (this feature is useful in cases in which a users is part of more then one unix group). Finally, the value ``T`` in the execute/access other field (also known as the sticky bit)indicates that despite the group write permissions on a file/directory only the user who has created the file/directory (and therefore owns it) can remove the file/directory (other users in the same group could however be able to modify it if the group has write permissions). .. Note:: By default on the Hoffman2 Cluster, users' directories are accessible only by the owner (that is the ``$HOME`` directory has ``drwx------`` permissions) therefore, no file or folder within them, despite their permissions, can be accessed by any other user. Working with files ================== Displaying files ---------------- Unix-like systems have more than one way to display the content of a file, or selected snippet of it, in your unix shell. Among some of the most useful commands are: ``cat``, ``less``, ``more``, ``head``, and ``tail``. To see how they work, issue each of these separately at Hoffman2 command prompt: .. code-block:: console $ cat $HOME/.bashrc $ less $HOME/.bashrc $ more $HOME/.bashrc To exit from displaying a file with ``more`` or ``less``, type: ``q``. To see just the first few lines of a file, type: .. code-block:: console $ head $HOME/.bashrc or, for the first two lines, type: .. code-block:: console $ head -n 2 $HOME/.bashrc To see the last few lines of a file, issue: .. code-block:: console $ tail $HOME/.bashrc or, for the last two lines: .. code-block:: console $ tail -n 2 $HOME/.bashrc To look on the fly at the last few lines of a file that is being written, issue: .. code-block:: console $ tail -f /u/local/licenses/LOGS/logit.matlab To exit, use: ``Control-C``. Editing files ------------- Unix-like systems have very powerful :ref:`editors ` packed with shortcuts that largely simplify the editing task. The learning curve on some of these editors, such as ``vi``, can be pretty steep. However, there are several simpler, if less powerful, alternatives. Text-based editors ^^^^^^^^^^^^^^^^^^ To start ``nano``, ``emacs``, or ``vi`` editors (listed in order of increasing complexity) from a terminal issue: .. code-block:: console $ nano $ emacs $ vi Graphical editors ^^^^^^^^^^^^^^^^^ Graphical editors open a GUI window (see: :ref:`Opening GUI applications`). To open the GUI window of the ``gedit``, ``emacs``, or ``gvim`` from a terminal issue: .. code-block:: console $ gedit & $ emacs & $ gvim & Miscellaneous commands ====================== Most unix commands have manual pages that can be accessed via the ``man`` command. For example, to learn more about the ``ls`` command, issue: .. code-block:: console $ man ls or: .. code-block:: console $ man vi This is only a superficial covering of the topic. Additional and in depth information can be found on internet. Problems with the instructions on this section? Please send comments `here `__.