You are here

hpc.Time Supercomputing

High-Performance Computing (HPC) is the use of large-scale, off-site computers and parallel processing techniques for solving complex computational problems. HPC technology focuses on using parallel processing algorithms and systems. HPC is typically used for solving advanced problems and performing research activities through computer modelling, simulation and analysis. The terms High-Performance Computing and Supercomputing are used interchangeably.


Uses of HPC

HPC systems are most often useful for researchers:

  • with parallelisable tasks (parallelism can be achieved on a fine or coarse grain level — on a fine level you have a program which is intrinsically parallel while on a coarse grain level you might run many instances of the same program for parameter studies)
  • to take advantage of fine/coarse grain parallelism, embarrassingly parallel and parameter studies techniques
  • with computing jobs needing more memory (RAM) than available on their local systems (if you run out of memory on your desktop system or can no longer process your data quickly enough)
  • needing access to software packages offered by the HPC facilities.
     

HPC jobs run in a non-interactive batch mode. These jobs  are submitted to a queue, and when the time comes for them to be executed they can run without any user intervention. This means that HPC systems rarely sit idle but utilise the hardware fully. It has the benefit that you can queue your jobs and get on with other things while you wait for them to execute. If you are accustomed to using your program interactively, you will have to learn how to use its batch mode in order to use it for HPC.


HPC Training

Intersect offers a series of HPC courses to meet the needs of all researchers, whether you are at a Beginner, Intermediate or Advanced HPC level. Check our Training page for more details. 

  • Intensive Beginner to Intermediate HPC - Introduction to UNIX for HPC
  • Intermediate HPC - From Unix to HPC
  • Advanced HPC - Parallel Programming 
     

Supercomputing Facilities for Intersect researchers

As a researcher you have access to a number of facilities:

  • The National Facility at NCI (National Computing Infrastructure), raijin.NCI.org.au, which serves researchers nationally
  • The NeCTAR and OwnTime Research Clouds, over 4,500 local and 32,000 distributed computing cores 
  • Facilities local to your institution or faculty

 

For details about the last option, please contact your institution.


The NeCTAR Research Cloud, rc.NeCTAR.org.au is powered by OpenStack.org, a cloud orchestration system that controls large pools of compute, storage, and networking resources throughout multiple Australian data centres, all managed through a dashboard that gives administrators control while empowering consumers with web self-serve resource provisioning. Pre-baked virtual machine images are catalogued at: https://support.rc.nectar.org.au/docs/images


Getting Started

Apply for a new project using Mancini on the NCI webpage. Intersect is an NCI partner. The compute and storage resources you ask for are supplied by Intersect.
Outside of the normal resource allocation round (RAR) you can ask for up to 20k SUs per quarter. Your project can get additional resources during the mid-term adjustment in the middle of each calendar quarter. The research argument can be brief.

 


How to use HPC

Connecting to Raijin

To connect to Raijin you'll need a Secure Shell (SSH) client. If you're on Linux or Mac OS X, chances are that you can just open a terminal and issue one of the following command:

ssh raijin@nci..org.au

When prompted, enter your username and password. If you're running Windows, you'll need to download an SSH client. A good free option is Putty.

To transfer files to and from your machine, you'll need a File Transfer Protocol (FTP) client that supports Secure-FTP (SFTP). You can do this from the command line on Linux, Mac OS X, and Windows:

sftp raijin.nci.org.au

But you might find it easier to use a graphical FTP program, such as FileZilla.


HPC Software Registries

A complete lists of all software installed are available at https://opus.nci.org.au/display/Help/Application+Software

Please note that software packages marked with a yellow dot carry license restrictions. Please contact us to establish whether you can use this software. 

Setting up software environments

To set up the environment for a software package you need to use the module system. This is identical on both machines but the module names might differ. To see the exact names of the modules visit the links given above or to get a list use the command:

module avail

To load, for example, the now latest version of the Intel compilers on Raijin use 

module load intel-fc/17.0.1.132

This sets up your environment (variables and path). The reason for using this module system is to allow for different versions of the same software package.

Other useful module commands include:

A list of already loaded modules: module list

To show you what a module does: module show package-name 

To unload a package module unload <package> ()

Software installation and requests

If you would like to have new software installed on Raijin, please email to time@intersect.org.au and specify the download site. You are also welcome to install software yourself in your home directory.