Recommendations for HPC Environment Design¶

SLURM Setup (From Controller VM)¶

The instructions below provide guidelines for installing the SLURM job-scheduler on your HPC cluster, and may be modified as required for alternative job-schedulers such as SGE, LSF, PBS or Torque/Maui.

Create a group for the slurm VM (add at least slurm1 as a node in the group, set additional groups of services,cluster,domain allows for more diverse group management):
```
metal configure group slurm
```
Customise slurm1 node configuration (set the primary IP address to 10.10.0.6):
```
metal configure node slurm1
```
Create /var/lib/metalware/repo/config/slurm1.yaml with the following network and server definition:
```
slurm:
  is_server: true
```

Add the following to /var/lib/metalware/repo/config/domain.yaml (set server to the hostname of the SLURM VM):

slurm:
  server: slurm1
  is_server: false
  mungekey: ff9a5f673699ba8928bbe009fb3fe3dead3c860c

Additionally, add the following to the setup: namespace list in /var/lib/metalware/repo/config/domain.yaml:
```
- /opt/alces/install/scripts/06-slurm.sh
```

Download the slurm.sh script to the above location:

mkdir -p /opt/alces/install/scripts/
cd /opt/alces/install/scripts/
wget -O 06-slurm.sh https://raw.githubusercontent.com/alces-software/knowledgebase/master/epel/7/slurm/slurm.sh

Build SLURM RPMs in custom repo (/opt/alces/repo/custom/Packages), a guide to building the SLURM RPMs can be found in the SLURM documentation. Once the packages have been moved to the previously mentioned custom repo directory, rebuild the repo with createrepo custom
Follow Client Deployment Example to setup the SLURM node

Note

All systems that are built will have SLURM installed and the SLURM daemon running which will allow that node to submit and run jobs. Should this not be desired then the service can be permanently stopped and disabled with systemctl disable slurmd && systemctl stop slurmd on the node which is no longer to run SLURM.

Once your job-scheduler is installed and running, follow the documentation specific to your chosen package for instructions on how to configure the software for your particular requirements.

Modules Setup (From Deployment VM)¶

The environment modules software allows users to control their environment variables for a particular login session. This enables flexibility to use a library of installed application software whilst maintaining the correct dependencies for each package. Modules can also control environment variables that deliver user assistance (e.g. manual pages, help-files, usage examples), curate software package usage terms and manage license availability for commercial packages.

The instructions below provide a simple modules installation for your HPC environment:

Create a group for the modules VM (add at least apps1 as a node in the group, set additional groups of services,cluster,domain allows for more diverse group management):
```
metal configure group apps
```
Customise apps1 node configuration (set the primary IP address to 10.10.0.7):
```
metal configure node apps1
```
Create /var/lib/metalware/repo/config/apps1.yaml with the following network and server definition:
```
modules:
  is_server: true
```
Add the following to /var/lib/metalware/repo/config/domain.yaml (set server to the IP of the apps VM):
```
modules:
  server: 10.10.0.7
  directory: /opt/apps
  is_server: false
```
Additionally, add the following to the setup: namespace list in /var/lib/metalware/repo/config/domain.yaml:
```
- /opt/alces/install/scripts/07-modules.sh
```

Download the modules.sh script to the above location:

mkdir -p /opt/alces/install/scripts/
cd /opt/alces/install/scripts/
wget -O 07-modules.sh https://raw.githubusercontent.com/alces-software/knowledgebase/master/epel/7/modules/modules.sh

Follow Client Deployment Example to setup the apps node

Note

The apps directory can be setup on the storage node if one was created, this allows for all NFS exports to come from a centralised server.