A Permanent Head Damage Student in Colorful Fluid Dynamics

An Enhanced Bash Shell Script for OpenFOAM Paralell Job Submission on TACC Lonestar 5

Bash

Bash is a Unix shell and command language.


Bash shell script

The bash shell commands below generates a SLURM job script which requests a parallel job with 110 threads spread over 5 node and 48 hours of run time in the normal queue on TACC’s Lonestar 5 system:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
#!/bin/bash
#-------------------------------------------------------------------------
# SLURM script generator for running OpenFOAM cases on TACC's Lonestar 5
# system.
#
# Note:
# Please put this file in the root directory of the case.
# Author:
# Dezhi Dai, MAE Department, UT Arlington.
#-------------------------------------------------------------------------

# Set the email, replace this by your own email address
email="dezhi.dai@mavs.uta.edu"

# Obtain current case directory and rootcase name
case_dir=`pwd`
case_name=`basename "$case_dir"`

# Set the file name of the SLURM job script
s_name="auto_sbatch_job_tacc"

# Set the flow solver
flow_solver="overInterPlicPhaseChangeDyMFoam"

# Set the No. of nodes requested and mpi tasks
n_nodes="5"
n_mpis="110"

# Write options and commands to the SLURM script
# Header
echo "#!/bin/bash" > $s_name

# SLURM options
echo "#SBATCH -J $case_name" >> $s_name
echo "#SBATCH -o $case_name.out" >> $s_name
echo "#SBATCH -e $case_name.err" >> $s_name
echo "#SBATCH -p normal" >> $s_name
echo "#SBATCH -N $n_nodes" >> $s_name
echo "#SBATCH -n $n_mpis" >> $s_name
echo "#SBATCH -t 48:00:00" >> $s_name
echo "#SBATCH -A Multiphase" >> $s_name
echo "#SBATCH --mail-user=$email" >> $s_name
echo "#SBATCH --mail-type=all" >> $s_name

# OpenFOAM commands
echo "decomposePar -case $case_dir" >> $s_name
echo "ibrun -np $n_mpis $flow_solver -parallel -case $case_dir" >> $s_name
echo "reconstructPar -case $case_dir" >> $s_name
echo "find $case_dir -name 'processor*' | xargs rm -rf" >> $s_name
echo "touch $case_dir/$case_name.foam" >> $s_name

Save the Bash shell script as slurm_script_generator_tacc_openfoam and run the following commands in the terminal:

1
2
bash slurm_script_generator_tacc_openfoam 
sbatch auto_sbatch_job_tacc

This post is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International (CC BY-NC-SA 4.0) License.

  1. 1. Bash
  2. 2. Bash shell script