Running STAR-CCM+ in Client-Server Mode

This tutorial shows how to run STAR-CCM+ on a remote HPC cluster in server mode and control it from a local GUI. The GUI connects to the compute nodes throgh an SSH tunnel.

1. Prerequisites

This instruction uses the LCRC Improv cluster with PBS for job scheduling. The local workstation has STAR-CCM+ installed. MobaXterm is used to create the SSH tunnel for the client-server connection.

Before you begin, confirm the following:

• A valid STAR-CCM+ licence.
• Same STAR-CCM+ version (including patch level) on both local and HPC sides.
• SSH access to the cluster (and to a bastion/jump host if required).
• Ability to submit and monitor jobs in the HPC login nodes.
• Do not run compute- or memory-intensive jobs on the login nodes.

The software and supporting utilities used in this workflow are listed below:

Software/utility	Version	Localtion
STAR-CCM+	18.04.009-R8	Remote / Local
PBS	2022.1.7.20241016134739	Remote
MobaXterm	v24.2	Local

2. Procedures

2.1 Launch the STAR-CCM+ server on the HPC cluster

Prepare a PBS job script that specifies the required nodes, cores per node (ppn), and walltime. Replace <queue_name>, <project>, <#>, <##>, <##>, <HH:MM:SS>, <your_email>, <port_number>, ${STARCCM+_LICENCE}, ${STARCCM+_FULL_PATH}, and ${STAR_SIM_FILE} with actual values in the following pbs_ccm_server script:

pbs_ccm_server in LCRC Improv cluster

#!/bin/bash -l
#PBS -N ccm_server
#PBS -o ccm_server.out
#PBS -e ccm_server.out
#PBS -j n
#PBS -q <queue_name>
#PBS -A <project>
#PBS -l select=<#>:ncpus=<##>:mpiprocs=<##>
#PBS -l walltime=<HH:MM:SS>
#PBS -m be
#PBS -M <your_email>


# Print general informaiton -----------------------------------------------
set -euo pipefail

NHOSTS=`cat $PBS_NODEFILE | wc -l`

echo Working directory is $PBS_O_WORKDIR
cd $PBS_O_WORKDIR

echo Jobid: $PBS_JOBID
echo Running on host `hostname`
echo Running with $NHOSTS mpi thread/threads
# -------------------------------------------------------------------------


# Set port number ---------------------------------------------------------
PORT=${PORT:-<port_number>}
# -------------------------------------------------------------------------


# Get compute nodes -------------------------------------------------------
if [[ -f "${PBS_NODEFILE:-}" ]]; then
  NP="$(wc -l < "$PBS_NODEFILE")"
  HEAD_NODE="$(head -n1 "$PBS_NODEFILE")"
else
  echo "WARN: PBS_NODEFILE not found, fallback to 1 node."
  NP="${NP:-1}"
  HEAD_NODE="$(hostname)"
fi

HEAD_FQDN="$(getent hosts "$HEAD_NODE" | awk '{print $2; exit}')"
if [[ -z "${HEAD_FQDN}" ]]; then HEAD_FQDN="$(hostname -f)"; fi

NODELIST=$(sort -u "$PBS_NODEFILE" 2>/dev/null || echo "$HEAD_NODE")
# -------------------------------------------------------------------------


# Print connection information --------------------------------------------
echo
echo '=============== STAR-CCM+ Server Info ==============='
echo "STAR-CCM+ server head node: ${HEAD_FQDN}"
echo "STAR-CCM+ server port: ${PORT}"
echo "Allocated nodes:"
echo "${NODELIST}" | nl -ba
echo "Total threads (NP): ${NP}"
echo '====================================================='
echo
# -------------------------------------------------------------------------


# Start STAR-CCM+ server --------------------------------------------------
export CDLMD_LICENSE_FILE=${STARCCM+_LICENCE}

${STARCCM+_FULL_PATH} -server -rsh ssh -np $NHOSTS -machinefile $PBS_NODEFILE -port ${PORT} -load ${STAR_SIM_FILE}
# -------------------------------------------------------------------------

The structure under the case root directory is

.
├── ${STAR_SIM_FILE}
└── pbs_ccm_server

Submit the script with qsub pbs_ccm_server. After the job starts, status and connection details are written to ccm_server.out, for example:

...

=============== STAR-CCM+ Server Info ===============
STAR-CCM+ server head node: i822.lcrc.anl.gov
STAR-CCM+ server port: <port_number>
Allocated nodes:
     1  i822.lcrc.anl.gov
     2  i823.lcrc.anl.gov
Total threads (NP): 256
=====================================================

...

Server::start -host i822.lcrc.anl.gov:<port_number>

2.2 Create the SSH tunnel

The SSH tunnel is created by using MobaXterm. In the Tunneling dialog, create a new tunnel of Local port forwarding with the following settings:

Property	Value
<Forwarded port>	<port_number> in pbs_ccm_server
<SSH server>	improv.lcrc.anl.gov
<SSH login>	$USER in improv.lcrc.anl.gov
<SSH port>	22 or leave it blank
<Remote server>	i822.lcrc.anl.gov
<Remort port>	<port_number> in pbs_ccm_server

After tunnel creation, add the SSH key file and enable Autoreconnect in Settings.

2.3 Connect remote server with local STAR-CCM+ GUI

Launch the local STAR-CCM+ GUI, then:

• Go to File -> Connect to Server...
• In the Connect to Server dialog:
  • Host: localhost
  • Port: <port_number> set in pbs_ccm_server
• Click OK.

Now the local GUI should be connected to the remote compute nodes.

---