How to run on ANEMONE

How to run on ANEMONE

How to run on ANEMONE

Here is a tutorial on how to run NEMO4.2-MEDUSA on ANEMONE (NOC supercomputer).

Environment

Luckily, Adam Blacker has already wrote a complete note on that subject here The explanations in this doc is about NEMO only, but it's almost the same procedure to run NEMO-MEDUSA.

In short :

load the right modules. This is easy as it requires loading only 1 package :

module load NEMO/prg-env

Get MEDUSA

Go to your work-directory (beware, ANEMONE's working disk is not backed up, so don't leave crucial files there) :

cd /dssgfs01/working/${username}/

To get MEDUSA, see the full description with explanation on the main page of the git project, but roughly :

git clone --recurse-submodules git@git.noc.ac.uk:acc/medusa_4.2.x.git
### or
git clone --recurse-submodules https://git.noc.ac.uk/acc/medusa_4.2.x.git
###  put MEDUSA at the right places :
cd medusa_4.2.x
./add_msa_to_nemo

Compile MEDUSA

Compile : \
- move to the nemo directory :
```
cd nemo
```

Compiling

Now you can compile as follows (With MY_NEW_MEDUSA your MEDUSA config if you want/need to create one) :
```
./makenemo -m x86-64_Anemone -r ORCA2_MEDUSA -n ${MY_NEW_MEDUSA} -j 16 
```

Subsequently when you compile you can instead run :

./makenemo -m x86-64_Anemone -r ${MY_NEW_MEDUSA} -j 16

Get your running directory ready

Once compiled, prepare your running directory, with ${running_dir} the name of your running directory :

mkdir cfgs/${MY_NEW_MEDUSA}/${running_dir}
cp /dssgfs01/scratch/jpp1m13/SETTING_RUNNING_DIR/eORCA1_ANEMONE/COPY_ME__NEMOMED-4.2_ORCA1_UKESM1_405.sh cfgs/ORCA2_MEDUSA/${running_dir}/.

Then go to your running directory and run this shell script. It should copy and link all necessary files in your running directory, to run NEMO4.2-MEDUSA on eORCA1 grid, forced with UKESM1.1 forcings.
```
cd cfgs/${MY_NEW_MEDUSA}/${running_dir}
./COPY_ME__NEMOMED-4.2_ORCA1_UKESM1_405.sh
```

Run your model

The submission script, hetjob_ORCA1, will likely need some edits. The only parts that should need changing in hetjob_ORCA1 are in the head of the file :

export OMP_NUM_THREADS=1
export EXP_NAM=RIVER2
export RUN_LEN_YR=1
export TOT_RUN_LEN_YR=10
export TS=2700
export MEAN_DIR=MEANS_RIV1
export MEAN_ROOT=$(grep "cn_exp" namelist_cfg | awk -F"'" '{print $2}')
export RUN_SCRIPT=hetjob_ORCA1

With RUN_LEN_YR the length (in year) of the submitted section of the run and TOT_RUN_LEN_YR the total run length in year. TS it the model time-step in seconds. Also, Make sure that cn_exp value in namelist_cfg is between ' and not ", the running script would not get the experiment name otherwise, and the updating script would not work properly.

The submission script will automatically update the restart and time-step variables in the namelist to start and restart the run.
Finally, the namelists should be edited as needed for the experiment. Remember that there are pairs of files for each component :
1. namelist_ref, the reference namelist for the default configuration
2. namelist_cfg, the configuration namelist for the experiment configuration The second should be used to make changes that override the default settings and parameter values in the first.
To submit the job, use sbatch hetjob_ORCA1.
To monitor the job, use squeue -u ${userid}. (If you can't remember who you are, type whoami from the prompt!)
You can restart (not starting from scratch, but from restart files), by simply adding an argument, like sbatch hetjob_ORCA1 2.
- To restart the run from scratch, don't forget to clean the RESTART and PREV directories before submitting the job. No need to reinitialize the namelists, it is done by the submission script.
Don't forget, the namelists here are adapted to run NEMO4.2-MEDUSA on eORCA1, forced with UKESM atmosphere and rivers ; starting on the 1850/01/01. You can change the namelists if you need to change anything to that.