How to run MEDUSA 1D on ARCHER2 and ANEMONE

Running MEDUSA 1D

Running MEDUSA 1D

To do this we assume you already have your environment ready and have checked out a NEMO4-MEDUSA branch. If not, check the appropriate page from this wiki page .

This how to page is valid for both ARCHER2 and ANEMONE. Just mind the path and libraries to load, depending the machine you are running with.

On ARCHER2

NEMO='/work/n01/n01/${userID}/medusa_4.2.x/nemo'
SETTING_RUN_DIR='/work/n01/n01/julpal/SETTING_RUNNING_DIR/1D_ARCHER2'
COMPILER='X86_ARCHER2-Gnu_4.2'

On ANEMONE

NEMO='/dssgfs01/working/${userID}/NEMO/medusa_4.2.x/nemo'
SETTING_RUN_DIR='/dssgfs01/scratch/jpp1m13/SETTING_RUNNING_DIR/1D_ANEMONE'
COMPILER='x86-64_Anemone'

Compiling MEDUSA 1D "the old way"

I call it "the old way", as you'll have to compile in few different steps, manually. You can use this way of doing to learn and understand how the config works. If you just want to compile and get something running, jump to the next section : Compiling MEDUSA 1D "the easy way"

Compile the 1D config

First thing is to compile a first time a new config based on C1D_PAPA : from your NEMO4.2 base directory (should be something like ${NEMO} defined just above, but of course, adapt to your own medusa path).

Some corrections listed in Andrews documentations for ARCHER2 only (here -- This should have already been done to compile your ORCA2_MEDUSA configuration, but better to double-check) :
- Update ext/FCM/lib/Fcm/Config.pm by changing FC_MODSEARCH => '', into FC_MODSEARCH => '-J',
- Add this arch file into your arch directory :
```
cp /work/n01/shared/nemo/ARCH/arch-X86_ARCHER2-Gnu_4.2.fcm arch/ 
```
Now you can compile a first time -- valid whatever the machine -- type :
```
./makenemo -m  ${COMPILER} -r C1D_PAPA -n C1D_MED -j 16
```
It should compile successfully the dynamic side only, but will create the right files at the right place. Here our 1D config is called C1D_MED

Compile MEDUSA 1D

Modify our new config component and cpp keys. At the moment they are based on C1D_PAPA which is dynamics only.

update cfgs/work_cfgs.txt change our config line to be (just add the TOP component) :
```
C1D_MED  OCE TOP
```
Then cfgs/C1D_MED/cpp_C1D_MED.fcm add the key_top, it should be like (not necessarily exactly the same, just add key_top) :
```
 bld::tool::fppkeys   key_xios key_linssh key_vco_1d3d key_top
```

To compile properly, MEDUSA still needs few things : the MEDUSA's adapted version of NEMO and NEMO-TOP subroutines. Just copy them from the ORCA2_MEDUSA config, with :
```
cp -d cfgs/ORCA2_MEDUSA/MY_SRC/* cfgs/C1D_MED/MY_SRC/
```
Now everything is ready to compile. You can do so with :
```
./makenemo -m ${COMPILER} -r C1D_MED -j 16
```

Compiling MEDUSA 1D "the easy way"

To make things easier, and reduce the risks of error, i've added a MEDUSA 1D reference configuration : C1D_MED_PAPA. That means that you just need to compile it to get MEDUSA running in 1D.

So, you still need to load the right libraries, and make sure you use the right compiler, before compiling your config. Adapting from the section above :

From your NEMO4.2 base directory (should be something like ${NEMO} defined just above, but of course, adapt to your own medusa path).

Some corrections listed in Andrews documentations for ARCHER2 only (here -- This should have already been done to compile your ORCA2_MEDUSA configuration, but better to double-check) :
- Update ext/FCM/lib/Fcm/Config.pm by changing FC_MODSEARCH => '', into FC_MODSEARCH => '-J',
- Add this arch file into your arch directory :
```
cp /work/n01/shared/nemo/ARCH/arch-X86_ARCHER2-Gnu_4.2.fcm arch/ 
```
Now you can compile your MEDUSA 1D configuration -- valid whatever the machine -- type :
```
./makenemo -m  ${COMPILER} -r C1D_MED_PAPA -n C1D_MED -j 16
```
It should compile successfully the dynamic and the passive tracer all at once. Here our 1D config is called C1D_MED. Compiling this way, All the files are at the right place already, so - still, double check - you should be able to happily skip the file management part of the next section.

Running The PAPA "ready to use" config.

From here, NEMO-Executable is ready, but we still need to include MEDUSA's namelist and other files to be able to run it.

go into cfgs/C1D_MED/EXP00.

copy and link the files :

   cp -rd ../../ORCA2_MEDUSA/EXPREF ..
   ln -s  ../../SHARED_MEDUSA/namelist_medusa_ref .
   cp    ../EXPREF/namelist_medusa_cfg .
   cp -d ../EXPREF/namelist_top_* .
   cp -d ../EXPREF/*medusa*xml .
   cp    ../EXPREF/xco2.atm .

All MEDUSA files are here. We need to adapt some.

Update context.xml : add references to field*medusa*xml and file*medusa*xml files. It should look like this afterward:

<!--
============================================================================================== 
NEMO context
============================================================================================== 
-->
<context id="nemo">
    <!-- $id$ -->

<!-- Fields definition -->
    <field_definition src="./field_def_nemo-oce.xml"/>   <!--  NEMO ocean dynamics                       --> 
    <field_definition src="./field_def_nemo-medusa.xml"/>   <!--  NEMO ocean dynamics                     -->

<!-- Files definition -->
    <file_definition src="./file_def_nemo-oce.xml"/>     <!--  NEMO ocean dynamics                     -->
    <file_definition src="./file_def_nemo-medusa.xml"/>     <!--  NEMO ocean dynamics                     -->

<!-- Axis definition -->
    <axis_definition src="./axis_def_nemo.xml"/>

<!-- Domain definition -->
    <domain_definition src="./domain_def_nemo.xml"/>

<!-- Grids definition -->
    <grid_definition   src="./grid_def_nemo.xml"/>


</context>

Update namelist_top_cfg : change ln_rsttr to .false. (we don't start MEDUSA from a restart file, you can if you want, but not at first, for that you'd need to extract the right vertical profile from a restart_trc file)
Update namelist_medusa_cfg : ln_read_dust to .false. (I'd recommend to change that to true when you get MEDUSA 1D to run. you would need to extract the right location from forcing fields).
Update file_def_nemo-medusa.xml : get the right output frequency. it is 1d in file_def_nemo-oce.xml. you can keep the same or update to what you prefer.

Now we only miss the initial conditions and forcing. Fortunately these already exits for the PAPA location. you can get them by running in your config EXP00 directory (which is the running directory in our test) :
```
wget https://zenodo.org/record/3386310/files/INPUTS_C1D_PAPA_v4.0.tar?download=1  
tar xvf 'INPUTS_C1D_PAPA_v4.0.tar?download=1'
```
The namelist are already adapted for these forcing and initial fields.
In our test MEDUSA starts from constant values. you can change that by extracting the right profile from a trc restart file.

Now you can run with ./nemo

Running from 3D-extracted conditions.

Prepare your running directory :

For this we'll use 3D initial conditions and forcings from UKESM1.1. Let's make a new running directory :

go to cfgs/C1D_MED/
then :

# make a new running directory :  
mkdir EXP_1D
cd EXP_1D

Copy and run the appropriate COPY_ME bash script that will prepare you running directory for the 1D runs forced with UKESM1 :

cp ${SETTING_RUN_DIR}/COPY_ME__NEMOMED-4.2_1D_UKESM1_405.sh .
./COPY_ME__NEMOMED-4.2_1D_UKESM1_405.sh

Extract the initial conditions and forcings

To run this 1D run, you want to extract the initial conditions and the forcings from a specific location. you need the coordinates (latitude and longitude) and well as the corresponding grid indices (i and j). If you look in your namelist_cfg file, you'll find within namc1d an example of famous site coordinates to use :

!-----------------------------------------------------------------------
&namc1d        !   1D configuration options                             (ln_c1d =T default: PAPA station)
!-----------------------------------------------------------------------
!! Localion name  |  real coords(lat-lon) | grid idices(j-i) | grid coords        | bathy depth
!!OWS India         (59.00N, -19.00E)       (273, 269)        (59.02N, -18.46E)    1516
!!PAP site          (49.00N, -16.50E)       (258, 272)        (49.11N, -16.07E)    4888
!!BATS              (31.67N, -64.17E)       (236, 224)        (32.07N, -64.52E)    4093
!!ESTOC             (29.07N, -15.25E)       (232, 273)        (28.72N, -15.50E)    3513
!!OWS Papa          (50.00N, -145.00E)      (262, 142)        (50.13N, -145.37E)   4290
!!HOTS              (22.75N, -158.00E)      (225, 130)        (22.41N, -158.50E)   4688
!!Kerfix            (-50.67N, 68.42E)       (113, 357)        (-50.70N, 68.50E)    1796
!! -- don't forget to run:  ./subset_init_forcing.sh ${i} ${j} to get the local files.
!--------------------------------------------------------------------------
   rn_lat1d    =    50.0     !  Column latitude
   rn_lon1d    =    -145.0    !  Column longitude
/

You need to update the rn_lat1d and rn_lon1d in this namelist. Here we use the coordinates of PAPA.

You also need to update the bathymetry depth in the namusr_def namelist in namelist_cfg. To make things easier, each site bathy depth is given with the coordinates. Here we adapt for the PAPA site:

!-----------------------------------------------------------------------
&namusr_def    !   C1D user defined namelist
!-----------------------------------------------------------------------
   rn_bathy    =  4290.   ! depth in meters 
/

Then, you need to extract forcings and initial conditions. in your running dir you'll find subset_init_forcing.sh Run this script by specifying the i and j indices to extract :
```
./subset_init_forcing.sh ${i} ${j} 

## for example, to extract what we need for the PAPA site, type :
./subset_init_forcing.sh 142 262
```

There, all should be ready. Then make sure

context.xml use the correct xml files;
The output frequency in the file*xml files match what you want,
one last look at the namelists. Here, all is ready to run 10y, with 5days frequency outputs.

Finally, to run :

sbatch hetjob_ORCA1

Running in high latitudes.

The C1D configuration runs without the sea-ice model in. For some reason, maybe because the test case doesn't need it. But because of this, trying to run NEMO 1D in high latitudes will end with abnormally low temperature.

Here are the additional steps needed if you need to run with the ice model ON.

Add the sea-ice component. for this : update cfgs/work_cfgs.txt change our config line to be (add the ICE component) :
```
C1D_MED  OCE TOP ICE
```
Add the sea-ice cpp key If you remember, this is done in cfgs/C1D_MED/cpp_C1D_MED.fcm add the key_si3 key, it should be like :
```
bld::tool::fppkeys   key_xios key_linssh key_top key_si3
```
Compile with the sea ice in: Just compile the model again as previously done :
- from your nemo directory, run :
```
./makenemo -m X86_ARCHER2-Gnu_4.2 -r C1D_MED -j 16
```
Add the ice namelists They should already be here, but if not you should be able to find both namelist_ice_ref and namelist_ice_cfg in cfgs/ORCA2_ICE_PISCES/EXPREF directory.
Activate the SI3 model at run time Change nn_ice in namelist_cfg to 2. If nn_ice is not in your namelist_cfg copy it from namelist_ref and add it in (make sure to add it in the correct namelist/section).

Include the ice xml files. if they are not in your running directory yet, copy the file_def_nemo-ice.xml and field_def_nemo-ice.xml from the cfgs/ORCA2_ICE_PISCES/EXPREF directory, and include them in the context.xml file. it should look like this :

<!-- Fields definition -->
    <field_definition src="./field_def_nemo-oce.xml"/>    <!--  NEMO ocean dynamics     -->
    <field_definition src="./field_def_nemo-ice.xml"/>    <!--  NEMO ocean dynamics     -->
    <field_definition src="./field_def_nemo-medusa.xml"/> <!--  NEMO-MEDUSA ocean biology -->

<!-- Files definition -->
    <file_definition src="./file_def_nemo-oce.xml"/>     <!--  NEMO ocean dynamics      -->
    <file_definition src="./file_def_nemo-ice.xml"/>     <!--  NEMO ocean dynamics      -->
    <file_definition src="./file_def_nemo-medusa.xml"/>  <!--  NEMO ocean biology       -->

From there you should be ready to run 1D with the ice model activated.