How to run in jasmin

This is an old version of this page. You can view the most recent version or browse the history.

Configuration file

After installing the environment in JASMIN you need to make sure to specify the correct paths to the code repository, input and output data. Make your own changes to the configuration file.

Note Check with Liz where the latest path location for the drifting buoys since we use her ICOADS output of data.

Griding process

The re-gridding of SST data and the interpolation of the data to the buoy's coordinates is done by a single script under: ~/scripts/coarse_cci_sst.py

The corresponding SLURM script to submit this job in JASMIN can be found under ~/src/run_gridding.slurm.

Several things need to be set up before running that script:

Make slurm-log output directories:

cd ~/orchestra-sst/
mkdir slurm_log_output
mkdir input_data

You should have the following directory configuration under ~/orchestra-sst/

input_data                 
scripts  
slurm_log_output
config.ini             
src
sst_tools

Modify paths and job array set up in the file ~/orchestra-sst/src/run_gridding.slurm:

#!/bin/bash
#SBATCH --partition=short-serial-4hr
#SBATCH --array=3001-6574
#SBATCH --job-name=sst_buoy_regrid
#SBATCH --output=../slurm_log_output/sst_buoy_regrid_%A_%a.out
#SBATCH --error=../slurm_log_output/sst_buoy_regrid_%A_%a.err
#SBATCH --mem=4000
#SBATCH --time=01:00:00

source activate ~/miniconda3/envs/sst

start_year=1993
start_month=1
start_day=1

echo "Analysing data for $SLURM_ARRAY_TASK_ID days"
echo "since $start_year $start_month $start_day minus a day"

python ~/orchestra-sst/scripts/coarse_cci_sst.py $start_year $start_month $start_day $SLURM_ARRAY_TASK_ID

echo "Done slurm task ID = $SLURM_ARRAY_TASK_ID"

Make sure you modify #SBATCH --array=1-3000 at the moment to process the entire time series you need to run it in two parts first from day 1 to 3000, and later from 3001 to 6574. So the job can be run in the short-serial-4hr queue (next part: #SBATCH --array=3001-6574).

After configuring the SLURM script you can run it by:

cd ~/orchestra-sst/src/
sbatch run_gridding.slurm

For more information on the processing workflow of this script read the following jupyter notebook

GitLab