|
Configuration file
|
|
Configuration file
|
|
------------------
|
|
------------------
|
|
|
|
|
|
After installing the environment in JASMIN you need to make sure you specify the corrects paths to the code repository, input and output data at the [configuration file]().
|
|
After installing the environment in JASMIN you need to make sure to specify the correct paths to the code repository, input and output data. Make your own changes to the [configuration file](https://git.noc.ac.uk/brecinosrivas/orchestra-sst/-/blob/master/config.ini).
|
|
|
|
|
|
> **Note** Check with Liz where the lastest version of SST data and drifters is located in JASMIN just to make sure if the paths above are correct.
|
|
> **Note** Check with Liz where the latest path location for the drifting buoys since we use her ICOADS output of data.
|
|
|
|
|
|
Gridding process
|
|
Griding process
|
|
----------------
|
|
----------------
|
|
|
|
|
|
The re-gridding of SST data and the interpolation of buoy coordinates to the satellite data is done by a single script under:
|
|
The re-gridding of SST data and the interpolation of the data to the buoy's coordinates is done by a single script under:
|
|
``~/scripts/coarse_cci_sst.py``
|
|
``~/scripts/coarse_cci_sst.py``
|
|
|
|
|
|
The corresponding SLURM script to submit this job in JASMIN can be found under ``~/src/run_gridding.slurm``.
|
|
The corresponding SLURM script to submit this job in JASMIN can be found under ``~/src/run_gridding.slurm``.
|
|
|
|
|
|
|
|
Several things need to be set up before running that script:
|
|
|
|
|
|
|
|
1. Make slurm-log output directories:
|
|
|
|
~~~
|
|
|
|
cd ~/orchestra-sst/
|
|
|
|
mkdir slurm_log_output
|
|
|
|
mkdir input_data
|
|
|
|
~~~
|
|
|
|
|
|
|
|
You should have the following directory configuration under `~/orchestra-sst/`
|
|
|
|
|
|
|
|
~~~
|
|
|
|
input_data
|
|
|
|
scripts
|
|
|
|
slurm_log_output
|
|
|
|
config.ini
|
|
|
|
src
|
|
|
|
sst_tools
|
|
|
|
~~~
|
|
|
|
|
|
|
|
2. Modify paths and job array set up in the file ``~/orchestra-sst/src/run_gridding.slurm``:
|
|
|
|
|
|
|
|
~~~
|
|
|
|
#!/bin/bash
|
|
|
|
#SBATCH --partition=short-serial-4hr
|
|
|
|
#SBATCH --array=3001-6574
|
|
|
|
#SBATCH --job-name=sst_buoy_regrid
|
|
|
|
#SBATCH --output=../slurm_log_output/sst_buoy_regrid_%A_%a.out
|
|
|
|
#SBATCH --error=../slurm_log_output/sst_buoy_regrid_%A_%a.err
|
|
|
|
#SBATCH --mem=4000
|
|
|
|
#SBATCH --time=01:00:00
|
|
|
|
|
|
|
|
source activate ~/miniconda3/envs/sst
|
|
|
|
|
|
|
|
start_year=1993
|
|
|
|
start_month=1
|
|
|
|
start_day=1
|
|
|
|
|
|
|
|
echo "Analysing data for $SLURM_ARRAY_TASK_ID days"
|
|
|
|
echo "since $start_year $start_month $start_day minus a day"
|
|
|
|
|
|
|
|
python ~/orchestra-sst/scripts/coarse_cci_sst.py $start_year $start_month $start_day $SLURM_ARRAY_TASK_ID
|
|
|
|
|
|
|
|
echo "Done slurm task ID = $SLURM_ARRAY_TASK_ID"
|
|
|
|
|
|
|
|
~~~
|
|
|
|
|
|
|
|
Make sure you modify ```#SBATCH --array=1-3000``` at the moment to process the entire time series you need to run it in two parts
|
|
|
|
first from day 1 to 3000, and later from 3001 to 6574. So the job can be run in the ``short-serial-4hr`` queue (next part: ``#SBATCH --array=3001-6574``).
|
|
|
|
|
|
|
|
3. After configuring the SLURM script you can run it by:
|
|
|
|
|
|
|
|
~~~
|
|
|
|
cd ~/orchestra-sst/src/
|
|
|
|
sbatch run_gridding.slurm
|
|
|
|
~~~
|
|
|
|
|
|
|
|
For more information on the processing workflow of this script read the following [jupyter notebook](https://git.noc.ac.uk/brecinosrivas/orchestra-sst/-/blob/master/notebooks/cci_at_buoy_coord_results.ipynb)
|
|
|
|
|
|
Computing time series
|
|
Computing time series
|
|
---------------------
|
|
---------------------
|
... | | ... | |