Skip to content

GitLab

  • Projects
  • Groups
  • Snippets
  • Help
    • Loading...
  • Help
    • Help
    • Support
    • Community forum
    • Submit feedback
    • Contribute to GitLab
  • Sign in
O ORCHESTRA-SST
  • Project overview
    • Project overview
    • Details
    • Activity
    • Releases
  • Repository
    • Repository
    • Files
    • Commits
    • Branches
    • Tags
    • Contributors
    • Graph
    • Compare
  • Issues 4
    • Issues 4
    • List
    • Boards
    • Labels
    • Service Desk
    • Milestones
  • Merge requests 0
    • Merge requests 0
  • CI/CD
    • CI/CD
    • Pipelines
    • Jobs
    • Schedules
  • Operations
    • Operations
    • Incidents
    • Environments
  • Analytics
    • Analytics
    • CI/CD
    • Repository
    • Value Stream
  • Wiki
    • Wiki
  • Snippets
    • Snippets
  • Members
    • Members
  • Activity
  • Graph
  • Create a new issue
  • Jobs
  • Commits
  • Issue Boards
Collapse sidebar
  • brivas
  • ORCHESTRA-SST
  • Wiki
  • How to run in jasmin

How to run in jasmin · Changes

Page history
added more information authored Aug 17, 2021 by Beartriz Recinos Rivas's avatar Beartriz Recinos Rivas
Hide whitespace changes
Inline Side-by-side
Showing with 62 additions and 4 deletions
+62 -4
  • How-to-run-in-jasmin.md How-to-run-in-jasmin.md +62 -4
  • No files found.
How-to-run-in-jasmin.md
View page @ 48bd7508
Configuration file
------------------
After installing the environment in JASMIN you need to make sure you specify the corrects paths to the code repository, input and output data at the [configuration file]().
After installing the environment in JASMIN you need to make sure to specify the correct paths to the code repository, input and output data. Make your own changes to the [configuration file](https://git.noc.ac.uk/brecinosrivas/orchestra-sst/-/blob/master/config.ini).
> **Note** Check with Liz where the lastest version of SST data and drifters is located in JASMIN just to make sure if the paths above are correct.
> **Note** Check with Liz where the latest path location for the drifting buoys since we use her ICOADS output of data.
Gridding process
Griding process
----------------
The re-gridding of SST data and the interpolation of buoy coordinates to the satellite data is done by a single script under:
The re-gridding of SST data and the interpolation of the data to the buoy's coordinates is done by a single script under:
``~/scripts/coarse_cci_sst.py``
The corresponding SLURM script to submit this job in JASMIN can be found under ``~/src/run_gridding.slurm``.
Several things need to be set up before running that script:
1. Make slurm-log output directories:
~~~
cd ~/orchestra-sst/
mkdir slurm_log_output
mkdir input_data
~~~
You should have the following directory configuration under `~/orchestra-sst/`
~~~
input_data
scripts
slurm_log_output
config.ini
src
sst_tools
~~~
2. Modify paths and job array set up in the file ``~/orchestra-sst/src/run_gridding.slurm``:
~~~
#!/bin/bash
#SBATCH --partition=short-serial-4hr
#SBATCH --array=3001-6574
#SBATCH --job-name=sst_buoy_regrid
#SBATCH --output=../slurm_log_output/sst_buoy_regrid_%A_%a.out
#SBATCH --error=../slurm_log_output/sst_buoy_regrid_%A_%a.err
#SBATCH --mem=4000
#SBATCH --time=01:00:00
source activate ~/miniconda3/envs/sst
start_year=1993
start_month=1
start_day=1
echo "Analysing data for $SLURM_ARRAY_TASK_ID days"
echo "since $start_year $start_month $start_day minus a day"
python ~/orchestra-sst/scripts/coarse_cci_sst.py $start_year $start_month $start_day $SLURM_ARRAY_TASK_ID
echo "Done slurm task ID = $SLURM_ARRAY_TASK_ID"
~~~
Make sure you modify ```#SBATCH --array=1-3000``` at the moment to process the entire time series you need to run it in two parts
first from day 1 to 3000, and later from 3001 to 6574. So the job can be run in the ``short-serial-4hr`` queue (next part: ``#SBATCH --array=3001-6574``).
3. After configuring the SLURM script you can run it by:
~~~
cd ~/orchestra-sst/src/
sbatch run_gridding.slurm
~~~
For more information on the processing workflow of this script read the following [jupyter notebook](https://git.noc.ac.uk/brecinosrivas/orchestra-sst/-/blob/master/notebooks/cci_at_buoy_coord_results.ipynb)
Computing time series
---------------------
......
Clone repository

Wiki pages

Home

Introduction
Installation
How to run the code