Skip to content

GitLab

  • Projects
  • Groups
  • Snippets
  • Help
    • Loading...
  • Help
    • Help
    • Support
    • Community forum
    • Submit feedback
    • Contribute to GitLab
  • Sign in
I ICOADS R HOSTACE
  • Project overview
    • Project overview
    • Details
    • Activity
    • Releases
  • Repository
    • Repository
    • Files
    • Commits
    • Branches
    • Tags
    • Contributors
    • Graph
    • Compare
  • Issues 7
    • Issues 7
    • List
    • Boards
    • Labels
    • Service Desk
    • Milestones
  • Merge requests 0
    • Merge requests 0
  • CI/CD
    • CI/CD
    • Pipelines
    • Jobs
    • Schedules
  • Operations
    • Operations
    • Incidents
    • Environments
  • Analytics
    • Analytics
    • CI/CD
    • Repository
    • Value Stream
  • Wiki
    • Wiki
  • Snippets
    • Snippets
  • Members
    • Members
  • Activity
  • Graph
  • Create a new issue
  • Jobs
  • Commits
  • Issue Boards
Collapse sidebar
  • brivas
  • ICOADS R HOSTACE
  • Wiki
    • Workflow
  • Data selection

Data selection · Changes

Page history
added liz changes authored Jun 11, 2020 by bearecinos's avatar bearecinos
Hide whitespace changes
Inline Side-by-side
Showing with 35 additions and 52 deletions
+35 -52
  • Workflow/Data-selection.md Workflow/Data-selection.md +35 -52
  • No files found.
Workflow/Data-selection.md
View page @ 3bbd1e55
In order to select data taken only by commercial ships, excluding In order to select data taken only by commercial ships, excluding
specialist ship data sources such as research vessels, a selection criteria has been established. specialist ship data sources such as research vessels, a selection criteria has been established.
Data selection rules are shown in following table. Reports require a day of the month present and to be valid (e.g. 30 Feb not used), but data with a missing hour are included for certain `dck` between 1850 and 1872. Data selection rules are shown in following table. Reports require a day of the month present and to be valid (e.g. 30 Feb not used), but data with a missing hour are included for certain `dck` between 1850 and 1872.
Each action in the following table corresponds to an `ifelse` statement in [split_by_type.R](https://git.noc.ac.uk/brecinosrivas/icoads-r-hostace/-/blob/master/rscripts/split_by_type.R) (all of the table corresponds to this script or there are other scripts that I should refer to here??) Each action in the following table corresponds to an `ifelse` statement in [split_by_type.R](https://git.noc.ac.uk/brecinosrivas/icoads-r-hostace/-/blob/master/rscripts/split_by_type.R)
________________ ________________
Rule | Action Rule | Action
:----- |:------------ :----- |:------------
`pt` missing & <br>`dck` = 128, 150, 151, 152, 155, 156, 192, 201, 255, 701, 721, 875, 897, 899 | `pt` = 0 (ship) `pt` missing & <br>`dck` = 128, 150, 151, 152, 155, 156, 192, 201, 255, 701, 721, 875, 897, 899 | `pt` = 0 (ship)
`pt` missing & <br>`dck` = 797 | `pt` = 13 (C-MAN) `pt` missing & <br>`dck` = 797 | `pt` = 13 (C-MAN)
`pt` missing & <br>`dck` = 896 & <br>`id` contains PLAT or RIG | `pt` = 15 (platform) `pt` missing & <br>`dck` = 896 & <br>`id` contains PLAT or RIG | `pt` = 15 (platform)
`pt` missing & <br>`dck` = 896 & <br> `id` contains SHIP | `pt` = 0 (ship) `pt` missing & <br>`dck` = 896 & <br> `id` contains SHIP | `pt` = 0 (ship)
`pt` missing & <br>`dck` = 896, 883 & <br> `id` starts with number | `pt` = 6 (moored buoy) `pt` missing & <br>`dck` = 896, 883 & <br> `id` starts with number | `pt` = 6 (moored buoy)
`pt` missing & <br> `dck` = 896 & <br> `id` contains 4Y or C7 | `pt` = 2 (OWS) `pt` missing & <br> `dck` = 896 & <br> `id` contains 4Y or C7 | `pt` = 2 (OWS)
`irf` = 2 except `dck` = 732 | Use ICOADS `irf` flag but retain `dck` 732 regional exclusions `irf` = 2 except `dck` = 732 | Use ICOADS `irf` flag but retain `dck` 732 regional exclusions
`pt` = 7 | Exclude drifters `pt` = 7 | Exclude drifters
`pt` = 6 | Exclude moored buoys `pt` = 6 | Exclude moored buoys
`pt` = 13 | Exclude C-MAN `pt` = 13 | Exclude C-MAN
`pt` = 14 | Exclude other coastal or island stations `pt` = 14 | Exclude other coastal or island stations
`pt` = 16 | Exclude tide gauges `pt` = 16 | Exclude tide gauges
`pt` = 18 | Exclude profiling floats `pt` = 18 | Exclude profiling floats
`pt` = 19 | Exclude undulating oceanographic recorders `pt` = 19 | Exclude undulating oceanographic recorders
`pt` = 20 | Exclude pinnepeds `pt` = 20 | Exclude pinnepeds
`pt` = 21 | Exclude gliders `pt` = 21 | Exclude gliders
`dck` = 795, 995 | Exclude C-MAN `dck` = 795, 995 | Exclude C-MAN
`dck` = 735, 740, 780, 782 | Exclude research vessels `dck` = 735, 740, 780, 782 | Exclude research vessels
`dck` = 793, 794, 993, 994 | Exclude moored buoys `dck` = 793, 794, 993, 994 | Exclude moored buoys
`id` = PLAT, BUOY, RIGG, BOUY | Exclude non-ship data `id` = PLAT, BUOY, RIGG, BOUY | Exclude non-ship data
`id` missing & <br> `dck` 700 | Exclude drifters `id` missing & <br> `dck` 700 | Exclude drifters
`id` = 5 digits & <br> `dck` = 700 & <br> `sid` = 147 & <br> `pt` = 5 | Exclude buoys `id` = 5 digits & <br> `dck` = 700 & <br> `sid` = 147 & <br> `pt` = 5 | Exclude buoys
`id` = 5 digits & <br> `dck` = 892 & <br> `sid` = 29 & <br> `pt` = 5 | Exclude buoys `id` = 5 digits & <br> `dck` = 892 & <br> `sid` = 29 & <br> `pt` = 5 | Exclude buoys
`id` contains TEST but /= CONTEST | Exclude suspect data `id` contains TEST but /= CONTEST | Exclude suspect data
Invalid day | Exclude data with invalid dates Invalid day | Exclude data with invalid dates
Hour missing | Except for `dck` = 246, 701, 721 Hour missing | Except for `dck` = 246, 701, 721
** At least one present of `sst`, `slp`, `at`, `w`, `d`, `ww`, `n`, `wbt`, `dpt`, `vv`, `rh`, `wh`, `nh`, `w1`, `osv` | Exclude reports with no selected variables. ** At least one present of `sst`, `slp`, `at`, `w`, `d`, `ww`, `n`, `wbt`, `dpt`, `vv`, `rh`, `wh`, `nh`, `w1`, `osv` | Exclude reports with no selected variables.
The data is groupped according to the following type of platform: The data is grouped according to the following type of platform:
- SHIP - SHIP
- DRIFT - DRIFT
- PLAT - PLAT
- COAST - COAST
- MOORED - MOORED
- SUBMARINE
- RESEARCH
- also REJECT (DY missing or HR missing for data after 1872 or ICOADS IRF reject)
Clone repository

Wiki pages

Home

Introduction
Installation
JASMIN tips

Workflow
- Data selection
- Processing of ID's
- Matching criteria
- Quality control
- Duplicate identification

API Reference

Releases

Examples