Skip to content

GitLab

  • Projects
  • Groups
  • Snippets
  • Help
    • Loading...
  • Help
    • Help
    • Support
    • Community forum
    • Submit feedback
    • Contribute to GitLab
  • Sign in
I ICOADS R HOSTACE
  • Project overview
    • Project overview
    • Details
    • Activity
    • Releases
  • Repository
    • Repository
    • Files
    • Commits
    • Branches
    • Tags
    • Contributors
    • Graph
    • Compare
  • Issues 7
    • Issues 7
    • List
    • Boards
    • Labels
    • Service Desk
    • Milestones
  • Merge requests 0
    • Merge requests 0
  • CI/CD
    • CI/CD
    • Pipelines
    • Jobs
    • Schedules
  • Operations
    • Operations
    • Incidents
    • Environments
  • Analytics
    • Analytics
    • CI/CD
    • Repository
    • Value Stream
  • Wiki
    • Wiki
  • Snippets
    • Snippets
  • Members
    • Members
  • Activity
  • Graph
  • Create a new issue
  • Jobs
  • Commits
  • Issue Boards
Collapse sidebar
  • brivas
  • ICOADS R HOSTACE
  • Wiki
    • Workflow
  • Data selection

Data selection · Changes

Page history
added liz changes authored Jun 11, 2020 by bearecinos's avatar bearecinos
Hide whitespace changes
Inline Side-by-side
Showing with 35 additions and 52 deletions
+35 -52
  • Workflow/Data-selection.md Workflow/Data-selection.md +35 -52
  • No files found.
Workflow/Data-selection.md
View page @ 3bbd1e55
In order to select data taken only by commercial ships, excluding
specialist ship data sources such as research vessels, a selection criteria has been established.
In order to select data taken only by commercial ships, excluding
specialist ship data sources such as research vessels, a selection criteria has been established.
Data selection rules are shown in following table. Reports require a day of the month present and to be valid (e.g. 30 Feb not used), but data with a missing hour are included for certain `dck` between 1850 and 1872.
Each action in the following table corresponds to an `ifelse` statement in [split_by_type.R](https://git.noc.ac.uk/brecinosrivas/icoads-r-hostace/-/blob/master/rscripts/split_by_type.R) (all of the table corresponds to this script or there are other scripts that I should refer to here??)
Each action in the following table corresponds to an `ifelse` statement in [split_by_type.R](https://git.noc.ac.uk/brecinosrivas/icoads-r-hostace/-/blob/master/rscripts/split_by_type.R)
________________
Rule | Action
:----- |:------------
`pt` missing & <br>`dck` = 128, 150, 151, 152, 155, 156, 192, 201, 255, 701, 721, 875, 897, 899 | `pt` = 0 (ship)
`pt` missing & <br>`dck` = 797 | `pt` = 13 (C-MAN)
`pt` missing & <br>`dck` = 896 & <br>`id` contains PLAT or RIG | `pt` = 15 (platform)
`pt` missing & <br>`dck` = 896 & <br> `id` contains SHIP | `pt` = 0 (ship)
`pt` missing & <br>`dck` = 896, 883 & <br> `id` starts with number | `pt` = 6 (moored buoy)
`pt` missing & <br> `dck` = 896 & <br> `id` contains 4Y or C7 | `pt` = 2 (OWS)
`irf` = 2 except `dck` = 732 | Use ICOADS `irf` flag but retain `dck` 732 regional exclusions
`pt` = 7 | Exclude drifters
`pt` = 6 | Exclude moored buoys
`pt` = 13 | Exclude C-MAN
`pt` = 14 | Exclude other coastal or island stations
`pt` = 16 | Exclude tide gauges
`pt` = 18 | Exclude profiling floats
`pt` = 19 | Exclude undulating oceanographic recorders
`pt` = 20 | Exclude pinnepeds
`pt` = 21 | Exclude gliders
`dck` = 795, 995 | Exclude C-MAN
`dck` = 735, 740, 780, 782 | Exclude research vessels
`dck` = 793, 794, 993, 994 | Exclude moored buoys
`id` = PLAT, BUOY, RIGG, BOUY | Exclude non-ship data
`id` missing & <br> `dck` 700 | Exclude drifters
`id` = 5 digits & <br> `dck` = 700 & <br> `sid` = 147 & <br> `pt` = 5 | Exclude buoys
`id` = 5 digits & <br> `dck` = 892 & <br> `sid` = 29 & <br> `pt` = 5 | Exclude buoys
`id` contains TEST but /= CONTEST | Exclude suspect data
`pt` missing & <br>`dck` = 128, 150, 151, 152, 155, 156, 192, 201, 255, 701, 721, 875, 897, 899 | `pt` = 0 (ship)
`pt` missing & <br>`dck` = 797 | `pt` = 13 (C-MAN)
`pt` missing & <br>`dck` = 896 & <br>`id` contains PLAT or RIG | `pt` = 15 (platform)
`pt` missing & <br>`dck` = 896 & <br> `id` contains SHIP | `pt` = 0 (ship)
`pt` missing & <br>`dck` = 896, 883 & <br> `id` starts with number | `pt` = 6 (moored buoy)
`pt` missing & <br> `dck` = 896 & <br> `id` contains 4Y or C7 | `pt` = 2 (OWS)
`irf` = 2 except `dck` = 732 | Use ICOADS `irf` flag but retain `dck` 732 regional exclusions
`pt` = 7 | Exclude drifters
`pt` = 6 | Exclude moored buoys
`pt` = 13 | Exclude C-MAN
`pt` = 14 | Exclude other coastal or island stations
`pt` = 16 | Exclude tide gauges
`pt` = 18 | Exclude profiling floats
`pt` = 19 | Exclude undulating oceanographic recorders
`pt` = 20 | Exclude pinnepeds
`pt` = 21 | Exclude gliders
`dck` = 795, 995 | Exclude C-MAN
`dck` = 735, 740, 780, 782 | Exclude research vessels
`dck` = 793, 794, 993, 994 | Exclude moored buoys
`id` = PLAT, BUOY, RIGG, BOUY | Exclude non-ship data
`id` missing & <br> `dck` 700 | Exclude drifters
`id` = 5 digits & <br> `dck` = 700 & <br> `sid` = 147 & <br> `pt` = 5 | Exclude buoys
`id` = 5 digits & <br> `dck` = 892 & <br> `sid` = 29 & <br> `pt` = 5 | Exclude buoys
`id` contains TEST but /= CONTEST | Exclude suspect data
Invalid day | Exclude data with invalid dates
Hour missing | Except for `dck` = 246, 701, 721
** At least one present of `sst`, `slp`, `at`, `w`, `d`, `ww`, `n`, `wbt`, `dpt`, `vv`, `rh`, `wh`, `nh`, `w1`, `osv` | Exclude reports with no selected variables.
Hour missing | Except for `dck` = 246, 701, 721
** At least one present of `sst`, `slp`, `at`, `w`, `d`, `ww`, `n`, `wbt`, `dpt`, `vv`, `rh`, `wh`, `nh`, `w1`, `osv` | Exclude reports with no selected variables.
The data is groupped according to the following type of platform:
The data is grouped according to the following type of platform:
- SHIP
- DRIFT
- DRIFT
- PLAT
- COAST
- MOORED
- SUBMARINE
- RESEARCH
- also REJECT (DY missing or HR missing for data after 1872 or ICOADS IRF reject)
Clone repository

Wiki pages

Home

Introduction
Installation
JASMIN tips

Workflow
- Data selection
- Processing of ID's
- Matching criteria
- Quality control
- Duplicate identification

API Reference

Releases

Examples