Skip to content

GitLab

  • Projects
  • Groups
  • Snippets
  • Help
    • Loading...
  • Help
    • Help
    • Support
    • Community forum
    • Submit feedback
    • Contribute to GitLab
  • Sign in
I ICOADS R HOSTACE
  • Project overview
    • Project overview
    • Details
    • Activity
    • Releases
  • Repository
    • Repository
    • Files
    • Commits
    • Branches
    • Tags
    • Contributors
    • Graph
    • Compare
  • Issues 7
    • Issues 7
    • List
    • Boards
    • Labels
    • Service Desk
    • Milestones
  • Merge requests 0
    • Merge requests 0
  • CI/CD
    • CI/CD
    • Pipelines
    • Jobs
    • Schedules
  • Operations
    • Operations
    • Incidents
    • Environments
  • Analytics
    • Analytics
    • CI/CD
    • Repository
    • Value Stream
  • Wiki
    • Wiki
  • Snippets
    • Snippets
  • Members
    • Members
  • Activity
  • Graph
  • Create a new issue
  • Jobs
  • Commits
  • Issue Boards
Collapse sidebar
  • brivas
  • ICOADS R HOSTACE
  • Wiki
  • Workflow

Workflow · Changes

Page history
fixed diagram authored Sep 15, 2020 by Beatriz Recinos's avatar Beatriz Recinos
Hide whitespace changes
Inline Side-by-side
Showing with 11 additions and 11 deletions
+11 -11
  • Workflow.md Workflow.md +11 -11
  • No files found.
Workflow.md
View page @ aa0f24a9
...@@ -44,17 +44,17 @@ More details on the data processing can be found in this [technical report](http ...@@ -44,17 +44,17 @@ More details on the data processing can be found in this [technical report](http
graph TB graph TB
A1[rscripts] A1[rscripts]
id1[(ICOADS v3.0)] --> |split_by_type.R|id2[<center> Separate records according <br> to the different platform types. </center>] id1[(ICOADS v3.0)] --> |split_by_type.R|id2[Separate records according <br> to the different platform types.]
id2 --> |simple_dup.R|id3[<center> Check for cross-type duplicates between <br> ship data and the different platform types. <br> Considers the records as duplicates if they <br> show matching date, time & position, <br> with DCK and ID specific selection criteria. </center>] id2 --> |simple_dup.R|id3[Check for cross-type duplicates between <br> ship data and the different platform types. <br> Considers the records as duplicates if they <br> show matching date, time & position, <br> with DCK and ID specific selection criteria.]
id3 --> |ship2plat.R|id4[<center> Exclude non-ship data identified in <br> cross-type duplicate analysis. </center>] id3 --> |ship2plat.R|id4[Exclude non-ship data identified in <br> cross-type duplicate analysis.]
id4 --> id5[<center> (ICOADS SHIP data) </center>] id4 --> id5[(ICOADS SHIP data)]
id5 --> |process_ships.R|id6[<center> Reformat selected ship IDs to homogenize <br> information between DCKs. <br> Uses IDs from Pub. 47 metadata <br> in ID prioritisation. <br> Corrects dates & times. </center>] id5 --> |process_ships.R|id6[Reformat selected ship IDs to homogenize <br> information between DCKs. <br> Uses IDs from Pub. 47 metadata <br> in ID prioritisation. <br> Corrects dates & times.]
id6 --> |get_pairs.R|id7[<center> Groups ship reports as potential duplicates <br> if the contents match within tolerance </center>] id6 --> |get_pairs.R|id7[Groups ship reports as potential duplicates <br> if the contents match within tolerance]
id7 --> |get_dups.R|id8[<center> Assesses the groups of potential duplicates, <br> accepting those where the ID match is appropriate. <br> Reports from DCK that are of lower quality, <br>or that are less complete, or that fail the <br> track check are flagged as the worst. </center>] id7 --> |get_dups.R|id8[Assesses the groups of potential duplicates, <br> accepting those where the ID match is appropriate. <br> Reports from DCK that are of lower quality, <br>or that are less complete, or that fail the <br> track check are flagged as the worst.]
id8 --> |merge_ids_year.R|id9[<center> Assesses IDs that have been associated <br> in previous processing to decide whether to replace <br> all IDs in the associated group with the preferred ID. </center>] id8 --> |merge_ids_year.R|id9[Assesses IDs that have been associated <br> in previous processing to decide whether to replace <br> all IDs in the associated group with the preferred ID.]
id9 --> |nrt_dup.R|id10[<center> Process near-real-time data collected after 2014. </center>] id9 --> |nrt_dup.R|id10[Process near-real-time data collected after 2014.]
id10 --> |clean_data.R|id11[<center> Runs track checking on data to produce <br>clean tracks for all IDs. </center>] id10 --> |clean_data.R|id11[Runs track checking on data to produce <br>clean tracks for all IDs.]
id11 --> |clean2track.R|id12[<center> Selects data for ship-tracking software <br>Carella et al. 2017,<br> choosing only data with missing or generic IDs. </center>] id11 --> |clean2track.R|id12[Selects data for ship-tracking software <br>Carella et al. 2017,<br> choosing only data with missing or generic IDs.]
id12 --> id13[(Output data)] id12 --> id13[(Output data)]
classDef pre-processing fill:#fcc679,stroke:#333,stroke-width:1px,font-size:16px,font-weight:100,text-align:center classDef pre-processing fill:#fcc679,stroke:#333,stroke-width:1px,font-size:16px,font-weight:100,text-align:center
......
Clone repository

Wiki pages

Home

Introduction
Installation
JASMIN tips

Workflow
- Data selection
- Processing of ID's
- Matching criteria
- Quality control
- Duplicate identification

API Reference

Releases

Examples