Merge branch '20-release-v0-1' into 'master'

Resolve "Release v0.1"

See merge request !16

.gitignore

-formats/__pycache__
+__pycache__/
+*.pyc
+

.gitlab-ci.yml

+include:
+  - project: communications-backbone-system/backbone-infrastructure-config
+    ref: master
+    file: gitlab/all.yml
+
+variables:
+  DOCKER_IMAGE_NAME: backbone-message-format
+  TEST_DOCKER_COMPOSE: 1
\ No newline at end of file

CHANGELOG.md

+# Changelog
+
+All notable changes to this project will be documented in this file.
+
+The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/)
+and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.html).
+
+## [Unreleased]
+
+## [v0.1.0] - 2023-03-24
+
+### Added
+
+JSON schema definitions for the SoAR project
+- outer wrapper MESSAGE definition 
+- header object 
+- payload object
+  - acknowledgement
+  - mission_plan (raw and encoded)
+  - observation (raw and encoded)
+  - platform_status (raw and encoded) 
+  - planning_configuration 
+
+Example messages matching the schema for each partner
+
+[v0.1.0]: https://git.noc.ac.uk/communications-backbone-system/backbone-message-format/compare/9e6ce245...v0.1.0
+[unreleased]: https://git.noc.ac.uk/communications-backbone-system/backbone-message-format/compare/v0.1.0...dev

LICENSE

 MIT License
 
-Copyright (c) 2022 National Oceanography Centre CLG
+Copyright (c) 2022 National Oceanography Centre
 
 Permission is hereby granted, free of charge, to any person obtaining a copy
 of this software and associated documentation files (the "Software"), to deal

README.md

@@ -11,6 +11,63 @@ Each message below will be wrapped in a `payload` field and will have a `header`
 * `acknowledgement`: level of acknowledgment where an acknowledgement is sent when a message is i. received, ii. sent to the next destination (e.g. platform in the water).
 * `planning_configuration`: sent from the GUI to initialise the AI model (autonomy engine).
 
+## Topics 
+
+Messages are routed using either a publish/subscribe or broadcast mechanism. 
+
+For published messages, the message is published for a given a topic. 
+
+Clients create a subscription (stored in their config) to a topic pattern which may include wildcards. 
+
+It's important to note that the client config documents the subscription. To change a client 
+subscription it must be updated with the backbone.
+
+Published messages are delivered to all clients where 
+the message topic matches the client's subscription 
+topic pattern.
+
+Broadcast messages are delivered to all clients regardless of their subscription. 
+
+### Field
+
+The topic should be set in the `message.header.destination` for published messages. 
+The `destination` field should be set to `broadcast` for any messages sent using broadcast. 
+
+Because the `destination` field is a topic it defines what the message relates to - not 
+who the intended recipient is. 
+
+This means that the clients are not typically represented in the topics. A client publishes a message about a platform and subscribers interested in that project, platform or message type will receive that message.
+
+### Structure
+
+`<prefix>.<operator>.<platform_type>.<platform_identifier>.<to_platform|from_platform>.<message_type>`
+
+e.g
+* `soar.noc.autosub.ah1.from_platform.platform_status`
+* `soar.planet-ocean.ecosub.eco1.to_platform.mission_plan`
+* `soar.hydrosurv.reav.reav1.to_platform.mission_plan`
+
+### Terms
+
+* `prefix` - an agreed hard-coded/config setting prefix shared by all adapters and client subscriptions eg soar 
+* `operator` - human readable organisation name for the platform operator one of [noc|planet-ocean|hydrosurv] 
+* `platform_type` - one of [autosub|ecosub|reav?]
+* `platform_identifier` - human readable identifier for the platform eg ah1, eco1, reav1 
+* `to_platform|from_platform` - eg ..to_platform.mission_plan, ..from_platform.platform_status
+missions are sent `to_platform` to be executed and status updates are received `from_platform` containing 
+information about its position and health status
+* `message_type` - one of `[acknowledgement|mission_plan|observation|planning_configuration|platform_status...]`
+
+### Subscriptions
+
+Subscriptions may contain multiword wildcards (#) or single word wildcards (*)
+eg
+* `soar.#` - everything 
+* `soar.planet-ocean.#` - anything relating to the planet-ocean operator
+* `soar.*.*.*.from_platform.*` - all messages inbound from platforms (autonomy engine)
+* `soar.*.*.*.to_platform.*` - all messages outbound to platforms (hermes)
+* `soar.*.slocum.#` - all messages relating to a slocum platform type
+
 ## Run Docs
 Run the command below and go to `http://127.0.0.1:5000`
 ```
@@ -22,8 +79,32 @@ Run the command below
 python3 -m unittest tests/test_schemas.py
 ```
 
-# Quick Links
+## Quick Links
 1. [Generated Swagger Docs (recommended to look at this)](https://git.noc.ac.uk/communications-backbone-system/backbone-message-format/-/blob/dev/project/soar/swagger.json)
 2. [Schema Fields Definitions](https://git.noc.ac.uk/communications-backbone-system/backbone-message-format/-/tree/dev/formats)
 3. [JSON Schema Examples](https://git.noc.ac.uk/communications-backbone-system/backbone-message-format/-/tree/dev/examples)
-4. Ongoing Project: [SoAR README.md](https://git.noc.ac.uk/communications-backbone-system/backbone-message-format/-/blob/dev/project/soar/SOAR_README.md)
\ No newline at end of file
+4. Ongoing Project: [SoAR README.md](https://git.noc.ac.uk/communications-backbone-system/backbone-message-format/-/blob/dev/project/soar/SOAR_README.md)
+
+## Installing 
+
+You can install the message formats as a package with pip
+
+```requirements.txt
+backbone-message-format @ git+https://git.noc.ac.uk/communications-backbone-system/backbone-message-format.git@v0.1.0#egg=backbone-message-format
+```
+
+### Importing the formats 
+
+```python
+import backbone_formats
+```
+
+### Importing the compiled schema 
+
+```python
+import importlib.resources
+import soar_schema
+import json
+with importlib.resources.open_text(soar_schema, "swagger.json") as file:
+    schema = json.load(file)
+```
\ No newline at end of file

__init__.py

@@ -4,6 +4,8 @@ from flask import Flask
 
 __all__ = [
     "formats",
+    "examples",
+    "project",
     "tests",
 ]
 app = Flask(__name__)

docker/Dev.Dockerfile

+FROM python:3.9.16-alpine
+
+WORKDIR /app
+
+COPY requirements-dev.txt requirements-dev.txt 
+
+RUN pip install -r requirements-dev.txt
\ No newline at end of file

docker/Dockerfile

+FROM python:3.9.16-alpine
+
+WORKDIR /app
+
+COPY . .
+
+RUN pip install -r requirements.txt
\ No newline at end of file

docker/docker-compose-test.yaml

+version: '3.8'
+services:
+  soar_flasgger_test:
+    build: 
+      context: ..
+      dockerfile: docker/Dev.Dockerfile
+    restart: unless-stopped
+    ports:
+      - "5000:5000"
+    container_name: soar_flasgger_test
+    environment:
+      - PYTHONDONTWRITEBYTECODE=1
+    volumes:
+      - ../:/app
+    command: "python3 -m unittest tests/test_schemas.py"
\ No newline at end of file

docker/docker-compose.yaml

+version: '3.8'
+services:
+  soar_flasgger:
+    build: 
+      context: ..
+      dockerfile: docker/Dockerfile
+    restart: unless-stopped
+    ports:
+      - "5000:5000"
+    container_name: soar_flasgger
+    environment:
+      - FLASK_HOST=0.0.0.0
+    volumes:
+      - ../:/app
+    command: "python generate_schema_config.py"
\ No newline at end of file

examples/autonomy_engine_adapter/acknowledgement.json

@@ -4,13 +4,14 @@
         "timestamp": "2022-11-16T00:00:00Z",
         "version": 1,
         "source": "hydrosurv_adapter",
-        "destination": "autonomy_engine",
+        "destination": "soar.hydrosurv.reav.reav1.from_platform.acknowledgement",
         "delivery_type": "publish",
         "encoded": false
     },
     "payload":{
         "message_type": "acknowledgement",
-        "acknowledged_message_ID": "11111111-7bc8-11ed-a1eb-0242ac999999",
-        "status": "c2_received"
+        "autonomy_engine_plan_ID": 1,
+        "platform_ID": "reav-x-1",
+        "approved": false
     }
 }
\ No newline at end of file

examples/autonomy_engine_adapter/mission_plan_AH1.json

@@ -4,7 +4,7 @@
     "timestamp": "2022-11-16T00:00:00Z",
     "version": 2,
     "source": "autonomy_engine",
-    "destination": "noc_c2",
+    "destination": "soar.noc.autosub.alr-52.to_platform.mission_plan",
     "delivery_type": "publish",
     "encoded": false
   },
@@ -15,39 +15,43 @@
     "plan": [
       {
         "action": "move",
-        "flight_style": "go to waypoint",
-        "latitude_waypoint": -3.237143188645706,
-        "longitude_waypoint": 52.37072283932642,
+        "start_point_latitude": -3.007143188645706,
+        "start_point_longitude": 50.37072283932642,
+        "target_waypoint_latitude": -3.237143188645706,
+        "target_waypoint_longitude": 52.37072283932642,
         "depth": 0.0,
         "activate_payload": false,
-        "send_environmental_data": false
+        "timeout": 200
       },
       {
         "action": "dive",
-        "flight_style": "dive to depth",
-        "latitude_waypoint": -3.237143188645706,
-        "longitude_waypoint": 52.37072283932642,
+        "start_point_latitude": -3.237143188645706,
+        "start_point_longitude": 52.37072283932642,
+        "target_waypoint_latitude": -3.337143188645706,
+        "target_waypoint_longitude": 56.37072283932642,
         "altitude": 10,
         "activate_payload": false,
-        "send_environmental_data": false
+        "timeout": 200
       },
       {
         "action": "scanline",
-        "flight_style": "go to waypoint",
-        "latitude_waypoint": -0.237143188645706,
-        "longitude_waypoint": 60.00000000000000,
+        "start_point_latitude": -3.337143188645706,
+        "start_point_longitude": 56.37072283932642,
+        "target_waypoint_latitude": -0.237143188645706,
+        "target_waypoint_longitude": 60.00000000000000,
         "altitude": 10,
         "activate_payload": true,
-        "send_environmental_data": false
+        "timeout": 200
       },
       {
         "action": "climb",
-        "flight_style": "go to surface",
-        "latitude_waypoint": -0.237143188645706,
-        "longitude_waypoint": 52.37072283932642,
+        "start_point_latitude": -0.237143188645706,
+        "start_point_longitude": 60.00000000000000,
+        "target_waypoint_latitude": -0.237143188645706,
+        "target_waypoint_longitude": 52.37072283932642,
         "depth": 0.0,
         "activate_payload": false,
-        "send_environmental_data": true
+        "timeout": 200
       }
     ]
   }

examples/autonomy_engine_adapter/mission_plan_ECOSUB.json

@@ -4,50 +4,54 @@
     "timestamp": "2022-11-16T00:00:00Z",
     "version": 2,
     "source": "autonomy_engine",
-    "destination": "ecosub_c2",
+    "destination": "soar.planetocean.ecosub.ecosub-1.to_platform.mission_plan",
     "delivery_type": "publish",
     "encoded": false
   },
   "payload":{
     "message_type": "mission_plan",
-    "platform_ID": "ecosub-5",
+    "platform_ID": "ecosub-2",
     "autonomy_engine_plan_ID": 1,
     "plan": [
       {
         "action": "move",
-        "flight_style": "go to waypoint",
-        "latitude_waypoint": -3.237143188645706,
-        "longitude_waypoint": 52.37072283932642,
+        "start_point_latitude": -3.007143188645706,
+        "start_point_longitude": 50.37072283932642,
+        "target_waypoint_latitude": -3.237143188645706,
+        "target_waypoint_longitude": 52.37072283932642,
         "depth": 0.0,
         "activate_payload": false,
-        "send_environmental_data": false
+        "timeout": 300
       },
       {
         "action": "dive",
-        "flight_style": "dive to depth",
-        "latitude_waypoint": -3.237143188645706,
-        "longitude_waypoint": 52.37072283932642,
+        "start_point_latitude": -3.237143188645706,
+        "start_point_longitude": 52.37072283932642,
+        "target_waypoint_latitude": -3.337143188645706,
+        "target_waypoint_longitude": 56.37072283932642,
         "altitude": 10,
         "activate_payload": false,
-        "send_environmental_data": false
+        "timeout": 300
       },
       {
         "action": "scanline",
-        "flight_style": "go to waypoint",
-        "latitude_waypoint": -0.237143188645706,
-        "longitude_waypoint": 60.00000000000000,
+        "start_point_latitude": -3.337143188645706,
+        "start_point_longitude": 56.37072283932642,
+        "target_waypoint_latitude": -0.237143188645706,
+        "target_waypoint_longitude": 60.00000000000000,
         "altitude": 10,
         "activate_payload": true,
-        "send_environmental_data": false
+        "timeout": 300
       },
       {
         "action": "climb",
-        "flight_style": "go to surface",
-        "latitude_waypoint": -0.237143188645706,
-        "longitude_waypoint": 52.37072283932642,
-        "depth": 0,
+        "start_point_latitude": -0.237143188645706,
+        "start_point_longitude": 60.00000000000000,
+        "target_waypoint_latitude": -0.237143188645706,
+        "target_waypoint_longitude": 52.37072283932642,
+        "depth": 0.0,
         "activate_payload": false,
-        "send_environmental_data": true
+        "timeout": 300
       }
     ]
   }

examples/autonomy_engine_adapter/mission_plan_HYDROSURV.json

@@ -4,7 +4,7 @@
     "timestamp": "2022-11-16T00:00:00Z",
     "version": 2,
     "source": "autonomy_engine",
-    "destination": "hydrosurv_adapter",
+    "destination": "soar.hydrosurv.reav-60.reav-60-1.to_platform.mission_plan",
     "delivery_type": "publish",
     "encoded": false
   },
@@ -14,16 +14,16 @@
     "autonomy_engine_plan_ID": 1,
     "plan": [
       {
-        "latitude_waypoint": -4.187143188645706,
-        "longitude_waypoint": 50.37072283932642
+        "target_waypoint_latitude": -4.187143188645706,
+        "target_waypoint_longitude": 50.37072283932642
       },
       {
-        "latitude_waypoint": -3.187143188645706,
-        "longitude_waypoint": 51.37072283932642
+        "target_waypoint_latitude": -3.187143188645706,
+        "target_waypoint_longitude": 51.37072283932642
       },
       {
-        "latitude_waypoint": -3.237143188645706,
-        "longitude_waypoint": 52.37072283932642
+        "target_waypoint_latitude": -3.237143188645706,
+        "target_waypoint_longitude": 52.37072283932642
       }
     ]
   }

examples/autonomy_engine_adapter/mission_plan_encoded.json

+{
+  "header": {  
+    "message_ID": "t1237003c-0000-11aa-a1eb-bvcdfghjgfdd",
+    "timestamp": "2023-03-16T00:00:00Z",
+    "version": 2,
+    "source": "autonomy_engine",
+    "destination": "soar.hermes.autosub.alr-52.to_platform.mission_plan_encoded",
+    "delivery_type": "publish",
+    "encoded": true
+  },
+  "payload": {
+    "message_type": "mission_plan_encoded",
+    "data": "SDQke4uwyP/YQQAgAhA2AND/nu8nvQAAAAAAAAAACtejPa5HHUGkcBlB/tRYQW3nez5HWYFBAAD+zwBySUAAAADVek72v1N2lUQAAAABAQAAAAAAAAAAAAAAAAAAIAAAAQAAAAAAAAAA9P2cP166ab+9cg==",
+    "file_name": "ah1_03837434286438.sbd",
+    "mime_type": "application/gzip",
+    "is_binary": true
+   }
+}   

examples/autonomy_engine_adapter/observation_encoded.json

+{
+  "header": {  
+    "message_ID": "e1237003c-0000-11aa-a1eb-bvcdfghjgfdd",
+    "timestamp": "2023-03-16T00:00:00Z",
+    "version": 2,
+    "source": "hermes.ecosub2",
+    "destination": "soar.planet-ocean.ecosub.ecosub-2.from_platform.observation_encoded",
+    "delivery_type": "publish",
+    "encoded": true
+  },
+  "payload": {
+    "message_type": "observation_encoded",
+    "data": "T1N2lUQAAAABAQAAAAAAAAAAAAAAAAAAIAAAAQAAAAAAAAAA9P2cP166ab+9cg==",
+    "file_name": "ecosub1_200002114.bin",
+    "mime_type": "application/gzip",
+    "is_binary": true
+   }
+}   

examples/autonomy_engine_adapter/planning_configuration.json

@@ -3,13 +3,13 @@
     "message_ID": "b427003c-0000-11aa-a1eb-bvcdfghjgfdd",
     "timestamp": "2022-11-16T00:00:00Z",
     "version": 2,
-    "source": "ecosub_c2",
-    "destination": "autonomy_engine",
+    "source": "gui",
+    "destination": "soar.all.all.all.from_platform.planning_configuration",
     "delivery_type": "publish",
     "encoded": false
   },
   "payload":{
-    "message_type": "platform_status",
+    "message_type": "planning_configuration",
     "planning_config_ID": 3,
     "exclusion_zones": [
       {
@@ -45,13 +45,12 @@
           {
             "model": "reav",
             "platform_ID": "reav-60-1",
-            "serial": "reav-60",
             "emergency": {
               "additional_data": {},
-              "latitude_waypoint": -7.432,
-              "longitude_waypoint": 50.365,
+              "target_waypoint_latitude": -7.432,
+              "target_waypoint_longitude": 50.365,
               "safe_command": "go_home",
-              "target_depth": 10
+              "target_depth": 10.0
             },
             "max_velocity": 0.9,
             "min_altitude": 15.2,
@@ -90,12 +89,11 @@
         "platforms": [
           {
             "platform_ID": "ecosub-1",
-            "serial": "ecosub-2",
             "model": "ecosub",
             "emergency": {
               "additional_data": {},
-              "latitude_waypoint": -7.432,
-              "longitude_waypoint": 50.365,
+              "target_waypoint_latitude": -7.432,
+              "target_waypoint_longitude": 50.365,
               "safe_command": "go_home",
               "target_depth": 10.0
             },
@@ -109,12 +107,11 @@
           },
           {
             "platform_ID": "ecosub-2",
-            "serial": "ecosubxyz-5",
             "model": "ecosub",
             "emergency": {
               "additional_data": {},
-              "latitude_waypoint": -0.432,
-              "longitude_waypoint": 20.365,
+              "target_waypoint_latitude": -0.432,
+              "target_waypoint_longitude": 20.365,
               "safe_command": "go_home",
               "target_depth": 0.0
             },
@@ -126,8 +123,7 @@
                 "swath_width": 10
             }
           }
-        ],
-        "region_of_interest": {}
+        ]
       },
       {
         "squad_ID": 3,
@@ -137,12 +133,11 @@
         "platforms": [
           {
             "platform_ID": "ah-1",
-            "serial": "autosubhover-1",
             "model": "autosub",
             "emergency": {
               "additional_data": {},
-              "latitude_waypoint": 20.432,
-              "longitude_waypoint": 50.365,
+              "target_waypoint_latitude": 20.432,
+              "target_waypoint_longitude": 50.365,
               "safe_command": "abort_now",
               "target_depth": 0.0
             },
@@ -153,8 +148,7 @@
                 "scan_type": "MBES"
             }
           }
-        ],
-        "region_of_interest": {}
+        ]
       }
     ]
   }

examples/autonomy_engine_adapter/platform_status-from_usbl_example.json

@@ -4,7 +4,7 @@
         "timestamp": "2022-11-16T00:00:00Z",
         "version": 2,
         "source": "hermes",
-        "destination": "autonomy_engine",
+        "destination": "soar.planet-ocean.ecosub.ecosub-5.from_platform.platform_status",
         "delivery_type": "publish",
         "encoded": false
     },
@@ -15,7 +15,7 @@
         "status_source": "usbl",
         "latitude": 178.2,
         "longitude": -10.122,
-        "depth": 50,
-        "altitude": 20
+        "depth": 50.0,
+        "altitude": 20.0
     }
 }

examples/autonomy_engine_adapter/platform_status.json

@@ -4,7 +4,7 @@
         "timestamp": "2022-11-16T00:00:00Z",
         "version": 2,
         "source": "ecosub_c2",
-        "destination": "autonomy_engine",
+        "destination": "soar.planet-ocean.ecosub.ecosub-2.from_platform.platform_status",
         "delivery_type": "publish",
         "encoded": false
     },
@@ -15,30 +15,28 @@
         "status_source": "onboard_platform",
         "autonomy_engine_plan_ID": 1,
         "battery_remaining_capacity": 80.2,
-        "active": true,
         "platform_state": "ABORT",
-        "mission_action_ID": 1,
+        "mission_plan_ID": 1,
         "mission_track_ID": 4,
         "latitude": 178.2,
         "longitude": -10.122,
-        "depth": 50,
+        "depth": 50.0,
         "altitude": 20,
         "heading": 90.0,
-        "health_status": "Warning",
-        "localisation_error": 0,
+        "health_status": false,
+        "localisation_north_error": 0,
+        "localisation_east_error": 0,
         "usbl_fix_seconds_ago": 0,
         "range_to_go": 124.3,
         "sensor_config": {
-            "sensor_ID": 22,
-            "serial": "sidescan-2x",    
+            "sensor_serial": "sidescan-2x",    
             "sensor_on": true,
             "additional_data": {
                 "whiskers_on": true
             }      
         },
-        "current_pitch": -4,
         "speed_over_ground": 4.3,
         "thrust_applied": 124.3,
-        "water_current_velocity": 124.3
+        "water_current_velocity": "124.3NE"
     }
 }