Merge branch '24-fix-readme-make-public-share-with-partners' into 'dev'

Resolve "Fix readme, make public, share with partners"

Closes #19 and #24

See merge request !12

LICENSE

 MIT License
 
-Copyright (c) 2022 National Oceanography Centre CLG
+Copyright (c) 2023 National Oceanography Centre
 
 Permission is hereby granted, free of charge, to any person obtaining a copy
 of this software and associated documentation files (the "Software"), to deal

README.md

@@ -2,84 +2,155 @@
 
 Communications Backbone by C2 Team (NOC)
 
+### Infrastructure 
+
+The backbone has 3 runtime components: 
+
+- RabbitMQ - run as a docker container 
+- The bus which handles the delivery of messages between queues 
+- The API which provides an interface to read from and write to the queues
+
 ### Data flow 
 
-- Client A sends to `client-a-outbox` (or POSTs to API /send - not yet implemented)
+- Client A sends to `client-a-outbox` (by POSTing to API /send)
 - Messages are forwarded from `client-a-outbox` to `soar-publish` 
 - Messages are published from `soar-publish` with the given topic read from the message 
 - Subscribers listen to for messages 
-- Subscription is delivered to `client-b-inbox` 
-- Client B reads from `client-b-inbox (or GETs from API /receive)
-
-There is a parallel flow when a client sends to `client-a-notify` in which case the 
-messages are delivered through the broadcast exchange to all clients `client-x-inbox`.
+- Subscription is delivered to `client-b-inbox` (if the client subscription matches the message topic) 
+- Client B reads from `client-b-inbox` (by GETting from API /receive)
 
-### Auth placeholder 
+There is a parallel flow when a client sends to `client-a-broadcast` (by POSTing to /notify). 
+In this case the messages are delivered through the broadcast exchange to all clients `client-x-inbox`.
 
-As a proxy for proper authentication, when you post a client a random secret is 
-returned in the response. To send to / receive from the bus you then call the API
-with the client_id and secret and it checks they match. The client_id determines 
-which queues it reads from. 
+### Prerequisites 
 
-Subsequent requests to the client endpoint return the client_id but not the secret.
-
-### Testing 
-
-Current coverage: 
+- Python >= 3.8
+- A virtual environment manager - virtualenv, venv or pipenv 
+- Docker and docker-compose 
 
-- API: yes 
-- Pika RabbitMQ implementation: no
+### Running via docker-compose
 
 ```
-pytest
+run-compose.sh
 ```
 
-### Running via docker-compose
-
 Using `docker-compose` will mean that everything is setup automatically, this includes the `rabbitmq` container, the backbone API, and the backbone bus. The `run-compose.sh` script has been provided to simplify this even further - all you have to do is set whatever env vars you need in the `.env` file and then run `./run-compose.sh` (the defaults in `.env` are fine for local dev work, but ones not labelled `optional` will need setting in a production setting). The env vars are:
 
 - `DATA_DIR` - Where to mount the volume of the API container on your local system. This defaults to the result of `pwd`, which should be within the `communications-backbone` repo
 - `SOAR_TOKEN_LIFETIME` (Optional) - The number of hours until a newly created token expires
 - `SOAR_TOKEN_SECRET` (Optional) - A secret key used to encrypt/decrypt token data. If specified the value should be set using TokenModel.getKey()
 
-### Running manually
+### Running the bus and API natively (without docker)
 
 #### Setup 
 
-In a virtual environment
+We recommend using some form of python virtual environment to maintain a consistent 
+python version and ring-fence the package management.
+
+In your virtual environment: 
 
 ```
 pip install -r requirements-dev.txt  
 ```
 
+This installs both the development and runtime dependencies for the project.
+
+### Testing 
+
+Current coverage: 
+
+- API: yes 
+- Pika RabbitMQ implementation: no
+
+In your virtual environment: 
+
+```
+pytest
+```
+
 #### RabbitMQ 
 
 `docker run --rm -p 5672:5672 -d --hostname rmq --name rmq rabbitmq:management`
 
 #### API 
 
+In your virtual environment:
+
 ```
 python api.py 
 ```
 
 #### Event bus 
 
+In your virtual environment: 
+
 ``` 
 python soar_bus.py
 ```
 
+At present the soar bus creates the clients defined in the API when it starts 
+but does not monitor for new clients so if you create a new client you will 
+need to restart the bus. This will be fixed in a later release.
+
+### Client Adapters 
+
+The intended use of the backbone is that multiple clients connect to the backbone 
+using adapters. An adapter handles: 
+
+- Authentication 
+- Sending and receiving over the backbone API 
+- Transforming messages between local formats and the backbone message formats
+
+We have implemented the following template client adapters: 
+
+- [Javascript](https://git.noc.ac.uk/communications-backbone-system/backbone-adapter-javascript)
+- [Python](https://git.noc.ac.uk/communications-backbone-system/backbone-adapter-python)
+
+If you need to port the adapter to another language please contact us. 
+
+The adapters can be installed as packages and sub-classed as required for your 
+client application.
+
+For install and usage instructions for the adapters see the READMEs. 
+
 ### Usage
 
-To use the backbone you have to create a client. 
-The client id and name should be unique but human 
-readable. A client secret is returned by the API 
-and client credential grants using this client_id 
-and secret are used to authenticate client 
-application connections.  
+- Run the backbone 
+- Create some clients
+- Restart the soar_bus service 
+  - `cd docker && docker-compose restart soar_bus` 
+- Save your client credentials 
+- Test reading and writing directly to the queues
+- Create an adapter 
+- Test sending and receiving via the adapter    
 
 #### Create some clients 
 
-`POST` to `http://localhost:3000/clients` 
+`POST` to `http://localhost:8087/client`
+
+The POST body should be JSON. 
+
+```json
+{ 
+  "client_id": "noc-c2",
+  "client_name": "NOC C2", 
+  "subscription": "soar.*.noc.#"
+}
+```
+
+- `client_id` - a project unique human readable name 
+- `client_name` - how to refer to that client on screen 
+- `subscription` - the topic pattern identifying the messages you want to receive 
+
+(`*` = single word wildcard, `#` = multi-word wildcard)
+
+The response from the post contains your client `secret`. This is only displayed once. 
+
+Subsequent GETS to `/client` or `/client/[client-id]` will not return the secret. 
+
+You should save the response as `soar-config.json` adding the additional `api` field 
+to specify the root URL of the API you're connecting to 
+(eg http://localhost:8087 for local development)
 
 #### Send / Receive directly 
 
@@ -93,12 +164,40 @@ python client_send.py noc-c2-outbox 'soar.noc.slocum.something' from noc-c2
 python client_read.py noc-sfmc-inbox
 ```
 
-#### Receive via API
+These scripts bypass authentication reading and writing directly to rabbitmq.
+
+#### Authentication 
+
+Authentication is handled by a client credentials grant which is a GET 
+request to `/token?client_id=[client-id]&secret=[secret]`
+
+The response includes a token and this token should be included as an 
+authorization header in requests to the API: 
+
+```json
+{
+  "Authorization": "Bearer [token]"
+}
+```
+
+#### Send and Receive via API
+
+Once you have a bearer token you can make requests to send and receive 
+messages via the API: 
+
+- GET `/receive?max_messages=[X]` - gets all messages in the queue 
+  - the default value of `max_messages` if not specified is 10
+- POST `/send` - publish a message to a given topic 
+- POST `/notify` - broadcast a message to all clients 
+
+Supplying the token as an Authorization header as described above.
+
+#### Send and Receive using adapters 
 
-As a placeholder for proper authentication you post the `client_id` and 
-`secret` and it checks that a client with that id exists and that the 
-secret matches before allowing the request. 
+If you have implemented one of the adapter templates then authentication 
+is handled for you.
 
-This should be replaced with a proper auth layer. 
+You will need to create a set of client credentials as above using the 
+POST `/client` endpoint and save the response. 
 
-`GET http://localhost:5000/receive?client_id=[client_id]&secret=[secret]`
+Then you pass your credentials when you create your adapter instance. 
\ No newline at end of file