Communications Backbone

Communications Backbone by C2 Team (NOC)

Data flow

• Client A sends to client-a-outbox (or POSTs to API /send - not yet implemented)
• Messages are forwarded from client-a-outbox to soar-publish
• Messages are published from soar-publish with the given topic read from the message
• Subscribers listen to for messages
• Subscription is delivered to client-b-inbox
• Client B reads from `client-b-inbox (or GETs from API /receive)

There is a parallel flow when a client sends to client-a-notify in which case the messages are delivered through the broadcast exchange to all clients client-x-inbox.

Auth placeholder

As a proxy for proper authentication, when you post a client a random secret is returned in the response. To send to / receive from the bus you then call the API with the client_id and secret and it checks they match. The client_id determines which queues it reads from.

Subsequent requests to the client endpoint return the client_id but not the secret.

Testing

Current coverage:

• API: yes
• Pika RabbitMQ implementation: no

pytest

Running via docker-compose

Using docker-compose will mean that everything is setup automatically, this includes the rabbitmq container, the backbone API, and the backbone bus. The run-compose.sh script has been provided to simplify this even further - all you have to do is set whatever env vars you need in the .env file and then run ./run-compose.sh (the defaults in .env are fine for local dev work, but ones not labelled optional will need setting in a production setting). The env vars are:

• DATA_DIR - Where to mount the volume of the API container on your local system. This defaults to the result of pwd, which should be within the communications-backbone repo
• SOAR_TOKEN_LIFETIME (Optional) - The number of hours until a newly created token expires
• SOAR_TOKEN_SECRET (Optional) - A secret key used to encrypt/decrypt token data. If specified the value should be set using TokenModel.getKey()

Running manually

Setup

In a virtual environment

pip install -r requirements-dev.txt  

RabbitMQ

docker run --rm -p 5672:5672 -d --hostname rmq --name rmq rabbitmq:management

API

python api.py 

Event bus

python soar_bus.py

Usage

To use the backbone you have to create a client. The client id and name should be unique but human readable. A client secret is returned by the API and client credential grants using this client_id and secret are used to authenticate client application connections.

Create some clients

POST to http://localhost:3000/clients

Send / Receive directly

# Send a message 
python client_send.py noc-c2-outbox 'soar.noc.slocum.something' from noc-c2

# Receive messages  
python client_read.py noc-sfmc-inbox

Receive via API

As a placeholder for proper authentication you post the client_id and secret and it checks that a client with that id exists and that the secret matches before allowing the request.

This should be replaced with a proper auth layer.

GET http://localhost:5000/receive?client_id=[client_id]&secret=[secret]