Communications Backbone

Communications Backbone by C2 Team (NOC)

DJ prototype

I did this in freedom-fire-day so it doesn't currently follow the flask template.

It's also a bit weird because it writes to a local ignored clients.json file instead of using a database. I did this as a placeholder because we've not yet decided what the infrastructure looks like.

Data flow

• Client A sends to client-a-outbox (or POSTs to API /send - not yet implemented)
• Messages are forwarded from client-a-outbox to soar-publish
• Messages are published from soar-publish with the given topic read from the message
• Subscribers listen to for messages
• Subscription is delivered to client-b-inbox
• Client B reads from `client-b-inbox (or GETs from API /receive)

There is a parallel flow when a client sends to client-a-notify in which case the messages are delivered through the broadcast exchange to all clients client-x-inbox.

Auth placeholder

As a proxy for proper authentication, when you post a client a random secret is returned in the response. To send to / receive from the bus you then call the API with the client_id and secret and it checks they match. The client_id determines which queues it reads from.

Subsequent requests to the client endpoint return the client_id but not the secret.

Setup

pipenv install 
pipenv shell 

Running

RabbitMQ

docker run --rm -p 5672:5672 -d --hostname rmq --name rmq rabbitmq:management

API

In a pipenv shell

python api.py 

Create some clients

POST to http://localhost:3000/clients

Event bus

In a pipenv shell

python soar_bus.py

Send / Receive directly

In a pipenv shell

# Send a message 
python client_send.py noc-c2-outbox 'soar.noc.slocum.something' from noc-c2

# Receive messages  
python client_read.py noc-sfmc-inbox

Receive via API

As a placeholder for proper authentication you post the client_id and secret and it checks that a client with that id exists and that the secret matches before allowing the request.

This should be replaced with a proper auth layer.

GET http://localhost:5000/receive?client_id=[client_id]&secret=[secret]

Components

• soar_bus.py - Run all the components threaded based on existing clients
• soar_forward.py - Listen for messages on queue A and forward messages to queue B
• soar_publish.py - Listen for messages on queue A and publish on exchange B
• soar_broadcast.py - Listen for messages on queue A and broadcast on exchange B
• soar_subscribe.py - Create subscriptions to both the publish and broadcast exchange - deliver to queue A (I think this should probably be 2 separate functions to keep things nice and simple)
• soar_push.py - Not yet implemented - Listen for messages on queue A and POST to the client's webhook URL