GitLab

Build wrapper to separate RabbitMQ from existing prototype by DJ !3 (merged)

changed milestone to %SoAR-2023W01

added Partner::NOC label

added Status::Sprint Backlog label

added Status::In Progress label and removed Status::Sprint Backlog label

changed title from Separate message queue protocol into wrapper(s) to Separate message queue protocol into wrapper(s) & structure

assigned to @jamki

changed title from Separate message queue protocol into wrapper(s) & structure to Separate message queue protocol into wrapper(s)

added Status::Sprint Backlog label and removed Status::In Progress label

created merge request !4 (closed) to address this issue

mentioned in merge request !4 (closed)

Had a chat with Dan about this one. Gonna be trying to write a bunch of super generic/low-level functions that the current soar_<whatever>.py stuff can use. Some ideas (feel free to add/remove stuff from it anyone ):

• readFromQueue
• writeToQueue
• publish
• subscribe
• broadcast
• acknowledge

Looks good!

• would we need a "read/write from/to queue" function though?
• maybe we could swap out accordingly to a receive / listen [e.g. zero mq works through send and receive function calls mainly]. Just an idea! I have no stand on this

• I think so, if we want to pop something directly onto a queue or read directly from it, we'd need to have that explicitly. Then the pub/sub is for interacting with the exchanges! soar_forward.py for example currently directly sends to a queue.
• zmq seems to declare itself as can do most messaging models inc. pub/sub and send/receive, so they should have those in common!

To some extent this is about the difference between the http and websockets implementation. When you do a GET /receive you can't have a constantly listening process. You're creating a listener to read messages from the client-inbox and then closing that connection. When we have a websocket we'll have a thread listening to the client-inbox queue and sending as soon as a message is received.

We always need a way to write to a queue to send from the client to the client-outbox - even if we don't use forward.

The way I see it the point of this library is to abstract away the queue provider native functions into a common interface. To some extent AMQP is doing that already but the other AMQP providers are not the things we'd likely move to as alternatives.

If we moved to something that abstracted away the queues entirely like MQTT / Amazon SNS and just handles pub/sub then potentially we could still provide the same interface - writing to a queue could be equivalent to publishing to a very specific topic per client.

The reason I didn't go down the treat-everything-as-pub/sub route is that delivering a message from the client to the backbone becomes a more complicated interaction and that's where we're likely to have limited bandwidth/unstable connections. I may be wrong but I think by just making that a simple write to queue it's more likely to succeed more often.

added Status::In Progress label and removed Status::Sprint Backlog label

mentioned in merge request !5 (merged)

added Status::In Review label and removed Status::In Progress label

mentioned in commit 298299d5

closed via merge request !5 (merged)