Journal#

Background#

The gateways generally use exchange features to carry enough information to reliably route order updates back to the origin (where the request was created). This is generally achieved by packing origin information into the exchange ClOrdID field.

However, some use-cases necessitates that a gateway can restart and enrich exchange downloaded orders with additional information persisted into a database.

For example, the FIX Bridge requries the clients to supply their own FIX ClOrdID field. This is propagated to the gateway using the routing_id field (Roq’s C++ API).

This introduces persistence in the hot path and will clearly affect latency.

The Journal solution is one way to reduce latency and at the same time reduce the probability of data loss.

Design#

  1. A Gateway may fetch working order history from the database at startup time. This happens before any orders are downloaded from the exchange.

  2. A Gateway may multicast OrderAck and OrderUpdate messages. Several instances of Journal will listen to multicast and receive the messages.

  3. Each Journal instance will appened messages to its own local cache file (useful for replay into database after restart).

  4. Each Journal instance may echo the message back to the peer IP address using UDP (port is contained in the received message).

  5. Each Journal instance will upload messages to a database.

Implementation#

It is not important to persist all OrderAck and OrderUpdate messages.

  • It is important to persist the OrderAck which is the Gateway’s response to a connected client that the order request has passed internal checks and is ready to forward to exchange. This OrderAck may have additional information to persist such as routing_id.

  • It is important to persist the OrderAck coming back from the exchange as it may be a reject. This may allow the Journal to mark an order “final”.

  • It is important to persist any OrderUpdate which positively changes important order attributes, routing_id in particular. This is often the case when the max_accepted_version is increased.

Constraints#

  • The gateway may not always be able to download recent working order history if the Journal(s) have not yet been able to persist these into the database. The guarantee is only to perform a datasync to the local filesystem. Saving to database happens asynchronously and may be buffered.

Limitations#

  • The gateway does not yet hold order requests in a queue. This is on the TODO list.