Journal¶
Background¶
The gateways generally use exchange features to carry enough information to reliably route order updates back to the origin (where the request was created). This is generally achieved by packing origin information into the exchange ClOrdID field.
However, some use-cases necessitates that a gateway can restart and enrich exchange downloaded orders with additional information persisted into a database.
For example, the FIX Bridge requries the clients to supply their own FIX ClOrdID
field.
This is propagated to the gateway using the routing_id
field (Roq’s C++ API).
This introduces persistence in the hot path and will clearly affect latency.
The Journal solution is one way to reduce latency and at the same time reduce the probability of data loss.
Design¶

A Gateway may fetch working order history from the database at startup time. This happens before any orders are downloaded from the exchange.
A Gateway may multicast
OrderAck
andOrderUpdate
messages. Several instances of Journal will listen to multicast and receive the messages.Each Journal instance will appened messages to its own local cache file (useful for replay into database after restart).
Each Journal instance may echo the message back to the peer IP address using UDP (port is contained in the received message).
Each Journal instance will upload messages to a database.
Implementation¶
It is not important to persist all OrderAck
and OrderUpdate
messages.
It is important to persist the
OrderAck
which is the Gateway’s response to a connected client that the order request has passed internal checks and is ready to forward to exchange. ThisOrderAck
may have additional information to persist such asrouting_id
.It is important to persist the
OrderAck
coming back from the exchange as it may be a reject. This may allow the Journal to mark an order “final”.It is important to persist any
OrderUpdate
which positively changes important order attributes,routing_id
in particular. This is often the case when themax_accepted_version
is increased.
Constraints¶
The gateway may not always be able to download recent working order history if the Journal(s) have not yet been able to persist these into the database. The guarantee is only to perform a
datasync
to the local filesystem. Saving to database happens asynchronously and may be buffered.
Limitations¶
The gateway does not yet hold order requests in a queue. This is on the TODO list.