How it works#
Client strategies must manage the order life-time using a single
This is a unique integer used to identify or reference the same original order.
In particular, these update events will contain the same
Also, the following order actions will use the same
The gateway will communicate the highest previously seen
order_id during the
The client is then simply expected to use
max_order_id + 1 for the next
Pros and Cons#
Makes it easy (and efficient) to reference local order state using a simple integer lookup
Does not need to know about exchange-specific conventions for managing order ID’s
Strategy implementation is easily transferrable and can be used with any gateway
Can not pipeline order actions (must wait for an ack before a subsequent action can be requested)
Can freely use the exchange’s client order ID to encode additional information about e.g. originating strategy (
user_id) and its local order reference (
Can decode the value from exchange’s client order ID and route ack, order updates, and fills to the originating strategy without using a database persist such information
The client order ID is not the only pass-through field which provides the gateway an opportunity to avoid database, sometimes other order creation fields can provide this functionality
Can “multiplex” requests and updates from/to any number of strategies using a single account
This only works when the exchange allows free-form client order ID’s (some exchanges only allow very specific ID’s such as UUID)
There is a small risk of re-using a client’s
order_id’s if the exchange only supports the download of working orders and the gateway is restarted. (There is a requirement for the client to use
max(local_max_order_id, DownloadEnd.max_order_id)which would prevent re-use as long as the strategy is not restarted. However, the gateway can not enforce this.)
The Roq gateways have more functionality than traditional order routing gateways. In particular (for the context being discussed here), they are designed to support
object caching so clients can reconnect at any time and quickly catch up to current state with both market and account data, then simply join the live stream of updates
multiplexing orders to a single account originated from any number of strategies
Some exchanges support a batch of multiple order requests.
Benefits may include
More relaxed rate-limiting
Atomic processing of order requests
The client dispatcher (
roq::client::Dispatcher) accepts an
is_last boolean flag.
The default is
true as an instruction for the gateway to immediately process the request.
However, setting this flag to
false will allow the gateway to optionally withhold the
request and await another later request having the flag set to
When the gateway has received all requests, it may elect to send all (or some of) the requests using the most optimal exchange API.
There is currently no generic implementation to support gateway batching. This is partially due to exchange API’s often being very custom and partially due to the very complex error handling.
Do not assume batching is implemented for the gateway you’re using
Do not assume requests are withheld by the gateway if you choose
Always read the reference documentation carefully to see if there is support
When creating a new order (using
routing_id string may
This field is pass-through and meant to capture an original order ID when routing orders from
If present, the gateway may encode the routing ID in the client order ID (or another free-form
All update events (see list above) will have this field present making it possible to communicate
the original order ID back to the other system.
What it doesn’t solve
The introduction of further client order ID’s generated by the originating system when modifying order attributes
Pipelining, i.e. when the originating system sends multiple order modifications without waiting for ack
When free-form text:
Roq will use 20 chars (base64 encoded) to carry important information about user_id, order_id and a nanosecond precision timestamp. This information is used to enable automatic download and recovery
The routing_id field will be appended. This may be an issue if (20 + len(routing_id)) exceeds any limit the exchange may have
Otherwise following exchange conventions with no option to have automatic download and recovery. For this case it is recommended to auto-cancel orders on exchange following a broken connection
We do not currently allow ModifyOrder to supply a new routing_id. There are multiple reasons
There is an inherent race between Ack and ExecutionReport, especially when the underlying communication happens on multiple connections, but also if fills are reported while the modification request is in-flight
Managing a chain of Ack’s (and the life-time of each routing_id) is very dependent on exchange sending back all acks, in the right order
The order_template is not encoded in any field sent to the exchange OrderUpdate can therefore only inform of this field until the gateway is restarted or download is initiated
Orders are cached per stream – but not yet cleared
Download (by gateway)#
When a gateway stream, supporting orders, experiences a disconnect, it will immediately try to reconnect. When connected, the download phase normally begins. This is what happens
All orders (known to the gateway) are internally marked as stale
Working orders are downloaded from the exchange
Orders that can be downloaded will be communicated with
When download completes, and before the stream becomes ready, remaining orders will be communicated with
Stale will only be communicated once, snapshot may be communicated following every reconnect.
During a disconnect, orders could be potentially be (partially or completely) filled or auto-canceled by the exchange. The procedure mentioned above helps us identify
current status for working orders, and
those orders which are in an unknown state
Since we can’t know if an order was completed or canceled during disconnect, it is necessary to use other means
to synchronize e.g. positions.
For that there’s potentially
A final solution might be to require human intervention when seeing stale order updates.