Order Management#

How it works#

Client strategies must manage the order life-time using a single order_id. This is a unique integer used to identify or reference the same original order.

In particular, these update events will contain the same order_id

Also, the following order actions will use the same order_id

The gateway will communicate the highest previously seen order_id during the download phase

roq::DownloadEnd has the max_order_id field

The client is then simply expected to use max_order_id + 1 for the next order_id.

Pros and Cons#

Client#

Pros

Makes it easy (and efficient) to reference local order state using a simple integer lookup
Does not need to know about exchange-specific conventions for managing order ID’s
Strategy implementation is easily transferrable and can be used with any gateway

Cons

Can not pipeline order actions (must wait for an ack before a subsequent action can be requested)

Gateway#

Pros

Can freely use the exchange’s client order ID to encode additional information about e.g. originating strategy (user_id) and its local order reference (order_id)
Can decode the value from exchange’s client order ID and route ack, order updates, and fills to the originating strategy without using a database persist such information
The client order ID is not the only pass-through field which provides the gateway an opportunity to avoid database, sometimes other order creation fields can provide this functionality
Can “multiplex” requests and updates from/to any number of strategies using a single account

Cons

This only works when the exchange allows free-form client order ID’s (some exchanges only allow very specific ID’s such as UUID)
There is a small risk of re-using a client’s order_id’s if the exchange only supports the download of working orders and the gateway is restarted. (There is a requirement for the client to use max(local_max_order_id, DownloadEnd.max_order_id) which would prevent re-use as long as the strategy is not restarted. However, the gateway can not enforce this.)

Note

The Roq gateways have more functionality than traditional order routing gateways. In particular (for the context being discussed here), they are designed to support

object caching so clients can reconnect at any time and quickly catch up to current state with both market and account data, then simply join the live stream of updates
multiplexing orders to a single account originated from any number of strategies

Batching#

Some exchanges support a batch of multiple order requests.

Benefits may include

More relaxed rate-limiting
Atomic processing of order requests

The client dispatcher (roq::client::Dispatcher) accepts an is_last boolean flag.

The default is true as an instruction for the gateway to immediately process the request.

However, setting this flag to false will allow the gateway to optionally withhold the request and await another later request having the flag set to true.

When the gateway has received all requests, it may elect to send all (or some of) the requests using the most optimal exchange API.

Warning

There is currently no generic implementation to support gateway batching. This is partially due to exchange API’s often being very custom and partially due to the very complex error handling.

In short

Do not assume batching is implemented for the gateway you’re using
Do not assume requests are withheld by the gateway if you choose is_last=true
Always read the reference documentation carefully to see if there is support

Routing#

When creating a new order (using roq::CreateOrder), a routing_id string may be supplied. This field is pass-through and meant to capture an original order ID when routing orders from another system. If present, the gateway may encode the routing ID in the client order ID (or another free-form field). All update events (see list above) will have this field present making it possible to communicate the original order ID back to the other system.

What it doesn’t solve

The introduction of further client order ID’s generated by the originating system when modifying order attributes
Pipelining, i.e. when the originating system sends multiple order modifications without waiting for ack

`ClOrdId`#

When free-form text:
- Roq will use 20 chars (base64 encoded) to carry important information about user_id, order_id and a nanosecond precision timestamp. This information is used to enable automatic download and recovery
- The routing_id field will be appended. This may be an issue if (20 + len(routing_id)) exceeds any limit the exchange may have
Otherwise following exchange conventions with no option to have automatic download and recovery. For this case it is recommended to auto-cancel orders on exchange following a broken connection

`routing_id`#

We do not currently allow ModifyOrder to supply a new routing_id. There are multiple reasons
- There is an inherent race between Ack and ExecutionReport, especially when the underlying communication happens on multiple connections, but also if fills are reported while the modification request is in-flight
- Managing a chain of Ack’s (and the life-time of each routing_id) is very dependent on exchange sending back all acks, in the right order

`order_template`#

The order_template is not encoded in any field sent to the exchange OrderUpdate can therefore only inform of this field until the gateway is restarted or download is initiated

streams#

Orders are cached per stream – but not yet cleared

Download (by gateway)#

When a gateway stream, supporting orders, experiences a disconnect, it will immediately try to reconnect. When connected, the download phase normally begins. This is what happens

All orders (known to the gateway) are internally marked as stale
Working orders are downloaded from the exchange
Orders that can be downloaded will be communicated with update_type=SNAPSHOT
When download completes, and before the stream becomes ready, remaining orders will be communicated with update_type=STALE
Stale will only be communicated once, snapshot may be communicated following every reconnect.