Release 0.9.8

2023-11-20

API Changes

Order Ack

The OrderAck message had quantity, price and stop_price added to better correlate order updates with request versions.

Cancel all Orders

The CancelAllOrders message was extended with more fields used to filter the set of orders that must be affected by the request.

The GatewaySettings message will communicate what filter fields are supported through the oms_cancel_all_orders bitset field.

Note

This change makes it possible to map order mass cancel requests to exchange messages.

Warning

An exchange may not support all combinations of filters, if any.

FlatBuffers

The auto-generated FlatBuffers interface was removed from the API.

Note

Now available from the roq-codec package (along with other protocols, e.g. SBE an FIX).

This was done to hide the FlatBuffers version dependency (the auto-generated code contains a check) thereby allowing a more flexible upgrade path for code dependencies.

Gateway Changes

Accounts

Important

BREAKING CHANGE

  • All gateways will now accept clients to subscribe to all accounts by not specifying any accounts for subscription.

Trades

  • Some gateways now support downloading trades: Binance, Bybit, Deribit, OKX.

    This is controlled by the --download_trades_lookback flag.

    When used, it is a time period offset (to current time), e.g. 24h for downloading all trades from the past day.

    Note

    There may be other relevant flags specific to each gateway. These will normally be prefixed with --download_trades_. Please consult each gateway’s reference documentation or --help from the command-line.

RoutingId

Important

BREAKING CHANGE

  • It is now possible to configure a gateway to persist RoutingId mapping to local file and/or broadcast mappings using multicast for consumption by Journal.

    The former is useful for convenient deployment but introduces a dependency on the filesystem with no redundancy, i.e. poor latency and some risk of losing the mappings.

    The latter design is distributed, low-latency and highly redundant, but more complex to deploy.

Note

RoutingId was previously appended to the ClOrdID we would send to the exchange.

This was a low-latency solution because it was using the exchange as a “database”. It was required by the FIX Bridge.

However, this was not always possible due to some exchanges imposing limits on the length of ClOrdID and/or the allowed characters.

Cancel all Orders

  • Changes were implemented to eventually support strategy_id. This meant support for filters and also forwarding the request to all relevant exchange connections. Multiple ack’s are also now supported (potentially one per each connection).

    Warning

    This is not yet fully implemented for strategy_id’s which will typically require a request for each order matching the filter.

Binance (spot)

  • Now supporting trades download.

  • Now defaulting to WS API for order management (revert to REST by using --ws_api=false).

Deribit

  • Market data connections (using WebSocket) will require authentication after 2023-12-05.

OKX

  • Now supporting trades download.

  • Order management was reviewed against exchange API’s.

  • VIP market data feeds were reviewed.

Auxiliary

FIX Bridge & FIX Proxy

The new FIX Proxy service can coordinate with a FIX Bridge service such that order updates are routed by a strategy_id associated with a FIX Proxy.

This design allows for dynamic routing and the FIX Proxy can therefore be used as a front-end to the FIX Bridge.

In other words: a load-balancing design can be implemented to support order routing for many connected clients.

Journal

A new Journal service has been added to support gateway persistence with very low latency being highly redundant.

Although this is a general tool to persist order attributes, it is in particular required by the FIX Bridge to persist ClOrdID (Roq’s RoutingId) which is central to the FIX order management protocol.

The relevant gateway flags are describer here.

SBE Publisher

The SBE publisher is now fully functional, including the configuration of a limited depth market by price (MbP) order book.

Python

  • Bindings for the roq-codec interfaces. In particular, the SBE messages can now be decoded directly from Python. Also partial support for FIX message encoding/decoding.

  • Bindings for some roq-io interfaces. In particular, the UDP re-order buffer and the MbP sequencer required to support multicast distribution of market data.

Some GitHub examples using Python’s asyncio

CHANGELOG

Milestone on GitHub.

Fixed

  • OrderID lookup was never fully implemented (FIX Bridge) (#420).

  • The position cache would treat zero as a missing value and therefore not update (API) (#419).

  • OrderMassStatusRequest didn’t return reason UNKNOWN_ORDER for no orders (FIX Bridge) (#418).

  • ModifyOrder with both price and quantity missing would trigger fatal error (Server) (#417).

  • Client handshake protocol didn’t support even-log-dir’s exceeding 64 characters (Service) (#416).

  • OMS order state was incorrectly updated following exchange reject (Deribit) (#413).

  • Revert to use exchange OrderID when our own OrigClOrdID is missing (rejects) (Deribit) (#412).

  • Incorrect lookup key was used for order caching (Core) (#411).

  • LocalMktDate not handled correctly for “empty” values (FIX) (#410).

  • OrderStatusRequest and OrderMassStatusRequest not fully FIX 4.4 compliant (FIX Bridge) (#408).

Changed

  • Market Data WebSocket now authenticating (Deribit) (#421).

  • Allow caching of {order_id, version, user_id, strategy_id} <==> {client_order_id, routing_id} mappings (Server) (#397).

  • Client will now receive all accounts if no accounts are configured for subscription (Server) (#415).

  • CancelAllOrders / CancelAllOrdersAck (API) (#414).

  • SHA and HMAC now using OpenSSL 3.x (Core) (#409).

Added

  • Capture OrderAck and OrderUpdate from multicast feed (Journal) (#405).

Removed

  • Appending routing_id to the generated exchange ClOrdID has been removed (Server) (#407).

Other