Release 0.8.5

2022-06-06

Gateways

OKX

TopOfBook is now sourced from the bbo-tbt channel.

Warning

It is no longer possible to use ws_books_depth=1 !!!

API Changes

roq::cache::Manager no longer a template class. The only downstream change should be the removal of template arguments.

Reference Data

Gateways can now (optionally) capture all reference data from the exchange, regardless what symbols regex-matching has been chosen.

A new flag --cache_all_reference_data has been added to support this.

“Idle” Streams

Sometimes it can happen that a connection remains open yet no “important” data seems to arrive. A new general flag has been introduced which may be used to force disconnect of a stream after a timeout has expired: --net_disconnect_on_idle_timeout.

Warning

This flag is not always used. The reason is that not all messages of a communication protocol can be considered “important”. For example, ping/pong messages should not be considered “important”. Streams dedicated to order entry / drop copy are other examples of “tricky” connections due to very little activity expected on these connections.

Note

This change was specifically implemented to support the Huobi gateway (#227).

Tools

A new query tool has been added: roq-query. This tool is useful to quickly create pivot tables like this

$ roq-query ftx.roq --type count --event_type top_of_book --period 5m --stream_id 6 --symbols "BTC.*,ETH.*"
receive_time_utc,"BTC-0624","BTC-0930","BTC-PERP","BTC/USD","BTC/USDT","ETH-0624","ETH-0930","ETH-PERP","ETH/USD","ETH/USDT"
1654086300000000000ns,5060,1796,3423,3201,4023,2482,5256,7504,7106,3579
1654086600000000000ns,6907,5140,5414,4218,5056,3113,7551,10478,11151,2510
1654086900000000000ns,5306,7986,5364,5562,6144,3561,7660,2714,10968,10240
1654087200000000000ns,6364,11089,6367,6409,4263,7116,9020,3787,10742,9150
[...]

Reference documentation can be found here.

CHANGELOG

Milestone on GitHub.

Added

  • New flag to control periodic download of reference data (Huobi Futures) (#233).

  • Increase/decrease log verbosity level through signals (Server) (#229).

  • Flag to disconnect market data connection after some time (Deribit) (#224).

  • Make it possible to capture all reference data (Server) (#225).

  • Deribit’s multicast protocol (Deribit) (#201).

Changed

  • Use MbP decimals (FIX Bridge) (#232).

  • TopOfBook now sourced from bbo-tbt (OKX) (#231).

  • Make cache::Manager a non-template (API) (#230).

  • Request throttling + force disconnect of “idle” streams (Huobi) (#227).

Fixed

  • Market data subscription was not working correctly after disconnect (OKX) (#226).

  • Symbol filtering was not used (BitMEX) (#223).

  • Download did not support a dynamic number of total pages (Core) (#222).

  • TradeSummary now correctly reports side and quantity (Gate Futures) (#221).

  • TradeSummary now reported as number of lots (Huobi Futures) (#220).

  • TradeSummary now using the (L3) match channel (KuCoin) (#219).

Other