roq-cme#

Stable

$ mamba install \
      --channel https://roq-trading.com/conda/stable \
      roq-cme

Unstable

$ mamba install \
      --channel https://roq-trading.com/conda/unstable \
      roq-cme

`roq-cme`#

$ roq-cme [FLAGS]

Description#

roq-cme is a gateway

Using the MDP3 (SBE/multicast) interface for market data.
- Supports MBOFD (Market by Order).
- Using both A and B channels for fastest access and to lower the probability of packet loss.
Using the iLink3 (SBE/TCP) interface for order management.

Supports#

Products

Spot
Futures	✓
Options
Combos

Market Data

Reference Data	✓
Market Status	✓
Top of Book	✓
Market by Price	✓
Market by Order	✓
Trade Summary	✓
Statistics	✓

Order Management

Create	✓
Modify	✓
Cancel	✓
Cancel All	✓
Auto-Cancel

Account Management

Positions
Funds

Flags#

$ roq-cme --help

Flags

--exchange (std::vector<std::string>(XCBTs,XNYMs)): Exchange identifier (list)

iLink

--ilink_firm_id: Firm Id
--ilink_port: Port
--ilink_location: Location
--ilink_config_file: Config file (path)
--ilink_market_segment_ids: Comma-separated list of market segments, e.g. 70

Multicast

--multicast_config_file: Config file (path)
--multicast_channel_ids: Comma-separated list of multicast channels, e.g. 310
--multicast_local_interface: Multicast local interface
--multicast_buffer_size (2048): Buffer size (at least the maximum possible size of a datagram)
--multicast_buffer_depth (8): Multicast buffer depth (used for re-ordering)
--multicast_timeout (90s): Timeout when waiting for snapshot

Misc

--enable_market_by_order (false): Enable market by order updates?
--filter_snapshot_from_incremental (1024): Used to remove low sequence number messages, e.g. snapshots
--secdef_config_file: Config file (path)
--mbp_allow_price_inversion (false): Allow price inversion?
--decode_buffer_size (10485760): Decode buffer size
--encode_buffer_size (4096): Encode buffer size

Test

--test_mbp_to_mbo_clear_price_level (false): Use MbP to delete MbO price levels?
--test_reordering (0): Test
--test_drop (0): Test

Environments#

$ $CONDA_PREFIX/share/roq-cme/flags

There is no specific environment setup.

You need to download the MDP config.xml and iLink MSGW_Config.xml files. These files are used to discover IP addresses for MDP multicast and iLink’s MSGW (market segment gateways).

The gateway flags --multicast_channel_ids and --ilink_market_segment_ids are used to join multicast streams and connect to specific MSGW end-points.

The multicast addresses must be made available from a local network interface. The actual IP address of that interface must be provided with the --multicast_local_interface flag.

Configuration#

$ $CONDA_PREFIX/share/roq-cme/config.toml

Important

The template will be replaced when the software is upgraded. Make a copy and modify to your needs.

symbols = ".*"

[accounts]

  [accounts.A1]
  master = true
  login = "YOUR_CME_LOGIN_GOES_HERE"
  secret = "YOUR_CME_SECRET_GOES_HERE"
  symbols = [ ".*" ]

[users]

  [users.test]
  password = "1234"
  symbols = ".*"

  [users.trader]
  password = "secret"
  accounts = [ "A1" ]
  symbols = [ ".*BTC.*", ".*ETH.*" ]

[rate_limits]

  [rate_limits.global]
    type = "ORDER_ACTION"
    aggregate = true
    request_limit = 100
    monitor_period = "10s"
    ban_period = "5m"

Market Data#

Live

Event	Stream	Messages
`roq::ReferenceData`	Incremental	MDInstrumentDefinitionFuture54, MDInstrumentDefinitionOption55, MDInstrumentDefinitionSpread56, MDInstrumentDefinitionFixedIncome57, MDInstrumentDefinitionRepo58, MDInstrumentDefinitionFX63
`roq::MarketStatus`	Incremental	SecurityStatus30, MDInstrumentDefinitionFuture54, MDInstrumentDefinitionOption55, MDInstrumentDefinitionSpread56, MDInstrumentDefinitionFixedIncome57, MDInstrumentDefinitionRepo58, MDInstrumentDefinitionFX63
`roq::TopOfBook`	Incremental	MDIncrementalRefreshBook46
`roq::MarketByPriceUpdate`	Incremental	MDIncrementalRefreshBook46
`roq::MarketByOrderUpdate`	Incremental	MDIncrementalRefreshBook46, MDIncrementalRefreshOrderBook47
`roq::TradeSummary`	Incremental	MDIncrementalRefreshTradeSummary48, MDIncrementalRefreshTradeSummaryLongQty65
`roq::StatisticsUpdate`	Incremental	MDIncrementalRefreshDailyStatistics49, MDIncrementalRefreshSessionStatistics51, MDIncrementalRefreshSessionStatisticsLongQty67, MDIncrementalRefreshVolume37

Download

Event	Stream	Messages
`roq::ReferenceData`	Instrument Definition	MDInstrumentDefinitionFuture54, MDInstrumentDefinitionOption55, MDInstrumentDefinitionSpread56, MDInstrumentDefinitionFixedIncome57, MDInstrumentDefinitionRepo58, MDInstrumentDefinitionFX63
`roq::MarketStatus`	Instrument Definition	MDInstrumentDefinitionFuture54, MDInstrumentDefinitionOption55
`roq::TopOfBook`
`roq::MarketByPriceUpdate`	MBP Market Recovery	SnapshotFullRefresh52, SnapshotFullRefreshLongQty69
`roq::MarketByOrderUpdate`	MBO Market Recovery	SnapshotFullRefreshOrderBook53
`roq::TradeSummary`
`roq::StatisticsUpdate`

Statistics#

TBD

Order Management#

Live

Event	Stream	Messages	Comments
`roq::OrderUpdate`	OrderEntry	ExecutionReportTradeOutright525, ExecutionReportTradeSpread526, ExecutionReportTradeSpreadLeg527
`roq::TradeUpdate`	OrderEntry	ExecutionReportTradeOutright525, ExecutionReportTradeSpread526, ExecutionReportTradeSpreadLeg527

Download

Event	Stream	Messages	Comments
`roq::OrderUpdate`	OrderEntry	ExecutionReportStatus532
`roq::TradeUpdate`			Unavailable

Request

Event	Stream	Messages
`roq::CreateOrder`	OrderEntry	NewOrderSingle514
`roq::ModifyOrder`	OrderEntry	OrderCancelReplaceRequest515
`roq::CancelOrder`	OrderEntry	OrderCancelRequest516
`roq::CancelAllOrders`	OrderEntry	OrderMassActionRequest529

Response

Event	Stream	Messages	Comments
`roq::OrderAck`	OrderUpdate	ExecutionReportNew522, ExecutionReportReject523, ExecutionReportModify531, ExecutionReportCancel534, BusinessReject521

Order Types#

Type	Comments
`MARKET`	Mapped to `'MarketwithProtection'` (SBE)
`LIMIT`	Mapped to `'Limit'` (SBE)

Time in Force#

Type	Comments
`GFD`	Mapped to `'Day'` (SBE)
`GTC`	Mapped to `'GoodTillCancel'` (SBE)
`IOC`	Mapped to `'FillAndKill'` (SBE)
`FOK`	Mapped to `'FillOrKill'` (SBE)

Position Effect#

Note

Not supported

Execution Instructions#

Note

Not supported

Templates#

Note

Not supported

Account Management#

Live

Event	Stream	Messages	Comments
`roq::PositionUpdate`			Unavailable
`roq::FundsUpdate`			Unavailable

Download

Event	Stream	Messages	Comments
`roq::PositionUpdate`			Unavailable
`roq::FundsUpdate`			Unavailable

Streams#

Instrument Definition

Type	Comments
UDP	Primary purpose Reference data

MBP Market Recovery

Type	Comments
UDP	Primary purpose MbP snapshot

MBO Market Recovery

Type	Comments
UDP	Primary purpose MbO snapshot

Incremental

Type	Comments
UDP	Primary purpose MbP incremental

OrderEntry (iLink)

Type	Comments
TCP	Primary purpose iLink

Constraints#

The OrderEntry (iLink) connections are established based on the provided list of market segments. This is a bit unusual: the list of accounts are normally 1:1 with the connections. It is only allowed to configure a single account until we know better how this all works with CME’s party-definitions.
The actual exchange (e.g. XCBT) is now used for order management. The reason is technical and due to most gateways only providing access to a single exchange. The exchange name used for creating, modify and canceling orders, must therefore match the name provided with the --exchange flag.

Comments#

ExternalLatency is currently only published when receiving a heartbeat from the exchange.
MarketByOrder is opt-in using the --enable_market_by_order flag.
ExecutionReportTradeOutright reports LastPx as the order’s limit price. This should normally be expected to be the (average) price of the fill(s).

`roq-cme-import`#

$ roq-cme-import [FLAGS] PCAP_FILE

Description#

roq-cme-import is a tool to create a .roq event-log by importing a .pcap file.

Requires the config.xml file (from CME’s FTP site) to map IP addresses to channels.
Requires the secdef.dat file (from CME’s FTP site) if the .pcap file does not contain the relevant instrument definitions.
Requires the .pcap file to contain all incremental messages from CME’s start-up (Sunday) if the recovery channels are not available.
Will not work if the .pcap file has gaps and the recovery channels are not available.

Flags#

$ roq-cme-import --help

Flags

--name: A name used to identify the service.
--type (pcap): Type of input file (only support type is pcap).
--channel_ids: Comma-separated list of multicast channels to include.
--symbols: Comma-separated list of symbols (regex) to include.
--cache_all_reference_data (false): This flag allows you (opt-in) to cache all reference data including symbols that would normally be discarded due to regex matching.

CME

--cme_config_file: CME’s multicast channel XML config file.
--cme_secdef_file: CME’s instrument definitions file (required when the .pcap file does not contain instrument definitions).

EventLog

--event_log_output_file: Event-log output file.
--event_log_dir: Event-log directory (optional).
--event_log_buffer_size (134217728): Event-log output buffer size.
--event_log_compression_level (4): Event-log compression level.

Misc

--encode_buffer_size (16777216): Encode buffer size.
--mbp_allow_price_inversion (true): Allow price inversion?
--mbp_max_depth (10): Max depth.

Test

--test_mbp_mbo (false): Compare MBP and MBO updates?
--test_depth (10): Depth to compare.
--test_mbp_to_mbo_clear_price_level (false): DEPRECATED

Example#

$ roq-cme-import \
    --name "cme" \
    --type "pcap" \
    --channel_ids 344 \
    --symbols "ZN[HMUZ][0-9]" \
    --cme_config_file "config.xml" \
    --cme_secdef_file "secdef.dat" \
    --event_log_output_file "test.roq" \
    all.pcap

`roq-cme-filter`#

$ roq-cme-filter [FLAGS]

Description#

roq-cme-filter is a tool to generate the PCAP filter required to capture specific channels.

Flags#

$ roq-cme-filter --help

Flags

--type: Type of output, e.g. tcpdump.
--channel_ids: Comma-separated list of multicast channels to include.

CME

--cme_config_file: CME’s multicast channel XML config file.

Example#

$ roq-cme-filter \
    --type "tcpdump" \
    --channel_ids 344 \
    --cme_config_file "config.xml"

(host 224.0.31.110 and port 14344) or (host 224.0.32.110 and port 15344) or (host 224.0.31.68 and port 14344) or (host 224.0.32.68 and port 15344) or (host 224.0.31.89 and port 14344) or (host 224.0.32.89 and port 15344) or (host 233.72.75.33 and port 23344) or (host 233.72.75.96 and port 22344)(dev)

This will output a tcpdump filter for channel_ids.

roq-cme#

`roq-cme`#

Description#

Supports#

Flags#

Environments#

Configuration#

Market Data#

Statistics#

Order Management#

Order Types#

Time in Force#

Position Effect#

Execution Instructions#

Templates#

Account Management#

Streams#

Constraints#

Comments#

`roq-cme-import`#

Description#

Flags#

Example#

`roq-cme-filter`#

Description#

Flags#

Example#

References#

Common#

CME#

roq-cme#

roq-cme#

Description#

Supports#

Flags#

Environments#

Configuration#

Market Data#

Statistics#

Order Management#

Order Types#

Time in Force#

Position Effect#

Execution Instructions#

Templates#

Account Management#

Streams#

Constraints#

Comments#

roq-cme-import#

Description#

Flags#

Example#

roq-cme-filter#

Description#

Flags#

Example#

References#

Common#

CME#

`roq-cme`#

`roq-cme-import`#

`roq-cme-filter`#