roq-cme

$ mamba install \
      --channel https://roq-trading.com/conda/stable \
      roq-cme
$ mamba install \
      --channel https://roq-trading.com/conda/unstable \
      roq-cme

roq-cme

$ roq-cme [FLAGS]

Description

roq-cme is a gateway

  • Using the MDP3 (SBE/multicast) interface for market data.

    • Supports MBOFD (Market by Order).

    • Using both A and B channels for fastest access and to lower the probability of packet loss.

  • Using the iLink3 (SBE/TCP) interface for order management.

Supports

Products

Spot

Futures

Options

Combos

Market Data

Reference Data

Market Status

Top of Book

Market by Price

Market by Order

Trade Summary

Statistics

Order Management

Create

Modify

Cancel

Cancel All

Auto-Cancel

Account Management

Positions

Funds

Flags

$ roq-cme --help
--exchange (XCBT,XNYM)

Exchange identifier (list)

--ilink_firm_id

Firm Id

--ilink_port

Port

--ilink_location

Location

--ilink_config_file

Config file (path)

--ilink_market_segment_ids

Comma-separated list of market segments, e.g. 70

--multicast_config_file

Config file (path)

--multicast_channel_ids

Comma-separated list of multicast channels, e.g. 310

--multicast_local_interface

Multicast local interface

--multicast_buffer_size (2048)

Buffer size (at least the maximum possible size of a datagram)

--multicast_buffer_depth (8)

Multicast buffer depth (used for re-ordering)

--multicast_timeout (90s)

Timeout when waiting for snapshot

--enable_market_by_order (false)

Enable market by order updates?

--filter_snapshot_from_incremental (1024)

Used to remove low sequence number messages, e.g. snapshots

--secdef_config_file

Config file (path)

--mbp_allow_price_inversion (false)

Allow price inversion?

--decode_buffer_size (10485760)

Decode buffer size

--encode_buffer_size (4096)

Encode buffer size

--test_reordering (0)

Test

--test_drop (0)

Test

Environments

$ $CONDA_PREFIX/share/roq-cme/flags

There is no specific environment setup.

You need to download the MDP config.xml and iLink MSGW_Config.xml files. These files are used to discover IP addresses for MDP multicast and iLink’s MSGW (market segment gateways).

The gateway flags --multicast_channel_ids and --ilink_market_segment_ids are used to join multicast streams and connect to specific MSGW end-points.

The multicast addresses must be made available from a local network interface. The actual IP address of that interface must be provided with the --multicast_local_interface flag.

Configuration

$ $CONDA_PREFIX/share/roq-cme/config.toml

Important

The template will be replaced when the software is upgraded. Make a copy and modify to your needs.

symbols = ".*"

[accounts]

  [accounts.A1]
  master = true
  login = "YOUR_CME_LOGIN_GOES_HERE"
  secret = "YOUR_CME_SECRET_GOES_HERE"
  symbols = [ ".*" ]

[users]

  [users.test]
  password = "1234"
  symbols = ".*"

  [users.trader]
  password = "secret"
  accounts = [ "A1" ]
  symbols = [ ".*BTC.*", ".*ETH.*" ]

[rate_limits]

  [rate_limits.global]
    type = "ORDER_ACTION"
    aggregate = true
    request_limit = 100
    monitor_period = "10s"
    ban_period = "5m"

Market Data

Event

Stream

Messages

Comments

roq::ReferenceData

Incremental

MDInstrumentDefinitionFuture54, MDInstrumentDefinitionOption55, MDInstrumentDefinitionSpread56, MDInstrumentDefinitionFixedIncome57, MDInstrumentDefinitionRepo58, MDInstrumentDefinitionFX63

roq::MarketStatus

Incremental

SecurityStatus30, MDInstrumentDefinitionFuture54, MDInstrumentDefinitionOption55, MDInstrumentDefinitionSpread56, MDInstrumentDefinitionFixedIncome57, MDInstrumentDefinitionRepo58, MDInstrumentDefinitionFX63

roq::TopOfBook

Incremental

MDIncrementalRefreshBook46

roq::MarketByPriceUpdate

Incremental

MDIncrementalRefreshBook46

roq::MarketByOrderUpdate

Incremental

MDIncrementalRefreshBook46, MDIncrementalRefreshOrderBook47

roq::TradeSummary

Incremental

MDIncrementalRefreshTradeSummary48, MDIncrementalRefreshTradeSummaryLongQty65

roq::StatisticsUpdate

Incremental

MDIncrementalRefreshDailyStatistics49, MDIncrementalRefreshSessionStatistics51, MDIncrementalRefreshSessionStatisticsLongQty67, MDIncrementalRefreshVolume37

Event

Stream

Messages

Comments

roq::ReferenceData

Instrument Definition

MDInstrumentDefinitionFuture54, MDInstrumentDefinitionOption55, MDInstrumentDefinitionSpread56, MDInstrumentDefinitionFixedIncome57, MDInstrumentDefinitionRepo58, MDInstrumentDefinitionFX63

roq::MarketStatus

Instrument Definition

MDInstrumentDefinitionFuture54, MDInstrumentDefinitionOption55

roq::TopOfBook

roq::MarketByPriceUpdate

MBP Market Recovery

SnapshotFullRefresh52, SnapshotFullRefreshLongQty69

roq::MarketByOrderUpdate

MBO Market Recovery

SnapshotFullRefreshOrderBook53

roq::TradeSummary

roq::StatisticsUpdate

Statistics

TBD

Order Management

Event

Stream

Messages

Comments

roq::OrderUpdate

OrderEntry

ExecutionReportTradeOutright525, ExecutionReportTradeSpread526, ExecutionReportTradeSpreadLeg527

roq::TradeUpdate

OrderEntry

ExecutionReportTradeOutright525, ExecutionReportTradeSpread526, ExecutionReportTradeSpreadLeg527

Event

Stream

Messages

Comments

roq::OrderUpdate

OrderEntry

ExecutionReportStatus532

roq::TradeUpdate

Unavailable

Event

Stream

Messages

Comments

roq::CreateOrder

OrderEntry

NewOrderSingle514

roq::ModifyOrder

OrderEntry

OrderCancelReplaceRequest515

roq::CancelOrder

OrderEntry

OrderCancelRequest516

roq::CancelAllOrders

OrderEntry

OrderMassActionRequest529

Event

Stream

Messages

Comments

roq::OrderAck

OrderUpdate

ExecutionReportNew522, ExecutionReportReject523, ExecutionReportModify531, ExecutionReportCancel534, BusinessReject521

Order Types

Type

Comments

MARKET

Mapped to 'MarketwithProtection' (SBE)

LIMIT

Mapped to 'Limit' (SBE)

Time in Force

Type

Comments

GFD

Mapped to 'Day' (SBE)

GTC

Mapped to 'GoodTillCancel' (SBE)

IOC

Mapped to 'FillAndKill' (SBE)

FOK

Mapped to 'FillOrKill' (SBE)

Position Effect

Note

Not supported

Execution Instructions

Note

Not supported

Templates

Note

Not supported

Account Management

Event

Stream

Messages

Comments

roq::PositionUpdate

Unavailable

roq::FundsUpdate

Unavailable

Event

Stream

Messages

Comments

roq::PositionUpdate

Unavailable

roq::FundsUpdate

Unavailable

Streams

Type

Comments

UDP

Primary purpose

  • Reference data

Type

Comments

UDP

Primary purpose

  • MbP snapshot

Type

Comments

UDP

Primary purpose

  • MbO snapshot

Type

Comments

UDP

Primary purpose

  • MbP incremental

Type

Comments

TCP

Primary purpose

  • iLink

Constraints

  • The OrderEntry (iLink) connections are established based on the provided list of market segments. This is a bit unusual: the list of accounts are normally 1:1 with the connections. It is only allowed to configure a single account until we know better how this all works with CME’s party-definitions.

  • The actual exchange (e.g. XCBT) is now used for order management. The reason is technical and due to most gateways only providing access to a single exchange. The exchange name used for creating, modify and canceling orders, must therefore match the name provided with the --exchange flag.

Comments

  • ExternalLatency is currently only published when receiving a heartbeat from the exchange.

  • MarketByOrder is opt-in using the --enable_market_by_order flag.

  • ExecutionReportTradeOutright reports LastPx as the order’s limit price. This should normally be expected to be the (average) price of the fill(s).

roq-cme-import

$ roq-cme-import [FLAGS] PCAP_FILE

Description

roq-cme-import is a tool to create a .roq event-log by importing a .pcap file.

  • Requires the config.xml file (from CME’s FTP site) to map IP addresses to channels.

  • Requires the secdef.dat file (from CME’s FTP site) if the .pcap file does not contain the relevant instrument definitions.

  • Requires the .pcap file to contain all incremental messages from CME’s start-up (Sunday) if the recovery channels are not available.

  • Will not work if the .pcap file has gaps and the recovery channels are not available.

Flags

$ roq-cme-import --help
--name

A name used to identify the service.

--type (pcap)

Type of input file (only support type is pcap).

--channel_ids

Comma-separated list of multicast channels to include.

--symbols

Comma-separated list of symbols (regex) to include.

--cache_all_reference_data (false)

This flag allows you (opt-in) to cache all reference data including symbols that would normally be discarded due to regex matching.

--cme_config_file

CME’s multicast channel XML config file.

--cme_secdef_file

CME’s instrument definitions file (required when the .pcap file does not contain instrument definitions).

--event_log_output_file

Event-log output file.

--event_log_dir

Event-log directory (optional).

--event_log_buffer_size (134217728)

Event-log output buffer size.

--event_log_compression_level (4)

Event-log compression level.

--encode_buffer_size (16777216)

Encode buffer size.

--mbp_allow_price_inversion (true)

Allow price inversion?

--mbp_max_depth (10)

Max depth.

--test_mbp_mbo (false)

Compare MBP and MBO updates?

--test_depth (10)

Depth to compare.

--test_mbp_to_mbo_clear_price_level (false)

DEPRECATED

Example

$ roq-cme-import \
    --name "cme" \
    --type "pcap" \
    --channel_ids 344 \
    --symbols "ZN[HMUZ][0-9]" \
    --cme_config_file "config.xml" \
    --cme_secdef_file "secdef.dat" \
    --event_log_output_file "test.roq" \
    all.pcap

roq-cme-filter

$ roq-cme-filter [FLAGS]

Description

roq-cme-filter is a tool to generate the PCAP filter required to capture specific channels.

Flags

$ roq-cme-filter --help
--type

Type of output, e.g. tcpdump.

--channel_ids

Comma-separated list of multicast channels to include.

--cme_config_file

CME’s multicast channel XML config file.

Example

$ roq-cme-filter \
    --type "tcpdump" \
    --channel_ids 344 \
    --cme_config_file "config.xml"

(host 224.0.31.110 and port 14344) or (host 224.0.32.110 and port 15344) or (host 224.0.31.68 and port 14344) or (host 224.0.32.68 and port 15344) or (host 224.0.31.89 and port 14344) or (host 224.0.32.89 and port 15344) or (host 233.72.75.33 and port 23344) or (host 233.72.75.96 and port 22344)(dev)

This will output a tcpdump filter for channel_ids.

References

Common

CME