Flags

Here we describe common options for all gateway solutions. You can see all options by using --help from the command-line.

Note

Most options have sensible defaults for a balanced configuration.

You should only have to override the defaults, if you want to tune the configuration for ultra low latency.

Basic

--name

A name used to identify the service.

--config_file

The config file path. See here for further details.

--secrets_file

An optional secrets file path. See here for further details.

--exchange

The exchange name. This name is copied to the exchange field of event messages.

Note

This flag is not supported by all gateways. In particular, brokers will route messages to multiple exchanges. Broker gateways will get the exchange information from the broker and will therefore typically not require you to specify the name.

Clients

--client_listen_address

A filesystem path used by clients to establish communication with the service.

Note

Unix domain sockets are used to exchange shared memory file descriptors between clients and gateway. You can therefore only connect clients running on the same host.

After an initial handshake, a Unix domain socket remains connected with the only purpose of checking the liveness of a connected client. All following communication is over shared memory.

--flatbuffers_listen_address

A TCP listen address used by clients to establish Flatbuffers based communication with the service.

Warning

This is currently work in progress and should therefore not be used.

Metrics

All gateways can export internal metrics using Prometheus’ exposition format.

--metrics_listen_address

The path or port used to expose service metrics.

--metrics_query_path

The HTTP path used to query for service metrics.

Loop

Gateway services are primarily implemented around a core event loop.

Note

The defaults will allow the gateway to sleep (and therefore preserve CPU). This is for low latency (response time: double-digit microseconds).

For ultra low latency (response time: single-digit microseconds) make sure to pin the dispatch thread to a CPU using --loop-cpu-affinity and choose the following options for busy-polling

  • --loop-sleep=0ns

  • --loop-timer-freq=250ns

--loop_cpu_affinity

Used to pin the main dispatch thread to a specific CPU.

--loop_sleep

Use to relinquish control to the kernel.

--loop_timer_freq

Gateway timer frequency.

Note

The core event loop looks a bit like this (pseudo code)

std::chrono::nanoseconds next_timer_update = {};
while (true) {
  auto now = get_monotonic_clock();
  if (now > next_timer_update) {
    next_timer_update = now + FLAGS_loop_timer_freq;
    drain_epoll_queues();  // socket processing (*no* wait!)
    gateway_timer();  // gateway specific, perhaps send ping to exchange
  }
  drain_shared_memory_queues();  // connected clients
  if (FLAGS_loop_sleep) {
    std::this_thread::sleep_for(FLAGS_loop_sleep);
  }
}
  • The core event loop is primarily designed for low latency, i.e. busy-polling.

  • --loop-sleep=0ns will never relinquish the CPU.

  • --loop-timer-freq=250ns will regularly drain socket queues. This is about achieving the best responsiveness, when communicating with the exchange, whilst reducing the time spent transitioning between user-space and kernel. It is here important to understand that syscall’s are expensive and will block the gateway from responding to client requests communicated over shared memory. You are encouraged to experiment and optimize for your own server configuration.

  • --loop-timer-freq=0ns will always drain socket queues. This may be a good choice, if you use a kernel bypass solution, e.g. something based on SolarFlare or DPDK. You are encouraged to experiment and optimize for your own server configuration.

Cache

The gateway will cache certain views (based on rolled up events) for the following purposes

  • Allow download of current state to newly established client connections.

  • Early detection of e.g. bad order book state. These situations may occur as a result of lost messages and/or programming mistakes. It is important that the gateways can act as a first order level of protection such that clients (trading strategies) don’t act on incorrect information.

--cache_fills_max_depth

The size of a pre-allocated vector required to hold a list of order fills.

--cache_mbp_allow_price_inversion

For various reasons, an exchange may allow choice or inverted order books to be disseminated. However, choice or inverted prices will most often indicate a problem somewhere in the stack (lost message, programming mistakes, …)

--cache_mbp_max_depth

The size of a pre-allocated vector required to cache a rolled-up view of MarketByPrice .

--cache_trades_max_depth

The size of a pre-allocated vector required to hold a list of exchange trades.

Event-Log

The gateway can automatically persist a log of all events. This is useful for simulation, back-testing and investigations.

--event_log_dir

Directory used to write an event-log. Default is current directory. Disable by using an empty directory name, i.e. --event_log_dir="".

Note

The event log is saved to this relative path

<iso-week>/<name>/<timestamp>.roq
  • The ISO week is formatted like this 2020-W37.

  • The name is what you have specified as --name.

  • The filename is constructed from startup time in milliseconds since the epoch.

--event_log_iso_week

Sub-directory as YYYY-WWww? (Alternative is YYYY-MM-DD.)

--event_log_sync_freq

Maximum time between buffering (encoding and compression) and write-to-disk.

--event_log_output_buffer_size

Size of the output buffer in bytes.

--event_log_encoding

Encoding method.

Note! Currently only supporting Flatbuffers.

--event_log_compression_level

Compression level. Depends on the compression method.

Note! Currently only supporting Brotli.

Note

The event log is not persisted to disk in real time. This is primarily to avoid wearing out your storage device and allow for better compression.

License Manager

Using a license manager is optional.

--license_manager_uri

A HTTP URI used to contact the license manager.

Note

The default is to not use a license manager.

--license_manager_refresh

Time between ticket refresh.

Warning

You should expect the gateway to operate with a reduced feature set, if it’s not possible to connect to a license manager to receive or refresh a ticket.

Rate-Limiter (EXPERIMENTAL)

Most exchanges implement a request rate-limiter.

Sometimes it’s possible to receive information from the exchange’s rate-limiter making it possible for us to at least try to mirror what is the current state.

Warning

The implementation is experimental and is likely to be removed again. You can effectively disable this check by settings --rest_rate_limit_max_requests to some high number.

--rest_rate_limit_interval

The time interval being monitored. We use a fixed window for efficiency reasons (as opposed to sliding window which requires us to track the full history of requests).

--rest_rate_limit_max_requests

The maximum limit for requests multiplied by their weights. The weights are hard-coded matching what can be found from exchange documentation. A weight of 1 is being used when such information can not be found. Disable by setting this to a high number.