Flags

Here we describe common options for all gateway solutions. You can see all options by using --help from the command-line.

Note

Most options have sensible defaults for a balanced configuration.

You should only have to override the defaults, if you want to tune the configuration for ultra low latency.

Basic

--name

A name used to identify the service.

--config-file

The config file path. See here for further details.

--exchange

The exchange name. This name is copied to the exchange field of event messages.

Note

This flag is not supported by all gateways. In particular, brokers will route messages to multiple exchanges. Broker gateways will get the exchange information from the broker and will therefore typically not require you to specify the name.

Clients

--client-listen-address

A filesystem path used by clients to establish communication with the service.

Note

Unix domain sockets are used to exchange shared memory file descriptors between clients and gateway. You can therefore only connect clients running on the same host.

After an initial handshake, a Unix domain socket remains connected with the only purpose of checking the liveness of a connected client. All following communication is over shared memory.

--flatbuffers-listen-address

A TCP listen address used by clients to establish Flatbuffers based communication with the service.

Warning

This is currently work in progress and should therefore not be used.

Metrics

All gateways can export internal metrics using Prometheus’ exposition format.

--metrics-listen-address

The path or port used to expose service metrics.

--metrics-query-path

The HTTP path used to query for service metrics.

Loop

Gateway services are primarily implemented around a core event loop.

Note

The defaults will allow the gateway to sleep (and therefore preserve CPU). This is for low latency (response time: double-digit microseconds).

For ultra low latency (response time: single-digit microseconds) make sure to pin the dispatch thread to a CPU using --loop-cpu-affinity and choose the following options for busy-polling

  • --loop-sleep-nsecs=0

  • --loop-timer-freq-nsecs=250

--loop-cpu-affinity

Used to pin the main dispatch thread to a specific CPU.

--loop-sleep-nsecs

Use to relinquish control to the kernel.

--loop-timer-freq-nsecs

Gateway timer frequency.

Note

The core event loop looks a bit like this (pseudo code)

std::chrono::nanoseconds next_timer_update = {};
while (true) {
  auto now = get_monotonic_clock();
  if (now > next_timer_update) {
    next_timer_update = now + std::chrono::nanoseconds { FLAGS_loop_timer_freq_nsecs };
    drain_epoll_queues();  // socket processing (*no* wait!)
    gateway_timer();  // gateway specific, perhaps send ping to exchange
  }
  drain_shared_memory_queues();  // connected clients
  if (FLAGS_loop_sleep_nsecs) {
    std::this_thread::sleep_for( std::chrono::nanoseconds { FLAGS_loop_sleep_nsecs });
  }
}
  • The core event loop is primarily designed for low latency, i.e. busy-polling.

  • --loop-sleep-nsecs=0 will never relinquish the CPU.

  • --loop-timer-freq-nsecs=250 will regularly drain socket queues. This is about achieving the best responsiveness, when communicating with the exchange, whilst reducing the time spent transitioning between user-space and kernel. It is here important to understand that syscall’s are expensive and will block the gateway from responding to client requests communicated over shared memory. You are encouraged to experiment and optimize for your own server configuration.

  • --loop-timer-freq-nsecs=0 will always drain socket queues. This may be a good choice, if you use a kernel bypass solution, e.g. something based on SolarFlare or DPDK. You are encouraged to experiment and optimize for your own server configuration.

Cache

The gateway will cache certain views (based on rolled up events) for the following purposes

  • Allow download of current state to newly established client connections.

  • Early detection of e.g. bad order book state. These situations may occur as a result of lost messages and/or programming mistakes. It is important that the gateways can act as a first order level of protection such that clients (trading strategies) don’t act on incorrect information.

--cache-fills-max-depth

The size of a pre-allocated vector required to hold a list of order fills.

--cache-mbp-allow-price-inversion

For various reasons, an exchange may allow choice or inverted order books to be disseminated. However, choice or inverted prices will most often indicate a problem somewhere in the stack (lost message, programming mistakes, …)

--cache-mbp-max-depth

The size of a pre-allocated vector required to cache a rolled-up view of MarketByPrice .

--cache-trades-max-depth

The size of a pre-allocated vector required to hold a list of exchange trades.

Event-Log

The gateway can automatically persist a log of all events. This is useful for simulation, back-testing and investigations.

--event-log-dir

Directory used to write an event-log. Default is current directory. Disable by using an empty directory name, i.e. --event-log-dir="".

Note

The event log is saved to this relative path

<iso-week>/<name>/<timestamp>.roq
  • The ISO week is formatted like this 2020-W37.

  • The name is what you have specified as --name.

  • The filename is constructed from startup time in milliseconds since the epoch.

--event-log-sync-freq-secs

Maximum time (in seconds) between buffering (encoding and compression) and write-to-disk.

--event-log-output-buffer-size

Size of the output buffer in bytes.

--event-log-encoding

Encoding method.

Note! Currently only supporting Flatbuffers.

--event-log-compression-level

Compression level. Depends on the compression method.

Note! Currently only supporting Brotli.

Note

The event log is not persisted to disk in real time. This is primarily to avoid wearing out your storage device and allow for better compression.

License Manager

Using a license manager is optional.

--license-manager-uri

A HTTP URI used to contact the license manager.

Note

The default is to not use a license manager.

--license-manager-refresh-secs

Time between ticket refresh.

Warning

You should expect the gateway to operate with a reduced feature set, if it’s not possible to connect to a license manager to receive or refresh a ticket.