FIX Architecture#
Note
Roq has reached a big milestone by releasing its version 1.
We always had the goal to offer solutions supporting a single person developing and deploying strategies all the way up to an industrial scale deployment. We believe we have finally reached this goal with the recent work described in this document.
The software that we now have is the result of having worked closely with key clients over the past several years. We thank all of our sponsors for the energy you gave back by using the software!
Background#
We have recently worked very closely with a broker to build the missing software components required to offer FIX order-routing capabilities.
This document outlines the key components and offers a high-level view of the architecture.
Please reach out on Email if you have further questions.
Deployment#
The client owns and deploys the entire software stack, including their own proprietary solutions.
This was always a key requirement from all our clients: Full control of everything.
FIX Bridge#
Roq natively uses a C++ API to support low latency communication over shared memory. The “server” side is often a gateway. The “client” side may be a market making strategy.
However, the “client”s can also bridge between protocols of which the FIX Bridge is one. This bridge supports standard FIX 4.4 protocol. This offers an easy path of integration with existing vendor products already supporting the standard.
Routing by Account#
This is our classic setup where the FIX Bridge maps 1:1 with (sub-)accounts on an exchange.
In this configuration, a FIX client must always be associated with a specific account.
This design is useful when a client needs full control over all aspects of an account. Examples include margin requirements and/or the exchange’s rate-limiter.
Routing by Strategy#
This is our new configuration allowing several FIX Proxies to connect to the same FIX Bridge.
In this configuration, a FIX Client is always associated with a strategy_id.
Note
The strategy_id is a generic identifier that can be used for any purpose.
For example, to identify a specific strategy or a real client (end-user).
The latter may typically be the case for order-routing.
Technically, a route is established between a FIX Proxy and the FIX Bridge when
a FIX Client has succesfully logged on.
The route is then used to forward all order events associated with that specific
strategy_id.
A route is torn down when the FIX Client logs out or disconnects.
The design can also support centralized control over dynamic account routing to better optimize per-account funding needs and/or optimize usage of the exchange’s rate-limiter.
In summary
Multi-tenant environment where any number of clients can trade on the same accounts.
Order routing is dynamic and associated with the life-time of a FIX Client session.
FIX Proxy offers horizontal scaling.
Option to overlay with bespoke solutions for sophisticated account and/or order management.
Journal#
The FIX protocol requires us to mange the ClOrdID as well as to manage the sequencing
of ClOrdID / OrigClOrdID rewrites.
Important
To guarantee uniqueness, we must map between an external ClOrdID and our own
internally generated order_id.
The internally generated order_id can be sent to the exchange so it can
be used to reference the order when any further order requests have to be sent.
However, it is seldom possible to find any space on the exchange order request to
persist the FIX ClOrdID on the exchange itself.
Furthermore, few exchanges (if any) allow us to support ClOrdID / OrigClOrdID
rewrites.
These are the main reasons for describing the journal in this section.
Roq’s Request Management was designed to support sequencing by assigning version numbers to order requests and the manage the various request and response versions on the order objects themselves.
To support gateway restarts, the roq-journal was introduced to support distributed writes of order requests and order updates.
The gateway will broadcast roq::OrderAck and roq::OrderUpdate events to a multicast address.
Important
This is single message being broadcast regardless how many instances of roq-journal has been deployed. This is about low latency.
Deployed versions of roq-journal listen to the multicast address and may receive the messages.
Important
There are no guarantees. Multicast allows for dropped datagrams.
When an instance of roq-journal receives a message it will perform these steps
Append to local file (the “journal”) which can be replayed after service restart.
Send an UDP message (an “ack”) back to the gateway.
Asynchronously upload to a database which can support de-duplication, e.g. ClickHouse.
Important
The journal can be configured to send an ack before or after appending to file. It can also be configured to issue a datasync before or after appending to file. These configurations allow for favouring low latency vs durability guarantees.
On gateway restart, the order (request and update) history can be downloaded from the database to
pre-populate order_id to ClOrdID mappings so they are available when the latest order state
is being downloaded from the exchange.
In summary
The journal multicast solution offers distributed persistence.
The design supports low latency.
Probability of loss can be minimized by having more journal instances deployed on different hosts, using different network, etc.
Summary#
We have given a very brief overview of some of the key components involved with the deployment of a FIX order-routing software stack.
If this is something you are interested in, please reach out on Email.