Capture and Inspect Event-Logs

You will learn to inspect the event-log, Roq’s native message capture solution.

Note

The event-log captures all events (messages) seen by a gateway.

Everything is APPENDED into the SAME event-log. Accurate time-ordering is therefore preserved.

The event-log is useful for inspecting “what-happened”, exporting into other database solutions and for HISTORICAL SIMULATION.

Conda

Install the roq-tools package

$ conda install -y -c https://roq-trading.com/conda/unstable roq-tools

Configure

Event-logs can potentially consume a lot of disk space and must therefore be enabled.

To enable the event-log, add the --event_log_dir flag the gateway’s command-line

$ roq-deribit \
    --name "deribit" \
    --config_file $HOME/deribit.toml \
    --client_listen_address unix://$HOME/deribit.sock \
    --flagfile $CONDA_PREFIX/share/roq-deribit/flags/test/flags.cfg
    --event_log_dir $HOME/roq

The gateway will then create the following directories and symlinks

$ ls -ltr $HOME/roq

total 8
drwxr-x--- 3 thraneh thraneh 4096 May  4 07:19 public
drwxr-x--- 3 thraneh thraneh 4096 May  4 07:19 private
lrwxrwxrwx 1 thraneh thraneh   56 May  4 07:19 deribit-public.roq -> public/2026-W19/deribit/deribit-public-1777879198185.roq
lrwxrwxrwx 1 thraneh thraneh   58 May  4 07:19 deribit-private.roq -> private/2026-W19/deribit/deribit-private-1777879198185.roq

The event-log is split into public and private data for various reasons, including

  • Separation of “sensitive” data (private) from “insensitive” data (public).

  • Speed of accessing private data when investigating execution issues. (The private files are typically much, much smaller than the public files.)

Tools

The “roq-tools” package has various tools to inspect and manipulate the event-logs.

Summary

We can get a summary overview of the public data

$ roq-summary $HOME/roq/deribit-public.roq

The output can be a bit length.

The first half contains meta-data and such

../../../_images/summary_public_1.png

The second half contains counters per event-type

../../../_images/summary_public_2.png

The similar summary overview of private data

$ roq-summary $HOME/roq/deribit-public.roq

The last half of the output

../../../_images/summary_private.png

Dump

We can see each event like this

$ roq-dump $HOME/roq/deribit-public.roq

This is quite verbose and we can limit the output in different ways. Here we just use grep.

Reference data like this

$ roq-dump $HOME/roq/deribit-public.roq | grep reference_data | grep BTC-PERPETUAL

1777879198521841930ns reference_data={stream_id=1, exchange="deribit", symbol="BTC-PERPETUAL", description="", security_type=SWAP, external_security_id=0, cfi_code="", base_currency="BTC", quote_currency="USD", settlement_currency="BTC", margin_currency="", commission_currency="", tick_size=0.5, tick_size_steps=[], multiplier=10, min_notional=nan, min_trade_vol=1, max_trade_vol=nan, trade_vol_step_size=1, option_type=UNDEFINED, strike_currency="", strike_price=nan, underlying="btc_usd", time_zone="", issue_date=17756d, settlement_date=0d, expiry_datetime=0s, expiry_datetime_utc=32503708800s, exchange_time_utc=0ns, exchange_sequence=0, sending_time_utc=0ns, discard=false}

Top of book like this

$ roq-dump $HOME/roq/deribit-public.roq | grep top_of_book | grep BTC-PERPETUAL

1777879198736497010ns top_of_book={stream_id=0, exchange="deribit", symbol="BTC-PERPETUAL", layer={bid_price=78170, bid_quantity=45011, ask_price=79866, ask_quantity=13652}, update_type=INCREMENTAL, exchange_time_utc=1777879198354000000ns, exchange_sequence=0, sending_time_utc=0ns, price_precision=UNDEFINED, quantity_precision=UNDEFINED}
1777879199265269465ns top_of_book={stream_id=0, exchange="deribit", symbol="BTC-PERPETUAL", layer={bid_price=78170, bid_quantity=45011, ask_price=78305, ask_quantity=3200}, update_type=INCREMENTAL, exchange_time_utc=1777879199248000000ns, exchange_sequence=0, sending_time_utc=0ns, price_precision=UNDEFINED, quantity_precision=UNDEFINED}
1777879199459126383ns top_of_book={stream_id=0, exchange="deribit", symbol="BTC-PERPETUAL", layer={bid_price=75933, bid_quantity=69, ask_price=76284, ask_quantity=34}, update_type=INCREMENTAL, exchange_time_utc=1777879199441000000ns, exchange_sequence=0, sending_time_utc=0ns, price_precision=UNDEFINED, quantity_precision=UNDEFINED}
[...]

Note

Event-types are named similarly to what you see with “roq-summary”.

We can also insepct order request and response messages in the private file

$ roq-dump $HOME/roq/deribit-private.roq | grep order_

1777879265753956422ns create_order={account="A1", order_id=421895982239, exchange="deribit", symbol="BTC-PERPETUAL", side=BUY, position_effect=UNDEFINED, margin_mode=UNDEFINED, quantity_type=UNDEFINED, max_show_quantity=nan, order_type=LIMIT, time_in_force=GTC, execution_instructions=, request_template="", quantity=1, price=50000, stop_price=nan, leverage=nan, routing_id="", strategy_id=0, release_time_utc=0ns}
1777879265753956422ns order_ack={stream_id=2, account="A1", order_id=421895982239, exchange="deribit", symbol="BTC-PERPETUAL", side=BUY, position_effect=UNDEFINED, margin_mode=UNDEFINED, quantity_type=UNDEFINED, request_type=CREATE_ORDER, origin=GATEWAY, request_status=FORWARDED, error=UNDEFINED, text="", request_id="sAAIn8z1OmIAAQAAAAAA", external_account="", external_order_id="", client_order_id="sAAIn8z1OmIAAQAAAAAA", quantity=1, price=50000, stop_price=nan, leverage=nan, routing_id="", version=1, risk_exposure=1, risk_exposure_change=1, traded_quantity=0, round_trip_latency=0ns, user="trader", strategy_id=0}
1777879265783459477ns order_ack={stream_id=2, account="A1", order_id=421895982239, exchange="deribit", symbol="BTC-PERPETUAL", side=BUY, position_effect=UNDEFINED, margin_mode=UNDEFINED, quantity_type=UNDEFINED, request_type=CREATE_ORDER, origin=EXCHANGE, request_status=ACCEPTED, error=UNDEFINED, text="success", request_id="sAAIn8z1OmIAAQAAAAAA", external_account="", external_order_id="96560775860", client_order_id="sAAIn8z1OmIAAQAAAAAA", quantity=1, price=50000, stop_price=nan, leverage=nan, routing_id="", version=1, risk_exposure=1, risk_exposure_change=nan, traded_quantity=0, round_trip_latency=29470174ns, user="trader", strategy_id=0}
1777879265783459477ns order_update={stream_id=2, account="A1", order_id=421895982239, exchange="deribit", symbol="BTC-PERPETUAL", side=BUY, position_effect=UNDEFINED, margin_mode=UNDEFINED, quantity_type=UNDEFINED, max_show_quantity=nan, order_type=LIMIT, time_in_force=GTC, execution_instructions=, create_time_utc=1777879265772000000ns, update_time_utc=1777879265772000000ns, external_account="", external_order_id="96560775860", client_order_id="sAAIn8z1OmIAAQAAAAAA", order_status=WORKING, quantity=1, price=50000, stop_price=nan, leverage=nan, risk_exposure=1, risk_exposure_change=nan, remaining_quantity=1, traded_quantity=0, average_traded_price=nan, last_traded_quantity=nan, last_traded_price=nan, last_liquidity=UNDEFINED, routing_id="", max_request_version=1, max_response_version=1, max_accepted_version=1, update_type=INCREMENTAL, exchange_time_utc=0ns, exchange_sequence=0, sending_time_utc=1777879265772000000ns, user="trader", strategy_id=0}
1777879270949814350ns modify_order={account="A1", order_id=421895982239, request_template="", quantity=nan, price=51515, routing_id="", version=0, conditional_on_version=0, release_time_utc=0ns}
1777879270949814350ns order_ack={stream_id=2, account="A1", order_id=421895982239, exchange="deribit", symbol="BTC-PERPETUAL", side=BUY, position_effect=UNDEFINED, margin_mode=UNDEFINED, quantity_type=UNDEFINED, request_type=MODIFY_ORDER, origin=GATEWAY, request_status=FORWARDED, error=UNDEFINED, text="", request_id="swAIn8z1OmIAAgAAAAAA", external_account="", external_order_id="96560775860", client_order_id="sAAIn8z1OmIAAQAAAAAA", quantity=nan, price=51515, stop_price=nan, leverage=nan, routing_id="", version=2, risk_exposure=1, risk_exposure_change=nan, traded_quantity=0, round_trip_latency=0ns, user="trader", strategy_id=0}
1777879270977689783ns order_ack={stream_id=2, account="A1", order_id=421895982239, exchange="deribit", symbol="BTC-PERPETUAL", side=BUY, position_effect=UNDEFINED, margin_mode=UNDEFINED, quantity_type=UNDEFINED, request_type=MODIFY_ORDER, origin=EXCHANGE, request_status=ACCEPTED, error=UNDEFINED, text="success", request_id="swAIn8z1OmIAAgAAAAAA", external_account="", external_order_id="96560775860", client_order_id="sAAIn8z1OmIAAQAAAAAA", quantity=nan, price=51515, stop_price=nan, leverage=nan, routing_id="", version=2, risk_exposure=1, risk_exposure_change=nan, traded_quantity=0, round_trip_latency=27830278ns, user="trader", strategy_id=0}
1777879270977689783ns order_update={stream_id=2, account="A1", order_id=421895982239, exchange="deribit", symbol="BTC-PERPETUAL", side=BUY, position_effect=UNDEFINED, margin_mode=UNDEFINED, quantity_type=UNDEFINED, max_show_quantity=nan, order_type=LIMIT, time_in_force=GTC, execution_instructions=, create_time_utc=1777879265772000000ns, update_time_utc=1777879270966000000ns, external_account="", external_order_id="96560775860", client_order_id="sAAIn8z1OmIAAQAAAAAA", order_status=WORKING, quantity=1, price=51515, stop_price=nan, leverage=nan, risk_exposure=1, risk_exposure_change=nan, remaining_quantity=1, traded_quantity=0, average_traded_price=nan, last_traded_quantity=nan, last_traded_price=nan, last_liquidity=UNDEFINED, routing_id="", max_request_version=2, max_response_version=2, max_accepted_version=2, update_type=INCREMENTAL, exchange_time_utc=0ns, exchange_sequence=0, sending_time_utc=1777879270967000000ns, user="trader", strategy_id=0}
1777879273867676120ns cancel_order={account="A1", order_id=421895982239, request_template="", routing_id="", version=0, conditional_on_version=0, release_time_utc=0ns}
1777879273867676120ns order_ack={stream_id=2, account="A1", order_id=421895982239, exchange="deribit", symbol="BTC-PERPETUAL", side=BUY, position_effect=UNDEFINED, margin_mode=UNDEFINED, quantity_type=UNDEFINED, request_type=CANCEL_ORDER, origin=GATEWAY, request_status=FORWARDED, error=UNDEFINED, text="", request_id="sgAIn8z1OmIAAwAAAAAA", external_account="", external_order_id="96560775860", client_order_id="sAAIn8z1OmIAAQAAAAAA", quantity=nan, price=nan, stop_price=nan, leverage=nan, routing_id="", version=3, risk_exposure=1, risk_exposure_change=nan, traded_quantity=0, round_trip_latency=0ns, user="trader", strategy_id=0}
1777879273893002914ns order_ack={stream_id=2, account="A1", order_id=421895982239, exchange="deribit", symbol="BTC-PERPETUAL", side=BUY, position_effect=UNDEFINED, margin_mode=UNDEFINED, quantity_type=UNDEFINED, request_type=CANCEL_ORDER, origin=EXCHANGE, request_status=ACCEPTED, error=UNKNOWN, text="notification", request_id="sgAIn8z1OmIAAwAAAAAA", external_account="", external_order_id="96560775860", client_order_id="sAAIn8z1OmIAAQAAAAAA", quantity=nan, price=nan, stop_price=nan, leverage=nan, routing_id="", version=3, risk_exposure=0, risk_exposure_change=nan, traded_quantity=0, round_trip_latency=25265900ns, user="trader", strategy_id=0}
1777879273893002914ns order_update={stream_id=2, account="A1", order_id=421895982239, exchange="deribit", symbol="BTC-PERPETUAL", side=BUY, position_effect=UNDEFINED, margin_mode=UNDEFINED, quantity_type=UNDEFINED, max_show_quantity=nan, order_type=LIMIT, time_in_force=GTC, execution_instructions=, create_time_utc=1777879265772000000ns, update_time_utc=1777879273881000000ns, external_account="", external_order_id="96560775860", client_order_id="sAAIn8z1OmIAAQAAAAAA", order_status=CANCELED, quantity=1, price=51515, stop_price=nan, leverage=nan, risk_exposure=0, risk_exposure_change=-1, remaining_quantity=1, traded_quantity=0, average_traded_price=nan, last_traded_quantity=nan, last_traded_price=nan, last_liquidity=UNDEFINED, routing_id="", max_request_version=3, max_response_version=3, max_accepted_version=3, update_type=INCREMENTAL, exchange_time_utc=0ns, exchange_sequence=0, sending_time_utc=1777879273882000000ns, user="trader", strategy_id=0}

These are all the messages around a create / modify / cancel scenario.