roq-clickhouse¶

Links¶

Purpose¶

Monitor event-logs streamed to a filesystem
Upload history to a Clickhouse timeseries database.

Conda¶

Using Conda

Install

$ conda install \
  --channel https://roq-trading.com/conda/stable \
  roq-clickhouse

Run

$ roq-clickhouse \
      --name "clickhouse" \
      --service_listen_address "$TCP_LISTEN_PORT" \
      --event_log_dir "$EVENT_LOG_DIR" \
      --flagfile "$FLAG_FILE"

Flags¶

Using Flags

$ roq-clickhouse --help

Flags

--print_ddl (false): Print ddl? (and exit)
--event_log_poll_freq (60s): Event-log poll frequency
--event_log_max_batch_events (1000): Max events to process before rotating to next event-log
--event_log_dir: Event-log directory
--event_log_include: Event-log include patterns (regex)
--event_log_exclude: Event-log exclude patterns (regex)
--market_by_price_cache_max_depth (32768): Market-by-price book depth
--market_by_price_view_depth (3): Market-by-price view depth
--market_by_order_cache_max_depth (32768): Market-by-order book depth
--market_by_order_view_depth (3): Market-by-order view depth
--snapshot_lookback_days (200): How many days to include when finding max(seqno) per session_id
--temp_drop_mbp (false): Do not populate MbP? (temporary work-around)

Database

--request_buffer_flush_after (30s): Flush request buffer after idle
--request_encode_buffer_size (4194304): Encode buffer size (bytes) for requests
--request_max_rows (32768): Max rows per request
--request_timeout (120s): Request timeout
--request_rate_limit_interval (5s): Rate limit interval
--request_rate_limit_max_requests (100): Rate limit maximum requests per interval
--server_ping_freq (30s): Ping frequency
--server_ping_path (/ping): Ping path
--request_retry_wait (10s): Retry wait period
--request_retry_max (3): Max retry attempts
--retention_policy_time_to_live_days (0): Time-to-live (days)
--uri (http://localhost:8123): Server URI
--proxy: Proxy URI
--username: Username
--password: Password
--database_name (roq): Database name
--create_database (false): Create database?

Environments¶

localhost

# flagfile

--uri=http://localhost:8123
--database_name=roq
--create_database=True
--retention_policy_time_to_live_days=2