Skip to main content

Pandas Utils

The Python dxapi client offers a utility module for integration with pandas. This module enables efficient reading of data from TimeBase into a pandas DataFrame and storing it from the DataFrame into stream.

Reading data to DataFrame

Utilize the pandas_utils.read_frame function from the dxapi to import data from streams or QQL query into a pandas.DataFrame.

import dxapi
from dxapi import pandas_utils

try:
db = dxapi.TickDb.createFromUrl("dxtick://localhost:8011")
db.open(True)

fields=['open', 'close', 'high', 'low'] # Specify fields to read from streams
symbols=['IBM', 'AAPL'] # Use None for all symbols
types=['deltix.timebase.api.messages.BarMessage'] # Use None for all types
df = pandas_utils.read_frame(db,
streams=['Daily'],
fields=fields,
tickers=symbols,
types=types,
from_time_str='2001-01-01',
to_time_str='2001-05-02')
print(df)
finally:
if (db.isOpen()):
db.close()

Valid arguments:

  • db (TickDb) TimeBase client instance;
  • streams (list[str]) List of streams to read data from;
  • query (str) QQL query to select data;
  • types (list[str]) type message filter (None means select all types);
  • tickers (list[str]) symbols filter (None means select all symbols);
  • fields (list[str]) fields list to select;
  • from_time (datetime) start time filter;
  • to_time (datetime) end time filter;
  • from_time_str (str) same as 'from_time', but in string format (ex, 2023-01-23 or 2023-01-23 10:30:00 UTC);
  • to_time_str (str) same as 'to_time', but in string format (ex, 2023-01-23 or 2023-01-23 10:30:00 UTC).

If the query parameter is specified, data will be read from the QQL result set; otherwise, it will be retrieved from the streams list.

Write DataFrame to stream

Employ pandas_utils.write_frame in conjunction with pandas_utils.bind_frame. The pandas_utils.bind_frame function enables you to construct DataFrame to stream bindings, thereby controlling how columns should be matched to the stream's fields.

import dxapi
from dxapi import pandas_utils

try:
db = dxapi.TickDb.createFromUrl("dxtick://localhost:8011")
db.open(False)

fields=['open', 'close', 'high', 'low']
symbols=['IBM', 'AAPL'] # Use None for all symbols
types=['deltix.timebase.api.messages.BarMessage'] # Use None for all types
df = pandas_utils.read_frame(db,
streams=['Daily'],
fields=fields,
tickers=symbols,
types=types,
from_time_str='2001-01-01',
to_time_str='2001-05-02')
print(df)

# Write DataFrame to new stream
binding = pandas_utils.bind_frame(db, 'Daily_new', df)

# change bindings if need
pandas_utils.write_frame(db, binding, df)
finally:
if (db.isOpen()):
db.close()