Skip to main content

Memory Usage

Large number of instruments in securities stream​

Large number (hundred thousands plus) instrument defined in TimeBase securities stream may take considerable memory foot print in Aggregator (since it maintains per instrument field-value cache).

You can reduce this memory footprint using two settings defined below

Instruments filter​

Reduces set of instruments will be loaded into SecurityMetadata cache aggregator connectors may use. Example:

Aggregator.smd.instrumentFilter=Bloomberg$Symbol IS NOT NULL

Provided value must be valid QQL Condition.

Fields filter​

Reduces set of fields SecurityMetadata cache will keep for each symbol (defined as comma-separated list). Example:

Aggregator.smd.fieldFilter=ActiveContract,Bloomberg$Symbol

Hybrid data connector buffer​

When Hybrid data connector is used pay attention to live message buffer sizes specified via liveMsgBufferSize parameter (default is 20000).

<liveMsgBufferSize>20000</liveMsgBufferSize>

This parameter defines how many live messages could be accumulated. For example for feeds that collect trades each trade message is encoded in 40-50 bytes. If you multiply this by buffer size (20000) and number of instruments that may undergo gap recovery (say 2000), we would need 2G memory requirement just for these live message buffers.