Skip to main content

Data Distribution

TimeBase standard use case is when data is written chronologically in streams in a form of messages. Messages are placed in Time Slice Files (TSF) also chronologically; therefore, each TSF represents (describes) a specific time range. As a result, a TimeBase stream may be seen as a chronological collection of TS files. You may use insert mode to subsequently rewrite or modify some data in TS files. Refer to Basic Concepts to learn more.

Introduction to Data Partitions (Spaces)​

TimeBase also allows working with data that is not chronologically arranged. Even though you may use insert mode to write such data, data distribution approach allows maximizing writing performance (increase throughput and speed of writing data). In case you plan to work with data that is not chronologically arranged, you can create dedicated data partitions (spaces) to physically distribute TS files based on their time range or other criteria. Using this approach, a loader can reference a specific space when writing data instead of just ingesting data into a stream. TimeBase allows multiple loaders to write concurrently to specific spaces and reading performance is not impacted by this approach. Similar to loaders, cursors can reference spaces to read from a specific time range. Chronological order is preserved when working with spaces. For example, you can keep all rarely-traded instruments in your portfolio in a specific space, or you can create space, each for a specific time range. In the code example below we write data to a space that stores all messages for the year 2020.

tip

Note, that one loader can write to just one TS file, one cursor can read from more than one TS file at the same time.

caution

Note, that a new space is created (options.space = "2020") only when you start writing messages to it. Empty space cannot be created.

Writing into Spaces​

Write into a space
LoadingOptions options = new LoadingOptions();
options.space = "2020";
try (TickLoader loader = stream.createLoader(options)) {
    InstrumentMessage message;
    // populate message attributes
    ...
    loader.send(message);
    ...
}
info
  • Refer to TickLoader API to learn how to create a space and write into one.
  • Refer to How To to learn more about spaces and how to work with them.

Reading from Spaces​

Select from space
SelectionOptions opts = new SelectionOptions();
opts.raw = false;
opts.live = false;
opts.space = "2020";
int count = 0;
try (TickCursor cursor = stream.select(Long.MIN_VALUE, opts)) {
    while (cursor.next())
        count++;
}
info
  • Refer to TickCursor API to learn how to read from spaces.
  • Refer to How To to learn more about spaces and how to work with them.