TickStream API

TickStream API allows getting information about streams, managing stream data, creating Cursors to read from and Loaders to write data into streams.

Stream Management

TickStream API provides methods to get, modify, and remove stream data.

Get Stream Information

With TickStream you can get information about streams such as name, description, schema, scope and more.

Java

Get a specific information about a stream.

//Get a stream key
String key = stream.getKey();

//Access a list of StreamOptions (list of stream properties): 
StreamOptions options = stream.getStreamOptions();
String name = options.name;
String description = options.description;
StreamScope scope = options.scope;

//Get stream schema
RecordClassSet metadata = options.getMetadata();

.NET

Get a specific information about a stream.

public class BBO : InstrumentMessage

//Get a stream key 
ITickStream stream = db.GetStream(key);

//Access a list of StreamOptions (list of stream properties): 
StreamOptions streamOptions = stream.GetStreamOptions();
String name = streamOptions.Name;
String description = streamOptions.Description;
StreamScope scope = streamOptions.Scope;

//Get stream schema
RecordClassSet classSet = streamOptions.GetMetaData();

Python

Get a specific information about a stream.

key = stream.key();

options = stream.options()
name = options.name()
description = options.description()
metadata = options.metadata() # xml stream schema

Update Stream Information

TickStream allows changing stream information. Note that stream key and name must remain unique.

Java

stream.setName("NewStreamName");
stream.setDescription("My awesome stream");
stream.setPolymorphic(descriptors);

.NET

stream.Name = "NewStreamName";
stream.Description = "My awesome stream";
RecordClassDescriptor[] descriptors;
...
stream.SetPolymorphicDescriptors(descriptors);

Rename Stream

Use rename() method to change a stream key.

Java

stream.rename("NewStreamKey");

.NET

stream.Rename("NewStreamKey");

info

Refer to TimeBase Shell to learn more about renaming streams.

Delete Stream

Use delete() method to remove the entire stream.

Java

Delete entire stream and all the data in the stream.

stream.delete();

.NET

Delete entire stream and all the data in the stream.

stream.Delete();

Python

Delete entire stream and all the data in the stream.

stream.deleteStream()

Entities

Each message has a "key" field called Symbol. Symbol represents domain-specific names such as IoT sensor name, trade instrument id, user id, etc. TimeBase streams use Symbols as permanent indices and allow filtering by Symbols. On an API level, each message implements IdentityKey interface holding Symbol string representation:

public interface IdentityKey {
    public CharSequence getSymbol ();
}

To collect information about stream data TickStream has following methods:

• getTimeRange() - use to get a time range for the entire stream or specific symbols.
• listEntities() - use to get a list of all symbols present in the stream.
• renameInstruments(from, to) - use to rename stream symbols.

Java

//Get a list of symbols
IdentityKey[] ids = stream.listEntities();

//Get timsetamps of the first and last message in the stream
long[] timerange = stream.getTimeRange(); 

//Get timsetamps of the first and last message for the "EPAM" symbol
long[] symbolRange = stream.getTimeRange(new ConstantIdentityKey("EPAM")); 

//Rename stream symbols
IdentityKey[] from = new IdentityKey[] { new ConstantIdentityKey("DELTIX") };
IdentityKey[] to = new IdentityKey[] { new ConstantIdentityKey("EPAM") };
stream.renameInstruments(from, to);

.NET

//Get a list of symbols
IIdentityKey[] ids = stream.ListEntities();

//Get timsetamps of the first and last message in the stream
DateTime[] range = stream.GetTimeRange();

//Get timsetamps of the first and last message for the "EPAM" symbol
DateTime[] symbolRange = stream.getTimeRange(new ConstantIdentityKey("EPAM"));

Python

# Get a list of symbols
entities = stream.listEntities()

# Get timsetamps of the first and last message in the stream
range = stream.getTimeRange()

# Get timsetamps of the first and last message for the "EPAM" symbol
symbol_range = stream.getTimeRange([dxapi.InstrumentIdentity(dxapi.InstrumentType.EQUITY, 'EPAM')])

Writing Data

TickStream API allows creating Loaders to write data into streams.

info

Refer to code samples to learn more how to create loaders and write into streams.

Reading Data

TickStream API allows creating Cursors to read from streams.

Pass to select() method such input parameters as start reading timestamp, selection options, message types and symbols you want to read from the stream. It is possible to dynamically change data subscription while reading stream data.

Java

try (TickCursor cursor = stream.select(Long.MIN_VALUE, new SelectionOptions(), types, entities)) {
...
    }

.NET

using (ITickCursor cursor = stream.Select(DateTime.MinValue, new SelectionOptions(), types, ids)){
...
    }

Python

with stream.trySelect(timestamp, dxapi.SelectionOptions(), types, entities) as cursor:
    # iterate cursor
    while cursor.next():
        message = cursor.getMessage()

info

Refer to code samples to learn more about readers (cursors) and how to read data from streams.

Data Modification

Use below methods to modify stream data.

Clear Stream Data

Use clear() command to clear all stream data or just the data recorded for specific symbols.

Java

//Clear all stream data
stream.clear();

//Remove data for "APPL" symbol only
IdentityKey[] identities = new IdentityKey[] { new ConstantIdentityKey("APPL") };
stream.clear(identities); 

.NET

//Clear all stream data
stream.Clear();

//Remove data for "APPL" symbol only
IIdentityKey[] identities = new IIdentityKey[] { new ConstantIdentityKey("APPL") };
stream.Clear(identities); 

Python

#Clear all stream data
stream.clear();

#Remove data for "APPL" symbol only
identities = [dxapi.InstrumentIdentity(dxapi.InstrumentType.EQUITY, 'APPL')]
stream.clear(identities)

Truncate

Use truncate () command to clear data after the defined timestamp. Truncate can be applied both to the entire stream and specific symbols.

Java

//Truncating using the current time
long timestamp = System.currentTimeMillis();

//Truncate all the stream data
stream.truncate(timestamp);

//Truncate all data for "APPL" symbol only
IdentityKey[] identities = new IdentityKey[] { new ConstantIdentityKey("APPL") };
stream.truncate(timestamp, identities);

.NET

//Truncate all stream data
DateTime time = DateTime.Now;
stream.Truncate(time);

//Truncate all the data for "APPL" symbol only
IIdentityKey[] identities = new IIdentityKey[] { new ConstantIdentityKey("APPL") };
stream.Truncate(time, identities);

Python

#Truncate all the stream data
stream.truncate(timestamp);

#Truncate all data for "APPL" symbol only
identities = [dxapi.InstrumentIdentity(dxapi.InstrumentType.EQUITY, 'APPL')]
stream.truncate(timestamp, identities);

info

Refer to TimeBase Shell to read more about truncating streams.

Purge

Use purge() command to clear data before the defined timestamp. Purge applies to the entire stream.

Java

//Purge all the data older than 1 day
long timestamp = LocalDateTime.now().minusDays(1).toEpochSecond(ZoneOffset.UTC) * 1000;
stream.purge(timestamp);

.NET

//Purge all the data older than 1 day
DateTime time = DateTime.Now.AddDays(-1);
stream.Purge(time);

Python

#Purge all the data older than 1 day
stream.purge(timestamp);

info

Refer to TimeBase Shell to read more about purging streams.

Delete

Use delete() command to clear stream data for specific symbols (applies to all symbols in case omitted) and within a defined time range. fromTimestamp and toTimestamp are inclusive parameters.

caution

If toTimestamp is later than the stream's last timestamp, the entire stream is deleted.

Java

stream.delete(fromTimestamp, toTimestamp, new IdentityKey[] { new ConstantIdentityKey("APPL") });

.NET

DateTime from = DateTime.Now.AddDays(-1);
DateTime to = DateTime.Now.AddDays(-2);
stream.Delete(from, to, new IIdentityKey[] { new ConstantIdentityKey("APPL") });

info

Refer to TimeBase Shell to read more about deleting streams.

Stream Locking

TimeBase supports two stream locking types: read and write locks.

info

Refer to TimeBase Shell to read more about locking streams.

Read Lock

Read Lock, when acquired, guarantees that stream data is not going to change until the lock is released. Multiple Read Locks can be acquired on a single stream, corresponding the number of readers (cursors).

Java

DBLock lock = null;
try {
    lock = stream.lock(LockType.READ);
    // ... do stream read stream
    
} finally {
    if (lock != null)
        lock.release();
}

// grabbing lock with timeout       

DBLock lock = null;
try {
    lock = stream.tryLock(LockType.READ, 30000); // 30 seconds to wait
    // ... do stream read stream
    
} finally {
    if (lock != null)
        lock.release();
}

.NET

IDbLock streamLock = null;
try
{
    streamLock = stream.Lock(LockType.Read);
}
finally
{
    if (streamLock != null)
        streamLock.Release();
}

// grabbing lock with timeout       
    
try
{
    streamLock = stream.TryLock(LockType.Read, 30000); // 30 seconds to wait
}
finally
{
    if (streamLock != null)
        streamLock.Release();
}

Write Lock

Write Lock is an exclusive right for a stream. When acquired, it is not possible to get a Read Lock or another Write Lock for this particular stream from another user. Write Lock allows modifying streams without anyone interfering in the process. It is possible to acquire a Write Lock only after all Read Locks are released.

Java

DBLock lock = null;
try {
    lock = stream.lock(LockType.WRITE);
    // ... do stream read stream
    
} finally {
    if (lock != null)
        lock.release();
}

// grabbing lock with timeout       

DBLock lock = null;
try {
    lock = stream.tryLock(LockType.WRITE, 30000); // 30 seconds to wait
    // ... do stream read stream
    
} finally {
    if (lock != null)
        lock.release();
}

.NET

IDbLock streamLock = null;
try
{
    streamLock = stream.Lock(LockType.Write);
}
finally
{
    if (streamLock != null)
        streamLock.Release();
}

// grabbing lock with timeout       
    
try
{
    streamLock = stream.TryLock(LockType.Write, 30000); // 30 seconds to wait
}
finally
{
    if (streamLock != null)
        streamLock.Release();
}

Locks are identified by the connection id, it means that when the connection is closed or dropped all locks owned by this connection are going to be released. In both cases stream locking is made per stream.

Working with Spaces (Data Partitions)

On a TickStream level you can rename and delete spaces. Refer to Data Distribution to learn how to create spaces using option.space.

Java

//Rename spaces
stream.renameSpace("newName", "oldName");

//Delete spaces
stream.deleteSpace("2019", "2020");
...

//Get space metadata

//Get time range
long[] range = stream.getTimeRange("2020");

//Get entities (symbols)
InstrumentIdentity[] ids = stream.listEntities("2020");

.NET

//Rename spaces
stream.RenameSpace("newName", "oldName");

//Delete spaces
stream.DeleteSpace("2019", "2020");
...

//Get space metadata

//Get time range
long[] range = stream.getTimeRange("2020");

//Get entities (symbols)
InstrumentIdentity[] ids = stream.ListEntities("2020");

Python

#Rename spaces
stream.renameSpace('newName', 'oldName');

#Delete spaces
stream.deleteSpaces('2019', '2020');

range = stream.getSpaceTimeRange('2020');

info

Refer to Data Distribution for more information about spaces.