org.apache.flink.contrib.streaming.state

Class AbstractRocksDBState<K,N,V,S extends State>

java.lang.Object

org.apache.flink.contrib.streaming.state.AbstractRocksDBState<K,N,V,S>

Type Parameters:

K - The type of the key.

N - The type of the namespace.

V - The type of values kept internally in state.

S - The type of State.

All Implemented Interfaces:

State, InternalKvState<K,N,V>

Direct Known Subclasses:

RocksDBAggregatingState, RocksDBFoldingState, RocksDBListState, RocksDBMapState, RocksDBReducingState, RocksDBValueState

public abstract class AbstractRocksDBState<K,N,V,S extends State>
extends Object
implements InternalKvState<K,N,V>, State

Base class for State implementations that store state in a RocksDB database.

State is not stored in this class but in the RocksDB instance that the RocksDBStateBackend manages and checkpoints.

Field Summary

Fields

Modifier and Type	Field and Description
protected RocksDBKeyedStateBackend<K>	backend Backend that holds the actual RocksDB instance where we store state.
protected org.rocksdb.ColumnFamilyHandle	columnFamily The column family of this particular instance of state.
protected V	defaultValue
protected DataOutputView	keySerializationDataOutputView
protected ByteArrayOutputStreamWithPos	keySerializationStream
protected org.rocksdb.WriteOptions	writeOptions

Constructor Summary

Constructors

Modifier	Constructor and Description
protected	AbstractRocksDBState(org.rocksdb.ColumnFamilyHandle columnFamily, TypeSerializer<N> namespaceSerializer, TypeSerializer<V> valueSerializer, V defaultValue, RocksDBKeyedStateBackend<K> backend) Creates a new RocksDB backed state.

Method Summary

Modifier and Type	Method and Description
void	clear() Removes the value mapped under the current key.
protected V	getDefaultValue()
byte[]	getSerializedValue(byte[] serializedKeyAndNamespace, TypeSerializer<K> safeKeySerializer, TypeSerializer<N> safeNamespaceSerializer, TypeSerializer<V> safeValueSerializer) Returns the serialized value for the given key and namespace.
void	setCurrentNamespace(N namespace) Sets the current namespace, which will be used when using the state access methods.
protected void	writeCurrentKeyWithGroupAndNamespace()
protected void	writeKeyWithGroupAndNamespace(int keyGroup, K key, N namespace, ByteArrayOutputStreamWithPos keySerializationStream, DataOutputView keySerializationDataOutputView)
protected void	writeKeyWithGroupAndNamespace(int keyGroup, K key, TypeSerializer<K> keySerializer, N namespace, TypeSerializer<N> namespaceSerializer, ByteArrayOutputStreamWithPos keySerializationStream, DataOutputView keySerializationDataOutputView)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.flink.runtime.state.internal.InternalKvState

getKeySerializer, getNamespaceSerializer, getValueSerializer

Field Detail

backend

protected RocksDBKeyedStateBackend<K> backend

Backend that holds the actual RocksDB instance where we store state.

columnFamily

protected org.rocksdb.ColumnFamilyHandle columnFamily

The column family of this particular instance of state.

defaultValue

protected final V defaultValue

writeOptions

protected final org.rocksdb.WriteOptions writeOptions

keySerializationStream

protected final ByteArrayOutputStreamWithPos keySerializationStream

keySerializationDataOutputView

protected final DataOutputView keySerializationDataOutputView

Constructor Detail

AbstractRocksDBState

protected AbstractRocksDBState(org.rocksdb.ColumnFamilyHandle columnFamily,
                               TypeSerializer<N> namespaceSerializer,
                               TypeSerializer<V> valueSerializer,
                               V defaultValue,
                               RocksDBKeyedStateBackend<K> backend)

Creates a new RocksDB backed state.

Parameters:

columnFamily - The RocksDB column family that this state is associated to.

namespaceSerializer - The serializer for the namespace.

valueSerializer - The serializer for the state.

defaultValue - The default value for the state.

backend - The backend for which this state is bind to.

Method Detail

clear

public void clear()

Description copied from interface: State

Removes the value mapped under the current key.

Specified by:

clear in interface State

setCurrentNamespace

public void setCurrentNamespace(N namespace)

Description copied from interface: InternalKvState

Sets the current namespace, which will be used when using the state access methods.

Specified by:

setCurrentNamespace in interface InternalKvState<K,N,V>

Parameters:

namespace - The namespace.

getSerializedValue

public byte[] getSerializedValue(byte[] serializedKeyAndNamespace,
                                 TypeSerializer<K> safeKeySerializer,
                                 TypeSerializer<N> safeNamespaceSerializer,
                                 TypeSerializer<V> safeValueSerializer)
                          throws Exception

Description copied from interface: InternalKvState

Returns the serialized value for the given key and namespace.

If no value is associated with key and namespace, null is returned.

TO IMPLEMENTERS: This method is called by multiple threads. Anything stateful (e.g. serializers) should be either duplicated or protected from undesired consequences of concurrent invocations.

Specified by:

getSerializedValue in interface InternalKvState<K,N,V>

Parameters:

serializedKeyAndNamespace - Serialized key and namespace

safeKeySerializer - A key serializer which is safe to be used even in multi-threaded context

safeNamespaceSerializer - A namespace serializer which is safe to be used even in multi-threaded context

safeValueSerializer - A value serializer which is safe to be used even in multi-threaded context

Returns:

Serialized value or null if no value is associated with the key and namespace.

Throws:

Exception - Exceptions during serialization are forwarded

writeCurrentKeyWithGroupAndNamespace

protected void writeCurrentKeyWithGroupAndNamespace()
                                             throws IOException

Throws:

IOException

writeKeyWithGroupAndNamespace

protected void writeKeyWithGroupAndNamespace(int keyGroup,
                                             K key,
                                             N namespace,
                                             ByteArrayOutputStreamWithPos keySerializationStream,
                                             DataOutputView keySerializationDataOutputView)
                                      throws IOException

Throws:

IOException

writeKeyWithGroupAndNamespace

protected void writeKeyWithGroupAndNamespace(int keyGroup,
                                             K key,
                                             TypeSerializer<K> keySerializer,
                                             N namespace,
                                             TypeSerializer<N> namespaceSerializer,
                                             ByteArrayOutputStreamWithPos keySerializationStream,
                                             DataOutputView keySerializationDataOutputView)
                                      throws IOException

Throws:

IOException

getDefaultValue

protected V getDefaultValue()