public abstract class AbstractStateBackend extends Object implements Serializable, Closeable
Modifier and Type | Class and Description |
---|---|
static class |
AbstractStateBackend.CheckpointStateOutputStream
A dedicated output stream that produces a
StreamStateHandle when closed. |
static class |
AbstractStateBackend.CheckpointStateOutputView
A dedicated DataOutputView stream that produces a
StateHandle<DataInputView> when closed. |
Modifier and Type | Field and Description |
---|---|
protected Object |
currentKey |
protected TypeSerializer<?> |
keySerializer |
protected HashMap<String,KvState<?,?,?,?,?>> |
keyValueStatesByName
So that we can give out state when the user uses the same key.
|
protected ClassLoader |
userCodeClassLoader |
Constructor and Description |
---|
AbstractStateBackend() |
Modifier and Type | Method and Description |
---|---|
abstract <S extends Serializable> |
checkpointStateSerializable(S state,
long checkpointID,
long timestamp)
Writes the given state into the checkpoint, and returns a handle that can retrieve the state back.
|
abstract void |
close()
Closes the state backend, dropping and aborting all I/O operations that are currently
pending.
|
abstract AbstractStateBackend.CheckpointStateOutputStream |
createCheckpointStateOutputStream(long checkpointID,
long timestamp)
Creates an output stream that writes into the state of the given checkpoint.
|
AbstractStateBackend.CheckpointStateOutputView |
createCheckpointStateOutputView(long checkpointID,
long timestamp)
Creates a
DataOutputView stream that writes into the state of the given checkpoint. |
protected abstract <N,T,ACC> FoldingState<T,ACC> |
createFoldingState(TypeSerializer<N> namespaceSerializer,
FoldingStateDescriptor<T,ACC> stateDesc)
Creates and returns a new
FoldingState . |
protected abstract <N,T> ListState<T> |
createListState(TypeSerializer<N> namespaceSerializer,
ListStateDescriptor<T> stateDesc)
Creates and returns a new
ListState . |
protected abstract <N,T> ReducingState<T> |
createReducingState(TypeSerializer<N> namespaceSerializer,
ReducingStateDescriptor<T> stateDesc)
Creates and returns a new
ReducingState . |
protected abstract <N,T> ValueState<T> |
createValueState(TypeSerializer<N> namespaceSerializer,
ValueStateDescriptor<T> stateDesc)
Creates and returns a new
ValueState . |
void |
dispose()
Releases all resources held by this state backend.
|
abstract void |
disposeAllStateForCurrentJob()
Disposes all state associated with the current job.
|
Object |
getCurrentKey() |
<N,S extends State> |
getPartitionedState(N namespace,
TypeSerializer<N> namespaceSerializer,
StateDescriptor<S,?> stateDescriptor)
Creates or retrieves a partitioned state backed by this state backend.
|
void |
initializeForJob(Environment env,
String operatorIdentifier,
TypeSerializer<?> keySerializer)
This method is called by the task upon deployment to initialize the state backend for
data for a specific job.
|
void |
injectKeyValueStateSnapshots(HashMap<String,KvStateSnapshot> keyValueStateSnapshots)
Injects K/V state snapshots for lazy restore.
|
<N,S extends MergingState<?,?>> |
mergePartitionedStates(N target,
Collection<N> sources,
TypeSerializer<N> namespaceSerializer,
StateDescriptor<S,?> stateDescriptor) |
void |
notifyOfCompletedCheckpoint(long checkpointId) |
void |
setCurrentKey(Object currentKey)
Sets the current key that is used for partitioned state.
|
HashMap<String,KvStateSnapshot<?,?,?,?,?>> |
snapshotPartitionedState(long checkpointId,
long timestamp) |
protected transient TypeSerializer<?> keySerializer
protected transient ClassLoader userCodeClassLoader
protected transient Object currentKey
public void initializeForJob(Environment env, String operatorIdentifier, TypeSerializer<?> keySerializer) throws Exception
env
- The Environment
of the task that instantiated the state backendoperatorIdentifier
- Unique identifier for naming states created by this backendException
- Overwritten versions of this method may throw exceptions, in which
case the job that uses the state backend is considered failed during
deployment.public abstract void disposeAllStateForCurrentJob() throws Exception
Exception
- Exceptions may occur during disposal of the state and should be forwarded.public abstract void close() throws IOException
close
in interface Closeable
close
in interface AutoCloseable
IOException
- Exceptions can be forwarded and will be logged by the systempublic void dispose() throws Exception
This method must make sure that all resources are disposed, even if an exception happens on the way.
Exception
- This method should report exceptions that occur.protected abstract <N,T> ValueState<T> createValueState(TypeSerializer<N> namespaceSerializer, ValueStateDescriptor<T> stateDesc) throws Exception
ValueState
.N
- The type of the namespace.T
- The type of the value that the ValueState
can store.namespaceSerializer
- TypeSerializer for the state namespace.stateDesc
- The StateDescriptor
that contains the name of the state.Exception
protected abstract <N,T> ListState<T> createListState(TypeSerializer<N> namespaceSerializer, ListStateDescriptor<T> stateDesc) throws Exception
ListState
.N
- The type of the namespace.T
- The type of the values that the ListState
can store.namespaceSerializer
- TypeSerializer for the state namespace.stateDesc
- The StateDescriptor
that contains the name of the state.Exception
protected abstract <N,T> ReducingState<T> createReducingState(TypeSerializer<N> namespaceSerializer, ReducingStateDescriptor<T> stateDesc) throws Exception
ReducingState
.N
- The type of the namespace.T
- The type of the values that the ListState
can store.namespaceSerializer
- TypeSerializer for the state namespace.stateDesc
- The StateDescriptor
that contains the name of the state.Exception
protected abstract <N,T,ACC> FoldingState<T,ACC> createFoldingState(TypeSerializer<N> namespaceSerializer, FoldingStateDescriptor<T,ACC> stateDesc) throws Exception
FoldingState
.N
- The type of the namespace.T
- Type of the values folded into the stateACC
- Type of the value in the state *namespaceSerializer
- TypeSerializer for the state namespace.stateDesc
- The StateDescriptor
that contains the name of the state.Exception
public void setCurrentKey(Object currentKey)
currentKey
- The current key.public Object getCurrentKey()
public <N,S extends State> S getPartitionedState(N namespace, TypeSerializer<N> namespaceSerializer, StateDescriptor<S,?> stateDescriptor) throws Exception
N
- The type of the namespace.S
- The type of the state.stateDescriptor
- The state identifier for the state. This contains name
and can create a default state value.Exception
- Exceptions may occur during initialization of the state and should be forwarded.public <N,S extends MergingState<?,?>> void mergePartitionedStates(N target, Collection<N> sources, TypeSerializer<N> namespaceSerializer, StateDescriptor<S,?> stateDescriptor) throws Exception
Exception
public HashMap<String,KvStateSnapshot<?,?,?,?,?>> snapshotPartitionedState(long checkpointId, long timestamp) throws Exception
Exception
public void notifyOfCompletedCheckpoint(long checkpointId) throws Exception
Exception
public void injectKeyValueStateSnapshots(HashMap<String,KvStateSnapshot> keyValueStateSnapshots) throws Exception
keyValueStateSnapshots
- The Map of snapshotsException
public abstract AbstractStateBackend.CheckpointStateOutputStream createCheckpointStateOutputStream(long checkpointID, long timestamp) throws Exception
checkpointID
- The ID of the checkpoint.timestamp
- The timestamp of the checkpoint.Exception
- Exceptions may occur while creating the stream and should be forwarded.public AbstractStateBackend.CheckpointStateOutputView createCheckpointStateOutputView(long checkpointID, long timestamp) throws Exception
DataOutputView
stream that writes into the state of the given checkpoint.
When the stream is closes, it returns a state handle that can retrieve the state back.checkpointID
- The ID of the checkpoint.timestamp
- The timestamp of the checkpoint.Exception
- Exceptions may occur while creating the stream and should be forwarded.public abstract <S extends Serializable> StateHandle<S> checkpointStateSerializable(S state, long checkpointID, long timestamp) throws Exception
S
- The type of the state.state
- The state to be checkpointed.checkpointID
- The ID of the checkpoint.timestamp
- The timestamp of the checkpoint.Exception
- Exceptions may occur during serialization / storing the state and should be forwarded.Copyright © 2014–2017 The Apache Software Foundation. All rights reserved.