@Internal public class BarrierBuffer extends Object implements CheckpointBarrierHandler
CheckpointBarrierHandler
that blocks inputs with barriers until
all inputs have received the barrier for a given checkpoint.
To avoid back-pressuring the input streams (which may cause distributed deadlocks), the BarrierBuffer continues receiving buffers from the blocked channels and stores them internally until the blocks are released.
Constructor and Description |
---|
BarrierBuffer(InputGate inputGate,
IOManager ioManager)
Creates a new checkpoint stream aligner.
|
BarrierBuffer(InputGate inputGate,
IOManager ioManager,
long maxBufferedBytes)
Creates a new checkpoint stream aligner.
|
Modifier and Type | Method and Description |
---|---|
void |
cleanup()
Cleans up all internally held resources.
|
long |
getAlignmentDurationNanos()
Gets the time that the latest alignment took, in nanoseconds.
|
long |
getCurrentCheckpointId()
Gets the ID defining the current pending, or just completed, checkpoint.
|
BufferOrEvent |
getNextNonBlocked()
Returns the next
BufferOrEvent that the operator may consume. |
boolean |
isEmpty()
Checks if the barrier handler has buffered any data internally.
|
void |
registerCheckpointEventHandler(StatefulTask toNotifyOnCheckpoint)
Registers the task be notified once all checkpoint barriers have been received for a checkpoint.
|
String |
toString() |
public BarrierBuffer(InputGate inputGate, IOManager ioManager) throws IOException
There is no limit to how much data may be buffered during an alignment.
inputGate
- The input gate to draw the buffers and events from.ioManager
- The I/O manager that gives access to the temp directories.IOException
- Thrown, when the spilling to temp files cannot be initialized.public BarrierBuffer(InputGate inputGate, IOManager ioManager, long maxBufferedBytes) throws IOException
The aligner will allow only alignments that buffer up to the given number of bytes. When that number is exceeded, it will stop the alignment and notify the task that the checkpoint has been cancelled.
inputGate
- The input gate to draw the buffers and events from.ioManager
- The I/O manager that gives access to the temp directories.maxBufferedBytes
- The maximum bytes to be buffered before the checkpoint aborts.IOException
- Thrown, when the spilling to temp files cannot be initialized.public BufferOrEvent getNextNonBlocked() throws Exception
CheckpointBarrierHandler
BufferOrEvent
that the operator may consume.
This call blocks until the next BufferOrEvent is available, or until the stream
has been determined to be finished.getNextNonBlocked
in interface CheckpointBarrierHandler
null
, if the stream is finished.IOException
- Thrown if the network or local disk I/O fails.InterruptedException
- Thrown if the thread is interrupted while blocking during
waiting for the next BufferOrEvent to become available.Exception
- Thrown in case that a checkpoint fails that is started as the result of receiving
the last checkpoint barrierpublic void registerCheckpointEventHandler(StatefulTask toNotifyOnCheckpoint)
CheckpointBarrierHandler
registerCheckpointEventHandler
in interface CheckpointBarrierHandler
toNotifyOnCheckpoint
- The task to notifypublic boolean isEmpty()
CheckpointBarrierHandler
isEmpty
in interface CheckpointBarrierHandler
True
, if no data is buffered internally, false
otherwise.public void cleanup() throws IOException
CheckpointBarrierHandler
cleanup
in interface CheckpointBarrierHandler
IOException
- Thrown if the cleanup of I/O resources failed.public long getCurrentCheckpointId()
public long getAlignmentDurationNanos()
CheckpointBarrierHandler
getAlignmentDurationNanos
in interface CheckpointBarrierHandler
Copyright © 2014–2018 The Apache Software Foundation. All rights reserved.