@Internal public class BufferSpiller extends Object
This implementation buffers data effectively in the OS cache, which gracefully extends to the disk. Most data is written and re-read milliseconds later. The file is deleted after the read. Consequently, in most cases, the data will never actually hit the physical disks.
IMPORTANT: The SpilledBufferOrEventSequences created by this spiller all reuse the same reading memory (to reduce overhead) and can consequently not be read concurrently.
Modifier and Type | Class and Description |
---|---|
static class |
BufferSpiller.SpilledBufferOrEventSequence
This class represents a sequence of spilled buffers and events, created by the
BufferSpiller . |
Constructor and Description |
---|
BufferSpiller(IOManager ioManager,
int pageSize)
Creates a new buffer spiller, spilling to one of the I/O manager's temp directories.
|
Modifier and Type | Method and Description |
---|---|
void |
add(BufferOrEvent boe)
Adds a buffer or event to the sequence of spilled buffers and events.
|
void |
close()
Cleans up the current spilling channel and file.
|
long |
getBytesWritten()
Gets the number of bytes written in the current spill file.
|
BufferSpiller.SpilledBufferOrEventSequence |
rollOver()
Starts a new sequence of spilled buffers and event and returns the current sequence of spilled buffers
for reading.
|
BufferSpiller.SpilledBufferOrEventSequence |
rollOverWithNewBuffer()
Starts a new sequence of spilled buffers and event and returns the current sequence of spilled buffers
for reading.
|
public BufferSpiller(IOManager ioManager, int pageSize) throws IOException
ioManager
- The I/O manager for access to teh temp directories.pageSize
- The page size used to re-create spilled buffers.IOException
- Thrown if the temp files for spilling cannot be initialized.public void add(BufferOrEvent boe) throws IOException
boe
- The buffer or event to add and spill.IOException
- Thrown, if the buffer of event could not be spilled.public BufferSpiller.SpilledBufferOrEventSequence rollOver() throws IOException
null
, if nothing was added since the creation of the spiller, or the
last call to this method.
NOTE: The SpilledBufferOrEventSequences created by this method all reuse the same
reading memory (to reduce overhead) and can consequently not be read concurrently with each other.
To create a sequence that can be read concurrently with the previous SpilledBufferOrEventSequence, use the
rollOverWithNewBuffer()
method.
IOException
- Thrown, if the readable sequence could not be created, or no new spill
file could be created.public BufferSpiller.SpilledBufferOrEventSequence rollOverWithNewBuffer() throws IOException
null
, if nothing was added since the creation of the spiller, or the
last call to this method.
The SpilledBufferOrEventSequence returned by this method is safe for concurrent consumption with any previously returned sequence.
IOException
- Thrown, if the readable sequence could not be created, or no new spill
file could be created.public void close() throws IOException
Does not clean up the SpilledBufferOrEventSequences generated by calls to
rollOver()
.
IOException
- Thrown if channel closing or file deletion fail.public long getBytesWritten()
Copyright © 2014–2018 The Apache Software Foundation. All rights reserved.