@NotThreadSafe public class PartitionSortedBuffer extends Object implements SortBuffer
SortBuffer
implementation which sorts all appended records only by subpartition index.
Records of the same subpartition keep the appended order.
It maintains a list of MemorySegment
s as a joint buffer. Data will be appended to the
joint buffer sequentially. When writing a record, an index entry will be appended first. An index
entry consists of 4 fields: 4 bytes for record length, 4 bytes for Buffer.DataType
and 8 bytes
for address pointing to the next index entry of the same channel which will be used to index the
next record to read when coping data from this SortBuffer
. For simplicity, no index entry
can span multiple segments. The corresponding record data is seated right after its index entry
and different from the index entry, records have variable length thus may span multiple segments.
SortBuffer.BufferWithChannel
Constructor and Description |
---|
PartitionSortedBuffer(Object lock,
BufferPool bufferPool,
int numSubpartitions,
int bufferSize,
int[] customReadOrder) |
Modifier and Type | Method and Description |
---|---|
boolean |
append(ByteBuffer source,
int targetChannel,
Buffer.DataType dataType)
Appends data of the specified channel to this
SortBuffer and returns true if all
bytes of the source buffer is copied to this SortBuffer successfully, otherwise if
returns false, nothing will be copied. |
SortBuffer.BufferWithChannel |
copyIntoSegment(MemorySegment target)
Copies data in this
SortBuffer to the target MemorySegment in channel index
order and returns SortBuffer.BufferWithChannel which contains the copied data and the
corresponding channel index. |
void |
finish()
Finishes this
SortBuffer which means no record can be appended any more. |
boolean |
hasRemaining()
Returns true if there is still data can be consumed in this
SortBuffer . |
boolean |
isFinished()
Whether this
SortBuffer is finished or not. |
boolean |
isReleased()
Whether this
SortBuffer is released or not. |
long |
numBytes()
Returns the number of bytes written to this
SortBuffer . |
long |
numRecords()
Returns the number of records written to this
SortBuffer . |
void |
release()
Releases this
SortBuffer which releases all resources. |
public PartitionSortedBuffer(Object lock, BufferPool bufferPool, int numSubpartitions, int bufferSize, @Nullable int[] customReadOrder)
public boolean append(ByteBuffer source, int targetChannel, Buffer.DataType dataType) throws IOException
SortBuffer
SortBuffer
and returns true if all
bytes of the source buffer is copied to this SortBuffer
successfully, otherwise if
returns false, nothing will be copied.append
in interface SortBuffer
IOException
public SortBuffer.BufferWithChannel copyIntoSegment(MemorySegment target)
SortBuffer
SortBuffer
to the target MemorySegment
in channel index
order and returns SortBuffer.BufferWithChannel
which contains the copied data and the
corresponding channel index.copyIntoSegment
in interface SortBuffer
public long numRecords()
SortBuffer
SortBuffer
.numRecords
in interface SortBuffer
public long numBytes()
SortBuffer
SortBuffer
.numBytes
in interface SortBuffer
public boolean hasRemaining()
SortBuffer
SortBuffer
.hasRemaining
in interface SortBuffer
public void finish()
SortBuffer
SortBuffer
which means no record can be appended any more.finish
in interface SortBuffer
public boolean isFinished()
SortBuffer
SortBuffer
is finished or not.isFinished
in interface SortBuffer
public void release()
SortBuffer
SortBuffer
which releases all resources.release
in interface SortBuffer
public boolean isReleased()
SortBuffer
SortBuffer
is released or not.isReleased
in interface SortBuffer
Copyright © 2014–2021 The Apache Software Foundation. All rights reserved.