public class RocksDBIncrementalCheckpointUtils extends Object
Constructor and Description |
---|
RocksDBIncrementalCheckpointUtils() |
Modifier and Type | Method and Description |
---|---|
static boolean |
beforeThePrefixBytes(byte[] bytes,
byte[] prefixBytes)
check whether the bytes is before prefixBytes in the character order.
|
static <T extends KeyedStateHandle> |
chooseTheBestStateHandleForInitial(List<T> restoreStateHandles,
KeyGroupRange targetKeyGroupRange,
double overlapFractionThreshold)
Choose the best state handle according to the
stateHandleEvaluator(KeyedStateHandle,
KeyGroupRange, double) to init the initial db. |
static void |
clipDBWithKeyGroupRange(org.rocksdb.RocksDB db,
List<org.rocksdb.ColumnFamilyHandle> columnFamilyHandles,
KeyGroupRange targetKeyGroupRange,
KeyGroupRange currentKeyGroupRange,
int keyGroupPrefixBytes,
boolean useDeleteFilesInRange)
The method to clip the db instance according to the target key group range using the
RocksDB.delete(ColumnFamilyHandle, byte[]) . |
static Optional<RunnableWithException> |
createRangeCompactionTaskIfNeeded(org.rocksdb.RocksDB db,
Collection<org.rocksdb.ColumnFamilyHandle> columnFamilyHandles,
int keyGroupPrefixBytes,
KeyGroupRange dbExpectedKeyGroupRange)
Returns a range compaction task as runnable if any data in the SST files of the given DB
exceeds the proclaimed key-group range.
|
static <T extends KeyedStateHandle> |
findTheBestStateHandleForInitial(List<T> restoreStateHandles,
KeyGroupRange targetKeyGroupRange,
double overlapFractionThreshold)
Choose the best state handle according to the
stateHandleEvaluator(KeyedStateHandle,
KeyGroupRange, double) to init the initial db from the given lists and returns its index. |
static boolean |
isSstDataInKeyGroupRange(org.rocksdb.RocksDB db,
int keyGroupPrefixBytes,
KeyGroupRange dbExpectedKeyGroupRange)
Returns true, if all entries in the sst files of the given DB is strictly within the expected
key-group range for the DB.
|
public static void clipDBWithKeyGroupRange(@Nonnull org.rocksdb.RocksDB db, @Nonnull List<org.rocksdb.ColumnFamilyHandle> columnFamilyHandles, @Nonnull KeyGroupRange targetKeyGroupRange, @Nonnull KeyGroupRange currentKeyGroupRange, @Nonnegative int keyGroupPrefixBytes, boolean useDeleteFilesInRange) throws org.rocksdb.RocksDBException
RocksDB.delete(ColumnFamilyHandle, byte[])
.db
- the RocksDB instance to be clipped.columnFamilyHandles
- the column families in the db instance.targetKeyGroupRange
- the target key group range.currentKeyGroupRange
- the key group range of the db instance.keyGroupPrefixBytes
- Number of bytes required to prefix the key groups.useDeleteFilesInRange
- whether to call db.deleteFilesInRanges for the deleted ranges.org.rocksdb.RocksDBException
public static boolean isSstDataInKeyGroupRange(org.rocksdb.RocksDB db, int keyGroupPrefixBytes, KeyGroupRange dbExpectedKeyGroupRange)
db
- the DB to check.dbExpectedKeyGroupRange
- the expected key-groups range of the DB.keyGroupPrefixBytes
- the number of bytes used to serialize the key-group prefix of keys
in the DB.public static Optional<RunnableWithException> createRangeCompactionTaskIfNeeded(org.rocksdb.RocksDB db, Collection<org.rocksdb.ColumnFamilyHandle> columnFamilyHandles, int keyGroupPrefixBytes, KeyGroupRange dbExpectedKeyGroupRange)
db
- the DB to check and compact if needed.columnFamilyHandles
- list of column families to check.keyGroupPrefixBytes
- the number of bytes used to serialize the key-group prefix of keys
in the DB.dbExpectedKeyGroupRange
- the expected key-groups range of the DB.public static boolean beforeThePrefixBytes(@Nonnull byte[] bytes, @Nonnull byte[] prefixBytes)
@Nullable public static <T extends KeyedStateHandle> T chooseTheBestStateHandleForInitial(@Nonnull List<T> restoreStateHandles, @Nonnull KeyGroupRange targetKeyGroupRange, double overlapFractionThreshold)
stateHandleEvaluator(KeyedStateHandle,
KeyGroupRange, double)
to init the initial db.T
- the generic parameter type of the state handles.restoreStateHandles
- The candidate state handles.targetKeyGroupRange
- The target key group range.overlapFractionThreshold
- configured threshold for overlap.public static <T extends KeyedStateHandle> int findTheBestStateHandleForInitial(@Nonnull List<T> restoreStateHandles, @Nonnull KeyGroupRange targetKeyGroupRange, double overlapFractionThreshold)
stateHandleEvaluator(KeyedStateHandle,
KeyGroupRange, double)
to init the initial db from the given lists and returns its index.T
- the generic parameter type of the state handles.restoreStateHandles
- The candidate state handles.targetKeyGroupRange
- The target key group range.overlapFractionThreshold
- configured threshold for overlap.Copyright © 2014–2024 The Apache Software Foundation. All rights reserved.