[ | ----------- Source Connector Thread ----------- | | --- KinesisAsyncClient Thread(s) -- | | FanOutRecordPublisher | FanOutShardSubscription | == blocking queue == | KinesisProxyV2 | KinesisAsyncClient | ]
Three types of message are passed over the queue for inter-thread communication:
SubscriptionNextEvent- passes data from the network to the consumer
SubscriptionCompleteEvent- indicates a subscription has expired
SubscriptionErrorEvent- passes an exception from the network to the consumer
The blocking queue has a maximum capacity of 1 record.
This allows backpressure to be applied closer to the network stack and results in record prefetch.
At maximum capacity we will have three
SubscribeToShardEvent in memory (per instance of this class):
Copyright © 2014–2020 The Apache Software Foundation. All rights reserved.