trait SequencerStore extends SequencerMemberValidator with NamedLogging with AutoCloseable
Persistence for the Sequencer. Writers are expected to create a SequencerWriterStore which may delegate to this underlying store through an appropriately managed storage instance.
- Alphabetic
- By Inheritance
- SequencerStore
- AutoCloseable
- NamedLogging
- SequencerMemberValidator
- AnyRef
- Any
- Hide All
- Show All
- Public
- Protected
Abstract Value Members
- abstract def acknowledge(member: SequencerMemberId, timestamp: CantonTimestamp)(implicit traceContext: TraceContext): FutureUnlessShutdown[Unit]
Write an acknowledgement that member has processed earlier timestamps.
Write an acknowledgement that member has processed earlier timestamps. Only the latest timestamp needs to be stored. Earlier timestamps can be overwritten. Acknowledgements of earlier timestamps should be ignored.
- abstract def batchingConfig: BatchingConfig
- Attributes
- protected
- abstract def blockSequencerMode: Boolean
Whether the sequencer store operates is used for a block sequencer or a standalone database sequencer.
- abstract def bufferedEventsMaxMemory: BytesUnit
Maximum memory usage by the events in the buffer before it starts dropping events.
Maximum memory usage by the events in the buffer before it starts dropping events. If set to 0, caching is disabled and all requests go to the underlying store (e.g. database).
- abstract def close(): Unit
- Definition Classes
- AutoCloseable
- Annotations
- @throws(classOf[java.lang.Exception])
- abstract def countRecords(implicit traceContext: TraceContext): FutureUnlessShutdown[SequencerStoreRecordCounts]
Count records currently stored by the sequencer.
Count records currently stored by the sequencer. Used for pruning tests.
- Attributes
- protected[canton]
- Annotations
- @VisibleForTesting()
- abstract def deleteEventsPastWatermark(instanceIndex: Int)(implicit traceContext: TraceContext): FutureUnlessShutdown[Option[CantonTimestamp]]
Delete all events that are ahead of the watermark of this sequencer.
Delete all events that are ahead of the watermark of this sequencer. These events will not have been read and should be removed before returning the sequencer online. Should not be called alongside updating the watermark for this sequencer and only while the sequencer is offline. Returns the watermark that was used for the deletion.
- abstract def disableMemberInternal(member: SequencerMemberId)(implicit traceContext: TraceContext): FutureUnlessShutdown[Unit]
- Attributes
- protected
- implicit abstract val executionContext: ExecutionContext
- Attributes
- protected
- abstract def fetchLowerBound()(implicit traceContext: TraceContext): FutureUnlessShutdown[Option[(CantonTimestamp, Option[CantonTimestamp])]]
Fetch the lower bound of events that can be read.
Fetch the lower bound of events that can be read. Returns
None
if all events can be read. - abstract def fetchOnlineInstances(implicit traceContext: TraceContext): FutureUnlessShutdown[SortedSet[Int]]
Fetch the indexes of all sequencers that are currently online
- abstract def fetchPreviousEventTimestamp(memberId: SequencerMemberId, timestampInclusive: CantonTimestamp)(implicit traceContext: TraceContext): FutureUnlessShutdown[Option[CantonTimestamp]]
Fetch previous event timestamp for a member for a given inclusive timestamp.
- abstract def fetchWatermark(instanceIndex: Int, maxRetries: Int = retry.Forever)(implicit traceContext: TraceContext): FutureUnlessShutdown[Option[Watermark]]
Read the watermark for this sequencer and its online/offline status.
Read the watermark for this sequencer and its online/offline status. Currently only used for testing.
- abstract def goOffline(instanceIndex: Int)(implicit traceContext: TraceContext, callerCloseContext: CloseContext): FutureUnlessShutdown[Unit]
Flag that we're going offline (likely due to a shutdown)
- abstract def goOnline(instanceIndex: Int, now: CantonTimestamp)(implicit traceContext: TraceContext): FutureUnlessShutdown[CantonTimestamp]
Mark the sequencer as online and return a timestamp for when this sequencer can start safely producing events.
Mark the sequencer as online and return a timestamp for when this sequencer can start safely producing events.
- now
Now according to this sequencer's clock which will be used if it is ahead of the lowest available timestamp from other sequencers.
- abstract def latestAcknowledgements()(implicit traceContext: TraceContext): FutureUnlessShutdown[Map[SequencerMemberId, CantonTimestamp]]
Return the latest acknowledgements for all members
Return the latest acknowledgements for all members
- Attributes
- protected[store]
- Annotations
- @VisibleForTesting()
- abstract def latestTopologyClientRecipientTimestamp(member: Member, timestampExclusive: CantonTimestamp)(implicit traceContext: TraceContext): FutureUnlessShutdown[Option[CantonTimestamp]]
For a given member and timestamp, return the latest timestamp of a potential topology change, that reached both the sequencer and the member.
For a given member and timestamp, return the latest timestamp of a potential topology change, that reached both the sequencer and the member. To be used by the topology snapshot awaiting, should there be a topology change expected to need to be taken into account for
timestampExclusive
sequencing timestamp. - abstract def locatePruningTimestamp(skip: NonNegativeInt)(implicit traceContext: TraceContext): FutureUnlessShutdown[Option[CantonTimestamp]]
Locate a timestamp relative to the earliest available event based on a skip index starting at 0.
Locate a timestamp relative to the earliest available event based on a skip index starting at 0. Useful to monitor the progress of pruning and for pruning in batches.
- returns
The timestamp of the (skip+1)'th event if it exists, None otherwise.
- abstract def loggerFactory: NamedLoggerFactory
- Attributes
- protected
- Definition Classes
- NamedLogging
- abstract def lookupMemberInternal(member: Member)(implicit traceContext: TraceContext): FutureUnlessShutdown[Option[RegisteredMember]]
Lookup member directly without caching.
Lookup member directly without caching.
- Attributes
- protected
- abstract def markLaggingSequencersOffline(cutoffTime: CantonTimestamp)(implicit traceContext: TraceContext): FutureUnlessShutdown[Unit]
Flag any sequencers that have a last updated watermark on or before the given
cutoffTime
as offline. - abstract def preloadBufferInternal()(implicit traceContext: TraceContext): FutureUnlessShutdown[Unit]
- Attributes
- protected
- abstract def previousEventTimestamp(memberId: SequencerMemberId, timestampExclusive: CantonTimestamp)(implicit traceContext: TraceContext): FutureUnlessShutdown[Option[CantonTimestamp]]
For a given member find the timestamp of the last event that the member has received before
timestampExclusive
. - abstract def pruneEvents(beforeExclusive: CantonTimestamp)(implicit traceContext: TraceContext): FutureUnlessShutdown[Int]
Prune events before the given timestamp
Prune events before the given timestamp
- returns
a best efforts count of how many events were removed. this value can be less than the number of events actually removed if technical issues prevent a full count from being returned (e.g. with a database we may retry a delete after a connectivity issue and find that all events were successfully removed and have 0 rows removed returned).
- Attributes
- protected[store]
- abstract def prunePayloads(beforeExclusive: CantonTimestamp)(implicit traceContext: TraceContext): FutureUnlessShutdown[Int]
Prune payloads before the given timestamp
Prune payloads before the given timestamp
- returns
a best efforts count of how many events were removed. this value can be less than the number of payloads actually removed if technical issues prevent a full count from being returned.
- Attributes
- protected[store]
- abstract def readEventsInternal(memberId: SequencerMemberId, fromExclusiveO: Option[CantonTimestamp], limit: Int)(implicit traceContext: TraceContext): FutureUnlessShutdown[ReadEvents]
Internal non-buffered implementation of
readEvents
.Internal non-buffered implementation of
readEvents
.- Attributes
- protected
- abstract def readPayloads(payloadIds: Seq[IdOrPayload], member: Member)(implicit traceContext: TraceContext): FutureUnlessShutdown[Map[PayloadId, Batch[ClosedEnvelope]]]
- abstract def readStateAtTimestamp(timestamp: CantonTimestamp)(implicit traceContext: TraceContext): FutureUnlessShutdown[SequencerSnapshot]
The state returned here is used to initialize a separate database sequencer (that does not share the same database as this one) using initializeFromSnapshot such that this new sequencer has enough information (registered members, previous event timestamps, etc) to be able to process new events from the same point as this sequencer to the same clients.
The state returned here is used to initialize a separate database sequencer (that does not share the same database as this one) using initializeFromSnapshot such that this new sequencer has enough information (registered members, previous event timestamps, etc) to be able to process new events from the same point as this sequencer to the same clients. This is typically used by block sequencers that use the database sequencer as local storage such that they will process the same events in the same order and they need to be able to spin up new block sequencers from a specific point in time.
- returns
state at the given time
- abstract def registerMember(member: Member, timestamp: CantonTimestamp)(implicit traceContext: TraceContext): FutureUnlessShutdown[SequencerMemberId]
Register the provided member.
Register the provided member. Should be idempotent if member is already registered and return the existing id.
- abstract def resetWatermark(instanceIndex: Int, ts: CantonTimestamp)(implicit traceContext: TraceContext): EitherT[FutureUnlessShutdown, SaveWatermarkError, Unit]
Reset the watermark to an earlier value, i.e.
Reset the watermark to an earlier value, i.e. in case of working as a part of block sequencer. Also sets the sequencer as offline. If current watermark value is before
ts
, it will be left unchanged. If a watermark doesn't yet exist, it won't be inserted, because it would effectively mean setting it to a future value. - abstract def safeWatermark(implicit traceContext: TraceContext): FutureUnlessShutdown[Option[CantonTimestamp]]
Return the minimum watermark across all online sequencers
- abstract def saveEvents(instanceIndex: Int, events: NonEmpty[Seq[Sequenced[PayloadId]]])(implicit traceContext: TraceContext): FutureUnlessShutdown[Unit]
Save a series of events to the store.
Save a series of events to the store. Callers should determine batch size. No batching is done within the store. Callers MUST ensure that event-ids are unique as no errors will be returned if a duplicate is present (for the sequencer writer see sequencer.PartitionedTimestampGenerator for use with their instance index).
- abstract def saveLowerBound(ts: CantonTimestamp, latestTopologyClientTimestamp: Option[CantonTimestamp])(implicit traceContext: TraceContext): EitherT[FutureUnlessShutdown, SaveLowerBoundError, Unit]
Save an updated lower bound of events that can be read.
Save an updated lower bound of events that can be read. Must be equal or greater than any prior set lower bound.
- Exceptions thrown
java.lang.IllegalArgumentException
if timestamp is lower than existing lower bound
- abstract def savePayloads(payloads: NonEmpty[Seq[BytesPayload]], instanceDiscriminator: UUID)(implicit traceContext: TraceContext): EitherT[FutureUnlessShutdown, SavePayloadsError, Unit]
Save a series of payloads to the store.
Save a series of payloads to the store. Is up to the caller to determine a reasonable batch size and no batching is done within the store.
- payloads
the payloads to save
- instanceDiscriminator
a unique ephemeral value to ensure that no other sequencer instances are writing conflicting payloads without having to check the payload body
- abstract def saveWatermark(instanceIndex: Int, ts: CantonTimestamp)(implicit traceContext: TraceContext): EitherT[FutureUnlessShutdown, SaveWatermarkError, Unit]
Write the watermark that we promise not to write anything earlier than.
Write the watermark that we promise not to write anything earlier than. Does not indicate that there is an event written by this sequencer for this timestamp as there may be no activity at the sequencer, but updating the timestamp allows the sequencer to indicate that it's still alive. Return an error if we find our sequencer is offline.
- abstract def sequencerMember: Member
- Attributes
- protected
- abstract def sequencerMetrics: SequencerMetrics
- Attributes
- protected
- abstract def status(now: CantonTimestamp)(implicit traceContext: TraceContext): FutureUnlessShutdown[SequencerPruningStatus]
Build a status object representing the current state of the sequencer.
- abstract def updatePrunedPreviousEventTimestampsInternal(updatedPreviousTimestamps: Map[SequencerMemberId, CantonTimestamp])(implicit traceContext: TraceContext): FutureUnlessShutdown[Unit]
- Attributes
- protected
- abstract def validateCommitMode(configuredCommitMode: CommitMode)(implicit traceContext: TraceContext): EitherT[FutureUnlessShutdown, String, Unit]
Validate that the commit mode of a session is inline with the configured expected commit mode.
Validate that the commit mode of a session is inline with the configured expected commit mode. Return a human readable message about the mismatch in commit modes if not.
Concrete Value Members
- final def !=(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
- final def ##: Int
- Definition Classes
- AnyRef → Any
- final def ==(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
- final def asInstanceOf[T0]: T0
- Definition Classes
- Any
- final def bufferEvents(events: NonEmpty[Seq[Sequenced[BytesPayload]]]): Unit
In case of single instance sequencer we can use in-memory fanout buffer for events
- def clone(): AnyRef
- Attributes
- protected[lang]
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.CloneNotSupportedException]) @IntrinsicCandidate() @native()
- def disableMember(member: Member)(implicit traceContext: TraceContext): FutureUnlessShutdown[Unit]
Prevents member from sending and reading from the sequencer, and allows unread data for this member to be pruned.
Prevents member from sending and reading from the sequencer, and allows unread data for this member to be pruned. It however won't stop any sends addressed to this member.
- final def eq(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
- def equals(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef → Any
- implicit def errorLoggingContext(implicit traceContext: TraceContext): ErrorLoggingContext
- Attributes
- protected
- Definition Classes
- NamedLogging
- val eventsBuffer: EventsBuffer
- Attributes
- protected
- lazy val eventsBufferEnabled: Boolean
- Annotations
- @VisibleForTesting()
- final def getClass(): Class[_ <: AnyRef]
- Definition Classes
- AnyRef → Any
- Annotations
- @IntrinsicCandidate() @native()
- def hashCode(): Int
- Definition Classes
- AnyRef → Any
- Annotations
- @IntrinsicCandidate() @native()
- def initializeFromSnapshot(initialState: SequencerInitialState)(implicit traceContext: TraceContext): EitherT[FutureUnlessShutdown, String, Unit]
- final def isInstanceOf[T0]: Boolean
- Definition Classes
- Any
- def isMemberRegisteredAt(member: Member, time: CantonTimestamp)(implicit tc: TraceContext): FutureUnlessShutdown[Boolean]
- Definition Classes
- SequencerStore → SequencerMemberValidator
- def logger: TracedLogger
- Attributes
- protected
- Definition Classes
- NamedLogging
- final def lookupMember(member: Member)(implicit traceContext: TraceContext): FutureUnlessShutdown[Option[RegisteredMember]]
Lookup an existing member id for the given member.
Lookup an existing member id for the given member. Will return a cached value if available. Return scala.None if no id exists.
- implicit def namedLoggingContext(implicit traceContext: TraceContext): NamedLoggingContext
- Attributes
- protected
- Definition Classes
- NamedLogging
- final def ne(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
- def noTracingLogger: Logger
- Attributes
- protected
- Definition Classes
- NamedLogging
- final def notify(): Unit
- Definition Classes
- AnyRef
- Annotations
- @IntrinsicCandidate() @native()
- final def notifyAll(): Unit
- Definition Classes
- AnyRef
- Annotations
- @IntrinsicCandidate() @native()
- def prune(requestedTimestamp: CantonTimestamp, status: SequencerPruningStatus, payloadToEventMargin: NonNegativeFiniteDuration)(implicit traceContext: TraceContext): EitherT[FutureUnlessShutdown, PruningError, SequencerPruningResult]
Prune as much data as safely possible from before the given timestamp.
Prune as much data as safely possible from before the given timestamp.
- requestedTimestamp
the timestamp that we would like to prune up to (see docs on using the pruning status and disabling members for picking this value)
- status
the pruning status that should be used for determining a safe to prune time for validation
- payloadToEventMargin
the maximum time margin between payloads and events. once we have a safe to prune timestamp we simply prune all payloads at
safeTimestamp - margin
to ensure no payloads are removed where events will remain. typically sourced from SequencerWriterConfig.payloadToEventMargin.- returns
the timestamp up to which the database sequencer has been pruned (lower than requestedTimestamp) and a human readable report on what has been removed.
- def readEvents(memberId: SequencerMemberId, member: Member, fromExclusiveO: Option[CantonTimestamp], limit: Int)(implicit traceContext: TraceContext, metricsContext: MetricsContext): FutureUnlessShutdown[ReadEvents]
Read all events of which a member is a recipient from the provided timestamp but no greater than the earliest watermark.
Read all events of which a member is a recipient from the provided timestamp but no greater than the earliest watermark. Passing both
member
andmemberId
to avoid a database query for the lookup. - final def resetAndPreloadBuffer()(implicit traceContext: TraceContext): FutureUnlessShutdown[Unit]
- final def synchronized[T0](arg0: => T0): T0
- Definition Classes
- AnyRef
- def toString(): String
- Definition Classes
- AnyRef → Any
- def updatePrunedPreviousEventTimestamps(updatedPreviousTimestamps: Map[Member, Option[CantonTimestamp]])(implicit traceContext: TraceContext): FutureUnlessShutdown[Unit]
Set the "pruned" previous event timestamp for a member.
Set the "pruned" previous event timestamp for a member. This timestamp is used to serve the oldest (earliest) event that sequencer has for the member:
- after pruning
- after the sequencer's onboarding
- final def wait(arg0: Long, arg1: Int): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.InterruptedException])
- final def wait(arg0: Long): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.InterruptedException]) @native()
- final def wait(): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.InterruptedException])