Decommissioning Canton nodes and Synchronizer entities

This guide assumes general familiarity with Canton, in particular Canton identity management concepts and operations from the Canton console.

Note that, while onboarding new nodes is always possible, a decommissioned node or entity is effectively disposed of and cannot rejoin a Synchronizer. Decommissioning is thus an irreversible operation.

In addition, decommissioning procedures are currently experimental; regardless, backing up nodes to be decommissioned before decommissioning them is strongly recommended.

Decommissioning a Sequencer

Sequencers are part of a Synchronizer’s messaging infrastructure and do not store app contracts, so they are disposable as long as precautions are taken to avoid disrupting the synchronization services. This means, concretely, ensuring that:

  1. No active Participant Node nor active Mediator is connected to the Sequencer to be decommissioned.

  2. All active Participant Nodes and Mediators are connected to an active Sequencer.

After that, the Sequencer can be decommissioned by removing it from the Synchronizer’s topology and finally disposed of.

Disconnecting all nodes from the Sequencer to be decommissioned

  • Change the Sequencer connection on the Mediators connected to the Sequencer to be decommissioned to use another active Sequencer, as per mediator connectivity:

mediator1.sequencer_connection.set(SequencerConnections.single(sequencer1.sequencerConnection))

  • Reconnect Participant Nodes to the Synchronizer, as described in Synchronizer connectivity, using a Sequencer connection to another active Sequencer:

participant2.synchronizers.disconnect(daName)
participant2.synchronizers.modify(
  daName,
  _.copy(sequencerConnections = SequencerConnections.single(sequencer1.sequencerConnection)),
)
participant2.synchronizers.reconnect(daName)

Decommissioning the Sequencer

Sequencers are part of the Synchronizer by virtue of having their node ID equal to the Synchronizer id, which also means they all have the same node ID. Since a Sequencer’s identity is the same as the Synchronizer’s identity, you should leave identity and namespace mappings intact.

A Sequencer may use its own cryptographic material. In that case, owner-to-key mappings must be removed for the keys it exclusively owns:

  1. Find the authentication and protocol keys on the Sequencer to be decommissioned using the keys.public.list command.

  2. Among those keys, find the ones not shared by other Sequencers. You can do this by issuing the keys.public.list command on each of them: the keys that appear only on the Sequencer node to be decommissioned correspond to its exclusively owned keys.

  3. Remove the mappings for its exclusively owned keys using the topology.owner_to_key_mappings.remove_key command.

// Remove all non-namespace exclusive keys
val sequencerToOffboardPubKeys =
  sequencerToOffboard.keys.public
    .list(filterUsage = Set(SequencerAuthentication, Protocol))
    .map(_.publicKey)
val sequencerToOffboardExclusivePubKeys =
  sequencerToOffboardPubKeys
    .diff(
      sequencers.all
        .filter(_.id != sequencerToOffboard.id)
        .flatMap(_.keys.public.list())
        .map(_.publicKey)
    )
sequencerToOffboardExclusivePubKeys.foreach { key =>
  sequencerToOffboard.topology.owner_to_key_mappings
    .remove_key(
      key.fingerprint,
      key.purpose,
    )
}

The cryptographic material exclusively owned by a decommissioned Sequencer must also be disposed of:

  • If it was stored only on the decommissioned Sequencer, it must be disposed of together with the decommissioned Sequencer node.

  • However, if a decommissioned Sequencer’s cryptographic material is managed via a KMS system, it must be disposed of through the KMS; refer to your KMS’ documentation and internal procedures to handle this. KMS-managed cryptographic material of Sequencer nodes.

A quorum of Synchronizer owners can then propose the Sequencer out of the Synchronizer topology:

// propose the SequencerSynchronizerState that removes the sequencer
synchronizerOwnersNE
  .foreach(
    _.topology.sequencers.propose(
      synchronizerId,
      threshold = seqState1.threshold,
      active = seqState1.active.filterNot(_ == sequencerToOffboard.id),
    )
  )

BFT Sequencers must also disappear from the ordering topology, before they are switched off; you can check that with the bft.get_ordering_topology command.

Decommissioning a Mediator

Mediators are also part of a Synchronizer’s messaging infrastructure and do not store app contracts, so they are disposable as long as precautions are taken to avoid disrupting the synchronization services. This means ensuring that at least one Mediator remains on the Synchronizer.

If other mediators exist on the Synchronizer, a mediator can be decommissioned by a quorum of Synchronizer owners proposing it out of the Synchronizer topology.

// propose the SequencerSynchronizerState that removes the sequencer
synchronizerOwnersNE
  .foreach(
    _.topology.mediators.propose(
      synchronizerId,
      threshold = medState1.threshold,
      active = medState1.active.filterNot(_ == mediatorToOffboard.id),
      group = medState1.group,
    )
  )