Compact Block Propagation

Let’s take a closer look at another new feature added with the recent dibbler testnet release.

New Compact Block Propagation

With the release of dibbler testnet comes a more efficient block propagation protocol that leverages the high likelihood that a base node receiving a new block already has all or the majority of the transaction data in its mempool.

Out with the old…

It is instructive to briefly understand how the previous protocol worked and some inefficiencies that the new protocol addresses. The process begins when a miner finds a new block. The hash for the block is calculated and transmitted to every connected peer. Each peer receiving this unrecognised block hash must request the full block of up to ~2mb from the peer that sent it. Once the full block is received, it is validated and the same block hash re-propagated to other peers.

Although this protocol is nice and simple, it has a number of drawbacks:

• The base node has no way to validate the block being transmitted, the hash could just be a bunch of random bytes!
• The base node transmitting the block hash may suddenly be inundated with requests for the block from many peers, creating delays and timeouts.
• The RTT for messages can be as much as a few seconds on connections with poor latency (e.g. tor).
• A base node likely already has all the required transactions in its mempool, but has no way to reconcile them with the new block.

… and in with the new.

To address these drawbacks, a new compact block message containing the minimum information required to assemble a full block is propagated to all peers. It is important that this message is kept as small as possible as this message is transmitted and received multiple times for each base node on the network.

This compact block contains:

• a block header,
• a coinbase transaction,
• and a list of transaction kernel excess signatures.

For a block fully loaded with transactions, this message comes to roughly 65Kb.

The base node fills its block with transactions matching the excess signatures from its mempool. If there are any missing transactions, they are requested from the peer and added to the block. In most cases, this won’t be necessary.

The block is validated and added to the blockchain - profit!.

Some points to note are:

• the base node can immediately validate that the PoW is valid from the block header.
• in the majority of cases, there is no need to ask the sending peer for any additional data, removing back-and-forth communication and effectively cutting the time taken before a base node can begin validating the block by two thirds.
• if a peer is missing any transactions, the transactions it requested as part of the protocol are stored in the published transaction mempool for the next peer to request, if required.