Road to Neo3: Decoupled state distribution through P2P protocol

In the previous Road to Neo3 article, we noted performance considerations and bug-fixing capabilities as the key factors affecting the ongoing state root implementation. Following on, we’ll look at the current approach under development—distribution of state through P2P messages.

Network protocol

Both consensus nodes and normal nodes on the Neo network are connected together in a peer-to-peer (P2P) structure. Each node will have multiple connections to other nodes on the network, ensuring that new information can be quickly propagated through it.

Communications between nodes is done in the form of messages, each carrying a command and payload to be processed. These messages could be a request for information such as the latest block details, or they could carry prioritized information such as consensus data.

Since all nodes on the network are constantly listening for incoming P2P messages, it is an ideal place for an agreed state to be distributed. This approach was first proposed by Igor Coelho:

“I think it’s better to have state hash pushed via P2P messages, signed by consensus nodes, so that nodes can still keep track of an “official” state, while not locking these forever on the blockchain.”

Decoupled state

Unlike the block header approach, distribution of state through the P2P network means that the agreed global state is never committed to the blockchain. This decouples state from blocks, meeting the original proposal goals by allowing room for potential performance improvements.

Additionally, since state is never committed on-chain, bug fixes that affect state can still be deployed without leading to conflicts with block data. If such a situation occurs, new signed hashes can be distributed. Neo core developer Jeff Solinsky explains:

“If an updated version of Neo needs to fix some issue that will change MPT hashes, an offline MPT hash file can be distributed that will contain updated MPT hashes signed by CN nodes. Clients can load this when resyncing the chain from an offline chain file.”

One potential downside to this approach is that the extra load on the network protocol could affect transaction throughput, since adding new messages or increasing the message frequency would decrease the time spent on other tasks. However, if state root hashes are distributed within existing messages, such as those associated with the consensus process, the burden would be minimized.

Implementation proposals

After deciding on state distribution through P2P messages, a post shared by Neo software engineer Zhang Tao outlined the potential implementation approaches.

Three directions were detailed:

1. Add state root to PrepareRequest consensus message and signature to Commit consensus message
2. Use a separate dBFT process for consensus on state roots
3. Have consensus nodes sign and broadcast current state root after block creation

Due to votes from other developers showing a split preference between the first and third option, Tao suggested a combination of the two. In this case, consensus nodes send the state root alongside a new block as in (1), however unconfirmed state roots can be verified by consensus node signature as in (3).

Tao also shared two issues which needed to be addressed. In the event of an upgrade, both the initial generation of a new state root chain and its broadcast to other nodes would require a large number of P2P messages, adversely affecting performance.

However, since each full node will still generate a local state root, only the latest official state root signed by consensus nodes needs to be checked. This removes the need for a large number of P2P messages to be generated, as each light or full node only requires the latest state root for verification purposes.