Block Data Gossiping

[liamsi]

There are some loose ideas of using libp2p for block data gossiping. Mainly because it easily enables a simple requirement we have: gossiping block data should only happen to validator nodes (who need all data) and not to full nodes (who can download a part of the data and rely on DAS for validity).

The team is planning to write an ADR that would also touch upon pubsub or gossiping (@Wondertan in particular expressed interest in writing this). This is needed anyways and will further clarify the pros&cons of different approaches (e.g. modifying the tendermint consensus reactor and using the tendermint native p2p for this purpose layer instead).

There are some concerns around libp2p this voiced in other places.

E.g. quoting @musalbas:

Re using gossipsub, I'm a bit wary of using that since Radicle had performance issues with it: radicle-dev/radicle-alpha#689

[liamsi]

From the linked radicle issue:

we relied on IPNS and IPFS PubSub

We here wouldn’t reply on IPNS and just plain pubsub (not IPFS pubsub - whatever that means exactly). Pubsub/gossipsub is used by a lot of blockchain projects to gossip (meta) data. It is known to perform reasonably well in these other contexts. I can provide a list of examples with links to code if that does not convince yet.

[liamsi]

In fact, we had to create our own IPFS network to make it work at all.

Yeah, we should actually prioritize this ☝🏼 In our case it seems like a no-brainer anyways.

[adlerjohn]

validator nodes (who need all data)

I think an additional distinction is that validators need all the data in a timely manner. Regular full nodes, not so much.

[liamsi]

Yeah, you are right. That is an important detail.

[tac0turtle]

Would full nodes lag a single block? I dont understand how other nodes in the network would get state?

[liamsi]

Would full nodes lag a single block?

My understanding of full nodes is: they run the consensus reactor (and hence do not lag behind a block), they download / request the state-relevant portion of the Tx (hence they should be able to apply state). They do not have to download or receive the other block data (which could be large). Anything to add here @adlerjohn?

I dont understand how other nodes in the network would get state?

Other node types than validators or full nodes do not recompute state necessarily (and only care about headers like light clients). Validators (like in tendermint) need all data and full nodes only require the reserved Txs. The API for the latter is not fully implemented yet. They could request it via RetrieveStateRelevantMessages once it's implemented, details here https://github.com/lazyledger/lazyledger-core/blob/master/docs/lazy-adr/adr-003-application-data-retrieval.md#detailed-design (the PR Rene has started will make this possible). Alternatively, if we switch to libp2p for block gossiping and gossip row by row instead, full nodes and whoever cares about state could actually subscribe to the "reserved Tx" topic. The reserved Txs will always be in the first row(s) and messages will be in the following rows (cc @Wondertan)

[liamsi]

It might actually be not too difficult to use the tendermint p2p stack for a first implementation. But it is questionable wether or not this is a good idea.
It also depends a bit on:

• which node types care about consensus messages (proposals, votes etc)
• which only care about the headers the moment they are proposed (also some form of consensus message as the header was not notarized yet)
• which only care about signed headers (headers plus signatures, plus validator sets)

After we have clarity around all these, we can make a pragmatic and quick decision. I still feel like libp2p is the right answer here though.

[liamsi]

This is no longer relevant in its current form: validators and fullnodes will continue to operate as before in tendermint mostly. They will be augmented to broadcast signed headers & fraud proofs to the other Celestia specific node types. Similarly they will serve the data to sampling nodes.