Nodeos Blocks Log Stride

A Telos mainnet blocks.log that contains all blocks from genesis up to the current chain state at the time of writing is now over 200GBytes in size. This prompted me to ask myself the question, “Is it possible to break the blocks.log into fragments that can subsequently be joined together again and the results of that recombined blocks.log be useable by nodeos?”

Of course there are lots of tools that will compress and split a large file to create an archive of that file. These will enable you to recreate the whole file. My previous article Nodeos Replays showed that nodes supports the use of a partial blocks.log, in the sense that they don’t have to go back to genesis. I reason that generic splitting tools, with no knowledge of the internal structure of a blocks.log, will be incapable of producing blocks.log fragments that are useable individually.

We saw in Nodeos Replays that the eosio-blocklog tool can be used to split a blocks.log up. However, there is no documented capability to recombine the fragments that it produces. In this article, I explore whether they can be recombined by simply concatenating them. This leads me on to the new blocks log stride feature in version 2.1.0 of EOSIO and things to be aware of when upgrading to this version and enabling this feature with an existing blocks.log in place.

As in Nodeos Replays, I will use my Docker Compose EOSIO services to step through various test scenarios and again I will focus on nodeos and not expand on Docker or Docker Compose concepts nor on generic command line actions. I will also skip over the detail of some actions if they were explained in Nodeos Replays, so I would recommend reading that article before reading this one.

Continue reading

Nodeos Replays

EOSIO is “a highly performant open-source blockchain platform, built to support and operate safe, compliant, and predictable digital infrastructures” and nodeos is “the core service daemon that runs on every EOSIO node”. This article is about the replaying of blockchain blocks using nodeos, which can be necessary to recover from failure or as a faster alternative to synchronising from a p2p network.

To demonstrate and test the concepts involved, I will use my Docker Compose EOSIO services, while referring to the published nodeos documentation. I will step through various test scenarios using a varilink/eosio Docker image based on version 2.0.12 of EOSIO and so any links to the EOSIO documentation within the body of the article will reference the EOSIO v2.0 manual. Since nodeos is the focus of this article, I will not expand on Docker or Docker Compose concepts nor generic command line actions.

Continue reading