A Telos mainnet blocks.log that contains all blocks from genesis up to the current chain state at the time of writing is now over 200GBytes in size. This prompted me to ask myself the question, “Is it possible to break the blocks.log into fragments that can subsequently be joined together again and the results of that recombined blocks.log be useable by nodeos?”
Of course there are lots of tools that will compress and split a large file to create an archive of that file. These will enable you to recreate the whole file. My previous article Nodeos Replays showed that nodes supports the use of a partial blocks.log, in the sense that they don’t have to go back to genesis. I reason that generic splitting tools, with no knowledge of the internal structure of a blocks.log, will be incapable of producing blocks.log fragments that are useable individually.
We saw in Nodeos Replays that the eosio-blocklog tool can be used to split a blocks.log up. However, there is no documented capability to recombine the fragments that it produces. In this article, I explore whether they can be recombined by simply concatenating them. This leads me on to the new blocks log stride feature in version 2.1.0 of EOSIO and things to be aware of when upgrading to this version and enabling this feature with an existing blocks.log in place.
As in Nodeos Replays, I will use my Docker Compose EOSIO services to step through various test scenarios and again I will focus on nodeos and not expand on Docker or Docker Compose concepts nor on generic command line actions. I will also skip over the detail of some actions if they were explained in Nodeos Replays, so I would recommend reading that article before reading this one.
Continue reading