Background
Many users reported their nodes were unable to sync and keep printing the log: “Sidechain written to disk”
It was first reported around Jan 29 2023.
We did some investigation and fixed it with PR-1301, which is included in the latest release: v1.1.19.
Suggest
Please upgrade your BSC binary to v1.1.19, if you still have the problem.
How It Happened
The currently downloader module of BSC will try to sync with a connected peer which claimed with highest TD(total difficulty) value. However, the claimed TD could be wrong, e.g. malicious nodes could broadcast a NewBlockPacket message with a future TD value which is not verified.
// NewBlockPacket is the network packet for the block propagation message.
type NewBlockPacket struct {
Block *types.Block
TD *big.Int
}
Actually, I have captured one peer that broadcasted such kind of malicious message over the network, the peer is:
"name":"Geth/v1.1.16-ae6d10cf-20221122/linux-amd64/go1.18.1",
"remoteAddress":"3.94.194.24:34806",
"difficulty":50892934,
"head":"0x06b12912021ddf03ab09b14894d96f7441ec10b92c1bfafecdaf7812c3e16166"
The peer claimed a future TD value(50892934) on Feb 6 2023, which was reached 6 days later, on block 25607109 (Feb 12 2023).
And the peer used a customized BSC binary, which is not officially released by BNB Chain, so it is uncertain yet if the peer did it on purpose or there is a bug in its binary.
How Is It Solved
We add flag for each Peer, it will be marked as lagging if its latest block height is less or equal to the local chain and is already included.
BSC nodes with v1.1.9 will skip these lagging peers even if they claimed a large TD value.
Ethereum could have the same issue, since Ethereum also use the packet NewBlockPacket to claim peer’s latest state, but the TD value in this packet is not verified.