BSC supports new version PebbleDB

BNB Smart Chain
1

LevelDB has been used as the default database to store the blockchain data for a long time for BSC network. And recently its repo is receiving very limited maintenance. And we had tried to use PebbleDB to replace it, but the testing result showed that the older version PebbleDB performance was not as good as LevelDB. Recently it is heard that PebbleDB has made some improvements with the new version. So we restarted to research if PebbleDB can be a good option. And below is the chasing block comparison testing we did with PebbleDB and LevelDB on BSC.

Eevironment

  1. Machine configuration
  • EC2: m6i.4xlarge
  • CPU: Intel(R) Xeon(R) Platinum 8375C CPU @ 2.90GHz 16 core
  • Mem: 64G
  • SSD: GP3 3000 IOPS, 250M/S

  1. Run mode

    ./geth --db.engine pebble(or leveldb) --syncmode full

Database Version

  1. LevelDB version: v1.0.1-0.20210819022825-2ae1ddf74ef7
  2. PebbleDB version: v0.0.0-20230928194634-aa077af62593

Configuration changes to PebbleDB

  1. Increased SST file size to improve block cache

The change is advised by the Pebble community. After adjusting the configuration it takes about 3 hours to warm up the read performance while 8 hours before.

Test Result

Type PebbleDB LevelDB
CPU(sys load) 116 120
Memory(GB) 5.8 9.6
Disk Read(avg) 42MB/s 127MB/s
Disk Write(avg) 17.5MB/s 18.2MB/s

The yellow one is LevelDB while the green one is PebbleDB.

  • The block chasing rate is nearly the same.

|614.9704212032384x321
|614.9704212032384x3211339Ă—700 42 KB

  • The import block cost is a little bit lower when using PebbleDB.

1600Ă—772 257 KB

  • Pebble avg: 371ms (avg value of 24 hours)

  • Leveldb avg: 380 ms (avg value of 24 hours)

  • PebbleDB used less memory.

1281Ă—579 141 KB

Pebble has two significant sources of memory usage:MemTables and the Block Cache.Originally, Pebble used regular GO memory allocation for the memory backing both MemTables and the Block Cache. It puts significant pressure on the GO GC. An “allocation cache” was introduced to lessen the pressure on the GO GC. It really produced a dramatic reduction in GC pressure and a measurable performance improvement.

  • Less compaction cost when using PebbleDB

  • CPU profile when using LevelDB

1600Ă—536 291 KB

  • CPU profile when using PebbleDB

1600Ă—696 273 KB

PebbleDB pacing mechanism uses separate rate limiters for flushes and compactions. It works by attempting to flush and compact only as fast as needed and no faster. While LevelDB has no throttle the rate of flushes and compactions, runs as fast as possible, which easily results in huge latency spikes for writes and reads.

Conclusion

PebbleDB with the new version is nearly the same speed with LevelDB with less memory usage and compaction cost. It is really one better option. You can try with the BSC new release.

2 Likes