Performance of juicefs dump with tikv metadata base

[zig]

Hi,

I have a juicefs FS using at the moment postgres for metadata. The juicefs dump command produce ~260000 entries and it takes 2m. I wanted to try tikv so I loaded the metadata dumped from postgres into a new tikv store.

The setup is quite simple, 3 replicas of pd and tikv, each on different machines, with SSD storage. It's on ARM server, there's plenty of RAM and CPU (about ~20 CPU and 60GB on each machine).

Then I tried dumping the metadata from tikv, I was surprised to find that it seems very slow : it took 12m, 6 times slower than it took postgres to dump the same metadata. On the other hand, loading the metadata in tikv was very fast ( a few seconds)

Is that kind of expected ? Or does it mean my tikv setup might not be right ?

[jiefenghuang]

You can use dump --fast, but this will increase memory usage. Version 1.3 will introduce binary backups, which are faster and consume less memory.

[zig]

Thanks for the tip.

I also did some benchmark with the mounted FS, just a simple du -s on the fs took a lot longer with tikv metadata. I'm using latest versions of everything (juicefs, postgres and tikv). I guess for now I'll stick with postgres (cnpg), which seems good enough for my use case, I'll revisit tikv later if the need arise.