DigitalVolcano Software Support

What could I do to make byte-to-byte comparison faster?

I had a big pool (151000 files, 3.87TB) and everything went fine until it got to 45%. When it got there, it started to take 20-40 minutes each single file. Suggestions?

With a large amount of files it's much faster to use hashes (SHA-1 should be fine)

Found the problem: about 40000 identical files. Maybe it was doing several unnecessary comparisons?

(I used byte-to-byte due to the high chance of corrupted files. it was a file recovery)

If you have a lot of identical files then using hashing will be an order of magnitude faster!