How to speed up hash calculation?

The best solution for finding and removing duplicate files.
Post Reply
tobias
Posts: 1
Joined: Fri May 05, 2023 12:39 pm

How to speed up hash calculation?

Post by tobias »

Hello,

I am currently testing DC5 - thus I am new to the program, and my evaluation period is on the run.

How can I speed up the calculation of MD5 hashes?

Currently, I am importing an external hdd to a virtual folder. My progress, however, is about 8 mb/s. In the Windows task manager I find that DC5 is using approximately 26% of my cpu, while the idle task is above 70%. Increasing task priorities did not improve these values.

The reading/writing speed of my hdd is far beyond this value. So there seems to be a bottleneck... somewhere else.

Since time matters I would appreciate any hints and tweaks.

Regards,
tobias
SiMoZ_287
Posts: 16
Joined: Tue Nov 16, 2021 5:42 pm

Re: How to speed up hash calculation?

Post by SiMoZ_287 »

It will probably never hit 100% for many reasons, one of them is likely that the disk is slower and cannot feed the cpu fast enogh, or that your cpu has many cores and I think the program just uses one (I have to check).
However a process like this it's pretty much expected to be slow, you have to deal with hard disk responsiveness, files fragmentation, high IO for many small files and the NTFS being old. I am already surprised you cpu got such a high usage, mine is at 7%
Post Reply