I'm computing a hash of a file on my NAS, but it's too slow.

The best solution for finding and removing duplicate files.
Post Reply
digicube
Posts: 2
Joined: Wed May 11, 2022 3:59 pm

I'm computing a hash of a file on my NAS, but it's too slow.

Post by digicube »

When I selected the NAS drives as a whole for the duplicate check, it was all done in a couple of hours.

However, I realized after the fact that I needed to protect a few folders, so I stopped it and started the duplicate check by selecting folders inside the same drive, but the hash calculation is very slow, around 1.5Kb/s, and I'm making very little progress.

I think the network speed is fast, but the hash calculation is slow and the CPU is barely being used, so what can I do?

Also, why is there a difference in speed when I select the same drive as a whole and when I select folders inside the drive one by one?
User avatar
therube
Posts: 615
Joined: Tue Jun 28, 2011 4:38 pm

Re: I'm computing a hash of a file on my NAS, but it's too slow.

Post by therube »

> When I selected the NAS drives as a whole for the duplicate check, it was all done in a couple of hours.

So at that point, it did complete, fully?

Was this done using a hash, & using the same hash method?

> I realized after the fact that I needed to protect a few folders, so I stopped it

Was this a second run?
Or did you actually abort the "it was all done in a couple hours" run?

> and started the duplicate check by selecting folders inside the same drive, but the hash calculation is very slow, around 1.5Kb/s

And this was yet a third run?
SiMoZ_287
Posts: 16
Joined: Tue Nov 16, 2021 5:42 pm

Re: I'm computing a hash of a file on my NAS, but it's too slow.

Post by SiMoZ_287 »

You can protect the folders even after the scan has completed, so it doesn't matter and I highly doubt it would take longer. Anyway, you can try to undestand why it is so slow by monitoring the network usage and, depending on the NAS, by monitoring the disk and cpu usage of your NAS. The hashing is usually not a problem for the cpu unless you are using complex hashing methods. In my experience, as I have a nas too, the main factor for the reduced performance is the fragmentation on the nas and the slow responsiveness of the hard drives. Sometimes with bigger files the bottleneck it's the speed of the network link even if I have a gigabit lan
Post Reply