Suggestion for md5/size and links

The best solution for finding and removing duplicate files.
Post Reply
User avatar
Hannie

Suggestion for md5/size and links

Post by Hannie »

I don`t know if it`s already done that way, but maybe search for duplicate content can be speeded up by searching for files with same size first, and only comparing hash from those afterwards.

And a feature I would suggest is the possibility to replace duplicates with regular links (.lnk)

Thanks for your great work!
User avatar
dv

Post by dv »

Hashing is only done for files of equal size.

Regular links is somthing I'll look at for the next version.
Thanks!
User avatar
Anonymous

Post by Anonymous »

If Duplicate Cleaner does not already employ a similar or better method, a potential scanning speed increase could be to hash a portion of the beginning of files, and if they match, then hash the entire file.

Example:
1. Check file size, if equal,
2. Hash first 32kB/[user-defined]kB of the files. If equal,
3. Hash entire files.

(Reference) FindDupe is a program that uses this method: http://www.sentex.net/~mwandel/finddupe/
User avatar
DV

Post by DV »

DC also uses the above 'quick hash' method.
dbague
Posts: 20
Joined: Mon Oct 02, 2017 6:35 pm

Re:

Post by dbague »

DV wrote:DC also uses the above 'quick hash' method.
In version 4, is the quick hash also cached (when option checked for hash cache)?

Also, an perhaps i should go in another threahd, is the hash cache database accessible, or less encompassing: when saving the list of duplicates in the format that can be reimported for further actions, could the hash and quick hash be included?

what ever happened to the "regular link" option that you had in your plans. i guess the hardlinks don't have preferred originals, making them easier to handle than regular links (when more than one dupe), nevertheless, since one has to leave one member of group unselected, that points to the original for regular links. wow, i just produced wind.... Anyway, what about regular links plans?
User avatar
DigitalVolcano
Site Admin
Posts: 1717
Joined: Thu Jun 09, 2011 10:04 am

Re: Suggestion for md5/size and links

Post by DigitalVolcano »

Yes, the quick hash is also cached. The 'Export' feature for the list does export the hash as well, but only the full hash.

Shortcuts and soft links are planned, but have been pushed back, probably to version 4.1.1.
Post Reply