Hardlink issues

The best solution for finding and removing duplicate files.
Post Reply
jackThom
Posts: 16
Joined: Tue May 27, 2014 7:21 am

Hardlink issues

Post by jackThom »

1) Choosing the hardlink option does not seem to actually do what it used to do, which is delete duplicates and put hardlinks in their place, freeing up space on the drive. Performing a hardlink operation actually takes up more space on the drive. (And the recycle bin stays the same). I have never had this happen before version 4.1 and cannot figure out what is going on.

Why would performing a hardlink operation leave less space on the drive??

2) The "Count Hard-links in File" option never seemed to work, regardless of which version I was on. Hard links always were counted regardless of whether that option was ticked. The only way I was ever able to have hardlinks not show up in the list is to tick that, and then tick the "Exclude Hard-linked files" option. This accomplishes the objective, but would seem to defeat the purpose of having the "count" option in the first place. (Also it would seem to take more time and CPU cycles to count them and then exclude them from the list than to simply avoid counting them in the first place.) I noticed this old post (viewtopic.php?f=4&t=1401) which says that the Hard link count function has issues with mapped network drives, but I'm only using local USB external drives.
User avatar
DigitalVolcano
Site Admin
Posts: 1717
Joined: Thu Jun 09, 2011 10:04 am

Re: Hardlink issues

Post by DigitalVolcano »

Are you sure the drive you are trying to hardlink is NTFS formatted (not FAT32 as some external drives are) ? Note you can't hardlink between drives.
jackThom
Posts: 16
Joined: Tue May 27, 2014 7:21 am

Re: Hardlink issues

Post by jackThom »

I am positive the drives are all NTFS, and I am only deduplicating within a single partition on a single drive.

I did some experimenting and was able to replicate similar behavior by running the hardlink operation multiple times on the same data set.

That is, scan a folder, and run a file removal operation and use the "create hard links" option...then run the scan again (with "Count Hard-links in File" NOT ticked, btw). I opted to remove all but one file in each group and created hard links again, and sure enough ended up with less free space on the drive.

So it is probable the issue that I am seeing stems from previously deduplicating a drive by creating hard links. At this later date I wanted to deduplicate the same drive and ran the same operation, unknowingly creating hard links for files which already had hard links created. (Again, the "Count Hard-links in File" was NOT ticked, so I was not thinking anything in the list was a hard link anyway.)

I still don't understand why Duplicate Cleaner would create duplicates in this case anyway, but even more confounding is how do I now get rid of them??

It would seem the reason there is less space is that DC created multiple file chunks which the hard links reference. But in my testing, even deleting the entire folder I was testing (thus removing all hardlinks), the free space on the drive remained smaller than it should be.
Post Reply