Name length error during "Create Hardlinks" fatally freezes DC

The best solution for finding and removing duplicate files.
Post Reply
SkyeFire
Posts: 3
Joined: Fri Sep 13, 2019 6:30 pm

Name length error during "Create Hardlinks" fatally freezes DC

Post by SkyeFire »

Pro version 4.1.2, under Win7Pro 64bit. NTFS file system.

I'm performing a find for 100% duplicate content. This works without any errors or warnings. I end up with several thousand duplicate files in various sub-trees that I need to keep, but want to save the raw drive space. So I use the "select all but one file in each group" option, then use the "Create Hard Links" option.

The Removal process will chew through its list to about 80-90% (judging by the progress bar) and then stop with an error pop-up: "Error in CreateHardlinks: Specified path, file name, or both are too long". After acknowledging this error, there's a long pause, then another error pop-up: "Error during post-removal process: database is locked database is locked" (no, that's not a typo, it repeats itself just like that).

At this point, DC is locked up so completely the only way to get out is to use the Task Manager to kill it, or force a full reboot. This error is 100% repeatable, across many different directory trees, and always locks up this fatally. It's not simply "waiting" either -- I tried leaving it "hung" for ~72hrs to see if it just needed time. No dice. This occurs regardless of whether I have the "skip problem files" option checked or not. Also, *none* of the selected files actually get converted to HLs, even the ones that are not running afoul of the character length limit.
User avatar
DigitalVolcano
Site Admin
Posts: 1717
Joined: Thu Jun 09, 2011 10:04 am

Re: Name length error during "Create Hardlinks" fatally freezes DC

Post by DigitalVolcano »

Do the files that are being converted have path+name longer than 260 characters? That could be the cause of the error.

I did think that Duplicate Cleaner used system calls that didn't have size limitations for hardlinks - something I need to look into. Will log a bug.

I'll also check why it isn't failing more gracefully when it hits a limit!
SkyeFire
Posts: 3
Joined: Fri Sep 13, 2019 6:30 pm

Re: Name length error during "Create Hardlinks" fatally freezes DC

Post by SkyeFire »

None of the file names are that long, but I have directory trees that have so many branches, I would be unsurprised if there were several trees whose length exceeded 240-260 characters (with or without the file name). I haven't gone digging to count characters, though. DC doesn't give me an indication of which directory it's failing on.

OTOH, when I've done a "Delete" clean on the same drive, instead of "Create Hardlinks", I do get a warning that it can't touch a particular directory due to the combined path/name length, and this error *does* give me the path (although I think it runs off the edge of the pop-up window). It would take me a few hours to re-create this fault, though. If I have the chance, I'll see if I can hand-delete this directory tree and see if that helps the Create Hardlinks option get past this error.
User avatar
DigitalVolcano
Site Admin
Posts: 1717
Joined: Thu Jun 09, 2011 10:04 am

Re: Name length error during "Create Hardlinks" fatally freezes DC

Post by DigitalVolcano »

OK thanks for the update. Another thing to try is to enable 'Long path support' in Advanced Settings. I can't recall if this affects hardlinking though.
User avatar
therube
Posts: 614
Joined: Tue Jun 28, 2011 4:38 pm

Re: Name length error during "Create Hardlinks" fatally freezes DC

Post by therube »

Everything (but it's down at the moment), https://www.voidtools.com/downloads/.

Search for: path:len:>240 (or whatever length interests you).
SkyeFire
Posts: 3
Joined: Fri Sep 13, 2019 6:30 pm

Re: Name length error during "Create Hardlinks" fatally freezes DC

Post by SkyeFire »

"Long Path Support" was already checked -- I suspect I may have done that a while back, while cleaning a different directory.

I ended up writing a quick script to list out and record the length of every directory+filename in my tree (24MB text file! :o ). Turns out I had LOTS of Really Long path&file names, from 260 up to 350. The great bulk of that is path, my filenames tend to be 24 characters max plus a 3-char extension.

Does DC put out log files for this kind of error?
User avatar
DigitalVolcano
Site Admin
Posts: 1717
Joined: Thu Jun 09, 2011 10:04 am

Re: Name length error during "Create Hardlinks" fatally freezes DC

Post by DigitalVolcano »

Check the log file in your Documents folder
Post Reply