Page 1 of 1

Name length error during "Create Hardlinks" fatally freezes DC

Posted: Fri Sep 13, 2019 6:41 pm
by SkyeFire
Pro version 4.1.2, under Win7Pro 64bit. NTFS file system.

I'm performing a find for 100% duplicate content. This works without any errors or warnings. I end up with several thousand duplicate files in various sub-trees that I need to keep, but want to save the raw drive space. So I use the "select all but one file in each group" option, then use the "Create Hard Links" option.

The Removal process will chew through its list to about 80-90% (judging by the progress bar) and then stop with an error pop-up: "Error in CreateHardlinks: Specified path, file name, or both are too long". After acknowledging this error, there's a long pause, then another error pop-up: "Error during post-removal process: database is locked database is locked" (no, that's not a typo, it repeats itself just like that).

At this point, DC is locked up so completely the only way to get out is to use the Task Manager to kill it, or force a full reboot. This error is 100% repeatable, across many different directory trees, and always locks up this fatally. It's not simply "waiting" either -- I tried leaving it "hung" for ~72hrs to see if it just needed time. No dice. This occurs regardless of whether I have the "skip problem files" option checked or not. Also, *none* of the selected files actually get converted to HLs, even the ones that are not running afoul of the character length limit.

Re: Name length error during "Create Hardlinks" fatally freezes DC

Posted: Mon Sep 16, 2019 9:38 am
by DigitalVolcano
Do the files that are being converted have path+name longer than 260 characters? That could be the cause of the error.

I did think that Duplicate Cleaner used system calls that didn't have size limitations for hardlinks - something I need to look into. Will log a bug.

I'll also check why it isn't failing more gracefully when it hits a limit!

Re: Name length error during "Create Hardlinks" fatally freezes DC

Posted: Wed Sep 18, 2019 7:15 pm
by SkyeFire
None of the file names are that long, but I have directory trees that have so many branches, I would be unsurprised if there were several trees whose length exceeded 240-260 characters (with or without the file name). I haven't gone digging to count characters, though. DC doesn't give me an indication of which directory it's failing on.

OTOH, when I've done a "Delete" clean on the same drive, instead of "Create Hardlinks", I do get a warning that it can't touch a particular directory due to the combined path/name length, and this error *does* give me the path (although I think it runs off the edge of the pop-up window). It would take me a few hours to re-create this fault, though. If I have the chance, I'll see if I can hand-delete this directory tree and see if that helps the Create Hardlinks option get past this error.

Re: Name length error during "Create Hardlinks" fatally freezes DC

Posted: Wed Sep 18, 2019 8:22 pm
by DigitalVolcano
OK thanks for the update. Another thing to try is to enable 'Long path support' in Advanced Settings. I can't recall if this affects hardlinking though.

Re: Name length error during "Create Hardlinks" fatally freezes DC

Posted: Thu Sep 19, 2019 12:34 am
by therube
Everything (but it's down at the moment), https://www.voidtools.com/downloads/.

Search for: path:len:>240 (or whatever length interests you).

Re: Name length error during "Create Hardlinks" fatally freezes DC

Posted: Fri Sep 20, 2019 6:28 pm
by SkyeFire
"Long Path Support" was already checked -- I suspect I may have done that a while back, while cleaning a different directory.

I ended up writing a quick script to list out and record the length of every directory+filename in my tree (24MB text file! :o ). Turns out I had LOTS of Really Long path&file names, from 260 up to 350. The great bulk of that is path, my filenames tend to be 24 characters max plus a 3-char extension.

Does DC put out log files for this kind of error?

Re: Name length error during "Create Hardlinks" fatally freezes DC

Posted: Fri Sep 20, 2019 7:57 pm
by DigitalVolcano
Check the log file in your Documents folder