Page 1 of 4

Beta: Duplicate Cleaner 5

Posted: Mon Dec 14, 2020 11:36 am
by DigitalVolcano
The Duplicate Cleaner 5 beta is currently closed to new testers, thanks for your interest!

Re: Beta: Duplicate Cleaner 5

Posted: Mon Dec 14, 2020 12:15 pm
by brew
I have a few questions about available features in this release. I have been dealing with de-duplicating corrupt files, so the motivation for these features pivot around this fact. Sorry in advance for the long list :P
  • Will there be support for video specific comparison? I would like to see thumbnail generation at least, which would help a lot. I have a large home video collection which needs to be organised.
  • Will we be able to highlight multiple files/folders/groups and press Ctrl+C or right-click to copy the selection to another place for external review/backup?
  • Can you add support (a filter maybe?) for finding files which are classed as duplicates in regards to file name/ext/date but the content is different? I have a drive with corrupt files and I need a way to find the okay copies to keep. It would be good to not use the 'compare file content' option, but still have the option to generate and display the hashes in the results.
  • As an extension to the point above, would you consider adding corruption detection or some way to handle this like an option to 'mark files that are corrupt'? I appreciate this may not be possible.

Re: Beta: Duplicate Cleaner 5

Posted: Mon Dec 14, 2020 1:24 pm
by DigitalVolcano
Thanks for your interest -

*There is a special video mode, with similar comparison and thumbnails available. See preview video here-

*You can mark/copy the files to another location using the File removal/copying tool in the program. Direct dragging to Windows Explorer is something which is still to be added.

*It doesn't yet generate hashes in 'Ignore content' mode, but this is an idea to look at.


Posted: Tue Dec 29, 2020 7:13 pm
by Rick
The beta is impressive! Thank you! Every time I think there is something to be added, I find it aready there

I have a lot of fonts and have to search in regular mode
Can fonts be added to the viewer capabilities at some point?
Double clicking on a TTF or OTF file does not launch it in the Windows font viewer - it can only be done via the context menu

Question: What good are the virtual folders? I really don't see any real use for it over adding a folder directly to the searches. If it held several folders or file from various places I could see it. Could someone please explain it a bit?


Re: Beta: Duplicate Cleaner 5

Posted: Mon Jan 04, 2021 1:25 pm
by DigitalVolcano
Glad you like it.

I'll look into the fixing the double click. At the moment it only sends a file to Windows to try and open when using the context menu.

The virtual drive/folder function is for making a complete map of a drive/folder which means that any part of it can be quickly compared even when the drive isn't present. For instance you could use this for cataloguing CD backups or offline USB storage drives, or slow network drives for quicker comparison later.

Re: Beta: Duplicate Cleaner 5

Posted: Mon Jan 18, 2021 10:51 pm
by cstern

Could you please briefly describe how to use the virtual folder feature? What steps to make it work, e.g. if I have files on "X:" which is on a NAS.

And will it be a "pre-compared" entity that does not need to be processed against itsself in future comparisons, for instance? I mean, if e.g. I have a collection of, say, 500.000 images I use as my main library and I want to compare that to batches of other files, and put that into a virtual folder, will DC5 then skip the part of the process where the 500.000 images are compared againt themselves on run #2+?

Also, is it a snapshot of a certain time point? So if I add more to the main library, would I then have to start over and "delete" the virtual folder and make a new one?

It would be really great if this feature could work like google image search, which is extremely fast.

BTW, I have been testing DC5 for a while, and it's awesome!


Re: Beta: Duplicate Cleaner 5

Posted: Wed Jan 20, 2021 10:48 am
by DigitalVolcano
The virtual folders are a snapshot of a drive/folder at a certain time, optionally with pre-calculated file hashes (Image/Audio/Video metrics are still TBD). There isn't currently a function to 'update' the snapshot without recreating, but this is something that can be added if the feature is popular.

It's main use is to create a virtual snapshot of something - e.g. offline archives/drives/optical media/slow network location - which can then be used in later (much faster) scans where the media isn't present.
It's up to the user whether the drive is pre-deduped, but when comparing it can be used with the 'External only' Duplicate option which means that any internal duplicates to that drive are ignored.

Re: Beta: Duplicate Cleaner 5

Posted: Tue Jan 26, 2021 1:42 am
by mjdalways
Just did a quick run of version 5 beta, its very nice. I like the new selection option (often wanted to be able to select (1) type files). It seems faster too, although the old version said it took less time (the old one felt longer - I'll do some more accurate tests at some point). Movie comparisons is amazing and the virtual folder is a great idea - cant wait for the attributes to be cached. I didnt think you had anything to add, but I was very wrong :)

Overall great job, and thanks, I'll be happy to pay for an upgrade.



Re: Beta: Duplicate Cleaner 5

Posted: Wed Feb 03, 2021 4:30 pm
by therube
Is it intended that we reply in a thread like this or via email?


(I've not looked at 5.0.3, yet, so the link parts may be working, now?)

/IF/ DC5 followed symbolic links
you could use Everything (of course) to find files
Link Shell Extension to "gather" a diverse set of files - from ALL OVER the place
& have it DROP those "files" into a single directory
which DC5 could then scan ( to find, dups ;-) )
- BUT, as it is, "shortcut" (symbolic) links are simply ignored
(perhaps it see's "0" bytes &... ?)

/IF/ all the files you wanted to scan were on a single drive (Volume)
/then/ you could create HARD-LINKS
from diverse directories, "placing" them all in a single directory
(not sure offhand what name dups would do)
that DC5 WILL scan, & find dups :-)
- BUT, that would rarely be the case - for me
with my diverse sets of files - from ALL OVER the place

(odd, first time i heard a [scan] "completion" sound) [i guess that is what it was?]

in a directory where i "put" some links
i created a COPY of 1 of the hardlinked files (file - Copy.mp4)
& then re-ran a scan on that directory
- & it did NOT find that newly created file as a duplicate
(& obviously, being an exact copy, it should have ?)
? maybe because of xxhash/metrics [tis a VIDEO scan], maybe because it is an IDENTICAL
file, it skipped the "rescan" [though it /DID/ create a new thumbnail.jpg file (which
since identical, obviously does IDENTICALLY match the earlier create (differently named)
thumbnail.jpg of that file ?

file/image metrics, [Clear cache], is (perpetually) grayed ?


a link to LOG.txt
should be available, from "everywhere" (or at least, more generally available)
i run a scan, it says, "scan complete"
& i want to see the LOG file - before anything else
& i have to hunt down HOW to view it

Re: Beta: Duplicate Cleaner 5

Posted: Wed Feb 03, 2021 5:13 pm
by therube
fix: Symlinks/Shortcuts now work across drives
Am I doing this correctly?

I symlink a file from \\othercomputer\C\LIB\NORTON\EROIICA.TXT
to C:\TMP\hardlink\EROIICA.TXT

I symlink a file from C:\TMP\hardlink\file.txt
to D:\WIN7ISO\file.txt

Should DC5 have found those? (It doesn't.)

I create a (physical) copy of C:\TMP\hardlink\file.txt
to C:\TMP\hardlink\file - Copy.txt

DC does find that pair.

Same Content.
(And if it is checking file size, first, which it should, & if the file is a symlink, then it will fail, cause a symlink shows as 0-bytes.)
(There are other settings in there; Mount points/Junctions, & Count hard-links, but they don't seem to have any effect on the situation.)