Page 1 of 1

Deleting files really slow

Posted: Mon Feb 08, 2010 12:21 am
by Chris
Hi there!

First: thanks a lot for that great piece of software! I've looked around for such a comparsion tool and this one fits best (on my hands :-) ).

I was glad to see how fat it works even on some more files - BUT I was astonished as I tried to delete the selected files - I can go and take a cup of coffee and another and another...

For example: looking through a folder and its several subfolders with about 9500 pdfs searching for duplicates lasts less than 10 minutes (AMD Athlon 64 X2 Dual Core 4000+ with 2 GB RAM, Windows 2000 SP4) - found about 6000 duplicates to delete (coming from several recovery runs after a disk crash) - for easyness I hit "keep one file of group", "remove - no bin". After 30 minutes (now) the fourth piece of the time bar appeared...

Is there some optimization possible?

I'm a little bit in fear to run the tool on my image folder with about 40.000 files.

Greetings,
Chris from Germany (so please excuse my fundamental :-) English)

Posted: Tue Feb 09, 2010 3:40 pm
by Chris
Hi it's me again,
I've done some tests against the clock :-)

And fact is that deleting one (1) file lasts somewhat exact one (1) second. Is there a halt implemented on Win2000? Am I the only one who has those problems?

Here are my figures:

58872 files searched (127,58 GB) on two different (internal) SATA-drives in two folders and their many subfoldes, found 4061 groups with duplicates, 8152 files in duplicates (647,27 MB), usage of RAM: 90 MB - searching and identified lasts only 21:48 minutes (that's real great!)
Marked 4091 files (323,87 MB) for deletion (using bin)
started at 10:46 AM, stopped at 11:56 AM

Greetings,
Chris

Posted: Wed Feb 10, 2010 8:57 am
by DV
I need to look into this - delete should be fast in theory as it uses Windows API calls. It doesn't look like this is the case though!
Will investigate.

Posted: Fri Feb 25, 2011 7:32 am
by Adam
I have experienced this as well - deleting 2112 pdf's has taken ~1.5 hours so far and it is only about 60% complete. This is on a high end quad core with 8 GB of RAM. Everything else is running smoothly on the system, no glitches or slowdowns of any kind. That said, DC is a great tool with wonderful features - the best I have seen in its class. The selection assistant is extraordinarily helpful. Thanks for all your hard work!

Posted: Sat Jun 11, 2011 11:03 pm
by Rene
Same problem here (still)! Duplicate cleaner deletes files at a rate of 1/sec and uses a lot of CPU power to do it. Otherwise an amazing tool, but this makes it very painful to use for a large number of duplicates.

Posted: Sun Jun 12, 2011 9:51 am
by DV
Is this the latest version (2.0.6)?

Posted: Sun Jun 12, 2011 3:41 pm
by therube
Are you deleting to Recycle Bin?

Recycle Bin maximum size limitations.

I believe XP has a 4 GB limit.
(Larger on Windows 7, generally 10% of disk size.)

As you hit the size limit, older entries are purged as newer are added.

Additionally as the number of items stored in a directory become larger, Windows becomes less efficient in handling them.

As a test, if you copy a set of files into another directory, open Window Explorer, & delete them from there, does the same 1/sec (@Rene) happen?

Posted: Sun Jun 12, 2011 6:16 pm
by Otto
First of all - excellent tool for modern digital libraries. Thank you.

Second- I agree that deleting high numbers of items to the recycle bin is painfully slow even to very high end systems. Copying to a folder and subsequently deleting the folder is a breeze. Is there a way to put a dialogue box on the delete portion that says "Hey, this is going to take a while because of windows..." ?

Re: Deleting files really slow

Posted: Sat May 29, 2021 8:29 am
by al_from_oz
I am having same problems (external USB drive though I do concede). I have >41,000 duplicate files to delete. I run for more than an hour and it's at 1% (not deleting to recycle bin).

Madness.

Re: Deleting files really slow

Posted: Thu Jun 03, 2021 4:04 pm
by therube
@al,

To confirm, you are not deleting to (Windows) Recycle Bin.

OK, then tell us about your external USB drive.

Some have their "own" recycle bins.
Like I've got an external Synology (NAS) drive (not NTFS or FAT...).
With that, deleted files do not delete to any (Windows) Recycle Bin, but the NAS itself has its' own #recycle (directory), where deleted files do go.

So if you, your external USB drive, has something like that set up on your end, then while you may not be deleting to (Windows) Recycle Bin, deleted files could still be going to some recycle bin.

Peruse through your drive & see if maybe you don't see something like that?