Page 1 of 1

Feature Request: Use hash directly from Google Drive API to hugely increase speed

Posted: Sat Oct 29, 2022 12:13 am
by Grady
Hi, from my understanding, most cloud drive solutions provide a checksum/hash via API calls.

For example:
https://stackoverflow.com/questions/234 ... -for-files

Some other products already do this (but not on Windows PC):
DeDuplicate: "It’s fast because it calls the APIs of the cloud services, retrieving useful information about the files but not the file itself. “How fast”? Well, I have more than 100.000 pictures on OneDrive and it takes 5 minutes to scan them."

Would it be possible for duplicate cleaner to utilize this mechanism for files on the cloud drive? At the moment, we need to download TBs of files from the cloud and then calculate the hash, hugely limited by download speeds. There are also download quota limitations that could be hit (e.g. 10TB/day from Google Drive, or some other limits imposed by ISPs) It would be 10000 times faster and cheaper if we can use existing hash/checksums provided by cloud drives like Google Drive via API.

Thanks!

Re: Feature Request: Use hash directly from Google Drive API to hugely increase speed

Posted: Mon Oct 31, 2022 3:25 pm
by therube
(While maybe not "particularly" important, kind of surprised that they use MD5 - given that MD5 can generate collisions.)