A bug of DC 1.4.7

kk · Post by kk » Sat Jan 22, 2011 8:07 am

From http://www.mathstat.dal.ca/~selinger/md5collision/
We can download two different exe files:
http://www.mathstat.dal.ca/~selinger/md ... /hello.exe
http://www.mathstat.dal.ca/~selinger/md ... /erase.exe

DC 1.4.7 consider they are duplicate files.

DV · Post by DV » Sat Jan 22, 2011 12:07 pm

Not a bug, it's an MD5 hash collision, and a limitation of using MD5 comparison. Very rare in the wild (the files you link to are specific examples of MD5 collision)

That said, Duplicate Cleaner 2.0 can use SHA-1 or Byte comparison which should avoid such rare cases!

kk · Post by kk » Tue Feb 22, 2011 1:49 am

It is strange that DC 1.4.7 succeeds to distinguish the two files below, whose MD5 values are the same too.

http://www.win.tue.nl/hashclash/SoftInt ... liding.exe

http://www.win.tue.nl/hashclash/SoftInt ... liding.exe

It is not safe to use DC 1.4.7, as MD5 collision is very rare, but it exist! Files with same MD5, should be checked byte by byte. It is a pain to use DC 2.0 which needs .Net Framework. Would DC 1.4.7 be updated?

CC · Post by CC » Tue Feb 22, 2011 9:17 pm

What's the problem with using .Net Framework? Lot's of programs use it. Also, it's best to use SHA-1 with ANY duplicate finder.