Error: database disk image is malformed

The best solution for finding and removing duplicate files.
User avatar
DigitalVolcano
Site Admin
Posts: 1717
Joined: Thu Jun 09, 2011 10:04 am

Re: Error: database disk image is malformed

Post by DigitalVolcano »

The database will still persist between sessions - there won't be any difference to the user!
User avatar
Spire
Posts: 5
Joined: Tue Jul 23, 2019 10:54 am

Re: Error: database disk image is malformed

Post by Spire »

Oh thank goodness. So then what did you mean by using a fresh database for each scan?

While I'm here, I have a second question: Does the database ever get pruned of stale entries? If not, it might not be a bad idea to do so. Then again, I don't know how you would deal things like external hard drives that are temporarily disconnected.
dbague
Posts: 20
Joined: Mon Oct 02, 2017 6:35 pm

Re: Error: database disk image is malformed

Post by dbague »

DigitalVolcano wrote: Thu Aug 15, 2019 10:07 am The database will still persist between sessions - there won't be any difference to the user!
I am also not sure, how to understand a fresh database each time that persists.

the advantage of the persistent database, is that files with hashes or prehashes (fasthashes?) already computed (i assume, it was assigning the encountered path to the hashes, so moved files, would recompute...getting side-tracked). So for external drives that are backup, or backups of backups, not always very organized over here, with minimal turnover, there was a great benefit.

still, please explain your concept, two type of databases? merging upon some criteria?. thanks for your work anyway.
User avatar
Spire
Posts: 5
Joined: Tue Jul 23, 2019 10:54 am

Re: Error: database disk image is malformed

Post by Spire »

dbague wrote: Thu Aug 15, 2019 9:34 pm I am also not sure, how to understand a fresh database each time that persists.
My best guess is that:
  1. There is a master database that is opened in read-only mode at the start of each scan.
  2. Any new items found during a scan are added to a fresh session database.
  3. After the scan is complete, the session database is merged into the master database and then deleted.
  4. The master database is always automatically backed up before a session database is merged into it.
  5. If the master database is ever corrupted, it can be restored from the most recent backup.
dbague
Posts: 20
Joined: Mon Oct 02, 2017 6:35 pm

Re: Error: database disk image is malformed

Post by dbague »

Spire wrote: Fri Aug 16, 2019 5:13 am My best guess is that:
  1. There is a master database that is opened in read-only mode at the start of each scan.
  2. Any new items found during a scan are added to a fresh session database.
  3. After the scan is complete, the session database is merged into the master database and then deleted.
  4. The master database is always automatically backed up before a session database is merged into it.
  5. If the master database is ever corrupted, it can be restored from the most recent backup.
I was thinking along those lines. but just to be make sure: the persistent databases would be loaded into memory so as not to recompute the wheel...

i have been doing the back ups every time i would do a big chunk in the past years. Nice if the software would do it, allowing for custom location, of course. Can the author confirm that direction or intent?

thanks.
User avatar
DigitalVolcano
Site Admin
Posts: 1717
Joined: Thu Jun 09, 2011 10:04 am

Re: Error: database disk image is malformed

Post by DigitalVolcano »

To clear up the confusion, the database refers to a single database that stores everything (in versions 3 to 4). The program settings, the hash caches and the current scan.
In version 5 this is split into separate database files - the settings, the hash/image caches, and the scans (each scan in a new file).

The advantages are that if some reason there is a crash and the scan database is damaged, the settings and caches are left alone. This also gives the option of storing and recalling previous scans, though I haven't figured out the best way of implementing it yet.
dbague
Posts: 20
Joined: Mon Oct 02, 2017 6:35 pm

Re: Error: database disk image is malformed

Post by dbague »

I was late to read your reply.

This is sound, to separate settings from hash cache. The hash cache is, in my mind, the interesting bit where not everything has to be recomputed
(or alternatively, one can go from smaller scope of drive "surface" to bigger, i often go from very restricted to wider, to get a sense of how long the computation will be, maybe an ancient habit from old library databases).

And at some point resetting the settings meant the cache as well in the past, right? if there were a bug with interface setting, or computer glitch, then the hash cache would get corrupted as well.

The scan, is currently (3 and 4) sharing memory space with the setting and hash cache? or also get stored with database as program closes?
The latter would not fit with having to save the file duplicate file list for later removal review (unless once integrated in the hash cache, the folder selections, of last session profile, are lost)

so i guess you mean that in 5, there will be three distinct data-sets, 2 databases: hash cache, current scan), perhaps allowing restart from previous state.

some kind of delay before integrating the current scan into the existing caches? Wow, the more i circle the idea, the more i circle it (constant radius).
or we are converging. What is the difficulty that you seem to mention in your post?
Post Reply