Finding occurrences in very large files

Tool for Search and Replace across multiple files.
Post Reply
User avatar
Fool4UAnyway

Finding occurrences in very large files

Post by Fool4UAnyway »

I get an error when just searching (extracting) occurrences in very large files. They are larger than 20 MB, the largest is around 90 MB. There are only less than 100 matches.

"Unable to show file: Argument Length must be larger than or equal to zero."

Perhaps this has something to do with the file's big size. I haven't seen this error before.

I am trying to find some changed occurrences between couples of binary files of around 10 MB. Because my file comparison tool has some problems with files this big, I just use the simple DOS Prompt FC command to write the different bytes found to output files. Those are the large files.

I then look for occurrences of two consecutive bytes, different in both files. One files has bytes AA and BB and the other has CC and DD, for instance. Bytes AA and BB are on two separate lines and so are bytes CC and DD, whether they are paired to AA and BB, or not.

I use Text Crawler (2) because of the ability to use a two-line regular expression and not having to open the files just to have a look at the occurrences. I just want to Extract them and I use that button.

The progress bar almost immediately hits 100%. I don't know if that means "all files" (I guess), or "all occurrences" processed. Then I have to wait for a couple of minutes. Apparently, Text Crawler is trying to display some matches in the pane below the progress bar. Eventually, it is filled with the first matches and then they are gone and the error message appears.

Then it may still take some time before the Scratch Pad is opened. It may, or may not, disappear on top. This may have to do with my system, though, perhaps because of memory limitations.

The error should be solved, but I would also like to be able to turn off this (pre)view functionality, because I want to Extract them only, anyway. And there are only less than 100 matches, so that can never be a problem.

I also used Text Crawler 1.1.4 to remove all redundant 00 digits at the start of the lines (offset addresses) in the binary comparison output to reduce the file's size first. This took quite some time, also, before a "not enough memory" error appeared. I would think Text Crawler would just create a new file by writing changed lines of the source to a new file and then rename or move this file to what the source file was. The source file can be backed up anyway.

By the way, it would be nice if this Comments text box could show some more lines instead of the current number of 5. It's hard to get a quick overview of what I just typed in order to review before posting.
User avatar
Fool4UAnyway

Post by Fool4UAnyway »

So far, not so good... :s

Anything on this, DV?
Post Reply