Help, Java Scanner Question

[power09]

Hello

I'm trying to code a Java program that will scan a text file and output the lines I need (Its all the source code from an Email Inbox and I only need the headers)

the whole text file is 172,066KB (3,048,394 lines in notepad)

The program Iv just written is working great until it tries to output line number 770, (which means its scanned 11,204 lines in the text file)

The scanner just stops working... (fails to scan the next line)
Is java.util.scanner the wrong tool for scanning a file this big? Or am I just using too much memory and the java virtual machine is stopping it?

 

[Black Flag]

does the code throw an exception? handle the error.... check error messages.

try your code with you own simple text file that you can control.

 

[power09]

Well I stopped using Scanner to read the file and started using BufferedReader and that solved the problem. (for a while)

Now with BufferedReader it throws an exception after writing line 162,731 (or failure reading line 3,043,244 )
I used e.getCause(); and it says its a java.lang.NullPointerException

I could tell from my attempt to debug it that BufferedReader starts reading the line but does not read the whole line
the program continues like normal so BufferedWriter writes the unfinished line to the file

then in the next iteration the BufferedReader does not read any of the next line (it should), so then the string that line gets assigned to is null and following operations cause the exception

why the hell would BufferedReader just fail in the middle of an operation?

 

[Sellyme]

The simple solution is to just split up the text file. I'm not /that/ good with Java, so I can't help much beyond that, but sometimes the simplest solutions are the best.

 

[power09]

I worked out the problem btw,

I was getting a null pointer exception at the end of the file when the scanner scanned a few/couple of empty lines. This exception caused the .close() method to never run. Meaning the BufferedFileWriter never was able to empty the whatever was in the current buffer at the time (which was always at the end of the file). I got around this by running .flush() (to empty the current buffer) every time the while loop ran and then weeks later found the cause of the nullpointerEx and fixed it.