I'm rewriting an app that pulls a dozen data items for about six million rows in an Oracle database. About half of these are strings, and a majority are unique (physical addresses and 10-digit phone numbers).
I've been profiling my memory usage, and the garbage collector gets me back down to a fairly constant size every time it runs, but it gets behind at the threshold, which means it allocates a little extra space before clearing the memory, and the heap gets bigger every cycle.
Is there a way of pulling them (or streaming them) directly into a StringBuffer, to prevent the
string pool from getting enormous from all the unique values? I've been trying getAsciiStream and getCharacterStream but not having much luck.
Alternately, is there a way to fiddle with the GC to prevent the reallocate before the collection?
Has anyone run into a problem like this before? Am I barking up the wrong tree?
Thanks in advance,
Bill