• Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other Pie Elite all forums
this forum made possible by our volunteer staff, including ...
Marshals:
  • Campbell Ritchie
  • Jeanne Boyarsky
  • Ron McLeod
  • Paul Clapham
  • Liutauras Vilda
Sheriffs:
  • paul wheaton
  • Rob Spoor
  • Devaka Cooray
Saloon Keepers:
  • Stephan van Hulst
  • Tim Holloway
  • Carey Brown
  • Frits Walraven
  • Tim Moores
Bartenders:
  • Mikalai Zaikin

suggestions on reading from file

 
Ranch Hand
Posts: 476
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Hello,
I have a pipe-delimited file that I want to read from. As I read each line of file, I tokenize it and put each token in an Object[][]. Everything works fine, the only problem is that it taks too long to read whole file. Is there a way to speed up this proccess?
Code is below.
thanks,
Alex

[ August 06, 2003: Message edited by: Alex Kravets ]
 
Wanderer
Posts: 18671
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
I don't see any code here that actually reads from a file. This seems to be some sort of processing after the file has been completely read. Chances are good that the IO from the file is slower than anything else you're doing, so let's look at that.
 
Alex Kravets
Ranch Hand
Posts: 476
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Actually I have a custom class that just calls standard Java I/O functions.
For example, grid.getLine(j) reads a single line of a file, and loop with: data = grid.getToken(columnLine,m); reads each token.
Alex
 
Chicken Farmer ()
Posts: 1932
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
I'm with Jim, I/O is usually one of the more processing intensive operations to go through. What sort of slow downs are you seeing?
 
Alex Kravets
Ranch Hand
Posts: 476
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
When my Applet reads a file I see it on the status bar of the browser, it says opening http://www.webmacaco.com/files/tmp/grid.1233 (grid.1233 is a name of a file with session id). So this stays in the status bar for about 3-4 minutes before applet goes to next step and loads. I figure this actual reading from a file is what's taking a lot of time. As a test I print out to the console all tokens that are being read (the printing itself also slows down the whole proccess -- I removed it later) I see that reading each token takes part of those 3-4 minutes. I just want to know if there a faster way to get data from a file into Object[][] or may be I should use other type of Collection like a map or hash?
thanks,
Alex
 
Jim Yingst
Wanderer
Posts: 18671
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Actually I have a custom class that just calls standard Java I/O functions.
Well, some standard I/O classes and methods are much more efficient than others. And there are plenty of opportunities to use them inefficiently if you're not careful. If you're using streams, the biggest optimization may be to switch to a FileChannel from 1.4. If you're unable to use 1.4 on your project, then at least try to use bulk read() methods like read(char[]) or readLine() rather than read() (which reads a single char or byte). That's a common mistake.
For example, grid.getLine(j) reads a single line of a file, and loop with: data = grid.getToken(columnLine,m); reads each token.
Well, I see a big possible slowdown right there. How does getLine(j) get to line j? Is it just reading the next line after whatever the last thing was that it read? (If so, what do you need the j argument for?) Or does it start at the beginning of the file an read consecutive lines until you reach line j? If it's the latter, you're reading all those lines a lot of times. When j = 100 you read 100 other lines before you get to line 100 (starting from 0, right?) Then when j = 101, you reread the same 101 lines you just read, before getting to line 101. That's got to be wasteful. There's a similar issue with the getToken() method - does it start from the beginning of the string each time you use it?
 
Alex Kravets
Ranch Hand
Posts: 476
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Ok, this is the whole nested loop. Outer loop reads a line depending on value of j (I set j = 2 because I don't want to read first 2 lines to the Object[][]), so if j = 2 I read line 3, if j = 3 read line 4 and so on (0 = line 1). grid.getLine(j) does not get all n<j lines before it reads j, it just reads line j.
Second loop just tokenizes columnLine and gets all tokens from there. Again I set to read from token 4 because I don't want to read previous 3 tokens into Object[][].

Also, this is my getLine() method:
----------------------------------

thanks,
Alex
 
jason adam
Chicken Farmer ()
Posts: 1932
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Well, I'm no expert, but if you're going out to a URL, having it read in a file and passing the stream back to you, and then iterating through the stream, that's quite a lot to do.
Assuming you're connecting to some sort of servlet, I'd look at putting as much processing on the server side and passing back the applet a meaningful object (through an ObjectInputStream) that contains the data you are expecting.
But then, I hate applets and never have them do anything except make a request and get an object (single or Collection) back. I would never have applet code iterate over a stream it gets from a URL.
 
Alex Kravets
Ranch Hand
Posts: 476
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
I'll give it a shot. Thanks!
 
jason adam
Chicken Farmer ()
Posts: 1932
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
No problem, let us know how it goes.
Now for something completely different, LOVE the quote. Do you have the Homer bottle opener? A friend of mine found a shirt with the "Seven Duffs" from a thrift store for $1. Told him I'd gladly pay 10 or 20 times the amount, but he wouldn't sell
 
Alex Kravets
Ranch Hand
Posts: 476
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Actually I almost did. My wife and I were shopping at Target here in Brooklyn, and I saw these two Mr. H.J.Simpson bottle openers $7.95 each. I bought two, but just when we were leaving after paying at the register...we forgot the bag with those two bottle openers. Next day when we came back there were no sign of them anywhere. Oh, well...may be there are some to be found at eBay?
Yup, eBay got them: http://cgi.ebay.com/ws/eBayISAPI.dll?ViewItem&item=3235904568&category=8616
[ August 06, 2003: Message edited by: Alex Kravets ]
 
jason adam
Chicken Farmer ()
Posts: 1932
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Yup, that's the one. Well worth the money, in my opinion
 
Alex Kravets
Ranch Hand
Posts: 476
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
true
 
Consider Paul's rocket mass heater.
reply
    Bookmark Topic Watch Topic
  • New Topic