• Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other Pie Elite all forums
this forum made possible by our volunteer staff, including ...
Marshals:
  • Campbell Ritchie
  • Tim Cooke
  • Ron McLeod
  • Jeanne Boyarsky
  • Paul Clapham
Sheriffs:
  • Liutauras Vilda
  • Henry Wong
  • Devaka Cooray
Saloon Keepers:
  • Tim Moores
  • Stephan van Hulst
  • Tim Holloway
  • Al Hobbs
  • Carey Brown
Bartenders:
  • Piet Souris
  • Mikalai Zaikin
  • Himai Minh

Parsing the log record

 
Greenhorn
Posts: 26
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Log message format :

Severity Time Generated Time Reported HostName Message

eg for the above.
info Mar 10 04:02:02 Mar 10 04:02:02 hostname message

Output on the UI should be in the format:
Time Generated Time Reported HostName Severity Message

I would be having some lakhs of records in my file.
While looping through all,I should take each log record mentioned in the first format,parse it or split it
and then display on the UI in the second format.

I am currently splitting the record using space and then looking for each thing.
In this process,I am using strings which is consuming more memory as there are so many records.

Can anyone define any pattern for this.
Please help.
 
Bartender
Posts: 9626
16
Mac OS X Linux Windows
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Weren't you already considering pagination to solve the memory use problem? That would be my first recommendation.
 
monica singh
Greenhorn
Posts: 26
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
My design implementation was asked not to use pagination.So,I cannot use this.
 
Joe Ess
Bartender
Posts: 9626
16
Mac OS X Linux Windows
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
You have two choices: make the VM heap larger or load less data into memory. Even if you make the heap larger, you will eventually run into the physical limit of the machine you are running on.
 
Ranch Hand
Posts: 2187
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Can anyone define any pattern for this.



You can easily write a performant conversion routine in Perl or Korn Shell.
 
Ranch Hand
Posts: 862
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

My design implementation was asked not to use pagination.So,I cannot use this.


You aren't providing many details, and are expecting people to solve your problem. Is this part of a university assignment and so your requirements are rigid? If not, part of what good developers do are to make suggestions or raise concerns up the management chain. The reality is that we always have to code around computer performance and memory limitations.

Not putting some cap on the upper end of lines that are returned to the browser will always lead to trouble.
 
Joe Ess
Bartender
Posts: 9626
16
Mac OS X Linux Windows
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
She's posted here and here about the same problem.
 
She's out of the country right now, toppling an unauthorized dictatorship. Please leave a message with this tiny ad:
Free, earth friendly heat - from the CodeRanch trailboss
https://www.kickstarter.com/projects/paulwheaton/free-heat
reply
    Bookmark Topic Watch Topic
  • New Topic