• Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other all forums
this forum made possible by our volunteer staff, including ...
Marshals:
  • Campbell Ritchie
  • Bear Bibeault
  • Ron McLeod
  • Jeanne Boyarsky
  • Paul Clapham
Sheriffs:
  • Tim Cooke
  • Liutauras Vilda
  • Junilu Lacar
Saloon Keepers:
  • Tim Moores
  • Stephan van Hulst
  • Tim Holloway
  • fred rosenberger
  • salvin francis
Bartenders:
  • Piet Souris
  • Frits Walraven
  • Carey Brown

unix shell script - search string and formatting to json

 
Ranch Hand
Posts: 32
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
I have a log file with multiple json message that is not formatted.
Trying to write a shell script to search for a string in the logs and format the matching json and write to a file.

For Example :- in the test.log file,

type_logfile 2015-04-14 18:06:15,284 INFO ResponseLogService -
{"header":{"time":"2015 06:06:13 PM EDT","reqId":"1","user":"baskar"}}

type_logfile 2015-04-14 18:06:15,284 INFO ResponseLogService -
{"header":{"time":"2015 06:06:13 PM EDT","reqId":"2","user":"raja"}}

type_logfile 2015-04-14 18:06:15,284 INFO ResponseLogService -
{"header":{"time":"2015 06:06:13 PM EDT","reqId":"3","user":"baskar"}}


I use the below command,

grep -i "baskar" test.log | grep -o "{\"header\".*" | python -m json.tool > ~/search_result.log



The above command works fine if the search returns only one json message. If there are multiple matches, this doesnt work and throw the following error.

Unable to decode the json message.

Whereas, if i redirect the matched output to some file and then read the file line by line to format the json works.

grep -i "baskar" test.log | grep -o "{\"header\".*" > ~/search_result.log

while read line
do
name=$line
echo $name | python -m json.tool >> ~/formatted_search_result.log
done < ~/search_result.log



Since, writing to a temp file and then reading from the temp file to format the json hits the performance, looking for some efficient way of doing this.

The expected output is,
When i search for

grep -i "baskar" test.log

,





Thanks,
Baskar.S
 
Saloon Keeper
Posts: 22273
151
Android Eclipse IDE Tomcat Server Redhat Java Linux
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Have you looked at this? http://www.skorks.com/2013/04/the-best-way-to-pretty-print-json-on-the-command-line/
 
CLUCK LIKE A CHICKEN! Now look at this tiny ad:
Thread Boost feature
https://coderanch.com/t/674455/Thread-Boost-feature
    Bookmark Topic Watch Topic
  • New Topic