get only a specific json object using Gson

[Naadiyaar Mansouri]

Hello everyone
so a web API is sending me a huge json file which most of it is useless to me, thus using JsonReader I'm trying to get only the required value.
But JsonReader after reading all values of the first object instead of traversing to next object finishes the process.
Here is part of my json (edited)
[  {    "meta": {      "id": "banana",      "uuid": "72664a65-19b1-45e0-9eaf-23408975939d",      "sort": "020045200",      "src": "collegiate",      "section": "alpha",      "stems": [        "banana",        "bananas"      ],      "offensive": false    },    "def": [      {        "sseq": [          [            [              "sense",              {                "sn": "1",                "dt": [                  [                    "text",                    "{bc}an elongated usually tapering tropical fruit with soft pulpy flesh enclosed in a soft usually yellow rind "                  ],                  [                    "vis",                    [                      {                        "t": "a bunch of {wi}bananas{/wi}"                      },                      {                        "t": "slipped on a {wi}banana{/wi} peel"                      }                    ]                  ]                ]              }            ]          ],          [            [              "sense",              {                "sn": "2",                "dt": [                  [                    "text",                    "{bc}any of several widely cultivated perennial often treelike {d_link|herbs|herb} (genus {it}Musa{/it} of the family Musaceae, the banana family) bearing bananas in compact {d_link|pendent|pendent} bunches"                  ]                ]              }            ]          ]        ]      }    ]  } ]
I'm trying to access "def" object members, more specifically second member of "dt".
I assume this should have a standard solution but I'm unable to find it.
this is my current (not working) code

JsonReader jsonReader = new JsonReader(new StringReader(json));
jsonReader.setStrictness(Strictness.LENIENT);

try
    {

            while (jsonReader.hasNext()) {
                JsonToken nextToken = jsonReader.peek();

                if (JsonToken.BEGIN_ARRAY.equals(nextToken)) {
                    jsonReader.beginArray();

                } else if (JsonToken.BEGIN_OBJECT.equals(nextToken)) {
                    jsonReader.beginObject();

                } else if (JsonToken.NAME.equals(nextToken)) {

                    String name = jsonReader.nextName();
                    System.out.println("Token KEY >>>> " + name);

                } else if (JsonToken.STRING.equals(nextToken)) {

                    String value = jsonReader.nextString();
                    System.out.println("Token Value >>>> " + value);

                } else if (JsonToken.NUMBER.equals(nextToken)) {

                    long value = jsonReader.nextLong();
                    System.out.println("Token Value >>>> " + value);

                } else if (JsonToken.NULL.equals(nextToken)) {

                    jsonReader.nextNull();
                    System.out.println("Token Value >>>> null");

                } else if (JsonToken.END_OBJECT.equals(nextToken)) {

                    jsonReader.endObject();

                } else if (JsonToken.END_ARRAY.equals(nextToken)) {
                    jsonReader.endArray();
                }

            }

        }

Any idea how to solve this?
thank you

[Carey Brown]

What jar file are you building with?

[Ron McLeod]

The JSON document is mal-formed - it missing a brace near the end.  Corrected:[  {    "meta":{      "id":"banana",      "uuid":"72664a65-19b1-45e0-9eaf-23408975939d",      "sort":"020045200",      "src":"collegiate",      "section":"alpha",      "stems":[        "banana",        "bananas"      ],      "offensive":false    },    "def":[      {        "sseq":[          [            [              "sense",              {                "sn":"1",                "dt":[                  [                    "text",                    "{bc}an elongated usually tapering tropical fruit with soft pulpy flesh enclosed in a soft usually yellow rind "                  ],                  [                    "vis",                    [                      {                        "t":"a bunch of {wi}bananas{/wi}"                      },                      {                        "t":"slipped on a {wi}banana{/wi} peel"                      }                    ]                  ]                ]              }            ]          ],          [            [              "sense",              {                "sn":"2",                "dt":[                  [                    "text",                    "{bc}any of several widely cultivated perennial often treelike {d_link|herbs|herb} (genus {it}Musa{/it} of the family Musaceae, the banana family) bearing bananas in compact {d_link|pendent|pendent} bunches"                  ]                ]              }            ]          ]        ]      }    ]  } ]

[Naadiyaar Mansouri]

Carey Brown wrote:What jar file are you building with?

I'm afraid I don't understand what you mean by jar file
but I'm using gson version 2.11.0 and building the code using maven exec plugin

[Ron McLeod]

I don't know what else you might need to do with the document, but when I have a complex document and I just want to extract a single item from it, I often use JsonPath.

With JsonPath you describe a path through the document to the location of the items of interest.  The downside is that is has a significant learning curve, and you can spend a lot of time in the beginning trying to form the expression that fits your needs.  There are on-line evaluators however which can help by showing you the results of the expression as you type (https://jsonpath.com/ and https://jsonpath.curiousconcept.com/#).

It may not be a good solution for what you are working on now, but it might be useful with some future project.

Here's an example (JSON document abbreviated):

import java.util.ArrayList;
import java.util.List;
import org.junit.jupiter.api.Test;
import com.google.common.reflect.TypeToken;
import com.google.gson.Gson;
import com.jayway.jsonpath.JsonPath;

public class JsonPathTest {
  String json = """
    [
    {
      "meta":{
      "id":"banana",
      "uuid":"72664a65-19b1-45e0-9eaf-23408975939d",
      "sort":"020045200",
		...
    }
    ]  
         """;

  @Test
  void find_in_path() {
    Gson gson = new Gson();
    var stringArrayType = new TypeToken<ArrayList<String>>() {}.getType();
    var document = JsonPath.parse(json);

    var unfilteredExpression = "$[0].def[0].sseq[*][0][1].dt[0][1]";
    var unfilteredStr = document.read(unfilteredExpression).toString();
    List<String> unfiltered = gson.fromJson(unfilteredStr, stringArrayType);
    System.out.println("unfiltered:");
    unfiltered.forEach(System.out::println);

    System.out.println("");

    var filteredExpression = "$[0].def[0].sseq[?(@[0][1].sn == '2')][0][1].dt[0][1]";
    var filteredStr = document.read(filteredExpression).toString();
    List<String> filtered = gson.fromJson(filteredStr, stringArrayType);
    System.out.println("filtered:");
    filtered.forEach(System.out::println);
  }
}

Example console output (long lines shortened):$ mvn -q verify -Dtest=JsonPathTest unfiltered: {bc}an elongated usually taper...rind {bc}any of several widely cult...ches filtered: {bc}any of several widely cult...ches
The first expression $[0].def[0].sseq[*][0][1].dt[0][1] breaks-down like this:
   $[0]  first element in document
   def[0]  contents of first occurance of def
   sseq[*]  contents of all occurrances of sseq
   [0]  contents of first container
   [1]  contents of second container (sense)
   dt[0]  contents of first occurance of dt
   [1]  value of second item

The secondexpression $[0].def[0].sseq[?(@[0][1].sn == '2')][0][1].dt[0][1] includes a filter to only include contents where sn has a value of 2.

These are the dependencies that I used:<dependency>    <groupId>com.jayway.jsonpath</groupId>    <artifactId>json-path</artifactId>    <version>2.9.0</version>    <scope>test</scope> </dependency> <dependency>    <groupId>com.google.code.gson</groupId>    <artifactId>gson</artifactId>    <version>2.11.0</version>    <scope>test</scope> </dependency>

[Carey Brown]

The hasNext() is returning false. This is my work around.

import com.google.gson.Strictness;
import com.google.gson.stream.JsonReader;
import com.google.gson.stream.JsonToken;

import java.io.StringReader;
import java.nio.file.Files;
import java.nio.file.Paths;

public class JsonReaderDemo
{
	static final String FILE_NAME = "data/JsonReaderDemo.json";

	public static void main( String[] args )
	{
		try
		{
			String json = Files.readString( Paths.get( FILE_NAME ) );
			JsonReader jsonReader = new JsonReader( new StringReader( json ) );
//			jsonReader.setLenient( true );
			jsonReader.setStrictness(Strictness.LENIENT);

			while( true ) // jsonReader.hasNext() )
			{
				JsonToken nextToken = jsonReader.peek();
//				System.out.println( "Token >>>> " + nextToken );

				if( JsonToken.BEGIN_ARRAY.equals( nextToken ) )
				{
					System.out.println( "Token BEGIN_ARRAY >>>>" );
					jsonReader.beginArray();
				}
				else if( JsonToken.BEGIN_OBJECT.equals( nextToken ) )
				{
					System.out.println( "Token BEGIN_OBJECT >>>>" );
					jsonReader.beginObject();
				}
				else if( JsonToken.NAME.equals( nextToken ) )
				{
					String name = jsonReader.nextName();
					System.out.println( "Token KEY >>>> " + name );
				}
				else if( JsonToken.STRING.equals( nextToken ) )
				{
					String value = jsonReader.nextString();
					System.out.println( "Token Value >>>> " + value );
				}
				else if( JsonToken.NUMBER.equals( nextToken ) )
				{
					long value = jsonReader.nextLong();
					System.out.println( "Token Value >>>> " + value );
				}
				else if( JsonToken.NULL.equals( nextToken ) )
				{
					jsonReader.nextNull();
					System.out.println( "Token Value >>>> null" );
				}
				else if( JsonToken.END_OBJECT.equals( nextToken ) )
				{
					System.out.println( "Token END_OBJECT >>>>" );
					jsonReader.endObject();
				}
				else if( JsonToken.END_ARRAY.equals( nextToken ) )
				{
					System.out.println( "Token END_ARRAY >>>>" );
					jsonReader.endArray();
				}
				else if( JsonToken.END_DOCUMENT.equals( nextToken ) )
				{
					System.out.println( "Token END_DOCUMENT >>>>" );
					break;
				}
				else
				{
					System.out.println( "Token unknown >>>> " + nextToken );
				}
			}
		}
		catch( Exception ex )
		{
			ex.printStackTrace();
		}
	}
}

[Carey Brown]

With indent and BOOLEAN.

import com.google.gson.Strictness;
import com.google.gson.stream.JsonReader;
import com.google.gson.stream.JsonToken;

import java.io.StringReader;
import java.nio.file.Files;
import java.nio.file.Paths;

public class JsonReaderDemo
{
	static final String FILE_NAME = "data/JsonReaderDemo.json";
	static int			indent = 0;
	
	public static void main( String[] args )
	{
		try
		{
			String json = Files.readString( Paths.get( FILE_NAME ) );
			JsonReader jsonReader = new JsonReader( new StringReader( json ) );
//			jsonReader.setLenient( true );
			jsonReader.setStrictness(Strictness.LENIENT);

			JsonToken	token;
			while( ! JsonToken.END_DOCUMENT.equals( token = jsonReader.peek() ) )
			{
//				System.out.println( "Token >>>> " + token );

				if( JsonToken.BEGIN_ARRAY.equals( token ) )
				{
					System.out.println( spaces() + "Token BEGIN_ARRAY >>>>" );
					indent++;
					jsonReader.beginArray();
				}
				else if( JsonToken.BEGIN_OBJECT.equals( token ) )
				{
					System.out.println( spaces() + "Token BEGIN_OBJECT >>>>" );
					indent++;
					jsonReader.beginObject();
				}
				else if( JsonToken.NAME.equals( token ) )
				{
					String name = jsonReader.nextName();
					System.out.println( spaces() + "Token KEY >>>> " + name );
				}
				else if( JsonToken.STRING.equals( token ) )
				{
					String value = jsonReader.nextString();
					System.out.println( spaces() + "Token Value >>>> " + value );
				}
				else if( JsonToken.NUMBER.equals( token ) )
				{
					long value = jsonReader.nextLong();
					System.out.println( spaces() + "Token Value >>>> " + value );
				}
				else if( JsonToken.BOOLEAN.equals( token ) )
				{
					boolean value = jsonReader.nextBoolean();
					System.out.println( spaces() + "Token Value >>>> " + value );
				}
				else if( JsonToken.NULL.equals( token ) )
				{
					jsonReader.nextNull();
					System.out.println( spaces() + "Token Value >>>> null" );
				}
				else if( JsonToken.END_OBJECT.equals( token ) )
				{
					System.out.println( spaces() + "Token END_OBJECT >>>>" );
					indent--;
					jsonReader.endObject();
				}
				else if( JsonToken.END_ARRAY.equals( token ) )
				{
					System.out.println( spaces() + "Token END_ARRAY >>>>" );
					indent--;
					jsonReader.endArray();
				}
				else
				{
					System.out.println( "Token unknown >>>> " + token );
				}
			}
		}
		catch( Exception ex )
		{
			ex.printStackTrace();
		}
	}
	
	static String spaces()
	{
		return "|  ".repeat( indent );
	}
}

[Tim Holloway]

I'm with Ron.

Two reasons:

1. Less code overall and the code is more to the point
2. A streaming reader is likely to be more memory efficient, since it doesn't have to load the entire document into RAM.

The jsonpath filter expression language is just one example of a path-seeking language. Another popular one it XML's xpath. Python's BeautifulSoup path processing also works on that basis for html, although it's document-based, not stream-based.

JSON complex data elements are one of two forms: dictionaries and arrays, and the jsonpath language reflects that. One good thing about JSON is that your browser might be able to view it with expand/collapse text, which makes it easier to home in on the elements of interest. So it needn't be that big a struggle.

[Tim Holloway]

Three reasons.

3. If the document structure changes, it's a lot easier to re-code a path than to manually re-code document search logic.

[Naadiyaar Mansouri]

Thank you all for great information!