• Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other Pie Elite all forums
this forum made possible by our volunteer staff, including ...
Marshals:
  • Campbell Ritchie
  • Jeanne Boyarsky
  • Ron McLeod
  • Paul Clapham
  • Liutauras Vilda
Sheriffs:
  • paul wheaton
  • Rob Spoor
  • Devaka Cooray
Saloon Keepers:
  • Stephan van Hulst
  • Tim Holloway
  • Carey Brown
  • Frits Walraven
  • Tim Moores
Bartenders:
  • Mikalai Zaikin

HttpURLConnection for multiple CURL commands

 
Greenhorn
Posts: 4
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
I am trying to execute a set of 200 curl commands on a remote host using Java. The jar file is deployed on the host and it is run on the remote host.
I am using the following code to do so.



The problem that I am facing is that some of the curl commands when executed, randomly show an error like "java.net.NoRouteToHostException: No route to host (Host unreachable)". I am able to ping the host and access the web interface and it seems to be working fine. There are a bunch of curl commands that I am executing. (About 200). Some of them work perfectly fine. Randomly, some command fails with the exception "java.net.NoRouteToHostException: No route to host (Host unreachable)". Retries may or may not work. The curl commands are spit out on a log.  When I notice that a curl command has failed, I copy the curl command and try to manually execute it. It works fine without any error even from another terminal on the same host.

One of the things I noticed was that increasing the wait time increased the chances of a retry working. It is like the garbage collector gets more time to free resources and hence it can execute the command without any errors on a retry. The jsonData that is passed to the function is formed using a complex SQL query and takes a few seconds to process. But all of the resources seem to have been freed after the jsonData is fetched.

I feel like this is a Java problem. Glad for any help in figuring out what is going on.

Thanks!
 
Marshal
Posts: 4499
572
VSCode Eclipse IDE TypeScript Redhat MicroProfile Quarkus Java Linux
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Aditi Deshpande wrote:I am trying to execute a set of 200 curl commands on a remote host using Java.


Are you doing the 200 operations concurrently or sequentially?

Have you tried snooping the network traffic using something like  Wireshark or tcpdump to see what is happening?
 
Aditi Deshpande
Greenhorn
Posts: 4
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
The operations are sequential. A for loop is used to call the function. They are not dependent on each other.
 
Ron McLeod
Marshal
Posts: 4499
572
VSCode Eclipse IDE TypeScript Redhat MicroProfile Quarkus Java Linux
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
I think you should be looking at your deployment platform's network connection or possibly the server you are trying to work with.

I tried 200 operations with your code and didn't hit any errors.  I was testing against a test server at httpbin.
 
reply
    Bookmark Topic Watch Topic
  • New Topic