aspose file tools*
The moose likes Java in General and the fly likes Problem to read output from running WGET command Big Moose Saloon
  Search | Java FAQ | Recent Topics | Flagged Topics | Hot Topics | Zero Replies
Register / Login


Win a copy of Soft Skills this week in the Jobs Discussion forum!
JavaRanch » Java Forums » Java » Java in General
Bookmark "Problem to read output from running WGET command" Watch "Problem to read output from running WGET command" New topic
Author

Problem to read output from running WGET command

Susan Smith
Ranch Hand

Joined: Oct 13, 2007
Posts: 224
Hello,

I'm trying to post a file to a URL using WGET. The server will return me "Status=1..(and some other information)" if the operation is successful.

I tried to read any output from the attempted command but the "response" variable below is empty when I print it using LOGGER.info.
I've done this below for "unrar" command and it works just fine. Could someone please advise?



Thanks in advance for all the help.
Paul Clapham
Bartender

Joined: Oct 14, 2005
Posts: 18986
    
    8

You're assuming you get something written to stdout. Maybe there's something wrong with your command and you only get stderr output. You should read When Runtime.exec() won't.
Susan Smith
Ranch Hand

Joined: Oct 13, 2007
Posts: 224
I added these lines but the "error" variable is empty.

Jim Yingst
Wanderer
Sheriff

Joined: Jan 30, 2000
Posts: 18671
Doesn't the -q option in the command indicate quiet mode? There's no output because you've told it to be quiet.

Incidentally it's generally good practice to read the stdio and stderr in separate threads. Otherwise it's possible for one to fill a buffer and block indefinitely while you're reading the other one. Alternately, you can use a ProcessBuilder from JDK 5+ and call redirectErrorStream(true) to simply send both error and standard out to the same place. ProcessBuilder has several other methods that make it more convenient than Runtime.exec().

Also, you may want to check the return value of Process.waitFor() or Process.exitValue(). In some cases that may contain a meaningful value, depending on the command.
[ February 15, 2008: Message edited by: Jim Yingst ]

"I'm not back." - Bill Harding, Twister
Susan Smith
Ranch Hand

Joined: Oct 13, 2007
Posts: 224
I removed the "-q" option and it shows the error output.
Before, I had the "-q" and I didn't try to print out the error output.

I'm not sure whether it's still ok to ask in this same posting or not: The error that I got is "Unsupported Scheme". But when I run it from LINUX command line, it's working okay. Could someone please advise?
Jim Yingst
Wanderer
Sheriff

Joined: Jan 30, 2000
Posts: 18671
Hm, suppressing error output sounds a bit overly aggressive for -q, for my taste. Oh well, glad at least one hurdle has been removed.

Asking a followup question in the same thread is fine, generally. Though it can depend on how closely related the new question is to the oringal one (and to the subject line). So far, we seem to be in the same ballpark. If the topic wanders too far off, you may be better off starting a fresh thread. It's not too big a deal either way.

What is the URL that you get this error on? The scheme is the first part of the URL, e.g. the "http:" or "mailto:" or whatever. So knowing what sort of URL it's complaining about may be helpful.

As for why you get different output from the command line, well the environment for a fresh Process from exec() may be a bit different than what you get from logging into a bash shell or something. Depends how your system is set up (and I'm harly a guru in that area; I just muddle through).

Try executing "which wget" from both exec() and the command line. Do you get the same result in both cases? If not, one simple solution may be to take the full path that your command line uses, and use that full path in the exec() command.

Simlarly, you can try running "echo $SHELL". Or "set" (or "setenv") to get a list of current environment variables. Likewise "alias" will list the currently-set aliases. If you discover substantial differences in these areas (and if my quick fix from the last paragraph didn't take), then we may be better off moving this to the unix forum for more expert help. Or maybe some unix gurus will catch the thread here; it's a polular forum after all.
Anand Hariharan
Rancher

Joined: Aug 22, 2006
Posts: 258

Originally posted by Jim Yingst:
Hm, suppressing error output sounds a bit overly aggressive for -q, for my taste.


From the output of 'wget --help':


-q, --quiet quiet (no output).
-v, --verbose be verbose (this is the default).
-nv, --non-verbose turn off verboseness, without being quiet.


and from wget's documentation:


-nv
--non-verbose

Non-verbose output--turn off verbose without being completely quiet (use -q for that), which means that error messages and basic information still get printed.



Originally posted by Jim Yingst:

<snip good advice regarding topicality and where-to-post>

Simlarly, you can try running "echo $SHELL". Or "set" (or "setenv") to get a list of current environment variables. Likewise "alias" will list the currently-set aliases.


I tried to find if there was anything grouped together in the wget's documentation regarding what environment variables could influence its behaviour, and the word "environment" was scattered at a number of places.

An incomplete list of variables that influence wget are 'WGETRC', 'http_proxy', 'ftp_proxy', and 'https_proxy'.

In general though, if you do not want your wget's behaviour to be influenced by the environment, make sure your defaults are set either in the command line or one of the wgetrc files.

What I would suggest is that you run your wget both within java and in the shell with verbose options and see if you get a clue as to what is going wrong based on the differences between them.

- Anand

[Edit: Fixed closing 'quote' tag]
[ February 16, 2008: Message edited by: Anand Hariharan ]

"Perfection is achieved, not when there is nothing more to add, but when there is nothing left to take away." -- Antoine de Saint-Exupery
Jim Yingst
Wanderer
Sheriff

Joined: Jan 30, 2000
Posts: 18671
Regarding the -q option, I don't disagree that that's how the wget command is actually implemented. I was just saying that's not what I would have done, which is what I would have expected without reading more closely. I often want a command to not distract me with any output at all, unless there's a problem - in which case of course I want to see the error message. Well, I suppose the -nv option may be OK there. If not I can always do something like > /dev/null, losing all standard out while not affecting error messages.
Susan Smith
Ranch Hand

Joined: Oct 13, 2007
Posts: 224
Answering Jim Yingst's suggestions:

(1.) Result of executing 'which wget'

From command line:
/usr/bin/wget
From exec():
/usr/bin/wget

(2.) Result of executing 'setenv'

From command line:
-bash: setenv: command not found

From exec():
java.io.IOException: setenv: not found

(3.) Result of executing 'set'
From command line:
--> It gives me a long list of variables along with its values

From exec():
java.io.IOException: set: not found

(4.) Result of executing 'alias'

From command line:
alias l.='ls -d .* --color=tty'
alias ll='ls -l --color=tty'
alias ls='ls --color=tty'
alias vi='vim'
alias which='alias | /usr/bin/which --tty-only --read-alias --show-dot --show-tilde'

From exec():
java.io.IOException: alias: not found

(5.) Result of executing 'echo $SHELL'

From command line:
/bin/bash

From exec():
$SHELL

(6.)
>>>>>>>>
The scheme is the first part of the URL, e.g. the "http:" or "mailto:" or whatever. So knowing what sort of URL it's complaining about may be helpful.
>>>>>>>>
It's http://
[ February 18, 2008: Message edited by: Susan Smith ]
Susan Smith
Ranch Hand

Joined: Oct 13, 2007
Posts: 224
Answering Anand Hariharan's suggestion:

(1.)
>>>>>>>>>>>>>>>
What I would suggest is that you run your wget both within java and in the shell with verbose options and see if you get a clue as to what is going wrong based on the differences between them.
>>>>>>>>>>>>>>>

From command line:
--> It gives me a bunch of output. And I see some values as I expected.

From exec():
--> 'http://.....': Unsupported scheme

[ February 18, 2008: Message edited by: Susan Smith ]
[ February 18, 2008: Message edited by: Susan Smith ]
Anand Hariharan
Rancher

Joined: Aug 22, 2006
Posts: 258

Originally posted by Susan Smith:
Answering Jim Yingst's suggestions:

(1.) Result of executing 'which wget'

From command line:
/usr/bin/wget
From exec():
/usr/bin/wget

(...)

(3.) Result of executing 'set'
From command line:
--> It gives me a long list of variables along with its values

From exec():
java.io.IOException: set: not found

(...)

(5.) Result of executing 'echo $SHELL'

From command line:
/bin/bash

From exec():
$SHELL
[ February 18, 2008: Message edited by: Susan Smith ]


Invoke wget from within your java code as -



If your URL contains characters that are special to the shell, make sure you quote them.

Let us know how this pans out. If it still does not work, you may want to continue this in the other Linux/UNIX forum.

good luck,
- Anand

[Edit: Fixed over-zealous tagging by the forum software.]
[ February 19, 2008: Message edited by: Anand Hariharan ]
Jesper de Jong
Java Cowboy
Saloon Keeper

Joined: Aug 16, 2005
Posts: 14423
    
  23

What are you exactly trying to do by executing wget from your Java program - are you trying to read a web page from some URL?

It is not so difficult to do this from Java, using classes in the java.net package. The advantage of doing it in pure Java is that it will work on all operating systems; if you use wget, it will only work on operating systems where that command is available.

See this list for examples - especially look at Getting Text from a URL.

*edit*: Sorry for not reading your question very well. I see that you want to post data. Have a look at this example: Sending a POST Request Using a URL

For more sophisticated file uploading via HTTP, you could use the Apache Commons FileUpload open source library.
[ February 20, 2008: Message edited by: Jesper Young ]

Java Beginners FAQ - JavaRanch SCJP FAQ - The Java Tutorial - Java SE 8 API documentation
Susan Smith
Ranch Hand

Joined: Oct 13, 2007
Posts: 224
Jasper, I already tried using your approach but it doesn't work. I'm working on with the engineer in the company that provides the service but we're not sure why. That's why I'm trying wget.

Anyway, I found the solution by accident. I'm still not sure why but I removed the single quote before and after the URL when I do it in my JAVA code then it's ok.
wget -q -O- --post-file=generatedFile URL
Daniel Jurado
Greenhorn

Joined: Apr 28, 2009
Posts: 8
Try this:

Rob Spoor
Sheriff

Joined: Oct 27, 2005
Posts: 19781
    
  20

Try not waking the zombies. This thread was sleeping nicely for over a year, and now you've kicked it and it's awake.


SCJP 1.4 - SCJP 6 - SCWCD 5 - OCEEJBD 6
How To Ask Questions How To Answer Questions
Daniel Jurado
Greenhorn

Joined: Apr 28, 2009
Posts: 8
Sorry,

was on forum's first page and not solved.
 
 
subject: Problem to read output from running WGET command