Win a copy of Escape Velocity: Better Metrics for Agile Teams this week in the Agile and Other Processes forum!

B Mayes

Ranch Hand
+ Follow
since Apr 28, 2010
B likes ...
Android Eclipse IDE Ubuntu
Cows and Likes
Cows
Total received
0
In last 30 days
0
Total given
0
Likes
Total received
0
Received in last 30 days
0
Total given
1
Given in last 30 days
0
Forums and Threads
Scavenger Hunt
expand Ranch Hand Scavenger Hunt
expand Greenhorn Scavenger Hunt

Recent posts by B Mayes

Steve Luke wrote:Hmm... So the returned text you expect is pretty minimal. It could be that the output buffer Windows uses is not being flushed before you call the stop() method because it never gets filled.


I think you mean the output buffer in Linux instead. I'm pretty sure the uname command doesn't work on Windows. ;)


Steve Luke wrote:
So this could be a situation where my preference to control the stop versus letting the process die (and join the gobblers) is not good.


It appears so. I guess I was right about joining the gobblers after all...at least I was right about something. I modified it to join the gobblers and presto it works. I ran it a few times and got 0 failures. So then I stuck it in a big bash loop and ran it 100 times:



$ for i in $(seq 1 100)
> do
> java SSCCE
> done
Counted 0 time(s) where the output was empty
Counted 0 time(s) where the output was empty
Counted 0 time(s) where the output was empty
Counted 0 time(s) where the output was empty
Counted 0 time(s) where the output was empty
...




You get the idea. IT WORKS!! Hopefully testing it 100k times and getting output in every single one is sufficient. I'm convinced at this point anyway.


Just FYI:

Steve Luke wrote:
/* completely untested example... maybe add a timeout, and better return or error handling or something... */


While I didn't show it for this SSCCE, I actually did wrap my process execution with a TimerTask as suggested here:
http://kylecartmell.com/?p=9


He also discusses Sun bug 6420270 so I call Thread.interrupted() to clear the interrupt flag as suggested. This bug isn't marked as fixed until Java 7 and most of our customers are on Java 6. I actually discovered one of our clients is still running Java 5...

Back to the timer though. My actual code has a default wait time of 5 seconds, and then the method is overloaded to have a single argument (the command), or 2 arguments (command and a timeout). If the timeout is null it simply falls back on the default. Yes, I use an int rather than long, but I cannot imagine anyone wanting to run a process longer than Integer.MAX_INT seconds anyway. That equates to like 596 hours or roughly 25 days! In any case, if the external process isn't done after the timeout expires then we simply interrupt the thread so that the darn thing doesn't block forever (perhaps due to an infinite loop). So here is the "complete" version of executeProcess():





Your input has been incredibly useful here and I really appreciate it. If we happen to meet up on the street then I owe you a beer, without question.

Steve Luke wrote:Have you checked the error stream output and the return value from the called application to see if the error was internal and not a problem with you Java code?



I'm not 100% sure I follow exactly what you want to look at, but I changed up the main method to print more information when a failing call occurs and also to print a successful call like this:





Here is the output that I got. Note that the return value for all "bad" calls was always zero...


$ java SSCCE
Counted 30 time(s) where the output was empty

[0,,]
[0,,]
[0,,]
[0,,]
[0,,]
[0,,]
[0,,]
[0,,]
[0,,]
[0,,]
[0,,]
[0,,]
[0,,]
[0,,]
[0,,]
[0,,]
[0,,]
[0,,]
[0,,]
[0,,]
[0,,]
[0,,]
[0,,]
[0,,]
[0,,]
[0,,]
[0,,]
[0,,]
[0,,]
[0,,]


Valid calls look like this: [0,x86_64
,]






I'm at home right now and don't really feel like rebooting into an old WinXP partition (plus I need to start getting ready for work and the kids ready for school, etc.). I will try this code on a Win7 VM when I get to the office and change the call to just run a directory listing. For Windows I believe the syntax would be something like this if anybody else wants to run it:

Steve Luke wrote:That's great it is working, thanks for the feedback :-)



Oops...maybe I spoke too soon. Below is a new SSCCE that just runs the uname -m command 1000 times and counts up the number of times the output returned is empty. It happens rather infrequently to me but the problem is that it does still sometimes happen, although during my web app initialization it seems to fail much more frequently than it actually works. In this SSCCE the max number of times it failed after running it a dozen or so times was 4.

Here is what I have. I have stripped out all of the javadoc to condense things, though I did give you and Michael Daconta credit for the StreamGobbler class. ;)

Did I screw something up when I made the tweaks to the class?


Steve,

I finally got a chance to work on this today and it appears to be working great. I tweaked a few things but otherwise I am essentially using it as-is. Code will be committed shortly and will go through some additional testing to verify that it's working but just on my own system it has been working every time. Thank you so much!
<---- Schooled.

Wow, great info.


Steve Luke wrote:
I wouldn't say it is bogus. First, he doesn't use br.ready() he uses br.readLine() != null. That works as long as you assume the gobblers should run until the process kills the streams. That is probably pretty safe, but a forced stop puts it under more control, in my opinion. And lets you close the streams on your end when you are done with them. I personally think a forced stop is cleaner, but but I don't fault the sample code because
1) it is an article with example code, not attempting to be production quality
2) I have never measured or experienced a problem with waiting for the process to close its streams, it just feels wrong to me.


Valid points.


Steve Luke wrote:
There is no real need to join the gobbler threads. Do you really need to block until the gobblers are done? Isn't knowing the process is done good enough? Especially if you can tell the gobbler to end.


I guess my whole idea behind joining the threads was to ensure that the main thread blocks until the spawned threads complete. On the surface I assumed that a successful completion would always indicate that the streams had been fully consumed, but as you have pointed out that may not always be the case when using br.ready(). In this case, the meaning of the word "consumed" is the tricky part.


Steve Luke wrote:
No, they can't run before main, because main thread creates the gobbler threads, so there is a happens-before relationship between code executed in the main thread (prior to start() being called) and code in the worker thread. But the gobbler threads can easily reach the br.ready() check before the process has put data in the streams.


Indeed you're correct. I followed the basic concept though. Thanks for clarifying the finer details.



Now I'm going to go hang in my head in shame for a bit before I go to bed. And maybe google "Steve Luke thread tutorials" as well. ;)


As I said, I will give this a shot tomorrow if I have the time. Right now the only thing that seems to be affected is that the dos2unix/fromdos command isn't being executed on a shell script at system startup. Apparently our clients have been running it manually each time they deploy a new war file for years anyway. I'm just trying to automate the process for them and ran into this snag. Cheers.

Steve Luke wrote:The StreamGobbler uses while(br.ready()) to perform the 'get data from stream' loop. This is not going to work. The br.ready() returns true when there is data in the buffer to be read. When the BufferedReader first starts and before anything is sent to it, the br.ready() will return false - indicating your StreamGobbler should shutdown before it ever started. You can't rely on that. In fact, you can't really rely on anything directly related to the BufferReader to tell you when you need to shutdown.



Man if that is true then the now-classic article "When Runtime.exec() won't" is completely bogus, and thus lots of people should be hosed by this. I'm sure you have read it but here is link for those who haven't:

http://www.javaworld.com/jw-12-2000/jw-1229-traps.html


I think their example is flawed in that it doesn't join the gobbler threads...but I see what you're saying. It's possible those threads could get scheduled and started prior to the "main" thread executing. So even though you call join, if those threads read the input stream before any data was written there, they would simply exit and calling join is quite ineffective at that point.

Thanks for the replies...I'll take a look and try some things out tomorrow if I get the chance.
I'm wrapping Runtime.exec() with a utility function that I created to execute an external process. I have read many articles on the pitfalls of Runtime.exec() and thought I had everything working, but I just recently discovered that while invoking external processes seems to work almost everywhere in my code, it's still failing in one spot. We basically have a C program that gets invoked in some situations, and on Unix/Linux the call is wrapped with a shell script. When our web app starts, it attempts to locate either the "fromdos" command or the "dos2unix" command to make sure that the shell script used has Unix EOL characters (if the file ends with DOS EOL characters then it fails to run properly). It also sets the script to be executable by running chmod +x on it but that part seems to be working.

It appears that my calls to the commands "which dos2unix" and "which fromdos" are not working. If I set a breakpoint and step through the debugger however -- then it returns the appropriate output and it all works great. This suggests that I am doing something wrong with the threads and there is a race condition.

I am using stream gobblers for stdout/stderr, join the threads, and feel like everything should be working. Below is an SSCCE (maybe not quite as small as I can make it) that fails on my system (64-bit Ubuntu 12.10 with Sun Java 1.6.0_26). Any idea why this doesn't work? Is it because which is a built-in or something? I tried changing the calls to use an argument array like but it still doesn't work. Maybe I should just finally move everything to use the ProcessBuilder class instead of Runtime.exec()? Any help is appreciated!

Here is an example of running it on my system:






And here is the entire SSCCE.java file. Try compiling it and you may/may not get the same output that I do above:


I realize this topic is a few months old, but have you seen this?

http://archive.dojotoolkit.org/nightly/dojotoolkit/dojox/gfx/demos/career_test.html


It's a joke of an aptitude test but all funny stuff aside, it actually has a lot of great dojo.gfx examples in it. It actually determines when the circles are dropped onto other objects which is exactly what are you looking for. Just view the page source and see if it helps you or not. Chances are you can use their example to accomplish what you want.
By the way, I went ahead and marked this as solved because the regex I posted above seems to be working great. I committed the change last week and haven't heard anything wrong from test yet. Essentially, I just split on " AND " since clauses from multiple places are always ANDed together, and then I build it back up with a StringBuilder object. The method is quite simple. Here is the basic idea:




An interesting observation is that if the word AND appears inside of a string literal, the method still returns the appropriate information. Note that I am splitting on " AND " which is padded by spaces. So a string like "FOO='BRAND'" will not get split up while a string like "BAR='ROCK AND ROLL!' will get split -- but it will be pieced back together in the exact same manner.

I know everyone thinks I am insane for doing this but there is one case where this is applicable, and the above code is working out great thus far.
11 years ago

Stephan van Hulst wrote:What about Associativity? If you drop part of a WHERE clause, you will have to figure out which ANDs and ORs to drop as well, etc. This will not be a very easy task. Why are you trying to do this?


I agree this isn't the easiest thing in the world, but it is still possible. I don't think figuring out the correct AND is going to be terribly difficult anyway. I'll try to get it working on Monday or Tuesday -- we'll see.

I don't mean to be rude or ungrateful but the motivation behind this seems irrelevant to me. Suffice it to say that it's for something at work. I got a bug request right before a big demo that our PMs/sales guys were doing. We wound up simulating the desired behavior by using a DB trigger on a particular table, but there is one valid case where arguments are not passed from JavaScript to the server (Java). In such a case I would like to just drop the offending portion of the WHERE clause.

Of course, now that I have this task in front of me I feel like I need to conquer it so I'm not going to give up on it...even if we don't ultimately wind up using it at work. Just FYI the following seems to work quite well (though I'll do some more rigorous testing to confirm). It's a slight modification of the above regex by Ninad. Thank you!

11 years ago
Thanks I will try that. I know it doesn't sound like it makes any sense but in this one context of this application it actually does. Just trust me. ;)
11 years ago
I have never been the best at regular expressions, so I'm hoping someone here can help. I have some SQL statements inside of String objects in Java. What I would like to do is drop any part of a WHERE clause that has a parameter in it like this:

SELECT * FROM SOME_TABLE WHERE VALUE=?;


I could check to see if the string contains the character '?' but what if it's a literal inside of a string like this?

SELECT * FROM SOME_TABLE WHERE VALUE='?';


In such a case I don't want to drop the WHERE clause just because it contains ? -- since ? is a character inside of the string. The WHERE clauses here are pretty specific and generally only contain constructs like AND and OR. There's nothing like a WHERE EXISTS and then correlated subqueries or anything. So my idea was to obtain the substring of everything after the word "WHERE", then split on AND and OR. Now with a set of strings to look through, I need to see if any of them contain a parameterized statement. If so, I would like to simply replace that particular portion of the WHERE clause with empty string (i.e. -- get rid of it!).

So any ideas how to match a String with ? that is *not* surrounded by single quotes? Thus, these strings should get matched (or at least the bold parts):

- VALUE=?
- VALUE=? AND OTHER=50


and these should not:

- VALUE='?'
- VALUE='abc?123'

Thanks!


EDIT: I'm thinking that the argument to matches should be something like:



But what if they have single quotes repeated (to escape the quote and use it as part of the string) like 'don''t touch that'? Hmm...
11 years ago
Nevermind. When using the matches method you have to put the string ".*" on both ends of it like this:




I made this mistake already yesterday (before moving everything to compiled patterns for better efficiency). Shame on me!

Thanks again Darryl!
11 years ago
Ok wait...I thought I was done but now I'm noticing some very strange behavior. Both matches called on a string object and called from the Pattern class return false while the Matcher.find() method returns true. I have been staring at this for like an hour now trying things out and I just don't get it. They should all behave the same shouldn't they?!?!!!??

What am I missing here?






Run this and the results are:


curr.matches: false
Pattern.matches: false
pdfFontName.find: true

11 years ago
Beautiful...it works! Thank you.
11 years ago