This week's giveaway is in the Android forum.
We're giving away four copies of Android Security Essentials Live Lessons and have Godfrey Nolan on-line!
See this thread for details.
The moose likes I/O and Streams and the fly likes Creating a nonblocking stream Big Moose Saloon
  Search | Java FAQ | Recent Topics | Flagged Topics | Hot Topics | Zero Replies
Register / Login


Win a copy of Android Security Essentials Live Lessons this week in the Android forum!
JavaRanch » Java Forums » Java » I/O and Streams
Bookmark "Creating a nonblocking stream" Watch "Creating a nonblocking stream" New topic
Author

Creating a nonblocking stream

Michael Morris
Ranch Hand

Joined: Jan 30, 2002
Posts: 3451
I'm stuggling to learn the new nio package and have a question about the nonblocking feature.
Can I use a Pipe to make an InputSream or OutputStream nonblocking?
I can create a ReadableByteChannel or WritableByteChannel with Channels.newChannel() but neither is an InterruptibleChannel. So can I create a Pipe with Pipe.open(), put the sink and source into nonblocking mode, register them with a Selector, write bytes into the sink when they come in from the ReadableByteChannel and finally write the bytes back to the WritableByteChannel from the source as it is notified by the Selector?
Thanks in advance,
Michael Morris


Any intelligent fool can make things bigger, more complex, and more violent. It takes a touch of genius - and a lot of courage - to move in the opposite direction. - Ernst F. Schumacher
Jim Yingst
Wanderer
Sheriff

Joined: Jan 30, 2000
Posts: 18671
Just so you know - I took a look at this, and couldn't seem to find a better way than your proposal - which seems like it will work, but it also seems like there ought to be a better way. So if you or anyone else learns more about this, please share.


"I'm not back." - Bill Harding, Twister
Michael Morris
Ranch Hand

Joined: Jan 30, 2002
Posts: 3451
It would sure be nice if you could dircetly attach streams to the sink and source of a Pipe.
Michael Morris
David Weitzman
Ranch Hand

Joined: Jul 27, 2001
Posts: 1365

[ February 23, 2003: Message edited by: David Weitzman ]
Michael Morris
Ranch Hand

Joined: Jan 30, 2002
Posts: 3451
Hi David,
I can't seem to get this to compile. What packages, besides the obvious ones do I need to import?
Thanks,
Michael Morris
David Weitzman
Ranch Hand

Joined: Jul 27, 2001
Posts: 1365
Originally posted by Michael Morris:
I can't seem to get this to compile. What packages, besides the obvious ones do I need to import?

That's because it isn't a valid class. I had a short comment at the top explaining that and giving a short burst of opinion, but somehow it must have disappeared when I edited the post. Strange.
Anyway, I said that InputStream and OutputStream aren't designed to support non-blocking IO. available() doesn't always work and unless it does you have to start a seperate thread that blocks on read(), thereby defeating the entire purpose of nonblocking IO. If available() does work, you could have a thread sort of like the Swing event thread that just copies bytes around all day long. It would look something like this (note that this code is incomplete):
* refer to code above *
[ February 23, 2003: Message edited by: David Weitzman ]
Michael Morris
Ranch Hand

Joined: Jan 30, 2002
Posts: 3451

That's because it isn't a valid class. ...

That much I figured out.

... thereby defeating the entire purpose of nonblocking IO. ...

But my original idea would simulate (for lack of a better term) a nonblocking stream. Or at least hide its true nature. Right?
Michael Morris
Jim Yingst
Wanderer
Sheriff

Joined: Jan 30, 2000
Posts: 18671
I had a short comment at the top explaining that and giving a short burst of opinion, but somehow it must have disappeared when I edited the post. Strange.
This is probably a UBB bug that seems to show up when you do a reply-with-quote followed by [code] tags. (I think.) At least in certain situations, the system manages to lose some text before the [code] tag. Very annoying. Personally I just avoid ever using reply-with-quote, and hand-edit text my UBB to include whatever previous text I want (which is usually much less than reply-with-quote gives you).
Re: using available() to guard against blocked reads:
I'm not sure what is meant by classes in which available() doesn't work correctly. I do know that if available() behaves according to its API and you use it to prevent blocked reads, you can't actually detect the end of a file. If a file has 50 bytes and they're all available, the first read will read 50 bytes (assuming buffer is large enough). The next time through the loop, available is 0, so read(buffer, 0, 0) will not even attempt to read anything, so it won't return -1. The program just loops forever, never trying to actually read a byte, and thus never seeing the -1. Very annoying. Is it possible this is why you say available() doesn't work? My impression is that it's the API which is broken; I don't know what they were thinking when they wrote it - but I guess they can't redefine available() (or read()) now.
I still need to spend more time with nio though - there may yet be some n alternative for Michael.
[ February 23, 2003: Message edited by: Jim Yingst ]
Michael Morris
Ranch Hand

Joined: Jan 30, 2002
Posts: 3451
Bumping this to the top.
Jim Yingst
Wanderer
Sheriff

Joined: Jan 30, 2000
Posts: 18671
But my original idea would simulate (for lack of a better term) a nonblocking stream. Or at least hide its true nature. Right?
Right. The problem is that it requires a separate thread to write bytes into the sink. It's always been possible to deal with blocking I/O by creating a new thread to manage each stream - the big point of nonblocking I/O is that you can manage a large number of different I/O sources with a single thread (or small number of threads), which often improves scalability for servers. It may still be useful to convert a blocking stream to a nonblocking channel at the cost of one thread, if you're interfacing with other code that expects a nonblocking channel. Just don't expect any great improvements over what you could have achieved if the other code had used only traditional I/O instead.
[ April 15, 2003: Message edited by: Jim Yingst ]
Ron Hitchens
Author
Ranch Hand

Joined: Mar 17, 2003
Posts: 30
Let me take a crack at this, since I'm the author of the NIO book (Java NIO).
NIO introduces non-blocking I/O, and non-blocking usage of stdin (System.in) is something that's often desirable to do. But it's important to remember that NIO operates on channels, not streams. The classic Java stream metaphor never included the notion of non-blocking - that's new with channels.
You can wrap a channel around a stream and then feed it through a pipe as a previous poster described, but you must dedicate a thread per stream to make it work properly. This is not a failing of NIO but rather a limitation of streams. You can't multiplex streams, you can only multiplex channels.
In a case like this you're multiplexing the channel the stream is feeding, not the stream itself. You must dedicate a thread to service each stream.
The most likely scenario is multiplexing a large number of socket connections, which should be instances of SocketChannel. Spinning off one thread to watch stdio is not too much of a penalty.
I have some sample code which encapsulates System.in as a selectable channel on the Java NIO site. Look at SystemInPipe.java and TestStdin.java


Author of <a href="http://www.amazon.com/exec/obidos/ASIN/0596002882/ref=jranch-20" target="_blank" rel="nofollow">Java NIO</a> (<a href="http://javanio.info" target="_blank" rel="nofollow">javanio.info</a>)
Michael Morris
Ranch Hand

Joined: Jan 30, 2002
Posts: 3451
Thanks Ron, I'll certainly take a look at the code. As I stated at the beginning of this thread, I'm struggling to learn NIO. By the way, welcome to JavaRanch.
Pho Tek
Ranch Hand

Joined: Nov 05, 2000
Posts: 761

Ron,
but you must dedicate a thread per stream to make it work properly. This is not a failing of NIO but rather a limitation of streams. You can't multiplex streams, you can only multiplex channels.

So, the fact that you can't multiplex streams is because streams block and NIO Channels decorate the streams with non-blocking features in addition to the ability to interrogate a stream whether its ready for read/write ?
Cheers,
Pho


Regards,

Pho
Jim Yingst
Wanderer
Sheriff

Joined: Jan 30, 2000
Posts: 18671
I wouldn't refer to channels as decorators. To me, that would imply that they access an underlying stream through Java's stream APIs - and as discussed here, that requires a new thread to avoid blocking. Instead, channels are an alternate API which has more direct access to native code, allowing more powerful and varied capabilities, such as non-blocking.
[ April 15, 2003: Message edited by: Jim Yingst ]
Michael Morris
Ranch Hand

Joined: Jan 30, 2002
Posts: 3451
Adding to what Jim said, this thread is about taking an inherently blocking stream and wrapping it in a layer of abstraction that makes it appear to be non-blocking. As I understand it InterruptibleChannels are truly non-blocking at (or near) the hardware level.
Ron Hitchens
Author
Ranch Hand

Joined: Mar 17, 2003
Posts: 30
As Jim points out, there is no relationship between streams and channels. They are completely distinct, a channel is not a type of stream nor is a stream a type of channel.
This is not the sort of decoration done within the java.io package where, for example, a BufferedReader adds functionality to a Reader.
What we're talking about here is more of an adapter. A stream object is accessed by one side of the adapter which presents a channel API to its clients.
The non-blocking capability provided by the Pipe and its dedicated thread could be considered a decorator of the adapter. It takes a simple channel (the adapted stream) and decorates it with a pipeline to present a non-blocking channel to its clients.
The thread is there to transfer data from the simple, blocking channel to the sink end of the Pipe. This allows the source end of the Pipe to present a ready indication when new data arrives. This essentially simulates what happens in the low-level implementations of SocketChannels when the socket data arrives.
[ April 16, 2003: Message edited by: Ron Hitchens ]
alzamabar
Ranch Hand

Joined: Jul 24, 2002
Posts: 379
Sorry about the question, but I looked on the internet and didn't find anything suitable, and I'm not a native English speaker, although I live in London...I'm studying Java NIO and would like to know what 'To multiplex' means, referring to non-blocking features of channels. What does it mean:'...This ability permits a process to multiplex many active streams...'?

Thanks for any help,


Marco Tedone<br />SCJP1.4,SCJP5,SCBCD,SCWCD
 
I agree. Here's the link: http://aspose.com/file-tools
 
subject: Creating a nonblocking stream
 
Similar Threads
pipe, Channel
Convert OutputStream to InputStream!
does a nio socket buffer data internally too?
FileChannel and thread safety
Converting outputstream to inputstream