Reader -> InputStream conversion

[Rob Spoor]

InputStreamReader can convert an InputStream into a Reader, and OutputStreamWriter can convert an OutputStream into a Writer. But why are there no classes for converting the other way?

Suppose you want to compress the data read from a Reader using GZIP compression. Now there is the GZIPInputStream, but that can only take an InputStream, not a Reader. There is also not a default class like ReaderInputStream.

Now I've looked at the source of InputStreamReader but that uses sun.nio.cs.StreamDecoder to do all of the actual work. Since I don't want to use any of the Sun internal classes, I'm kinda stuck.

Anyone have an idea?

[Brian Cole]

Originally posted by Rob Prime:
InputStreamReader can convert an InputStream into a Reader, and OutputStreamWriter can convert an OutputStream into a Writer. But why are there no classes for converting the other way?

Suppose you want to compress the data read from a Reader using GZIP compression. Now there is the GZIPInputStream, but that can only take an InputStream, not a Reader. There is also not a default class like ReaderInputStream.

Now I've looked at the source of InputStreamReader but that uses sun.nio.cs.StreamDecoder to do all of the actual work. Since I don't want to use any of the Sun internal classes, I'm kinda stuck.

Anyone have an idea?

Well, do you know how you want to convert the 16-bit chars returned by the reader into 8-bit bytes?

Deciding on that is the hard part, but you shouldn't have to use any sun internal classes. possibilities:

strip off the highest 8 bits: just cast from char to byte

convert each char to two bytes (you'll have to choose which endian)

use a specific Charset: use CharsetEncoder or String.getBytes()

Code is out there if you look. For example, this one seems to use String.getBytes(charsetName).

For the GZIP situation you mention, it would be best to just read the data with an InputStream to begin with.
[ November 13, 2007: Message edited by: Brian Cole ]

[Rob Spoor]

Originally posted by Brian Cole:
Well, do you know how you want to convert the 16-bit chars returned by the reader into 8-bit bytes?

Deciding on that is the hard part, but you shouldn't have to use any sun internal classes. possibilities:

strip off the highest 8 bits: just cast from char to byte

convert each char to two bytes (you'll have to choose which endian)

use a specific Charset: use CharsetEncoder or String.getBytes()

Code is out there if you look. For example, this one seems to use String.getBytes(charsetName).

Well that was why I posted this here, since this is not a simple issue; some characters can be represented using a single byte, while some others need two bytes. Therefore, options 1 and 2 are not good. As for CharsetEncoder, this is an abstract class without any known subclasses, and since I'm not that good at encoding I can't really write a subclass myself.

I also though about using Strings, but didn't want to use that yet because it would require a lot of new String objects. I've checked the code of the ReaderInputStream on Koders.com and guess that's the way to go.

For the GZIP situation you mention, it would be best to just read the data with an InputStream to begin with.

Well if I could I certainly would have. The problem is, sometimes you can't use an InputStream because some other API only returns a Reader.

I guess I'll base my code on the Koders.com example. Thanks for the link.
[ November 14, 2007: Message edited by: Rob Prime ]

[Jim Yingst]

I expect the easiest thing to do here would be to simply read from the Reader and write to a file or ByteArrayOutputStream, then when you're done, read from the file or array. You can use an OutputStreamWriter to control the encoding used. This may not work for you if the stream is very large - a byte array may take too much memory, and writing and reading from a file may be slower than you want. Also for some applications you may want to be able to start reading before all the content has gone through the stream. In that case, the Koders code (actually from Apache, it seems) looks like a decent bet, and there are other examples out there using the same name.

[Rob]: As for CharsetEncoder, this is an abstract class without any known subclasses, and since I'm not that good at encoding I can't really write a subclass myself.

Not necessary. Use

CharsetEncoder cse = Charset.forName(enc).newEncoder();

With this, you can probably improve on getBytes() since you won't have to create as many objects.

[Rob Spoor]

Originally posted by Jim Yingst:
Not necessary. Use

CharsetEncoder cse = Charset.forName(enc).newEncoder();

With this, you can probably improve on getBytes() since you won't have to create as many objects.

Ah thanks, I always hate it when the creating methods are somewhere else

I really need to press that "Use" link some more.

[Jim Yingst]

Aha - I found my own old code for this. I don't know if this is the best way to do it, but I wanted to try using piped streams for once. Worth a shot at least. The exception handling should probably be replaced with logging, whatever you use. Rethrowing as an unchecked exception is no good if you're in a separate thread; it will just go to the uncaught handler and print a stack trace.

import java.io.*;
 
public class ReaderInputStream extends InputStream {
    private final Reader reader;
    private final Writer writer;
    private final PipedInputStream inPipe;
 
    public ReaderInputStream(Reader reader) throws IOException {
        this(reader, null);
    }
 
    public ReaderInputStream(final Reader reader, String encoding) throws IOException {
        synchronized (this) {
            this.reader = reader;
            inPipe = new PipedInputStream();
            OutputStream outPipe = new PipedOutputStream(inPipe);
            writer = (encoding == null) ? new OutputStreamWriter(outPipe) : new OutputStreamWriter(outPipe, encoding);
        }
        new Thread(new Copier()).start();
    }
 
    public int read() throws IOException {
        return inPipe.read();
    }
 
    public int read(byte b[]) throws IOException {
        return inPipe.read(b);
    }
 
    public int read(byte b[], int off, int len) throws IOException {
        return inPipe.read(b, off, len);
    }
 
    public long skip(long n) throws IOException {
        return inPipe.skip(n);
    }
 
    public int available() throws IOException {
        return inPipe.available();
    }
 
    public synchronized void close() throws IOException {
        close(reader);
        close(writer);
        close(inPipe);
    }
 
    private static void close(Closeable cl) {
        try {
            cl.close();
        } catch (IOException e) {
            e.printStackTrace();
        }
    }
 
    private class Copier implements Runnable {
        public void run() {
            char[] buffer = new char[8192];
            try {
                while (true) {
                    int n;
                    synchronized (ReaderInputStream.this) {
                        n = reader.read(buffer);
                    }
                    if (n == -1)
                        break;
                    synchronized (ReaderInputStream.this) {
                        writer.write(buffer, 0, n);
                        writer.flush();
                    }
                }
            } catch (IOException e) {
                e.printStackTrace();
            } finally {
                close(reader);
                close(writer);
            }
        }
    }
 
    public static void main(String[] args) throws IOException {
        Reader r = new StringReader("The quick brown fox jumped over the lazy dog");
        InputStream in = new ReaderInputStream(r);
        Reader r2 = new InputStreamReader(in);
        for (int c; (c = r2.read()) != -1;)
            System.out.write(c);
        System.out.println();
    }
}

[ November 14, 2007: Message edited by: Jim Yingst ]