Character Encoding

[dhoni Gibson]

Hi
I had problem in converting blob object (which was stored as zip) to string. since i used different encoding format while reading from the database vs encoding used by database to convert the data to blob object.

below code creates problem if i use different encoding

final int DATA_BLOCK_SIZE = 2048;
			
while ((byteCount = sourceStream.read(data, 0, DATA_BLOCK_SIZE)) != -1) {
       targetStream.write(data, 0, byteCount);
}

i fix the issue by replacing the above code with the below one

BufferedInputStream bis = new BufferedInputStream(zis);
while((avail = bis.available() ) > 0) {
	byte[] b = new byte[avail];
	bis.read(b);
	bos.write(b);
}

so please advice me whether it is correct approach to fix the problem in case of reading the file in different encoding . i feel there is no much difference between code, but second code fix my problem (educate me how does the second code fix the issue).


i appreciate you reply on this
Thanks
Dhoni

[Paul Clapham]

dhoni Gibson wrote:I had problem in converting blob object (which was stored as zip) to string.

If the BLOB data was originally a ZIP archive, then it doesn't make sense to convert it to a string. It isn't text data so it shouldn't be treated as a string.

However it's impossible to answer your question specifically about character encoding in your code, because the fragments you posted don't show any evidence of character encodings being used. Could you perhaps show us where you think you are using a character encoding?

[dhoni Gibson]

hi paul

appreciate your apply on this.

1) i want to convert the zip data (byte) into the character later i will form the string object.

2) i was passing the encoding as jvm argument as below.

-Dfile.encoding=UTF8

hope i answered your question


Thanks
Dhoni

[Paul Clapham]

No, you didn't answer my question.

Character encodings are only used when bytes are converted to chars, or vice versa. Your code doesn't have any such examples. And BLOBs are binary objects, so producing a BLOB from bytes doesn't involve any character encoding.

[dhoni Gibson]

i got it. i am pasting the code with the byte to char conversion code here.


Old code here

		ZipEntry ze = null;

		while ( ( ze = zis.getNextEntry() ) != null )
		{
			StringBuffer sb = new StringBuffer();
			String fileName = null;
			if( ze != null )
			{
				int i = 0; byte[] temp = new byte[2048];
				while((i=zis.read(temp,0,2048)) != -1)
				{
					sb.append(new String(temp,0,i));
				}
				fileName = ze.getName();
				zis.closeEntry();

New code with byte to char conversion

	ZipInputStream zis = new ZipInputStream(new ByteArrayInputStream(zipin));
	ZipEntry ze = null;
	while ((ze = zis.getNextEntry()) != null) {
		byte[] allbytes = null;
		String fileName = null;
		if (ze != null) {
			ByteArrayOutputStream bos = new ByteArrayOutputStream();
			BufferedInputStream bis = new BufferedInputStream(zis);
			int avail = 0;
			while((avail = bis.available() ) > 0) {
				byte[] b = new byte[avail];
				bis.read(b);
				bos.write(b);
			}
			
			allbytes = bos.toByteArray();
			bos.close();
			fileName = ze.getName();
			zis.closeEntry();

once i got all the bytes.. i will convert the bytes into the text data in the new code

[Greg Charles]

I'm not sure if this affecting you, but Java had a problem with foreign characters in ZipEntry. Apparently, that's fixed in Java 7, but if you're using an earlier JDK, that could be causing you problems. http://bugs.sun.com/view_bug.do?bug_id=4244499

[dhoni Gibson]

Hi Greg

Thanks for your post. i read through the link and learned issue was between two different platforms while encoding characaters.my issue is bit different, I am running my application in unix environment where i had issue while reading through zip file. i happen where my zip data is persisited in db and while reading through the java code i couldnt recreate the data inside the data.

i have fixed the issue which i have mentioned in the previous post.

i want to know whether i need to use same encoding format which my database is using.

i am using oracle database which uses US7ASCII as encoding . where as my jvm uses US-ASCII.

[Greg Charles]

No, I don't think it matters. Blobs are binary data and therefore not encoded. You might have to match encodings between whatever produced the zip file entry names and whatever is consuming them, but I can't see how Oracle's preferred encoding would make any difference.