why CharConversionException only happens at the Linux environment?

[Caly LeeAnn]

I have a thread which reads a file, parse the data line by line, and write the field values to the database.

There are some German characters in the file.

The thread runs on JBoss application server.

When the file resides at Windows file system and JBoss also runs on Windows OS, I can see the German characters on a web page after I do a search against the database. But when the file resides at Linux file system and JBoss runs on Linux OS, the follwoing exception is thrown.

How can I fix this problem?

Any advices will help. Thank you.


12:25:38,979 ERROR [STDERR] java.io.CharConversionException: Not an ISO 8859-1 character: ���
12:25:38,980 ERROR [STDERR] at javax.servlet.ServletOutputStream.print(ServletOutputStream.java:133)
12:25:38,980 ERROR [STDERR] at javax.servlet.ServletOutputStream.println(ServletOutputStream.java:286)
12:25:38,980 ERROR [STDERR] at com.liq.contacts.COCompanyListServlet.doPost(Unknown Source)
12:25:38,980 ERROR [STDERR] at javax.servlet.http.HttpServlet.service(HttpServlet.java:760)
12:25:38,980 ERROR [STDERR] at javax.servlet.http.HttpServlet.service(HttpServlet.java:853)
12:25:38,980 ERROR [STDERR] at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:366)
12:25:38,980 ERROR [STDERR] at org.mortbay.jetty.servlet.WebApplicationHandler$Chain.doFilter(WebApplicationHandler.java:332)
12:25:38,980 ERROR [STDERR] at com.liq.diagnostics.DiagnosticsFilter.doFilter(Unknown Source)
12:25:38,980 ERROR [STDERR] at org.mortbay.jetty.servlet.WebApplicationHandler$Chain.doFilter(WebApplicationHandler.java:324)
12:25:38,980 ERROR [STDERR] at com.liq.login.MultiLoginFilter.doFilter(Unknown Source)
12:25:38,980 ERROR [STDERR] at org.mortbay.jetty.servlet.WebApplicationHandler$Chain.doFilter(WebApplicationHandler.java:324)
12:25:38,980 ERROR [STDERR] at com.liq.login.WelcomePageFilter.doFilter(Unknown Source)
12:25:38,980 ERROR [STDERR] at org.mortbay.jetty.servlet.WebApplicationHandler$Chain.doFilter(WebApplicationHandler.java:324)
12:25:38,980 ERROR [STDERR] at org.mortbay.jetty.servlet.WebApplicationHandler.dispatch(WebApplicationHandler.java:276)
12:25:38,980 ERROR [STDERR] at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:577)
12:25:38,980 ERROR [STDERR] at org.mortbay.http.HttpContext.handle(HttpContext.java:1674)
12:25:38,980 ERROR [STDERR] at org.mortbay.jetty.servlet.WebApplicationContext.handle(WebApplicationContext.java:544)
12:25:38,980 ERROR [STDERR] at org.mortbay.http.HttpContext.handle(HttpContext.java:1624)
12:25:38,981 ERROR [STDERR] at org.mortbay.http.HttpServer.service(HttpServer.java:875)
12:25:38,981 ERROR [STDERR] at org.jboss.jetty.Jetty.service(Jetty.java:541)
12:25:38,981 ERROR [STDERR] at org.mortbay.http.HttpConnection.service(HttpConnection.java:785)
12:25:38,981 ERROR [STDERR] at org.mortbay.http.HttpConnection.handleNext(HttpConnection.java:935)
12:25:38,981 ERROR [STDERR] at org.mortbay.http.HttpConnection.handle(HttpConnection.java:802)
12:25:38,981 ERROR [STDERR] at org.mortbay.http.SocketListener.handleConnection(SocketListener.java:200)
12:25:38,981 ERROR [STDERR] at org.mortbay.util.ThreadedServer.handle(ThreadedServer.java:294)
12:25:38,981 ERROR [STDERR] at org.mortbay.util.ThreadPool$JobRunner.run(ThreadPool.java:743)
12:25:38,981 ERROR [STDERR] at java.lang.Thread.run(Thread.java:534)

[Edwin Dalorzo]

You seem to be using ISO-8859-1 as the default encoding. This enconding is also known as Latin Alphabet N� 1 and may not contain the characters that you intend to represent.

Is there any possibility that your JBoss configured for the Windows environment is using a differnt enconding than that used by your JBoss in the Linux environment and that is why the error is never happening in Windows?

[Caly LeeAnn]

Thank you, Edwin.

Can you tell me where I can check what encoding the JBoss is using? Thanks again.

[Edwin Dalorzo]

Well, I am not an expert on JBoss, but you can find out what encoding you JVM is currently using by means of evaluating the value of the file.enconding property or using java.nio.charset.Charset class.

System.out.println(java.nio.charset.Charset.defaultCharset());
System.out.println(System.getProperty("file.encoding"));

They may produce different results since the enconding may have different names in IO and NIO. (See Supported Encodings for further details).

Now, in a typical application you could set the default encoding at JVM startup by means of setting the proper variable:

-Dfile.encoding="UTF-8"

There must be some kind of configuration file for JBoss that lets you set the default encoding.
[ January 16, 2007: Message edited by: Edwin Dalorzo ]

[Paul Clapham]

When you read this file, are you specifying its encoding (via an InputStreamReader) or are you allowing the Reader to use the default encoding of the system? I ask because Windows and Linux do not have the same default encoding, so non-ASCII characters would cause the problem you describe.