my dog learned polymorphism*
The moose likes Java in General and the fly likes Problem encoding japanese character Big Moose Saloon
  Search | Java FAQ | Recent Topics | Flagged Topics | Hot Topics | Zero Replies
Register / Login


Win a copy of OCM Java EE 6 Enterprise Architect Exam Guide this week in the OCMJEA forum!
JavaRanch » Java Forums » Java » Java in General
Bookmark "Problem encoding japanese character" Watch "Problem encoding japanese character" New topic
Author

Problem encoding japanese character

Hema Nandhini
Ranch Hand

Joined: Aug 04, 2008
Posts: 31
Hi All,

I have had good solutions for my problems here. I hope to get one for the problem i am facing now.

I have to store japanese data in postgres database. Mine is a web application. I get the values using request.getParameterMap(). When we pass japanese character in request, the value that is retrieved has some junk characters [like è£?ç½®ã?®ã??ã??ã?®è£?ç½®]. I am not sure how to convert this into a proper japanese character to store in database. I searched on net and tried to do something using the below code. But i am getting the output like this

�?置�?��??�??�?��?置




Can someone help me to do this??? Thanks in advance...
James Sabre
Ranch Hand

Joined: Sep 07, 2004
Posts: 781

Hema Nandhini wrote:Hi All,

I have had good solutions for my problems here. I hope to get one for the problem i am facing now.

I have to store japanese data in postgres database. Mine is a web application. I get the values using request.getParameterMap(). When we pass japanese character in request, the value that is retrieved has some junk characters [like è£?ç½®ã?®ã??ã??ã?®è£?ç½®]. I am not sure how to convert this into a proper japanese character to store in database. I searched on net and tried to do something using the below code. But i am getting the output like this

�?置�?��??�??�?��?置




Can someone help me to do this??? Thanks in advance...


Strings in Java are UNICODE encoded as UTF-16 always always ALWAYS. There is no such thing as a UTF-8 String and you should not need to do any conversion at all so your method is pointless. Your JDBC driver should perform any character encoding required.
Shanky Sohar
Ranch Hand

Joined: Mar 17, 2010
Posts: 1051

we can use Sql to do this..but if the charaters are not so long so we can use one of the java hashmap to do this.

E.g.


This is efficiently what a database would do anyways

SCJP6.0,My blog Ranchers from Delhi
James Sabre
Ranch Hand

Joined: Sep 07, 2004
Posts: 781

shanky sohar wrote:we can use Sql to do this..but if the charaters are not so long so we can use one of the java hashmap to do this.


I fail to see the relevance of this in relation to the original problem. The original problem is a false assumption that a String object can be encoded as UTF-8 when it is always UNICODE encoded as UTF-16!
Hema Nandhini
Ranch Hand

Joined: Aug 04, 2008
Posts: 31
Thanks for your replies...

Now, my database [Postgres] is created with Encoding as "UTF8", Collation as "POSIX" and Ctype as "POSIX". Without doing any conversion myself, the data is inserted as is ie., the junk data è£?ç½®ã?®ã??ã??ã?®è£?ç½® is inserted. I also tried to do "SET CLIENT_ENCODING TO UTF8" before insert. But still no improvement...

Is there something that i am missing here??
James Sabre
Ranch Hand

Joined: Sep 07, 2004
Posts: 781

Hema Nandhini wrote:Thanks for your replies...

Now, my database [Postgres] is created with Encoding as "UTF8", Collation as "POSIX" and Ctype as "POSIX". Without doing any conversion myself, the data is inserted as is ie., the junk data è£?ç½®ã?®ã??ã??ã?®è£?ç½® is inserted. I also tried to do "SET CLIENT_ENCODING TO UTF8" before insert. But still no improvement...

Is there something that i am missing here??


One possibility - does the software being used to view the database know how to display utf-8 encoded data and does it have a font with glyphs for Japanese characters? Can you view the table contents as hex encoded bytes and see if the content corresponds to utf-8 characters?
Paul Sturrock
Bartender

Joined: Apr 14, 2004
Posts: 10336

This is always good background reading for people struggling with this sort of issue. Not a direct answer to your issue but still very much worth a read.


JavaRanch FAQ HowToAskQuestionsOnJavaRanch
Hema Nandhini
Ranch Hand

Joined: Aug 04, 2008
Posts: 31
Thanks James and Paul... the link was really helpful. "struggling" was exactly the word for me that day. Now i am clear on the encoding concepts and i was able to get it right in POST requests. My storage and retrieval all worked well in POST request. However, i was not able to make it in GET. request.getParameterMap() returns junk values when the input is given thru javascript encodeURIComponent... i am trying. Will post the solution if ever i find the solution.
 
I agree. Here's the link: http://aspose.com/file-tools
 
subject: Problem encoding japanese character