This week's giveaway is in the Android forum.
We're giving away four copies of Android Security Essentials Live Lessons and have Godfrey Nolan on-line!
See this thread for details.
The moose likes JSP and the fly likes Problem with encoding and polish characters Big Moose Saloon
  Search | Java FAQ | Recent Topics | Flagged Topics | Hot Topics | Zero Replies
Register / Login


Win a copy of Android Security Essentials Live Lessons this week in the Android forum!
JavaRanch » Java Forums » Java » JSP
Bookmark "Problem with encoding and polish characters" Watch "Problem with encoding and polish characters" New topic
Author

Problem with encoding and polish characters

Karol Matysiak
Greenhorn

Joined: Aug 30, 2012
Posts: 20
Hello.

So:
I'm using windows 7 (sadly),
I've got MySQL server with database encoded to latin2
I've got glassfish3 server with hibernate (3.2.5 ? or something like that, I can't check it exactly right now)
I'm using JSP/Servlets to create webapp

When I select data in my mysql console, everything works great, I have a polish characters, but when I want to retrive data from the db and put it into the JSP page, data displays without polish characters. I have set the encoding to latin2everywhere I could (jsp pages, hibernate.cfg - also in the hibernate.connerction.url property, filter), but without any success.

I had a problem with displaying the polish characters in the page title, but when I have added a filter (found on the net) everything was fine, but the filter changed encoding to UTF-8. So I have also changed it to latin2 encoding.

When I set utf-8 in my mysql database I don't get polish characters when I'm selecting data, so I can't use utf-8 ;/

Do I need to change encoding somewhere else? In glassfish server or jdbc driver?
jatan bhavsar
Ranch Hand

Joined: Jul 23, 2008
Posts: 296

Hi Karol,

Right direction is to work with the filter and UTf-8.
Try to work this way and let me know what type of error are you getting i might be able to help you.
Also have you set the UTf-8 encoding type for the glassfish server like <parameter-encoding default-charset="UTF-8"/>.

It is require in tomcat to change the default encoding to UTF-8 with encodeuri when you are working for i18n so check the same for the glassfish .

Regards
jatan
Vishal Shaw
Ranch Hand

Joined: Aug 09, 2012
Posts: 179
Karol Matysiak wrote: When I set utf-8 in my mysql database I don't get polish characters when I'm selecting data, so I can't use utf-8


This is wrong, as UTF-8 is the most accepted encoding, until unless you want to use UTF-16 :)

As for the character encoding filter, is it the first filter (in the filter chain) in your xml? Otherwise it will not work as intended. Also did you tried setting the connection encoding to the required encoding, like

I did it for UTF-8, you can use your encoding , if you desire.
Also is your page encoding using the required one, or is it using the default one?

Try these and let us know.


Programming is about thinking, NOT coding
Karol Matysiak
Greenhorn

Joined: Aug 30, 2012
Posts: 20
jatan bhavsar wrote:Hi Karol,

Try to work this way and let me know what type of error are you getting i might be able to help you.
Also have you set the UTf-8 encoding type for the glassfish server like <parameter-encoding default-charset="UTF-8"/>.


I don't get error. Data is displayed, but I've got something like Imię: Andrzej Nazwisko: Krokosiäski tel.: 721265273, instead of Imię: Andrzej Nazwisko: Krokosiński tel.: 721265273 . Word "imię" got polish character, but that word is not from the database.
I just notice, that I didn't changed the filter. After change (it's now encoding to latin2, same as mysql server) the data which was in the database is displaying as earlier - Krokosiäski, but when I will add somethign through the form on index page, everything is fine (it displays Krokosiński ).

This is wrong, as UTF-8 is the most accepted encoding, until unless you want to use UTF-16 :)


I know, but I can't set up my db server to utf-8 :/ I'm trying this since few days.

As for the character encoding filter, is it the first filter (in the filter chain) in your xml? Otherwise it will not work as intended. Also did you tried setting the connection encoding to the required encoding, like

I did it for UTF-8, you can use your encoding , if you desire.
Also is your page encoding using the required one, or is it using the default one?

Try these and let us know.


I have that in my hibernate.cfg.xml - but it didn't solved my problem. Of course in each page I have set up encoding to latin2.
And about my filters, I have only that one filter, so it must be first and the last filter in the chain :)
Here is my hibernate.cfg.xml

hibernate reveng.xml

JSP with the form

JSP with the data from db

servlet


I don't post the java mapping file because it is a bunch of setters and getters.>
Vishal Shaw
Ranch Hand

Joined: Aug 09, 2012
Posts: 179
Karol Matysiak wrote: I know, but I can't set up my db server to utf-8

Did you mean, you are trying but can't set it, or you have been forbidden to set it?

Karol Matysiak wrote: After change (it's now encoding to latin2, same as mysql server) the data which was in the database is displaying as earlier - Krokosiäski, but when I will add somethign through the form on index page, everything is fine (it displays Krokosiński )

I think your filter is encoding only request parameters not the response object. So, you can try setting it on response objects too, alternatively try adding

in your HibernateTest.java
Karol Matysiak
Greenhorn

Joined: Aug 30, 2012
Posts: 20
Vishal Shaw wrote:
Did you mean, you are trying but can't set it, or you have been forbidden to set it?


Yes, I tried, it's not any buisness project so I can set up what only I want, but I have set up everything to utf-8 in my database and polish characters wasn't displayed well.
Vishal Shaw wrote:

I think your filter is encoding only request parameters not the response object. So, you can try setting it on response objects too, alternatively try adding

in your HibernateTest.java


I will try to add this when I will be at home.
Karol Matysiak
Greenhorn

Joined: Aug 30, 2012
Posts: 20
Nope, it still doesn't work. Only records which I add from the mysql server console display without polish characters.
Bear Bibeault
Author and ninkuma
Marshal

Joined: Jan 10, 2002
Posts: 60785
    
  65

"rok"?


[Asking smart questions] [Bear's FrontMan] [About Bear] [Books by Bear]
Paul Clapham
Bartender

Joined: Oct 14, 2005
Posts: 18541
    
    8

Bear Bibeault wrote:"rok"?


The Polish word for "year". (Hope that helps!)
Karol Matysiak
Greenhorn

Joined: Aug 30, 2012
Posts: 20
My mistake, it should be "work". Now it's ok;p
Paul Clapham
Bartender

Joined: Oct 14, 2005
Posts: 18541
    
    8

Karol Matysiak wrote:I'm using windows 7 (sadly)


So am I.

I've got MySQL server with database encoded to latin2


My instance of MySQL is configured to use UTF-8. This handles letters specific to Polish (and Slovak for that matter) correctly.

When I set utf-8 in my mysql database I don't get polish characters when I'm selecting data, so I can't use utf-8 ;/


You can't just change the configuration and expect it to work correctly. All that does is to assert that data which was already encoded in latin2 is now encoded in UTF-8.
Which of course it isn't. You have to export all the data from the database to a text file, then change the configuration, then import all the data back from that text file.
Karol Matysiak
Greenhorn

Joined: Aug 30, 2012
Posts: 20
Paul Clapham wrote:
You can't just change the configuration and expect it to work correctly. All that does is to assert that data which was already encoded in latin2 is now encoded in UTF-8.
Which of course it isn't. You have to export all the data from the database to a text file, then change the configuration, then import all the data back from that text file.


Of course your're right, but I have even created new database to check it, without any success. In best configuration I had something like that on my server:
query:
result :


+----------------------+-----------------+
| Variable_name | Value |
+----------------------+-----------------+
| collation_connection | utf8_general_ci |
| collation_database | utf8_polish_ci |
| collation_server | utf8_polish_ci |
+----------------------+-----------------+



Also it was set everywhere utf8_general_ci, and still no polish characters
And result for query


Result was:
+--------------------------+----------------------------------------------------
-----+
| Variable_name | Value
|
+--------------------------+----------------------------------------------------
-----+
| character_set_client | utf8
|
| character_set_connection | utf8
|
| character_set_database | utf8
|
| character_set_filesystem | BINARY
|
| character_set_results | utf8
|
| character_set_server | utf8
|
| character_set_system | utf8
|
| character_sets_dir | C:\Program Files\MySQL\MySQL Server 5.5\share\chars
ets\ |
+--------------------------+----------------------------------------------------


And I have also created db in InnoDB engine, with setchar=utf8 attributes, I created even tables with encoding set to utf8 - and still it didn't worked. I was searching solution for few days, and the only thing i found was for PHP developing to send mysql query "SET NAMES 'utf-8' ". Here's my.ini file:

[WinMySQLAdmin]
Server="C:/Program Files/MySQL/MySQL Server 5.5/bin/mysqld.exe"

[client]
default-character-set=utf8

[mysql]
default-character-set=utf8

[mysqladmin]
default-character-set=utf8

[mysqlcheck]
default-character-set=utf8
[mysqldump]
default-character-set=utf8

[mysqlimport]
default-character-set=utf8

[mysqlshow]
default-character-set=utf8
[mysqld]
character-set-server=utf8
collation-server=utf8_polish_ci
init_connect='SET collation_connection = utf8_polish_ci'
init_connect = 'SET NAMES utf8'


Paul Clapham
Bartender

Joined: Oct 14, 2005
Posts: 18541
    
    8

I think you're going to have to tell us what "didn't work" means.

If you're using the Windows console to test with, that quite likely isn't going to show you non-ASCII characters correctly unless you have your Windows code page set correctly. But instead of messing about with that, you could use a small Swing application to check your database configuration. That's the context I was describing, which works perfectly well with a MySQL database configured to use UTF-8.

In any case you shouldn't use a web application to test that, because now you're testing the web app and the database connection at the same time, so you don't know where to look when you don't get the correct result.
Karol Matysiak
Greenhorn

Joined: Aug 30, 2012
Posts: 20
Yes, I'm using windows console.

Ok but if I set encoding to latin2 - in windows console I have all characters which I want, when I will change it to utf-8, those characters are missing.

Anyway, like you said, I wrote simple app that is connecting to the same server as my JSP, and still the polish characters are not displayed :|
Karol Matysiak
Greenhorn

Joined: Aug 30, 2012
Posts: 20
Sorry for double post, but I have to update few things. So:

I have switched to linux, now I have mysql server 5.1, jboss 7.1.1, eclipse ide.

MySQL encoding is now set to utf-8 and I have polish characters in linux command line, but of course i don't have those characters on my web page, and not only in data retrived from database, but also on template text in jsp, also in title of jsp. Everything is the same but it's of course doesn't work...

Any ideas what can be wrong now?
 
I agree. Here's the link: http://aspose.com/file-tools
 
subject: Problem with encoding and polish characters
 
Similar Threads
dealing with non ascii characters
Message bundle encoding
Unicode and MSSQL
JSF and Character Sets (UTF-8 mainly)
Character Encoding with DBCP pool