Internationalization

[Migrated From Jforum.net]

Hi all,

I am trying to configure jforum to work with Russian letters and having some problems.

When I post a message I can see Russian letters only in topic. For example here is a page with topics: http://www.jforum.net/forums/show/5.page. So topics are displayed with Russian letters but when I go to body of the message I see questio marks (???). Here is an example of body of a message: https://coderanch.com/t/576296

Why is it happening?

What can be done to correct this? I assume the problem when I try to save in to a database but I am not sure.

Thanks!
[originally posted on jforum.net by AnotherGuest]

[Migrated From Jforum.net]

You are not alone buddy ,
I've got the same problem https://coderanch.com/t/576302 .
I applied my own filter to jforum web application to reinforce client's encoding to UTF-8 (see how other guys solved similar problem: http://www.jforum.net/confluence/display/configuration/charEncodingContainer http://forum.springframework.org/archive/index.php/t-10431.html).

However, it did not help me to get rid of the ??? in message body. If you look at the database, jforum_topics table, topic_title column is a VARCHAR(100), so it works fine, you can see topic titles in Russian. Try to check the

 SELECT HEX(post_text)  FROM jforum_posts_text j;

, if you see 3F in the post_text value, it means there is garbage content was written to the db from the client - not UTF-8 friendly value!
Again, could it be something related to the http://dev.mysql.com/doc/refman/5.0/en/cjk-faq-question-marks.html?

By the way, the webapp works fine on windows box, however the problem appears only when it was deployed in linux environment.

Let's help to each other!
Appreciate for your time and assistance in advance!
onsh76
[originally posted on jforum.net by onsh76]

[Migrated From Jforum.net]

Hi all,

yeah, I am not sure what the problem is but I'd appreciate any feedback on this.

I installed jforum on my Linux box (Fedora Core 5) and I have the same problem

I took a look around and found out this: http://dan.solutions.lv/forum/forums/list.page
Apparently it can work with Russian letters but front end is still in English
I made a post on the above site with no luck - nobody has replied just yet

I do see 3F when I ran that query.

How did you make it work on Windows machine?

In either case Linux or Windows I get question marks.

Please advise.

By the way in Configuration which encoding are you using?

At this point I am trying things out. I'll keep you updated on the progress



Cheers


[originally posted on jforum.net by AnotherGuest]

[Migrated From Jforum.net]

I am back,

Here is what I thouth - if jforum_topics table topic_title column is a VARCHAR(100) and it works fine why not change post_text's type from text to VARCHAR(some value here)?

Needed tables can be altered but I am not sure if that create any problems down the road. There should be a limitation to how many chars you can type in. I guess this question is for Rafael.

Anyway, what do you think about altering needed tables?
[originally posted on jforum.net by AnotherGuest]

[Migrated From Jforum.net]

By the way I got nullPointer exception when I tried to post A LOT of text

HTTP Status 500 -

type Exception report

message

description The server encountered an internal error () that prevented it from fulfilling this request.

exception

java.lang.NullPointerException
	net.jforum.JForumExecutionContext.enableRollback(JForumExecutionContext.java:272)
	net.jforum.JForum.service(JForum.java:209)
	javax.servlet.http.HttpServlet.service(HttpServlet.java:802)
	net.jforum.util.legacy.clickstream.ClickstreamFilter.doFilter(ClickstreamFilter.java:59)

note The full stack trace of the root cause is available in the Apache Tomcat/5.5.9 logs.

[originally posted on jforum.net by AnotherGuest]

[Migrated From Jforum.net]

what am I supposed to put for mail configuration settings?

it says [e-mail] for smtp name.
Could you prvide an example plz?
[originally posted on jforum.net by AnotherGuest]

[Migrated From Jforum.net]

Ch��a vi���t h�?a ho��n to��n, kh̫ng bi���t ��em code v��? Vi���t H�?a �Ԯ��ȣc kh̫ng ��̢y, b�� con n��o x��i r���i cho bi���t �? ki���n nhe, c�? g�� m��nh h�ȣp t��c
[originally posted on jforum.net by BlackWolf88]

[Migrated From Jforum.net]

not sure what you mean by this ...
[originally posted on jforum.net by AnotherGuest]

[Migrated From Jforum.net]

I'm too, but when I post by Vietnameses in this, it is very good but when I post in my forum, it not display Vietnamese
Why so???
[originally posted on jforum.net by BlackWolf88]

[Migrated From Jforum.net]

I guess we need some help from the author...
[originally posted on jforum.net by AnotherGuest]

[Migrated From Jforum.net]

hello,

I have JForum 2.1.6 configured under Tomcat 5.0.28 hosted on Solaris 10 x86. We use mysql 4.1.7 as a database server. Though we mostly use JForum for English language communication when I just tried to post both message title and body in Russian they were displayed correctly both in preview and when saved to db.

Concerning your internationalization question..Do you have other webapps under the same web container (I guess you also use Tomcat) configured to display Cyrillic characters correctly? I do remember what a pain was to get Ukrainian characters displayed correctly under our other web app.
At that time it appeared that to get Cyryllic displayed well you need to configure it both at database server level and in your web app.
So,
1. Our mysql database server is started with --default-character-set=utf8 option. Then you need to set a default character set for your specific database to utf8. If you already have them created with latin1, pls make sure you issue alter table on all tables and set utf8 as a default encoding for all tables in your database. After long hrs (or days) of investigation of this problem in our other web app I noticed that though I converted my database to utf8 some tables still had latin1 as a default encoding and that was the problem.
2. At a web app level, we use a SetCharacterEncodingFilter to set utf8 at request and response objects before they are accessed by a user code. Our installation of JForum seems to work without it. Next, you should check your JDBC connection URL to see whether characterEncoding=utf8 is added to it (this is mysql specific but there should be smth similar for other RDBMSs).

Basically, if you do steps 1, 2 it should work for you. But there is also an option of the system's default encoding. If you're using linux, checkout the file /etc/sysconfig/i18n. I think you need some UTF8 encoding there too (not just cp1251, though I'm not sure). At my laptop it is:
LANG=uk_UA.UTF-8.
BTW, do you use any Java ide on your linux? Can you type Cyrillic symbols in the IDE editors? JDK versions prior to 1.5.0 seemed to have some bug which prevented proper Cyrillic display.

And finally, concerning examining db tables in console. Though we have Ukrainian characters correctly displayed in our web app and JForum installation I still cannot read them through my console at Solaris box. I get those familiar ??? marks. It is because console also has to be configured to display UTF8 correctly. So, I think if you're on linux you'll have to play a little with the system's default encoding property.

Hope this helps.

Regards,
Andrew Stepanenko
[originally posted on jforum.net by andrew.stepanenko]

[Migrated From Jforum.net]

Hi Andrew,

Thanks a lot for the hints you gave!
I altered tables and now I see the following:

jforum_posts_text | CREATE TABLE jforum_posts_text (
post_id int(11) NOT NULL default '0',
post_text text character set latin1,
post_subject varchar(100) character set latin1 default NULL,
PRIMARY KEY  (post_id)
ENGINE=InnoDB DEFAULT CHARSET=utf8 |

utf8 is there which is good.

I am a bit confused about second point you made.
You said that I need to "check your jdbc connection URL to see whether characterEncoding=utf8 is added to it...." Where would I check this? I have 4.1 MySQL version.

Yes, I have NetBeans on Fedora Core 5. I can type and see Cyrillic symbols with no problems. I got the same problem where I cannot see Cyrillic symbols on console window.

Regards,

AnotherGuest
[originally posted on jforum.net by AnotherGuest]

[Migrated From Jforum.net]

hello,

in the snippet you posted you still have latin1 on columns:

post_text text character set latin1

I think you need to convert you table to utf8 not just set default encoding. The ALTER TABLE clause in MySQL has an option to CONVERT TO CHARACTER SET in addition to [DEFAULT] CHARACTER SET.

Also, checkout in the admin control panel of your jforum a 'Character Encoding' property of General purpose section. It should be set to UTF-8.
In jforum probably you do not need to modify jdbc connection url manually, the above setting should do that. I mentioned characterEncoding=utf8 in the case of a stand alone web app which you want to display Cyrillic correctly.

Regards,
Andrew
[originally posted on jforum.net by andrew.stepanenko]

[Migrated From Jforum.net]

Hi Andrew,

I changed the table but still get question marks.
I have another app that sits there and I use the following in my other tables:

description text CHARACTER SET koi8r,

It works fine and I see Russian letters perfectly fine on output.

I tried to do the same for jforum_posts_text table but it did not help.

Is there anything else that I should try?

Thank you.

[originally posted on jforum.net by AnotherGuest]

[Migrated From Jforum.net]

Hi Andrew,

I changed the DB but it did not help. So I guess I have to look somewhere else.

Your second point says "chtck JDBC connection URL to see whether characterEncoding=utf8 is added to it". Where would I check it?

Thanks.
[originally posted on jforum.net by AnotherGuest]

[Migrated From Jforum.net]

hello,

for your question:

Your second point says "chtck JDBC connection URL to see whether characterEncoding=utf8 is added to it". Where would I check it?

Check out this post: https://coderanch.com/t/576302 . But still, I'm not sure you need to edit that URL. At our site JForum works with Ukrainian fine as is. That my second point was related to the case of your own web app which you want to make Cyrillic aware.

In your situation I suggest you install another copy of db server and configure it from scratch with utf8, then install another fresh copy of Tomcat and make a simple web app with pages in Russian which reads/posts content in DB. Once you get this working you can easily add JForum web app to this new Tomcat installation or at least you would know what was wrong.

Regards,
Andrew Stepanenko
[originally posted on jforum.net by andrew.stepanenko]

[Migrated From Jforum.net]

Hi Andrew,

Thanks for following up.
I do not see a benefit or reinstalling db server because encodings of a db server can be changed.

I am not sure if I mentioned that above but I do have another web app sitting on this db with Tomcat and it works fine. I can strore and retrive letters in Russian without any problems. Therfore, I do not understand why jforum does not work on the same db :?:

I had to do some tricks in the code though.

The main point if that I thought that some jforum code needs to be changed but I have no idea where to look at.

If you have any ideas what I might consider doing please share with me.

Thanks a lot.



[originally posted on jforum.net by AnotherGuest]

[Migrated From Jforum.net]

to AnotherGuest
Vam udalos reshit vopros s kodirovkoi?
U nas takaja je problema
[originally posted on jforum.net by Makoff]