Win a copy of Functional Design and Architecture this week in the Functional programming forum!
  • Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other Pie Elite all forums
this forum made possible by our volunteer staff, including ...
Marshals:
  • Campbell Ritchie
  • Ron McLeod
  • Rob Spoor
  • Tim Cooke
  • Junilu Lacar
Sheriffs:
  • Henry Wong
  • Liutauras Vilda
  • Jeanne Boyarsky
Saloon Keepers:
  • Jesse Silverman
  • Tim Holloway
  • Stephan van Hulst
  • Tim Moores
  • Carey Brown
Bartenders:
  • Al Hobbs
  • Mikalai Zaikin
  • Piet Souris

Printing latin characters not working

 
Greenhorn
Posts: 2
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
I have a program which works perfectly using desktop java. It prints strings which include Latin characters such as  "û"  "ý"   "þ"  "ÿ"
I have modified the program to run in Tomcat but now characters line these are simply printing as "?", each one.
I've checked the web.xml files and everything seems to be set to UTF-8
I've been using System.out.println to write to the tomcat log and FileWriter to write to files, both give the same result

George
 
Saloon Keeper
Posts: 24283
167
Android Eclipse IDE Tomcat Server Redhat Java Linux
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
UTF-8 can handle letters such as thorn. Your problem is almost certainly in what program you display the generated text with. .If you don't have an appropriate font selected, yes, you'll almost certainly see boxes, "?", or other replacement glyphs.

And for the record, I þink it's high time we brought back þe þorn! Too long has the English-printing world been tyrannized by the limitations of Shakespearean-era printing.

Just to be pedantic (again), thorn isn't actually a "Latin" character. It's English. But it's part of most Latin font sets. Also, System.out.println may be OK for crude problem solving, but I do hope you use proper logging when you get seerious about the app. It's not going to be any different in what it displays. but System.out is not officially supported in Enterprise Java and it's not as civilized as a true logger, anyway.
 
George Thompson
Greenhorn
Posts: 2
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
found a clue online:
The answer is that a java.io.FileWriter doesn't use UTF-8 encoding by default. Instead use the following code to create the writer:
def writer = new OutputStreamWriter(new FileOutputStream(outputFile),"UTF-8")
Hat tip to http://www.malcolmhardie.com/weblogs/angus/2004/10/23/java-filewriter-xml-and-utf-8/ for the answer.
 
Tim Holloway
Saloon Keeper
Posts: 24283
167
Android Eclipse IDE Tomcat Server Redhat Java Linux
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
You might find this informative: https://www.baeldung.com/java-char-encoding

Actually, in most cases, I'd be using a PrintWriter rather than a FileWriter for outputting text. The PrintWriter has the advantage of supporting OS-independent output of end-of-line characters and automatic buffer flushing of lines (which is important for console output in interactive apps).
 
Marshal
Posts: 73953
332
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

George Thompson wrote:. . . java.io.FileWriter doesn't use UTF-8 encoding by default. . . ..

The documentation (Java14 version) says it uses

. . . either a specified charset or the platform's default charset.

This has changed since the comment about it in the blog: Java6 version of documentation.
I am a bit worried that the blog you quoted suggested mixing XYZWriters and XYZStreams.

. . . new OutputStreamWriter(new FileOutputStream(path),"UTF-8");

This is what its documentation says:-

. . . FileOutputStream is meant for writing streams of raw bytes such as image data. For writing streams of characters, consider using FileWriter. . . .

You can probably convert that to a byte stream with the output stream writer, but nowadays I would use a Formatter or this sort of code to get a buffered writer.Plain simple
 
Tim Holloway
Saloon Keeper
Posts: 24283
167
Android Eclipse IDE Tomcat Server Redhat Java Linux
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Actually, my go-to is a PrintWriter constructed on a FileOutputStream. It's very rare that I'd be outputting nothing but text and not have it arranged in lines.
 
Master Rancher
Posts: 4023
53
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
For what it's worth, for some time now (JDK 1.5+) PrintWriter has had constructors that allow you to directly specify both an output file and a character encoding.   With a BufferedWriter in between for good measure. You don't have to go explicitly make 2-3 streams chained together anymore, most of the time.  But, there's not a constructor that includes autoflush with the file and encoding - if you want autoflush, you still need at least two streams.  Presumably that's why Tim still does it...
 
Campbell Ritchie
Marshal
Posts: 73953
332
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Tim, please explain more about autoflush.

Yesterday, I wrote:Plain simple

I think I was supposed to follow that with something else, but I have forgotten what.
 
Tim Holloway
Saloon Keeper
Posts: 24283
167
Android Eclipse IDE Tomcat Server Redhat Java Linux
  • Likes 1
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Campbell Ritchie wrote:Tim, please explain more about autoflush.

Yesterday, I wrote:Plain simple

I think I was supposed to follow that with something else, but I have forgotten what.



Java borrows heavily from C, as we all know. One of the characteristics of the C printf function was that whenever a newline is encountered in the process of rendering output, the flush() function would be automatically invoked. This is why we have:


Instead of this:

The Java PrintWriter class also conforms to this convention. As I said, it's a lot more convenient when you're writing conversational terminal applications if you don't have to explicitly invoke flush() after every println() in order to get the latest response or prompt to display.
 
Campbell Ritchie
Marshal
Posts: 73953
332
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Thank you You don't notice you ae using it all the time on standard out.
 
Mike Simmons
Master Rancher
Posts: 4023
53
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Well, you're probably not using autoflush all the time... unless you explicitly set it up.  PrintStream and PrintWriter have a variety of constructors - those that accept a boolean are the ones that allow you to use that boolean to force autoflush, by setting the boolean to true.  All the other constructors, however, do not enable autoflush - the default is false.

As for System.out and System.err, I don't think it's documented whether those have autoflush enabled or not.  It may be platform dependent, I think.  But I suspect they are not, in most or all cases.  In any event, you should never assume it's enabled.  If you want it, set it.  (You can even use System.setOut() to ensure it's true for all subsequent uses of System.out.)

However, be aware that you don't necessarily want autoflush enabled all the time.  It has the potential to degrade performance, especially if you're writing a long file with a lot of short lines.  That's part of why they encouraged us to include a BufferedOutputStream or BufferedWriter, back in the days before it could come bundled automatically.  If you're writing a "conversational" app as Tim says, you almost certainly do want autoflush - but if you're writing a high volume of lines to files, you probably don't.

Also, note that there's a small difference between PrintWriter and PrintStream, with respect to autoflush.  While PrintStream does ensure that with autoflush on, flushing occurs after any \n character, PrintWriter does not.  Instead, it only enforces flushing after a println, printf, or format method call.  You can still bypass it using a print or write call.  Usually that's fine, since most of us use println, printf or format most of the time anyway.  But the difference is there nonetheless...
 
Don't get me started about those stupid light bulbs.
reply
    Bookmark Topic Watch Topic
  • New Topic