bad characters (MS Windows-only chartacters) in two Java files

[Amandeep Singh]

These characters are not interpreted correctly in Linux.

export/bea/Win-cm-int/cc/projects/cycle/core/src/main/com/cycleus/cycleton/wizard/entryvalidation/ValidateManifest.java:1845: warning: unmappable character for encoding UTF8

this was recently notified by my colleague. i don't know why linux complains. In my window using ant i compiled fine. is there any way before i check in my code, i could know if there are such bad characters present or not ?

[William Brogden]

Is it possible that these could be the MS Word "smart punctuation" characters - is somebody is editing source code using Word?

They also cause trouble for XML documents.

Bill

[Amandeep Singh]

yes you are right. other than compiling through the linux box. is there other tool available in windows to get to know about the bad characters.

[William Brogden]

Here is what I use:

   // replace Word processor smart quotes and ' with ascii if flag set
  String fixWordSmart( String s ){
    if( !fixSmart ||
         s == null ||
         s.length() == 0 ){
           return s ;
    }
    s = s.replace( (char)145, (char)'\'');
    s = s.replace( (char)8216, (char)'\''); // left single quote
    s = s.replace( (char)146, (char)'\'');
    s = s.replace( (char)8217, (char)'\''); // right single quote
    s = s.replace( (char)147, (char)'\"');
    s = s.replace( (char)148, (char)'\"');
    s = s.replace( (char)8220, (char)'\"'); // left double
    s = s.replace( (char)8221, (char)'\"'); // right double
    s = s.replace( (char)8211, (char)'-' );
    return s.replace( (char)150, (char)'-' );
  }   

[Amandeep Singh]

thanks cool...but to check whether bad characters are there or not, explicitly is to complie in cygwin or linux.

[Campbell Ritchie]

You can try reading the text file with the usual techniques, then use regular expressions to look for the smart quotes.