encoding [?] problem - different content on Win and Linux after downloading file

[Maciej Opala]

Hi,
Don't know where to write but guess it might be the best forum.
I've written small command line tool in groovy to download/upload file to drools.
When I download file I receive different content on linux and win just after converting the stream to string (before saving file to local filesystem).

Linux:

2012:02:06 11:33:19,563 INFO  [Main] Processing started
2012:02:06 11:33:19,783 INFO  [Main] No properties files specified, using default
2012:02:06 11:33:19,794 INFO  [Main] Working in download mode
2012:02:06 11:33:19,821 INFO  [Main] Downloading file -> http://localhost:8080/drools-guvnor/org.drools.guvnor.Guvnor/api/packages/com.test.config/products.csv
headers: [Date: Mon, 06 Feb 2012 10:34:55 GMT, Transfer-Encoding: chunked, Content-Type: application/x-download, Content-Disposition: attachment; filename=data;, Set-Cookie: JSESSIONID=dLlZPvsPGhvshsGlzHQKjRcrNvQRs16Bh269jM9hDlYhlVZnPzph!1498084790; path=/, X-Powered-By: Servlet/2.5 JSP/2.1]
1;Prod1 „SH”;EUR,CHF,USD;A;SH;
2;Prod2 „SH”;EUR,CHF,USD;B;SH;
3;Prod3 „SH”;EUR;C;SH;
4;Prod4 „NH”;A;NH;
5;Prod5 „NH”;B;NH;
6;Prod6 „NH”;C;NH;
2012:02:06 11:33:20,396 INFO  [Main] Request status -> 200
2012:02:06 11:33:20,410 INFO  [Main] Saving file -> ~/lol/products.csv

Win:

2012:02:06 11:55:38,734 INFO  [Main] Processing started
2012:02:06 11:55:38,921 INFO  [Main] No properties files specified, using default
2012:02:06 11:55:38,937 INFO  [Main] Working in download mode
2012:02:06 11:55:38,937 INFO  [Main] Downloading file -> http://localhost:8080/drools-guvnor/org.drools.guvnor.Guvnor/api/packages/com.test.config/products.csv
headers: [Date: Mon, 06 Feb 2012 10:57:11 GMT, Transfer-Encoding: chunked, Content-Type: application/x-download, Content-Disposition: attachment; filename=data;, Set-Cookie: JSESSIONID=xk9mPvyHVZphkbKv3FV0mvnPccrHDblQncBvkJP2SnT2JDBnCL3Q!1498084790; path=/, X-Powered-By: Servlet/2.5 JSP/2.1]
1;Prod1 �SH�;EUR,CHF,USD;A;SH;
2;Prod2 �SH�;EUR,CHF,USD;B;SH;
3;Prod3 �SH�;EUR;C;SH;
4;Prod4 �NH�;A;NH;
5;Prod5 �NH�;B;NH;
6;Prod6 �NH�;C;NH;
2012:02:06 11:55:39,750 INFO  [Main] Request status -> 200
2012:02:06 11:55:39,781 INFO  [Main] Saving file -> .\products.csv

Here's the code:

 def get(params) {
        def m = new HttpGet(params.url)
        add_headers(m, params.headers)
        def r = dhc.execute(m)
        def o = [:]
        println "headers: ${r.allHeaders}"
        o.status = r.statusLine.statusCode
        o.content = IOUtils.toString(r.entity.content, 'UTF-8')
        println o.content
        return o
    }

I use HttpComponents to handle net tasks. Does anybody know the explanation? I'm really annoyed with this problem and run out of possible solutions.

[Alex Armenteros]

I see this is not java but I encountered a similar problem last week.

be careful what programs do you use to transfer between SOs, WinSCP change the size of text files.

Also that way the file was read differently on Win than on Linux.

Finally I got it reading the file byte by byte.

File f = new File(args[1]);
			FileInputStream fis = new FileInputStream(f);
			int size = 0;
			while (fis.read() != -1) {

				size++;
			}
			byte[] ba = new byte[size];
			int b = 0;
			fis.close();
			fis = new FileInputStream(f);
			int i = 0;

			while ((b = fis.read()) != -1) {
				ba[i] = (byte) b;
				i++;
			}

			String res = new String(ba, "latin1");

[Maciej Opala]

Thanks for the hint.
I'd consider it strange if it changed anything but will try nevertheless

[Maciej Opala]

Have done it as You advised Alex Armenteros. Thanks for the solution! But it's strange that Apache IOUtils couldn't handle these stream operations correctly. Or I don't know how to use it.