This week's book giveaway is in the Functional programming forum.
We're giving away four copies of Functional Design and Architecture and have Alexander Granin on-line!
See this thread for details.
Win a copy of Functional Design and Architecture this week in the Functional programming forum!
  • Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other Pie Elite all forums
this forum made possible by our volunteer staff, including ...
Marshals:
  • Campbell Ritchie
  • Ron McLeod
  • Rob Spoor
  • Tim Cooke
  • Junilu Lacar
Sheriffs:
  • Henry Wong
  • Liutauras Vilda
  • Jeanne Boyarsky
Saloon Keepers:
  • Jesse Silverman
  • Tim Holloway
  • Stephan van Hulst
  • Tim Moores
  • Carey Brown
Bartenders:
  • Al Hobbs
  • Mikalai Zaikin
  • Piet Souris

Tomcat is blocking javax.mail under 11.1 but not under 10.13

 
Bartender
Posts: 1829
17
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Not sure what is going on, but the same SpringBoot WAR file under Tomcat (9.0.41) in Mac OS 11.1 when sending email is failing with "Exception reading response.  Cause: java.net.SocketTimeoutException: Read timed out", but works perfectly OK under Mac 10.13 or Windows. Have even copied the working Tomcat folder from the Mac 10.13 machine to the 11.1 machine. It works on the 10.13 machine, but, again, same error as above on the 11.1 machine.
Even re-installed entire 11.1 machine to test before enabling Mac firewall. Same error.

Same code, same Tomcat, etc., only the machine OS seems different.
Took code from SpringBoot app to standalone Java app and it worked on the 11.1 machine.

Clearly seems to be a Tomcat issue of some kind.

What else to try or how to figure out what's blocking javax.mail in Tomcat on the 11.1 machine?

Thanks,

- mike
 
Saloon Keeper
Posts: 24283
167
Android Eclipse IDE Tomcat Server Redhat Java Linux
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Tomcat doesn't have even the remotest idea of what JavaMail is - just one of the things that shows it's not a full-stack JEE implementation.

If you're getting a timeout, it's almost certainly either because you have a firewall issue or because you're talking to the wrong port or wrong server. These days mail isn't always handled on port 25.

One of the best ways to test that is to use telnet to try and connect to the upstream mailserver. For example:

telnet mail.mousetech.com 25

Basic SMTP is a text-based protocol and it's actually possible to send mail directly via telnet, but the important thing is whether it connects and responds. If it doesn't then definitely the problem is not in Tomcat.
 
Mike London
Bartender
Posts: 1829
17
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Tim Holloway wrote:Tomcat doesn't have even the remotest idea of what JavaMail is - just one of the things that shows it's not a full-stack JEE implementation.

If you're getting a timeout, it's almost certainly either because you have a firewall issue or because you're talking to the wrong port or wrong server. These days mail isn't always handled on port 25.

One of the best ways to test that is to use telnet to try and connect to the upstream mailserver. For example:

telnet mail.mousetech.com 25

Basic SMTP is a text-based protocol and it's actually possible to send mail directly via telnet, but the important thing is whether it connects and responds. If it doesn't then definitely the problem is not in Tomcat.



Thanks, but ... same code, same ports, etc., as noted above, in a standalone Java program works (port 465),

Same EXACT Tomcat install, as noted above, on other machine works, not on 11.1.

Also, as noted above, completely re-installed MacOS and before any Firewall enabling, etc, same error on 11.1.

Also, same WAR file works fine on AWS Windows EC2 instance.

The only difference is Tomcat that I can see.

---

Hopefully, someone has experienced this issue and can suggest some helpful things to try.

Thanks,

- mike
 
Tim Holloway
Saloon Keeper
Posts: 24283
167
Android Eclipse IDE Tomcat Server Redhat Java Linux
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Well, again, Tomcat knows nothing. Tomcat does not interject itself into network traffic. It either owns ports outright (8080, 8443, etc.) or the application does. No sharing, packet conditioning, traffic routing/shaping. Nothing.

So, taking you at your word (though I do recommend a telnet test on port 465), I would look at the larger environment. Is there a virtual storage issue? Has the Tomcat JVM been allocated enough object memory? Is the machine it's running in being starved?

And last, but definitely not least, have you attempted to run the system using a different JVM?
 
Mike London
Bartender
Posts: 1829
17
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Tim Holloway wrote:Well, again, Tomcat knows nothing. Tomcat does not interject itself into network traffic. It either owns ports outright (8080, 8443, etc.) or the application does. No sharing, packet conditioning, traffic routing/shaping. Nothing.

So, taking you at your word (though I do recommend a telnet test on port 465), I would look at the larger environment. Is there a virtual storage issue? Has the Tomcat JVM been allocated enough object memory? Is the machine it's running in being starved?

And last, but definitely not least, have you attempted to run the system using a different JVM?



Here is the Tomcat memory usage:


G1 Eden Space Heap memory 26.00 MB 219.00 MB -0.00 MB 123.00 MB
G1 Old Gen Heap memory 230.00 MB 136.00 MB 4096.00 MB 28.21 MB (0%)
G1 Survivor Space Heap memory 0.00 MB 15.00 MB -0.00 MB 15.00 MB
CodeHeap 'non-nmethods' Non-heap memory 2.43 MB 4.06 MB 7.24 MB 2.20 MB (30%)
CodeHeap 'non-profiled nmethods' Non-heap memory 2.43 MB 6.68 MB 116.37 MB 6.62 MB (5%)
CodeHeap 'profiled nmethods' Non-heap memory 2.43 MB 18.62 MB 116.37 MB 18.58 MB (15%)
Compressed Class Space Non-heap memory 0.00 MB 7.00 MB 1024.00 MB 6.23 MB (0%)
Metaspace Non-heap memory 0.00 MB 58.25 MB -0.00 MB 56.23 MB

JDKs tried:

Latest official Oracle JDK Java 8
Latest Corretto JDK 8
Latest Corretto JDK 11

In the email log on the server, I see: "SMTP connection from (IP):59819 lost while reading message data (after header)"


I need to try another email server also, I suppose.

Thanks for the suggestions.

- mike


 
Tim Holloway
Saloon Keeper
Posts: 24283
167
Android Eclipse IDE Tomcat Server Redhat Java Linux
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Actually, that message reads to me like a problem with the mailserver. It's also mostly reported from other systems than tomcat. For example:

https://lists.exim.org/lurker/message/20090831.204705.8e09002e.en.html

In this particular case, it looked like the mailserver was doing a blacklist check against a third-party server and didn't get an answer back fast enough to complete the transmission.

So one possibility would be to make sure that the destination mailserver locally whitelists the machine that Tomcat lives on.
 
Mike London
Bartender
Posts: 1829
17
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Tim Holloway wrote:Actually, that message reads to me like a problem with the mailserver. It's also mostly reported from other systems than tomcat. For example:

https://lists.exim.org/lurker/message/20090831.204705.8e09002e.en.html

In this particular case, it looked like the mailserver was doing a blacklist check against a third-party server and didn't get an answer back fast enough to complete the transmission.

So one possibility would be to make sure that the destination mailserver locally whitelists the machine that Tomcat lives on.



Yep, tried that as well. Whitelisted the public IP from the machine Tomcat is running on. No dice...

Thanks,

-- mike
 
Mike London
Bartender
Posts: 1829
17
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
FWIW, I copied the WAR file to a Windows 10 virtual machine (also Tomcat 9 on the same computer and the email send worked almost instantly.

Totally baffled.

I wonder if it's worth trying to disable the Mac's System Integrity Protection as a test?

-mike
 
Tim Holloway
Saloon Keeper
Posts: 24283
167
Android Eclipse IDE Tomcat Server Redhat Java Linux
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Lacking direct access, I cannot offer any better suggestions at the moment. Except to note that if your machine has both internal and external IP addresses, the mailserver probably should whitelist them all.
 
Mike London
Bartender
Posts: 1829
17
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Tim Holloway wrote:Lacking direct access, I cannot offer any better suggestions at the moment. Except to note that if your machine has both internal and external IP addresses, the mailserver probably should whitelist them all.



No worries. I'm setting up a Big Sur VM now for testing. If that doesn't work, I'll try a 10.15 virtual machine.

Will post back results.

Thanks Tim.

-- mike
 
Mike London
Bartender
Posts: 1829
17
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Same problem on clean VM install (Big Sur).
 
Tim Holloway
Saloon Keeper
Posts: 24283
167
Android Eclipse IDE Tomcat Server Redhat Java Linux
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
My condolences. Have you a friendly network expert who can do packet tracing for you? Also, look at mailserver logs (preferably at debug level). You can also set debug-level logging in the webapp, although that probably won't tell you anything.
 
Mike London
Bartender
Posts: 1829
17
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Tim Holloway wrote:My condolences. Have you a friendly network expert who can do packet tracing for you? Also, look at mailserver logs (preferably at debug level). You can also set debug-level logging in the webapp, although that probably won't tell you anything.



I downloaded wireshark and ran it, but its output isn't "intuitive". I'll try to find somebody who can interpret its output.

Thanks Tim.

- mike
 
Saloon Keeper
Posts: 7089
165
  • Likes 1
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
For JavaMail problems I find that turning on debug mode quite often provides sufficient information to pinpoint the problem. See https://javaee.github.io/javamail/FAQ#debug for which flag to set (and the following few questions for additional things to test).
 
Mike London
Bartender
Posts: 1829
17
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Tim Moores wrote:For JavaMail problems I find that turning on debug mode quite often provides sufficient information to pinpoint the problem. See https://javaee.github.io/javamail/FAQ#debug for which flag to set (and the following few questions for additional things to test).



Thanks Tim! I'll do that next.

FWIW, I copied a little JavaMaiil send email hard-coded method into another (non SpringBoot) Tomcat webapp and it worked fine.


Will report back.

Thanks again.

--(bleary eyed) mike
 
Mike London
Bartender
Posts: 1829
17
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Here is the debug output. I replaced some of the personal stuff with << .... >> tags.




Here is a portion of the Javax.mail code if this helps:




----

Is any of this helpful? I don't see anything obvious.

Thanks,

-- mike
 
Tim Moores
Saloon Keeper
Posts: 7089
165
  • Likes 1
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
That looks pretty normal - the client seems to connect successfully to the server, and transmit the message. Only when it tries to close the connection does the exception occur. Odd.
 
Mike London
Bartender
Posts: 1829
17
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Tim Moores wrote:That looks pretty normal - the client seems to connect successfully to the server, and transmit the message. Only when it tries to close the connection does the exception occur. Odd.



That last line is my own e.getMessage() " cause: " + e.getCause() line I'm writing to log4j.

Approaching debug-proof?

Do you still think that wireshark could be helpful?

It's odd that this only happens in the Springboot application but not in a separate war file using the "sparkjava.com" framwork.

Appreciate your replies.

-- mike
 
Mike London
Bartender
Posts: 1829
17
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Just to eliminate the CentOS mail server, I created a iCloud email account, but had the same timeout issues.

The thing that all the failures have in common:

1. Being on the Big Sur 2019 iMac,
2. SpringBoot (two versions tried: 2.4.2 and earlier 2.2.6.RELEASE).

The simpler sparkjava.com framework works fine on the 2019 iMac for sending email.
Sending email using the SpringBoot app works fine once I move it basically to any other computer.

That's all I know for now.

-- mike
 
Tim Holloway
Saloon Keeper
Posts: 24283
167
Android Eclipse IDE Tomcat Server Redhat Java Linux
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Well, again, it does sound like you might need the services of a network expert. About the only other theory I i can propose would be if the reply port was blocked.

Every tcp connection requires 2 network ports. One is the server port on the destination server. The other is the reply port on the source client. The reply port is typically a high-numbered port number assigned at random where responses from the server are returned. It is not a fixed port since a single client might have multiple server connections concurrently active. In fact, web browsers might be running 10 different requests in parallel for a single web page.

So the reply port needs to be visible through the local firewalls. In Linux using iptables, this is covered as part of a pair of stock rules for state=ESTABLISHED and state=RELATED. I don't know what the MacOS equivalents are, but it's possible that these rules aren't set up quite right on the offending machine. It doesn't make sense in that supposedly without that sort of access all connections on the client machine would potentially fail, but it's about my best guess.

Other than that, I suppose that somehow the mail client in the Spring Boot app is subtly different from the stand-alone and it's invoking something that makes the mailserver pause. I am, incidentally, assuming that your problems come when running the offending application directly on the client OS and not in a container, where networking gets an extra layer of complexity.

Also see if you can get the maillog from the mailserver for your connections. It may shed some light.
 
Mike London
Bartender
Posts: 1829
17
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Tim Holloway wrote:Well, again, it does sound like you might need the services of a network expert. About the only other theory I i can propose would be if the reply port was blocked.

Every tcp connection requires 2 network ports. One is the server port on the destination server. The other is the reply port on the source client. The reply port is typically a high-numbered port number assigned at random where responses from the server are returned. It is not a fixed port since a single client might have multiple server connections concurrently active. In fact, web browsers might be running 10 different requests in parallel for a single web page.

So the reply port needs to be visible through the local firewalls. In Linux using iptables, this is covered as part of a pair of stock rules for state=ESTABLISHED and state=RELATED. I don't know what the MacOS equivalents are, but it's possible that these rules aren't set up quite right on the offending machine. It doesn't make sense in that supposedly without that sort of access all connections on the client machine would potentially fail, but it's about my best guess.

Other than that, I suppose that somehow the mail client in the Spring Boot app is subtly different from the stand-alone and it's invoking something that makes the mailserver pause. I am, incidentally, assuming that your problems come when running the offending application directly on the client OS and not in a container, where networking gets an extra layer of complexity.

Also see if you can get the maillog from the mailserver for your connections. It may shed some light.



SOLVED! Well, sort of.

If I downgrade the JavaMail maven library from 1.6.2 to 1.4.4, it works.

I saw some offhanded remark in one of the hundreds of links I was reading and somebody just mentioned that.

Checking the various Maven repos, it's unclear to me what the actual current version is.

In any case, since 1.4.4 works, I'll change my maven imports to:



I can think of no good reason for this probable bug and certainly can't spend another several days trying to figure out how to report it, what's wrong with the library, etc.

I appreciate you both sticking with me on this!

Thanks Tim!

-- mike
 
Tim Holloway
Saloon Keeper
Posts: 24283
167
Android Eclipse IDE Tomcat Server Redhat Java Linux
  • Likes 1
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Congratulations!

Was the working stand-alone app using the 1.6.2 version also?

I don't know if this means that the newer version was trying some sort of fancy tricks that annoyed the server or if there's an actual bug there - and since it's OS-specific, possibly even involving something within the network stack. Would be interesting to investigate if I wasn't busy on other stuff - and, of course, had a Big Sur system. But we'll take a win however we can get it.

Don't forget to put a comment about this in your POM against the day when things get upgraded!

 
Mike London
Bartender
Posts: 1829
17
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Tim Holloway wrote:Congratulations!

Was the working stand-alone app using the 1.6.2 version also?

I don't know if this means that the newer version was trying some sort of fancy tricks that annoyed the server or if there's an actual bug there - and since it's OS-specific, possibly even involving something within the network stack. Would be interesting to investigate if I wasn't busy on other stuff - and, of course, had a Big Sur system. But we'll take a win however we can get it.

Don't forget to put a comment about this in your POM against the day when things get upgraded!



Good question. No, in the standalone class, it's using 1.4.3. No wonder it worked.

I did add a comment in the POM so I won't reapply that 1.6.2 version at some point after I've (hopefully) forgotten about all this "fun".

Thanks Tim....you rock!

-- mike
 
Tim Moores
Saloon Keeper
Posts: 7089
165
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
That's quite odd. In my experience, upgrading JavaMail from version to version has been seemless for a looong time, certainly since the 1.4 days.
 
Tim Holloway
Saloon Keeper
Posts: 24283
167
Android Eclipse IDE Tomcat Server Redhat Java Linux
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Tim Moores wrote:That's quite odd. In my experience, upgrading JavaMail from version to version has been seemless for a looong time, certainly since the 1.4 days.



Unfortunately, I know of no product and no vendor that has been completely immune to an occasional "dud" release. It's possible that none of the testers had a Big Sur machine to run under.
 
Mike London
Bartender
Posts: 1829
17
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Tim Holloway wrote:

Tim Moores wrote:That's quite odd. In my experience, upgrading JavaMail from version to version has been seemless for a looong time, certainly since the 1.4 days.



Unfortunately, I know of no product and no vendor that has been completely immune to an occasional "dud" release. It's possible that none of the testers had a Big Sur machine to run under.




Well put.

-- mike
 
You showed up just in time for the waffles! And this tiny ad:
Building a Better World in your Backyard by Paul Wheaton and Shawn Klassen-Koop
https://coderanch.com/wiki/718759/books/Building-World-Backyard-Paul-Wheaton
reply
    Bookmark Topic Watch Topic
  • New Topic