How do I read the data from a telnet connection without waiting for the connection to end?

[Nicholas Bostaph]

I'm a web developer that works primarily in cfml (ColdFusion and Lucee), which is a tag based server side language.  Because both of these servers are built upon java, I'm able to use java when necessary, but I have nearly zero knowledge or experience with it.  Unfortunately, nothing in cfml seems to handle telnet at all, and I am trying to write a script to interact with a game server that can only be done over telnet, so into the java I go...

With the help of chatGPT I have cobbled together three cfml functions that use Apache Commons Net to initiate a telnet session.  My functions look like this:

    <cffunction returntype="Struct" name="telnetConnect" access="public" output="no">
		<cfargument name="SID" 			type="string"	required="yes">

		<cfquery name="getServers">
			select	telnetAdd,
					telnetPort
			from	servers
			where 	ID = <cfqueryparam cfsqltype="integer" value="#arguments.SID#">
		</cfquery>

        <cfset retVal			= structNew()>
        <cfset retVal.success 	= true>

        <cftry>

        	<cfparam name="application.telnetClient" default="#structNew()#">
        	<cfparam name="application.telnetData" default="#structNew()#">

			<cfset application.telnetData[#arguments.SID#]	= "">

			<cfscript>
			    // Load the Apache Commons Net TelnetClient class
			    application.telnetClient[#arguments.SID#] = createObject("java", "org.apache.commons.net.telnet.TelnetClient").init();

			    // If there is an existing connection for this server; close it.
			    telnetClose(arguments.SID);

		        // Connect to the Telnet server
		        application.telnetClient[#arguments.SID#].connect(getServers.telnetAdd, getServers.telnetPort);

		        // Get input and output streams
		        application.inputStream[#arguments.SID#] = application.telnetClient[#arguments.SID#].getInputStream();
		        application.outputStream[#arguments.SID#] = application.telnetClient[#arguments.SID#].getOutputStream();
			</cfscript>

        	<cfcatch>
        		<cfset retVal.success 	= false>
        		<cfset retVal.cfcatch	= cfcatch>
        	</cfcatch>
        </cftry>

		<cfreturn retVal />
	</cffunction>


    <cffunction returntype="Struct" name="telnetRead" access="public" output="no">
		<cfargument name="SID" 			type="string"	required="yes">

		<cfquery name="getServers">
			select	telnetAdd,
					telnetPort,
					telnetPword
			from	servers
			where 	ID = <cfqueryparam cfsqltype="integer" value="#arguments.SID#">
		</cfquery>

        <cfset retVal			= structNew()>
        <cfset retVal.success 	= true>

        <cftry>

			<cfscript>
				// Create an InputStreamReader from the InputStream
				inputStreamReader = createObject("java", "java.io.InputStreamReader").init(application.inputStream[#arguments.SID#]);

				// Create a BufferedReader from the InputStreamReader
				bufferedReader = createObject("java", "java.io.BufferedReader").init(inputStreamReader);

				// Create a StringBuilder for the output
				stringBuilder = createObject("java", "java.lang.StringBuilder").init();

				// Read lines from the BufferedReader
				line 	= bufferedReader.readLine();
				x		= 0;
				while (line NEQ "" && x < 5) {
				    // Append the line to the StringBuilder
				    stringBuilder.append(line);
				    
				    // Read the next line
				    line = bufferedReader.readLine();

					x		= x + 1;
				}

				// Get the final output as a string
				output = stringBuilder.toString();
			</cfscript>

			<cfset application.test	= stringBuilder>
			<cfset application.telnetData[#arguments.SID#]	= "#application.telnetData[#arguments.SID#]# #output#">

        	<cfcatch>
        		<cfset retVal.success 	= false>
        		<cfset retVal.cfcatch	= cfcatch>
        	</cfcatch>
        </cftry>

		<cfreturn retVal />
	</cffunction>


    <cffunction returntype="Struct" name="telnetClose" access="public" output="no">
		<cfargument name="SID" 			type="string"	required="yes">

        <cfset retVal			= structNew()>
        <cfset retVal.success 	= true>

        <cftry>

			<cfscript>
		        if (application.telnetClient[#arguments.SID#].isConnected()) {
		            application.telnetClient[#arguments.SID#].disconnect();
		        }
			</cfscript>

        	<cfcatch>
        		<cfset retVal.success 	= false>
        		<cfset retVal.cfcatch	= cfcatch>
        	</cfcatch>
        </cftry>

		<cfreturn retVal />
	</cffunction>

What I'm trying to accomplish is to establish a connection to the game server and send commands.  I will also need to be able to read what the game server sends back.

The above code seems to work to establish the telnet connection and to close it; I can see that on the console of the game server.  However, I don't understand how to read the output of that session and I think my code is all wrong there.  Line 62-87 is what I'm primarily looking for help with here (I provided the rest for context).

When I call the read function I get back nothing but a number, such as 13.  If I call the read function twice it hangs the server until a timeout occurs.  ChatGPT has been no help here and I feel like I'm banging my head against the wall.

Can someone please give me some direction as to what I want to do in the telnetRead function?  Any suggestions would be really appreciated.

[Carey Brown]

It's hard  to tell what parts are actual Java. "NEQ" is obviously not but it's in the middle of stuff that could be. I don't see you closing BufferedReader anywhere, that could be leaving a resource open.

[Tim Holloway]

wow. I though ColdFusion was long, long dead.

Item #1. Putting logic in your markup is generally considered to be a Very Bad Idea. Whether it's Java Scriptlets, PHP, or whatever, separating the presentation from the processing is almost always going to make your life simpler. Since you're using a JVM-based engine, see if you can't push the code into a Java class, instead and let the cfm tags invoke that the way that JavaServer Pages (JSPs) use Exression Language (EL). If Java doesn't appeal, maybe another JVM-based language such as Kotlin or Jython might work.

Item #2. Be sure know where the request is coming from. If you want the backend JVM to do the telnet-talking, the above applies. If you need the remote client to make the connection, you might do better to code in JavaScript.

Item #3. If you want full interactivity you need unbuffered I/O. telnet was designed (as its name implies) to emulate a TeleType machine and TTYs worked with single character transmissions, not packets.

Item #4. Don't forget that telnet has crap security and transmits unencrypted. Don't use it if Bad Things can be done with the Telnet client. If they can, keep the telnet behind a firewall, over an encrypted VPN, use SSH, or at least something.  

[Campbell Ritchie]

Welcome to the Ranch

I don't think this is a “beginning Java®” question, so I shall try moving you to our IO forum.

[Nicholas Bostaph]

Thank you for the responses!

Tim Holloway wrote:
Item #3. If you want full interactivity you need unbuffered I/O. telnet was designed (as its name implies) to emulate a TeleType machine and TTYs worked with single character transmissions, not packets.

Ah, so when I call java.io.BufferedReader on line 67 I'm heading down the wrong path already, and that is waiting for some kind of stop command or for the buffer to fill before continuing?  Google indicates there is no such thing as UnbufferedReader; could you point me to the right subclass to use in its place?  Would InputStreamReader (https://docs.oracle.com/javase%2F8%2Fdocs%2Fapi%2F%2F/java/io/InputStreamReader.html) be the right approach?

Tim Holloway wrote:
Item #4. Don't forget that telnet has crap security and transmits unencrypted. Don't use it if Bad Things can be done with the Telnet client. If they can, keep the telnet behind a firewall, over an encrypted VPN, use SSH, or at least something.  

Unfortunately, this is the only way that is supported to send commands to the game server.

[Paul Clapham]

Yeah, you won't want buffering for a Telnet conversation.  But you do want a Reader and not a Stream, so InputStreamReader will work just fine. (The remark in the docs about "efficiency" isn't aimed at you. Ignore it.)

[Tim Holloway]

Looks like my earlier response failed to post. Paul said much the same thing but I should add that in a Unix/Linux system, you may also need to "stty" the serial device you're talking over as otherwise the OS itself may buffer.

[Nicholas Bostaph]

Thank you for pointing me in the right direction!

I've managed to cobble together an extremely messy, but working cfc for communicating with a telnet server.  Still much cleanup to do, but for anyone else who is trying to do this and happens across this thread, here is the PoC:

<cfcomponent>

    <cffunction returntype="Struct" name="telnetConnect" access="public" output="no">
		<cfargument name="SID" 			type="numeric"	required="yes">

        <cfset retVal			= structNew()>
        <cfset retVal.success 	= true>

        <cftry>
        	<cfparam name="application.telnet" default="#arrayNew()#">
        	<cfparam name="application.telnet[arguments.SID]" default="#structNew()#">
        	<cfparam name="application.telnet[arguments.SID].telnetClient" default="">
        	<cfparam name="application.telnet[arguments.SID].lastData" default="#Now()#">

        	<cfset application.telnet[arguments.SID].telnetData	= "">
        	<cfset application.telnet[arguments.SID].isReading	= false>

			<cfscript>
			    // If there is an existing connection for this server; close it.
			    telnetClose(arguments.SID);
			    sleep(25);

			    // Load the Apache Commons Net TelnetClient class
			    application.telnet[arguments.SID].telnetClient = createObject("java", "org.apache.commons.net.telnet.TelnetClient").init();

		        // Connect to the Telnet server
		        application.telnet[arguments.SID].telnetClient.connect(telnetAdd, telnetPort);

		        // Get input and output streams
		        application.telnet[arguments.SID].inputStream = application.telnet[arguments.SID].telnetClient.getInputStream();
		        application.telnet[arguments.SID].outputStream = application.telnet[arguments.SID].telnetClient.getOutputStream();
			</cfscript>

        	<cfcatch>
        		<cfset retVal.success 	= false>
        		<cfset retVal.cfcatch	= cfcatch>
        	</cfcatch>
        </cftry>

		<cfreturn retVal />
	</cffunction>

    <cffunction returntype="Struct" name="telnetSend" access="public" output="no">
		<cfargument name="SID" 			type="numeric"	required="yes">
		<cfargument name="message"		type="string"	required="yes">

        <cfset retValSend			= structNew()>
        <cfset retValSend.success 	= true>

        <cftry>

			<cfscript>
		        command = "#arguments.message##chr(13)#";
		        application.telnet[arguments.SID].outputStream.write(command.getBytes());
		        application.telnet[arguments.SID].outputStream.flush();
			</cfscript>

        	<cfcatch>
        		<cfset retValSend.success 	= false>
        		<cfset retValSend.cfcatch	= cfcatch>
        	</cfcatch>
        </cftry>

		<cfreturn retValSend />
	</cffunction>

	<cffunction returntype="Struct" name="telnetRead" access="public" output="no">
	    <cfargument name="SID" type="string" required="yes">

	    <cfset var retVal = structNew()>
	    <cfset retVal.success = true>

	    <cftry>
	    	<cfset application.telnet[arguments.SID].isReading	= true>

	        <cfscript>
	            // Create an InputStreamReader from the InputStream
	            inputStreamReader = createObject("java", "java.io.InputStreamReader").init(application.telnet[arguments.SID].inputStream, "UTF-8");
	            
	            // Create a char array for reading
	            charBuffer = createObject("java", "java.lang.reflect.Array").newInstance(createObject("java", "java.lang.Character").TYPE, 1024);
	            charsRead = 0;
	            totalRead = 0;
	            maxRead = 5120; // Max chars to read at a time
	        </cfscript>

	        <cfloop condition="totalRead lt maxRead">
	        	<cfset charsRead = inputStreamReader.read(charBuffer, 0, 1024)>
	        	<cfif charsRead is -1>
	        		<cfbreak>
	        	</cfif>

				<cfset dataChunk = createObject("java", "java.lang.String").init(charBuffer, 0, charsRead)>
				<cfset totalRead += charsRead>

				<cfset application.telnet[arguments.SID].telnetData &= dataChunk>

				<cfif dataChunk contains chr(13)>
					<cfloop list="#dataChunk#" delimiters="#chr(13)#" index="thisLine">
						<cfset recordLine 		= false>
						<cfif thisLine contains "Please enter password"
							or dataChunk contains "{Add other text here}... ">
							<cfset recordLine 	= true>
						</cfif>

						<cfif recordLine>
							<cfquery name="saveChunk">
								insert into ...
							</cfquery>
							<cfset application.telnet[arguments.SID].lastData	= Now()>
						</cfif>
					</cfloop>
				<cfelse>
					<cfset recordLine 		= false>
					<cfif dataChunk contains "Please enter password"
							or dataChunk contains "{Add other text here}... ">
						<cfset recordLine 	= true>
					</cfif>

					<cfif recordLine>
						<cfquery name="saveChunk">
								insert into ...
						</cfquery>
						<cfset application.telnet[arguments.SID].lastData	= Now()>
					</cfif>
				</cfif>
	        </cfloop>

	    	<cfset application.telnet[arguments.SID].isReading	= false>

	        <cfcatch>
	    		<cfset application.telnet[arguments.SID].isReading	= false>
	            <cfset retVal.success = false>
	            <cfset retVal.cfcatch = cfcatch>
	        </cfcatch>
	        <cffinally>
	    		<cfset application.telnet[arguments.SID].isReading	= false>
			</cffinally>
	    </cftry>

	    <cfreturn retVal />
	</cffunction>

    <cffunction returntype="Struct" name="telnetClose" access="public" output="no">
		<cfargument name="SID" 			type="numeric"	required="yes">

        <cfset retValClose			= structNew()>
        <cfset retValClose.success 	= true>

        <cftry>
			<cfscript>
		        if (application.telnet[arguments.SID].telnetClient.isConnected()) {
		            application.telnet[arguments.SID].telnetClient.disconnect();
		        }
			</cfscript>

        	<cfcatch>
        		<cfset retValClose.success 	= false>
        		<cfset retValClose.cfcatch	= cfcatch>
        	</cfcatch>
        </cftry>

		<cfreturn retValClose />
	</cffunction>

</cfcomponent>

[Mike Simmons]

I'm glad you have something working, at least somewhat.  I don't follow all the ColdFusion stuff very well, so I haven't looked super carefully at what you have now.  But, I'm thinking the BufferedReader solution probably wasn't so far off.  I suspect the main problem was around here:

               while (line NEQ "" && x < 5) {
                   // Append the line to the StringBuilder
                   stringBuilder.append(line);
                    
                   // Read the next line
                   line = bufferedReader.readLine();
 
                   x       = x + 1;
               }

The problem was that if the server is waiting for a command from you (before writing at least five lines), then it's stuck waiting forever.  Probably you need to check, after each individual line is read, does that line require some sort of response from you?  If so, call some other code to do whatever needs to be done, before continuing the loop to read the next line.

There are potentially some situations where a BufferedReader and readLine() won't work for you.  But in many cases, it will.  You need to look at: how does the sequence of commands and responses work?  How does one side know when it needs to do something?  Many common protocols are line-based, meaning that once you start reading one character, you can safely continue reading until you've read a whole line, at least, before you might need to send any response.  If that's the case, then BufferedReader and readLine() will work find for you.  However, if you ever need to send a response at a point where you have not received a new line (e.g. if you need to report improper spelling in a sentence which is not yet finished with a "return"), then a line-based solution using readLine() will not work for you.  In that case you need to read the individual characters, and decide after each character whether something has occurred that requires a response from you.  

But, a readLine() solution will probably be simpler, if the protocol allows it.  The good news is, many protocols do.  And it looks like yours does too, based on what I can discern about your code that checks for chr(13), a carriage return.  You're looking for new lines and reacting to those.  Great - chances are, you can use a BufferedReader / readLine() solution that will be cleaner to work with.   Enjoy!