A Counter in StdOut

[Neil Barton]

I want to print a counter from a program to indicate how it is progressing. I want it to say:
"Processing 01" and then update with "Processing 02" but overwriting the previous entry (obviously only the last two chars!).
I have tried this:

int sct = subjects.size();
System.out.println(String.format("%2d",sct));
System.flush();
for (subject S:subjects){
   sct--;
   System.out.println(String.format("\b\b%2d",sct));
   System.flush();
}

Firstly this wipes out the word "processing" when it flushes the buffer the second time.
Secondly it doesn't update every time the program goes through (it seems to be random, perhaps when java is tidying up?)
This program is run from the command line and the output is going there so I don't have the option to put a counter in a window.
Is there a way to do it across all systems? Linux and windows?

[Carey Brown]

There is no platform independent way to backspace over text on different console implementations.
My solution was to come up with a character based ProgressBar that always moves forward till it hits 100%.
Here "Beep" is not platform independent and should be removed.

public class ProgressBarLong
{
	int				percent = 0;
	long			max;
	int				length;
	long			start, stop;
	Beep			beep = new Beep( Beep.INSTRUMENT, 60, Beep.MAXVELOCITY, Beep.DURATION );
	
	public ProgressBarLong( long max, int length )
	{
		this.max = max;
		this.length = length /  4 * 4;
	}
	
	public void printHeader()
	{
		printHeader("");
	}
	
	public void printHeader( String str )
	{
		int			len = length / 4 - 1;
		System.out.print( "|" + "-".repeat( len ) + "+" );
		System.out.print( "-".repeat( len ) + "+" );
		System.out.print( "-".repeat( len ) + "+" );
		System.out.println( "-".repeat( len ) + "+| " + str );
		System.out.print( "|" );
		beep.play();
		start = System.currentTimeMillis();
	}
	
	public void update( long amount )
	{
		int			pct = (int)( amount * length / max );
		if( pct > length )
			pct = length;
		if( pct > percent )
		{
			System.out.print( "=".repeat( pct - percent ) );
			System.out.flush();
			percent = pct;
			if( percent >= length )
			{
				stop = System.currentTimeMillis();
				System.out.println( "| " + duration() );
				beep.play();
			}
		}
	}
	
	public void end()
	{
		if( percent < length )
		{
			stop = System.currentTimeMillis();
			System.out.println( "| " + duration() );
			percent = length;
			beep.play();
		}
	}
	
	private String duration()
	{
		long	millis = stop - start;
		long	hours = millis / (60L * 60L * 1_000L);
		millis %= (60L * 60L * 1_000L);
		
		long	minutes = millis / (60L * 1_000L);
		millis %= (60L * 1_000L);
		
		long	seconds = millis / (1_000L);
		millis %= 1_000L;
		
		return String.format( "%02d:%02d:%02d.%03d", hours, minutes, seconds, millis );
	}
}

[Carey Brown]

A small demo program.

public class ProgressBarDemo
{
	public static void main( String[] args ) throws InterruptedException
	{
		System.out.println( "Copying 20 files" );
		ProgressBarLong pb = new ProgressBarLong( 20L, 32 );
		pb.printHeader( "20  files" );
		for( int i=0 ; i <= 20 ; i++ )
		{
			pb.update( i );
			Thread.sleep(1000);
		}
		
		System.out.println( "\nCopying 50 files" );
		ProgressBarLong pb2 = new ProgressBarLong( 50L, 32 );
		pb2.printHeader( "50  files" );
		for( int i=0 ; i <= 50 ; i++ )
		{
			pb2.update( i );
			Thread.sleep(1000);
		}
	}
}

Screenshot:

[Neil Barton]

Thanks Carey, That looks great. I'll give it a try.

[Tim Holloway]

Don't use println. It's defined to output whatever the local OS/hardware recognizes as carriage-return/newline (CR/LF), dropping the output down to a new line.

Use out.print(), instead, followed by out.flush(). You don't need to explicitly flush the print buffer when using println, since a newline automatically triggers an internal flush(), but out.print() generally is used to build output out of fragments, hence the need to explicitly flush.

[Carey Brown]

@Tim, you make a valid point about println() but it doesn't change the fact that backspaces are not dealt with consistently across platforms.

public class ProgressCounter
{
	public static void main( String[] args ) throws InterruptedException
	{
		System.out.print( "10" );
		for( int i = 10 ; i > 0 ; i-- )
		{
			System.out.printf( "\b\b%2d", i );
			Thread.sleep(500);
		}
	}
}

[Neil Barton]

Thanks Carey, it works exceedingly well!

[Neil Barton]

Thanks Carey, it works exceedingly well!

[Campbell Ritchie]

I got it to work with JShell on Linux (Bash/Fedora39). As Carey sais, you might get different behaviours on different terminals, particularly on Windows®, I recommend you try it on different boxes. You may find that \r (=carriage return) is much more variable than \b (=backspace).
An important part of the procedure is slowing down the output; since you are not doing anything else on the Main thread, Thread.sleep() as Carey showed, works nicely. I had no need to use flush()./* Using JShell. Created dummy local class “Subject” */ void foo() {    class Subject {}    Subject[] subjects = new Subject[100];    Arrays.fill(subjects, new Subject());    int sct = subjects.length;    System.out.print("Processing    ");    for (Subject S : subjects)    {        System.out.printf("\b\b\b%3d", sct--);        try        {            Thread.sleep(333L);        }        catch (InterruptedException ex)        {            ex.printStackTrace();        }    }    System.out.print(String.format("\b\b%2d", sct)); }Note that you would want your processing in that loop if you want a correct display of progress. Also, you will only get a legible display if processing takes several hundred milliseconds or more. If each Subject is processed in, say, 1μs, your progress indicator will move too fast to be legible, and displaying the progress will probably take much much longer than the processing.

[Tim Holloway]

Control characters are handled by the output device driver. In Unix-like OS's the driver emulates an old-time TTY or CRT screen device and which one is used is the user's preference. Linux tended to default to emulating the DEC VT-100, for example, although I think that there's nuances to that these days. Control characters were also subject to a translation process called Termcaps, though Termcaps is long obsolete and I forget what the replacement's called. Terminfo, I believe.

And if that wasn't complicated enough, some terminals can select proportional-width fonts. Since the standard terminals don't track characters once rendered to the display, a backspace cannot accurately re-position. If a backspace moves back 8 pixels, but some characters are 6 pixels wide and some are 10, it can get messy fast, so it's important to ensure that you're outputting a fixed-width font, And if you think that's bad, tab characters are even worse.

CR is generally going to do what you expect, but if you use it to overwrite all or part of a line, again, proportional fonts can ding you. The character widths of "-" versus (often used as progress markers is often different for example.

[Neil Barton]

Thanks Tim & Carey, it gets more interesting...
I copied the code into my app and called it as my processing was running. It worked perfectly on my development PC.

Processing Subjects 20220623.TXT...Loading type : 15 35 points found.
Doing PostCaptureCalcs
|-------------------+-------------------+-------------------+-------------------+| 35  points
|================================================================================| 00:00:13.224
:Done

Wonderful, exactly what I wanted. I copied the new code across to the production machine which is also a W10 machine. When I run on this machine I want to monitor what is going on so the command is:
java -jar processSubjects.jar > log.txt 2>&1
Then I start a Powershell window and run: Get-Content log.txt -wait
Because I'd already run a bit the output was the same as above until it started processing when it did this:

Processing Subjects 20220623.TXT...Loading type : 15 35 points found.
Doing PostCaptureCalcs
|-------------------+-------------------+-------------------+-------------------+| 35  points
|================================================================================| 00:00:13.224
:Done
Processing Subjects 20220624.TXT...Loading type : 15 48 points found.
Doing PostCaptureCalcs
|-------------------+-------------------+-------------------+-------------------+| 48  points
|==============================
===
==
=
====
===
==
=
====
=
====
===
==
=
==
==
===
==
=
==
==
=
==
==| 00:00:21.201
:Done

So it seems the -wait is screwing it up! Get-content puts it out properly but as soon as it's receiving output as it runs it cannot process it correctly. I don't know Powershell well enough (& none of the places I've looked knows either) but is there some sort of setting you can change in Powershell to stop this happening?

[Carey Brown]

If it were me, I'd try to write a 30 line program that reads from a file and writes to System.out. It may not be any better but worth a shot.

[Tim Holloway]

Well, if I'm seeing what I think I'm seeing, it looks like PowerShell auto-flushes the output buffer after a fixed interval. And compounds the offence by adding an unsolicted newline sequence in the process.

You'll likely need to look for a shell window configuration parameter to override that.

For Unix systems, the culprit would most probably be in the "stty" command, but I've no clue what PowerShell may demand.

[Carey Brown]

You could try a command window (cmd.exe) instead of PowerShell and see if that helps.

[Neil Barton]

How do you replicate tail -f in a cmd window? (which is what I'm trying to do with PShell)

[Tim Holloway]

Neil Barton wrote:How do you replicate tail -f in a cmd window? (which is what I'm trying to do with PShell)

Urk.

It's one thing to use control characters on direct stdout and quite another to expect them to work when running it through tail.

I'm fairly sure that there's a tail command for DOS, just that DOS doesn't come with a lot of the amenities that Unix/Linus and later Powershell does.

[Neil Barton]

Thanks Tim,
I'm sure I looked a while ago but it must have been what my wife calls 'man looking' as I didn't find the unixUtils here: https://sourceforge.net/projects/unxutils/

The tail in this package fulfils my requirements completely and stops me having to mess about with PShell. Win win.

Thanks for your help folks,
Neil