Transform a unformatted paragraph to a multiple-line length-limited paragraph

[Ron McLeod]

I want to take an unformatted paragraph of text and transform it in to multiple lines (line breaks on word boundaries only), with the content of each line not exceeding a specified maximum length.  I also optionally want to be able to specify a string to prefix each line (I'm calling this indentation).

I am solving this using a collector which consumes a stream of words and produces a formatted string.  Internally the collector uses a list of StringBuilder objects which are created dynamically during the processing.

Functionally it is working as expected, but I am wanting to get some feedback on the design/implementation.  For example - would you solve this problem differently?  Would you use StringBuilders or something more primitive?

Here's what I did:

import java.util.ArrayList;
import java.util.Collections;
import java.util.List;
import java.util.Set;
import java.util.function.BiConsumer;
import java.util.function.BinaryOperator;
import java.util.function.Function;
import java.util.function.Supplier;
import java.util.stream.Collector;

public class LengthLimitingCollector implements Collector<String, List<StringBuilder>, String> {
	private static final String WORD_DELIMITER = " ";
	private static final int WORD_DELIMITER_LENGTH = WORD_DELIMITER.length();
	private final int maxLineLength;
	private final String lineBreak;
	private final String indentation;

	public LengthLimitingCollector(int maxLineLength, String lineBreak) {
		this(maxLineLength, lineBreak, "");
	}

	public LengthLimitingCollector(int maxLineLength, String lineBreak, String indentation) {
		this.maxLineLength = maxLineLength;
		this.lineBreak = lineBreak;
		this.indentation = indentation;
	}

	@Override
	public Supplier<List<StringBuilder>> supplier() {
		return () -> {
			List<StringBuilder> acc = new ArrayList<>();
			acc.add(new StringBuilder());
			return acc;
		};
	}

	@Override
	public BiConsumer<List<StringBuilder>, String> accumulator() {
		return (acc, ele) -> {
			StringBuilder lastBuilder = acc.get(acc.size() - 1);
			if (lastBuilder.length() + ele.length() + WORD_DELIMITER_LENGTH > maxLineLength) {
				acc.add(new StringBuilder());
				lastBuilder = acc.get(acc.size() - 1);
			}
			if (lastBuilder.length() > 0) {
				lastBuilder.append(WORD_DELIMITER);
			}
			lastBuilder.append(ele);
		};
	}

	@Override
	public BinaryOperator<List<StringBuilder>> combiner() {
		return (left, right) -> {
			left.addAll(right);
			return left;
		};
	}

	@Override
	public Function<List<StringBuilder>, String> finisher() {
		return acc -> {
			StringBuilder resultBuilder = new StringBuilder();
			int lastAccBuilderIndex = acc.size() - 1;
			for (int i = 0; i < lastAccBuilderIndex; i++) {
				resultBuilder.append(indentation).append(acc.get(i)).append(lineBreak);
			}
			resultBuilder.append(indentation).append(acc.get(lastAccBuilderIndex));
			return resultBuilder.toString();
		};
	}

	@Override
	public Set<Characteristics> characteristics() {
		return Collections.emptySet();
	}

}

Test:

import java.util.regex.Pattern;

public class Test {
	public static void main(String[] args) {
		String originalText = "The limit of the line length in 70–80 characters "
				+ "may well be originated from various technical limitations "
				+ "of various equipment. The American teletypewriters could "
				+ "type only 72 CPL, while the British ones even less, 70 CPL.";
		int maxLineLength = 40;
		String lineBreak = "\n";
		String indentation = "     ";

		String formattedText = Pattern.compile("\\s").splitAsStream(originalText)
				.collect(new LengthLimitingCollector(maxLineLength, lineBreak, indentation));

		System.out.println(formattedText);
	}
}

Output:     The limit of the line length in 70–80     characters may well be originated from     various technical limitations of various     equipment. The American teletypewriters     could type only 72 CPL, while the     British ones even less, 70 CPL.

[Paul Clapham]

This sounds like something I did a long time ago. I can't track it down now, maybe it was for work but I don't think so. Anyway I used (or thought of using) a java.text.BreakIterator which sounds a lot like your requirement.

[salvin francis]

I came up with a shorter form using regex:

		String originalText = "The limit of the line length in 70–80 characters "
		        + "may well be originated from various technical limitations "
		        + "of various equipment. The American teletypewriters could "
		        + "type only 72 CPL, while the British ones even less, 70 CPL.";
		String indentation = "     ";
		int maxCharsPerLine = 40 - indentation.length();
		Pattern pattern = Pattern.compile(".{0," + maxCharsPerLine + "}\\b");
		Matcher matcher = pattern.matcher(originalText);

		StringBuilder sb = new StringBuilder();
		while (matcher.find()) {
			sb.append(indentation);
			sb.append(originalText.substring(matcher.start(), matcher.end()));
			sb.append("\n");
		}

		System.out.println(sb);

[Tim Holloway]

In other words, you're looking to implement a bog-standard old-time text formatting utility.

You've got 2 possible strategies here. One is to use a scanner to break the text down into tokens and keep appending tokens into a StringBuilder until you reach the limit, then output the StringBuilder contents as a String. Reset the StringBuilder to empty and start appending from there. There really isn't anything more primitive that StringBuilder in Java.

The other is faster, but maybe a bit more brutal - use a regex to suck up as large a set of characters from the source string(s) as will fill out the StringBuilder buffer, rinse and repeat until you run out of sources.

And, of course, either way, don't forget to flush out the buffer at the end! (I always forget).

The regex way is probably more performant and more likely to preserve non-semantic constructs such as multiple adjacent spaces (depending on how your scanner works). The scanner, on the other hand, can be more intelligent about how it handles tokens. And, unlike regex'es is less likely to injure your sanity. So it's a matter of which suits you better.

[Ron McLeod]

Tim Holloway wrote:You've got 2 possible strategies here. One is to use a scanner to break the text down into tokens and keep appending tokens into a StringBuilder until you reach the limit, then output the StringBuilder contents as a String. Reset the StringBuilder to empty and start appending from there. There really isn't anything more primitive that StringBuilder in Java.

That is basically what I did, but as a stream collector.

[Knute Snortum]

Just a thought (and I having tried it), I don't think any of these solutions handle when a "word" is longer than the maximum characters per line.

[Tim Holloway]

Knute Snortum wrote:Just a thought (and I having tried it), I don't think any of these solutions handle when a "word" is longer than the maximum characters per line.

I'll take "Things that will 'never' happen for $20", Alex.

Yep. Something that I usually do pay attention to when designing algorithms of that sort. Much to Management's annoyance.

The accepted solution is to simply break the work. You can do that crudely, by just snapping the two (or more!) segments apart, or you can back up 1 and append a word-continuation hyphen on the affected line.

OR you can get really fancy and try and backtrack to a syllable boundary. Although assuming your output lines are a civilized length (60 characters or so), the English language doesn't have a lot of potential offenders and therefore syllable-backtracking isn't likely to be worth it. Not so sure about German, though. Different syllabification rules there, anyway, I'd expect. And stuff like code samples is quite another matter.

[Paul Clapham]

Paul Clapham wrote:This sounds like something I did a long time ago. I can't track it down now, maybe it was for work but I don't think so. Anyway I used (or thought of using) a java.text.BreakIterator which sounds a lot like your requirement.

I just remembered why I didn't end up using that class -- it was because my data was going to be rendered by fonts which might not be fixed-width fonts.

[Knute Snortum]

Tim Holloway wrote:Although assuming your output lines are a civilized length (60 characters or so), the English language doesn't have a lot of potential offenders and therefore syllable-backtracking isn't likely to be worth it.

Well, I was thinking of very-long-over-hyphenated-pseudo-adjectives or a list like Democrat/Republican/Independent/Green/Pacific/Socialist/Communist, any list of words that don't have spaces between them.

[Tim Holloway]

Knute Snortum wrote:

Tim Holloway wrote:Although assuming your output lines are a civilized length (60 characters or so), the English language doesn't have a lot of potential offenders and therefore syllable-backtracking isn't likely to be worth it.

Well, I was thinking of very-long-over-hyphenated-pseudo-adjectives or a list like Democrat/Republican/Independent/Green/Pacific/Socialist/Communist, any list of words that don't have spaces between them.

As long as you know what you're getting, though, adding a word-breaker function should be fairly easy. In fact I think I've run into a text formatter or two that specifically supported plug-in word-breakers.