Java Get Ngrams Method - Performance Improvements

[Simon Ritchie]

Hi all,

I’ve written a method to get the number of n-grams of a specified size n from a String.  For example, the number of 2-grams in the sentence “How now brown cow” is 3:

How now
now brown
Brown cow

Here is my method:

private List<String> generateNgrams(List<String> words, int sequenceSize) {
	if (sequenceSize <= 1) {
		return new ArrayList<>(words);
	}

	ArrayList<String> ngrams = new ArrayList<>();
	int numberOfWords = words.size();

	for (int index = 0; index < numberOfWords; index ++) {

		if ((index + sequenceSize - 1) < numberOfWords) {
			int stop = index + sequenceSize;
			StringBuilder ngramWords = new StringBuilder(words.get(index));

			for (int next = index + 1; next < stop; next ++) {
				ngramWords.append(" ").append(words.get(next));
			}
			ngrams.add(ngramWords.toString());
		}
	}
	return ngrams;
}

My concern is that the nested for loop here might adversely affect performance and that there would be a faster, more memory-efficient way to perform this operation.  Are there any suggestions?  Or are there any common libraries out there that perform this function that I've overlooked?