Java Study Guide OCP "Real World Scenaria: Avoiding Stateful Streams"

[Nico van de Kamp]

Hello,

Chapter 143 - Concurrency:

"A stateful lambda expression is one whose result depends on any state that might change during the execution of a pipeline."

public List<Integer> addValues(IntStream source) {
   var data = Collections.synchronizedList(new ArrayList<Integer>());
   source.filter(s -> s % 2 == 0)
      .forEach(i -> { data.add(i); }); // STATEFUL: DON'T DO THIS! 
   return data;
}

What is stateful here? A Intstream is passed and if that change, yes you will get other results. I should not expect anything others. But I don't think that is what they by STATEFUL.
Yes and if you add some operations to the lambda the result will change, but that is also not what is meant here I think.
Why is

forEach(i -> { data.add(i); 

STATEFUL?

Were depends the result on any state? Or what is meant by state here

[Mikalai Zaikin]

In my opinion, the author wanted to demonstrate that a stream should not modify external to pipeline objects.

A data variable in our case.

Ideally pipeline should get a value, reduce it, and return a result, for example, do not accumulate in some external collection.

In this code it's safe, but if we do not know the source of the collection, it can be modified in parallel for example.

Just my thoughts.....

[Stephan van Hulst]

A stateful operation is an operation that causes "side effects".

In this case, the side effect caused by the lambda expression is that the data list is modified. "How is that a side effect?", you may ask. "I intended to modify the list!"

Well, anything other than computing a new value from the input parameter of the lambda expression is considered a side effect. Most operations on a Stream require you to pass a stateless function; or in other words, a "pure function".

Unfortunately, the example in the study guide is wrong. The forEach() method does not require a stateless function. If a stream operation requires a stateless function, it will explicitly say so in the Javadocs. The only requirement on the lambda expression is that it is "non-interfering". A non-interfering function is one that doesn't modify the source collection, i.e. the collection that generates the stream.

Compare the Javadocs for the Stream.map() and Stream.forEach(). You will notice that map() requires a stateless function, forEach() does not.

[Paul Clapham]

As an exercise, consider rewriting that code to use functional programming properly. Creating an empty list and adding to it via the stream is bad form, there's a more straightforward way to produce the desired list.

[Liutauras Vilda]

Even method name addValues() implies bad practices.

A much better name would be:

public static List<Integer> findEven(IntStream source) {
    return source.filter(Test::isEven).boxed().toList();
}

private static boolean isEven(int n) {
    return (n & 1) == 0;
}

[Campbell Ritchie]

Liutauras, please explain the advantages you might get from using & rather than %.

[Liutauras Vilda]

Campbell Ritchie wrote:Liutauras, please explain the advantages you might get from using & rather than %.

We know you know, but maybe other readers don't.

In short, modulus operation uses number division and returns a remainder, and if the number appears to be big, that can be quite intensive computationally.

On the other hand, the form I used uses bitwise AND operation  (remember, numbers in computers are stored in binary format).

So here is how that looks in practice AND'ing two binary numbers where one of those is 1. For even numbers AND'ing any number with 1 you'd always get 0, likewise, for the odd numbers, you'd always get 1.

Remember from boolean algebra:
0 AND 0 is 0
1 AND 0 is 0
0 AND 1 is 0
1 AND 1 is 1

Pay attention to least significant bit (rightmost).
Decimal 1 0001 0001 ---- 0001 (1 so it is odd) Decimal 2 0010 0001 ---- 0000 (0 so it is even) Decimal 3 0011 0001 ---- 0001 Decimal 4 0100 0001 ---- 0000 Decimal 7 0111 0001 ---- 0001
If you have anything to add - please do so.

[Paul Anilprem]

Paul Clapham wrote:Creating an empty list and adding to it via the stream is bad form, there's a more straightforward way to produce the desired list.

Curious to know why. The given code seems fine to me in that resect, if the method is meant to selectively build a list of items from an input stream.

[Paul Clapham]

Paul Anilprem wrote:Curious to know why. The given code seems fine to me in that resect, if the method is meant to selectively build a list of items from an input stream.

Like Mikalai said:

In my opinion, the author wanted to demonstrate that a stream should not modify external to pipeline objects.

Liutauras's post showed the better way that I had in mind.

[Paul Clapham]

Although I have to confess, I have even made database updates from inside stream-processing code.

[Stephan van Hulst]

As I mentioned before, there is no problem with doing any of this from a forEach() method. After all, the entire point of this method is to cause side-effects.

Even so, I would say that when the goal is to populate a new collection, it's idiomatic to use collect() or toList() instead of forEach().

[Campbell Ritchie]

Liutauras Vilda wrote:. . . If you have anything to add - please do so.

Yes: (Minor point which Liutauras mentioned) The & operator will give a slight performance enhancement.
(Major point) What happens if you are looking for odd numbers and you use i % 2 == 1?
[edit]Add underlined text above.

[Liutauras Vilda]

Campbell Ritchie wrote:(Major point) What happens if you are looking for odd numbers and you use i % 2 == 1?

Good I did not use modulus operation, and instead, used bitwise AND.

Using bitwise AND to check whether number even a most habitual approach is to use the following check (n & 1) == 0, likewise, for the odd numbers one can use opposite (n & 1) == 1, and that works as expected and that's what to most (no scientific proof) people's brains would come I suppose.

But then there is modulus operation, which to define whether number is even you'd use i % 2 == 0, but then problem comes because of the habits when you try to define whether number is odd, and as Campbell pointed out, using  i % 2 == 1 would work only in 3/4 cases. Whoever has a book Java Puzzlers by Joshua Bloch they could find out themselves there, but for the others...

There are negative numbers, which half of them are ODD and half of them are EVEN. For the even numbers you get the right outcome only because of coincidence, but for the ODD you get incorrect, because -3 % 2 is equal to -1 and not 1. So the right approach to check for the odd numbers would be i %2 != 0.

...getting few sentences back, so why using bitwise AND we can't fall into this trap? Because in binary, two's complement notation, to define whether number is positive or negative is being used a reserved bit (leftmost), 0 for positive, 1 for negative, so it has no effect on bitwise AND operation outcome (please correct me if my wording is not accurate or even wrong here).

[Campbell Ritchie]

Liutauras Vilda wrote:. . . Java Puzzlers by Joshua Bloch . . .

Now, that is a noce book. Maybe a bit old, but well worth reading.

[Campbell Ritchie]

Liutauras Vilda wrote:. . . using bitwise AND we can't fall into this trap . . .

That applies even more for something like a hash map, where buckets are often chosen by array indices. You can use the formula h & c − 1 where h is the hash code (maybe rehashed) and c is the capacity (=potential number of buckets =length of backing array). Remember that − has a higher precedence than &. This works especially well when c is an exact power of two. Using % would risk passing negative indices to the array and the program will crash with an index out of bounds exception.

[Jeanne Boyarsky]

I stand by what we wrote. This is stateful. It changes the state of data.

data.add(i);

Using a collector instead of forEach() allows Java to better use concurrency to optimize your problem. (And in some cases, to ensure it is in fact thread safe and accurate.)

In real program, I do use forEach() sometimes. But it is for things like parallelizing a bunch of rest api calls; not for making alist.