This week's book giveaway is in the HTML Pages with CSS and JavaScript forum.
We're giving away four copies of Testing JavaScript Applications and have Lucas da Costa on-line!
See this thread for details.
Win a copy of Testing JavaScript Applications this week in the HTML Pages with CSS and JavaScript forum!
  • Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other all forums
this forum made possible by our volunteer staff, including ...
Marshals:
  • Campbell Ritchie
  • Bear Bibeault
  • Ron McLeod
  • Jeanne Boyarsky
  • Paul Clapham
Sheriffs:
  • Tim Cooke
  • Liutauras Vilda
  • Junilu Lacar
Saloon Keepers:
  • Tim Moores
  • Stephan van Hulst
  • Tim Holloway
  • fred rosenberger
  • salvin francis
Bartenders:
  • Piet Souris
  • Frits Walraven
  • Carey Brown

Parallel Reduction in Java 8 Streams API

 
Ranch Hand
Posts: 135
5
Eclipse IDE Postgres Database Java
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hi everyone,

I am preparing for my OCP Java SE 8. So far, I have done 2 enthuware mock exams and I got failed those both. I'm kind of disparate now. (Cos I was working with Java almost 3 years now. But I couldn't pass this mock exams at least getting 65%)

Can anyone teach me about the reduce() in case of a parallelSteam()

Below caption is taken from Oracle docs.

The identity value must be an identity for the accumulator function. This means that for all t, accumulator.apply(identity, t) is equal to t. The accumulator function must be an associative function.



I can't understand what this means> "that for all t, accumulator.apply(identity, t) is equal to t."

I don't know how the parallel streams process each element in reduce(). Can someone help me?

Regards,
Jude
 
Ranch Hand
Posts: 145
4
Hibernate Spring Java
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hi Jude,

I don't know how the parallel streams process each element in reduce(). Can someone help me?  



reduce method third version method declaration is

When you are giving arguments to the above method , you must take care of the following points.
1.identity:  This is the initial value to accumulator.
2. accumulator :It takes two values from Stream and performs operation as it is implemented. identity  is the initial value for this.
3.combiner : It works on intermediate results of accumulator . This function should be compatible
accumulator otherwise you can't expect the results.

If you follow these principles when building your reduce() arguments, then the
operations can be performed using a parallel stream and the results will be ordered as they
would be with a serial stream.



In the above example
1.identity : empty string
2.accumulator : (c1,s1) -> c1 + s1
3.combiner : (s2,s3) -> s2 + s3

On parallel streams, the reduce() method works by applying (accumulator ) the reduction to pairs of
elements within the stream to create intermediate values and then combining(combiner) those intermediate
values to produce a final result.If you give different accumulator , combiner then you can't expect the results.

In serial stream, OCP8 was built one
character at a time, in a parallel stream, the intermediate strings OC and P8 could have been
created and then combined.


Hope it helps !
 
Ranch Hand
Posts: 51
1
Eclipse IDE Java
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hello there. I know this is a week old, so please forgive me if you have already cracked this, but just to add to the theoretical explanation offered above (which is quite the same as in the OCP book), to help in understanding how parallel streams are working with reduce, debug always helps... though I appreciate with streams this can be a bit tricky. Augmenting your example below. If you change parallelStream() to stream(), you'll see that combiner is never used. However, with parallel stream, first the accumulator is used (I added a call to a simple "process" method that is just doing some substring within the lambda to illustrate the kind of thing I think you might be doing in real life, though probably still a bit contrived) and then the combiner adds the resulting elements together in the same order as they originally appeared in the stream (though note AFTER having been "processed" in the accumulator). Anyway, if you run the code, hopefully the output should illustrate better what is actually happening (and of course you can play around with it). As an aside, I don't know whether you are familiar with Hadoop, but I think this can be considered analogous to the "map/reduce" paradigm therein (except in a single JVM with multi-threading instead of across a HDFS cluster etc).

 
Jude Niroshan
Ranch Hand
Posts: 135
5
Eclipse IDE Postgres Database Java
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hi Theo,

Appreciate your reply.  I understood how the parallel reductions happens. You have pointed some important facts about reduce().

reduce() has 3 versions. But the most complicated one is 3 argument version which accept:

  • Identity U
  • accumulator BiFunction<U, ? super T, U>
  • combiner BinaryOperator<U>

  • Most important fact that I learnt was, even though accumulators may run in parallel, combiners will always run sequentially which they were appeared to be in the List.
     
    Theo van Kraay
    Ranch Hand
    Posts: 51
    1
    Eclipse IDE Java
    • Mark post as helpful
    • send pies
    • Quote
    • Report post to moderator
    Glad it helped Jude
     
    Saloon Keeper
    Posts: 12165
    258
    • Mark post as helpful
    • send pies
    • Quote
    • Report post to moderator

    Jude Niroshan wrote:Most important fact that I learnt was, even though accumulators may run in parallel, combiners will always run sequentially which they were appeared to be in the List.



    This is NOT true. Combiners will combine partial reductions as if all elements were reduced in order, but this process still occurs in parallel. That's why the combiner function needs to be associative and stateless.

    Let's say you have a stream [1,2,3,4,5,6] and at some point during the reduction you end up with three partial reductions [1,2,3], [4] and [5,6] that need to be combined. It's perfectly legal for the reduce function to first combine [4] and [5,6] before it combines [1,2,3] with [4,5,6].
     
    Theo van Kraay
    Ranch Hand
    Posts: 51
    1
    Eclipse IDE Java
    • Mark post as helpful
    • send pies
    • Quote
    • Report post to moderator

    Stephan van Hulst wrote:

    Jude Niroshan wrote:Most important fact that I learnt was, even though accumulators may run in parallel, combiners will always run sequentially which they were appeared to be in the List.



    This is NOT true. Combiners will combine partial reductions as if all elements were reduced in order, but this process still occurs in parallel. That's why the combiner function needs to be associative and stateless.

    Let's say you have a stream [1,2,3,4,5,6] and at some point during the reduction you end up with three partial reductions [1,2,3], [4] and [5,6] that need to be combined. It's perfectly legal for the reduce function to first combine [4] and [5,6] before it combines [1,2,3] with [4,5,6].



    To be honest, I kind of thought this is what Jude meant (though I agree the language/wording does imply that the combiner will run sequentially). I think she just meant that the combiner will always ensure that items are ultimately combined in the order they first appear in the list (which they will, as long as the function is stateless and associative which is an important point). Perhaps I gave too much latitude, so your clarification was certainly very worthwhile
     
    CLUCK LIKE A CHICKEN! Now look at this tiny ad:
    Building a Better World in your Backyard by Paul Wheaton and Shawn Klassen-Koop
    https://coderanch.com/wiki/718759/books/Building-World-Backyard-Paul-Wheaton
      Bookmark Topic Watch Topic
    • New Topic