Distributed Groovy with ProActive: GPars - Part 2

Since we leveraged ProActive to execute remote Groovy closure in the previous blog post, we will, in this one, integrate ProActive with GPars to allow the distribution of concurrent workloads.

GPars is a nice library written in Groovy to develop parallel applications. It leverages closures to provide a nice and concise API, something that we will “soon” benefit from in Java with the Lambda project. GPars covers many concepts of concurrent programming such as concurrent collections, actors, fork/join, agents, STM,... Here we will focus on concurrent collections processing.

GPars in its implementation details relies on Threads, plain old school Java synchronization and amongst other Fork/Join framework. To avoid changing GPars itself, I’ve chosen to implement the distribution as a decoration of the closure that we want to parallelize. The basic API should stay the same and the execution on the client side will keep the same concept, a thread pool that GPars uses to run the closure, except this time it will call a ProActive node to run the closure and get the result back. This simplistic approach is of course limited but it will be enough to illustrate the concept and the potential of mixing Groovy with ProActive.

Let’s start with an example of what GPars can do:

Here we ask Wally’s friends where they are and since they are not very disciplined they can all answer at the same time and will do! We see that they each live in a different thread, yet on the same JVM.

$> [Wally is here: ForkJoinPool-1-worker-1, Wilma is here: ForkJoinPool-1-worker-2, Woof is here: ForkJoinPool-1-worker-3]

Since Wally discovered worm holes (seriously?) his friends are able to travel to remote locations, such as a ProActive node!

Here we ask again where Wally’s friends are and again they answer as they wish. Now they are in different universes, aka different JVMs.

$> [Wally is here: RemoteClosure on rmi://jily.local:1099/PA_JVM1522417505_GCMNode-0, Wilma is here: RemoteClosure on rmi://jily.local:1099/PA_JVM139013877_GCMNode-0, Woof is here: RemoteClosure on rmi://jily.local:1099/PA_JVM1680423209_GCMNode-0]

If you want to get into the gory details of it, you can take a look at ProActivePool where it mimics GParsPool class to add the distributed call to a ProActive node.

With this simple example, we have demonstrated how GPars could be extended using ProActive to actually distribute the parallel computations. Groovy helps us to keep a simple API and to hide the implementation details to the user.

On a side note, we are adding Groovy as a script engine in ProActive latest release. Combined with the new script task, we hope it will help you to build ProActive application faster while still benefiting from a tight integration with Java.