ParallelR—Parallelizing Loops

One common approach to parallelization is to see if the iterations within a loop can be performed independently, and if so, then try to run the iterations concurrently rather than sequentially. ParallelR 2.0 includes two packages, foreach and iterators, that can help you do this loop parallelization quickly and easily. The new foreach command provides a looping construct that can be viewed as a hybrid of the standard for loop and lapply function.

 

A key innovation of foreach is migration of the portions of the R environment that are required by the loop statement to worker R processes. The practical implication is that R programmers and users don't need to explicitly define functions and variables that may be required by the worker processes. The result is that parallel programs look and run similarly to their sequential counterparts, making it easier to migrate from sequential to parallel without extensive re-coding.

 

Simple foreach Example

# Load the Parallel R stack
require ('doNWS')

# We define the function f in our local environment
f <- function (x) { sqrt (x) }

# Start up two R worker processes and register them with foreach/parallel version
setSleigh (sleigh (workerCount=2))
registerDoNWS ()

# Run a simple foreach loop in parallel on the two workers
foreach (j=1:9, .combine=c) %dopar% f(j)

# Note that the workers use the function f from our local environment, even though it was not explicitly
# defined on the workers!

 

Foreach is open-source and a variety of sequential and parallel back-ends are available. Foreach provides a way for package authors to write sequential code that can easily use parallel computing when available--we encourage authors to use foreach where appropriate.

 

Contact us for more information and pricing on fully supported REvolution R Enterprise.

Have questions regarding any product or service? Contact our sales team for more information.