Parallel.For and Silver Bullets

When the Task Parallel Library arrived in .NET, one method caught everyone's imagination: Parallel.For. Replace a for loop with it and your code runs on all cores. It looks like a silver bullet. Like every silver bullet, it works wonderfully on the right target and does damage on the wrong one.

The basic idea

A normal loop runs iterations one after another:

for (int i = 0; i < images.Length; i++)
{
    Process(images[i]);
}

The parallel version hands the iterations to a pool of worker threads:

using System.Threading.Tasks;

Parallel.For(0, images.Length, i =>
{
    Process(images[i]);
});

The framework partitions the range, schedules the pieces across threads and waits until all have finished before continuing. You do not create threads yourself. If you only have a collection, Parallel.ForEach does the same thing.

When it helps

Parallelism pays off when three conditions hold:

1. The iterations are independent. No iteration needs the result of another.
2. There is enough work in each iteration. Each must cost more than the scheduling overhead.
3. The work is CPU-bound, so more cores means more throughput.

Image processing, number crunching, compressing many files and parsing many documents are typical candidates. If the machine has several cores and the loop is long enough, the speed-up can be substantial, though how much depends on your hardware and workload, so measure rather than assume.

When it hurts

Shared mutable state

The classic failure is writing to a shared variable from several iterations:

int total = 0;
Parallel.For(0, numbers.Length, i =>
{
    total += numbers[i];   // race condition!
});

The increment is not atomic. Two threads read the same old value and both write back, so some additions are lost. The result is wrong and different from run to run. The remedies are to avoid sharing, to use Interlocked, or to use the overload with thread-local state that combines results at the end:

long total = 0;
Parallel.For(0, numbers.Length,
    () => 0L,                                  // per-thread starting value
    (i, state, local) => local + numbers[i],   // body
    local => Interlocked.Add(ref total, local)); // combine once per thread

Locks work too, but a lock inside a hot loop can serialize the iterations and erase the gain.

Tiny iterations

If each iteration does almost nothing, the overhead of dispatching work costs more than the work. A loop adding two numbers will usually run slower in parallel. The framework tries to batch iterations, but it cannot work miracles.

I/O-bound work

Waiting on a database or a web service is not CPU work. Blocking many pool threads while they wait can starve the rest of the application and may overwhelm the service you are calling. Asynchronous APIs with limited concurrency are a better fit than parallel loops here.

Ordering and side effects

Iterations run in no guaranteed order. Anything that writes to a list, a file or the console in loop order will produce a jumble. Collections that are not thread-safe will corrupt silently; use the concurrent collections or collect results by index.

Exceptions

If iterations throw, the loop gathers the errors and throws an AggregateException after stopping. Catch and unpack it deliberately, since the exception you expect may be wrapped.

Measure before and after

Do not guess. A tiny harness is enough:

var sw = Stopwatch.StartNew();
RunSequential(data);
Console.WriteLine("Sequential: " + sw.ElapsedMilliseconds + " ms");

sw.Restart();
RunParallel(data);
Console.WriteLine("Parallel:   " + sw.ElapsedMilliseconds + " ms");

Run it several times, with realistic data, on hardware like your target. Include the first run separately, since startup costs can skew it. If the difference is small, keep the simple sequential version; simple code has its own value.

Testing parallel code

Parallel code is hard to test because failures depend on timing. Keep the work function pure and test it on its own, then keep the parallel wrapper thin. Passing the work in as a delegate, as in funcy testing, lets you check the wrapper with a trivial function. For code that touches the clock or other statics, see isolating code with Moles and Fakes. Once you trust the pieces, you can look further at testing beyond units for concurrency-specific problems.

A short checklist

1. Is the loop actually the bottleneck? Profile first.
2. Are iterations independent?
3. Does each iteration do enough work?
4. Is all shared state protected or removed?
5. Did you measure a real improvement?

No silver bullets

Parallel.For is a good tool, and part of a bigger principle: there are no silver bullets, only trade-offs you must understand. It will not fix a slow algorithm, an inefficient query or a design problem. Often a better data structure beats extra cores. Reach for parallelism after the obvious wins, not before, and you will be using it where it earns its keep.