C#: Parallel.For with Interlocked

Spread loop iterations over cores and combine results safely with an atomic add.

using System;
using System.Linq;
using System.Threading;
using System.Threading.Tasks;

class Program {
  static void Main() {
    long total = 0;
    Parallel.For(1, 1001, i => {
      Interlocked.Add(ref total, (long)i * i);
    });
    Console.WriteLine("Parallel.For: " + total);

    long viaLinq = Enumerable.Range(1, 1000).AsParallel().Sum(i => (long)i * i);
    Console.WriteLine("PLINQ:        " + viaLinq);
  }
}

Output

Parallel.For: 333833500
PLINQ:        333833500

Explanation

Parallel.For runs the iterations of a loop on several threads. The body must be safe to run concurrently: iterations may not depend on each other, and shared variables must be protected. Adding to one shared total from every iteration is a data race if done with a plain +=.

Interlocked.Add performs the addition as one atomic operation, so no update is lost. PLINQ is a higher-level alternative: AsParallel().Sum combines partial sums for you. Both give the same exact answer here because integer addition is associative, so the order of the additions does not matter.

Explore the dry run

Guided walkthroughReady

1using System;

2using System.Linq;

3using System.Threading;

4using System.Threading.Tasks;

6class Program {

7 static void Main() {

8 long total = 0;

9 Parallel.For(1, 1001, i => {

10 Interlocked.Add(ref total, (long)i * i);

11 });

12 Console.WriteLine("Parallel.For: " + total);

14 long viaLinq = Enumerable.Range(1, 1000).AsParallel().Sum(i => (long)i * i);

15 Console.WriteLine("PLINQ: " + viaLinq);

16 }

17}

Follow the example

Start to inspect the source, values, and output. These prepared steps explain the published sample; grouped loop steps describe multiple iterations.

View the complete dry-run table

Prepared trace of the published program. Some rows group repeated iterations.
What happens	State after step	Output so far
A shared total updated from many threads.	`total = 0`	—
Each iteration adds its square atomically, so no addition is lost.	`total grows by i² per iteration`	—
The loop returns only when every iteration is done.	`total = 333833500`	Parallel.For: 333833500
PLINQ computes the same sum with partial sums per thread.	`total = 333833500`	Parallel.For: 333833500 PLINQ: 333833500

Cost and common mistakes

O(n) work with ideal elapsed time O(n / p).

total += i * i inside Parallel.For is a data race and usually gives a smaller, changing result.
Contending on one atomic variable limits the speed-up. Thread-local partial sums combined at the end scale better, which is what PLINQ does.
Floating-point addition is not associative, so parallel sums of doubles can differ slightly from a serial sum.

Try it yourself

What is the sum of squares from 1 to 10?

Reveal answer and reasoning

385, from 10 · 11 · 21 / 6.

C#: Parallel.For with Interlocked

Explanation

Follow the example

Cost and common mistakes

Try it yourself

Further reading