Visual C++ 2008’s /MP flag: "I typically get linear speedups on the compile phase. The link phase is still sequential, but on most projects compilation dominates."

Visual Studio's parallel project builds in Batch Build mode, "though that feature didn’t let you compile multiple files in the same project in parallel."

Excel 2007's parallel recalculation: "Assuming the spreadsheet is large and doesn’t just contain sequential dependencies between cells, it usually scales linearly up to at least 8 cores."

I implored commenters who felt strongly about the benefits of quad-core to point me to multitasking benchmarks that showed a profound difference in performance between 2 and more-than-2 CPU cores. It's curious. The web is awash in zillions of hardware review websites, yet you can barely find any multitasking benchmarks on any of them. I think it's because the amount of multitasking required to seriously load more than two CPU cores borders on the absurd...

"It is, nevertheless, very real and very active in other aspects of computing. It is important to understand that multi-core processors need concurrency, but that doesn't mean concurrent programs. In fact, if you can get concurrent processes and lots of them --  such as in a web server environment -- you are probably better off than having to write concurrent programs."
"I just would like to add that even in enterprise database servers that have a roughly similar architecture, multi-threading is considered important," Haghighat explains. "Initially, some of the major databases were using process-level parallelism as this was common in SMP systems. That is, each client would get a separate process on the server. Soon DB architects realized by moving from process model to thread model, there's a good amount of potential performance gain due to the sharing that happens in the memory subsystem and micro-architecture.  Care for performance justified the burden of making the database servers thread-safe. I would not be surprised to see the same thing in future on the web servers (e.g., PHP) and middleware."

This is an excellent, easily digestible checklist for managed-code programmers. Most interesting is his focus on the .NET Common Language Runtime memory model: "I explicitly permute (often on a whiteboard or in notepad) the sections of the code that involve shared loads and stores, using knowledge of the legal reorderings given our memory model, to see if the code breaks."

A useful resource, and an encouraging sign...

One answer to this problem is to replace those data structures with ones that impose less serialization. For instance, the synchronous queue has been replaced in Java 6 by a better implementation for concurrency.

The other issue is how to hold locks for less time. Goetz suggested moving thread-local computation out of synchronization blocks, "but don’t make them so small as to split atomic operations." Another trick is to replace synchronization counters with AtomicInteger.

Finally, lock splitting can reduce contention: "You often write one very large lock, and sometimes that big fat lock on your hash map is the bottleneck. You can improve things by replacing with finer granularity locks; replace a synchronized Map with ConcurrentHashMap, which has separate locks for each access."