Task Parallelism

Definitions

1. A form of parallel computing in which different tasks, which may run different code, execute at the same time on separate cores or machines. It contrasts with data parallelism, in which the same operation is applied simultaneously to different parts of a dataset, as in SIMD instructions or GPU kernels. A web server handling separate requests concurrently, or a build system compiling independent files, is task-parallel. Programs usually express it as a graph of tasks with dependencies, run by a scheduler over a pool of worker threads rather than one thread per task. Many runtimes use work stealing, where idle workers take queued tasks from busy ones, including Cilk, Intel TBB, Java's ForkJoinPool, .NET's Task Parallel Library, and Rust's Rayon. The fork-join pattern splits work into subtasks and waits for them to finish. Speedup is limited by the longest chain of dependent tasks and by the serial portion of the program, as expressed by Amdahl's law.

   In plain English: Running different jobs at the same time on different processors, rather than splitting one big job into identical pieces.

   Example: "We used task parallelism to run the image resize, DB query, and email send all at once."

Related Terms

• Message Passing
• Parallelism
• Thread
• Web Worker
• Hot Path
• Latency vs Throughput
• Context Switch
• NUMA
• Lock-Free Data Structure
• Thread Pool
• Work Stealing
• Concurrency Control
• Concurrency Model
• Concurrency Pattern
• Lock Contention
• Multi-Threading
• Non-Blocking
• Parallel Processing
• Pool
• Resource Pool
• Thread Starvation
• Threading Model
• Worker Pool
• Heavyweight
• Concurrency Limit