Editing Thread pool

{{Short description|Software design pattern}}
[[File:Thread pool.svg|thumb|400px|A sample thread pool (green boxes) with waiting tasks (blue) and completed tasks (yellow)]]

In [[computer programming]], a '''thread pool''' is a [[software design pattern]] for achieving [[Concurrency (computer science)|concurrency]] of execution in a computer program. Often also called a '''replicated workers''' or '''worker-crew model''',<ref>Garg, Rajat P. & Sharapov, Ilya ''Techniques for Optimizing Applications - High Performance Computing'' Prentice-Hall 2002, p. 394</ref> a thread pool maintains multiple [[thread (computer science)|threads]] waiting for [[task (computers)|tasks]] to be allocated for [[concurrent computing|concurrent]] execution by the supervising program. By maintaining a pool of threads, the model increases performance and avoids latency in execution due to frequent creation and destruction of threads for short-lived tasks.<ref>{{Cite book |first=Allen |last=Holub |authorlink=Allen Holub |title=Taming Java Threads |publisher=Apress |year=2000 |page=209}}</ref> Another good property - the ability to limit system load, when we use fewer threads than available. The number of available threads is tuned to the computing resources available to the program, such as a parallel  task queue after completion of execution.

==Performance==
The size of a thread pool is the number of threads kept in reserve for executing tasks. It is usually a tunable parameter of the application, adjusted to optimize program performance.<ref name="ACM, Thread pool size" /> Deciding the optimal thread pool size is crucial to optimize performance.

One benefit of a thread pool over creating a new thread for each task is that thread creation and destruction overhead is restricted to the initial creation of the pool, which may result in better [[performance tuning|performance]] and better system [[Stability model|stability]].  Creating and destroying a thread and its associated resources can be an expensive process in terms of time. An excessive number of threads in reserve, however, wastes memory, and context-switching between the runnable threads invokes performance penalties.  A socket connection to another network host, which might take many CPU cycles to drop and re-establish, can be maintained more efficiently  by associating it with a thread that lives over the course of more than one network transaction.

Using a thread pool may be useful even putting aside thread startup time.  There are implementations of thread pools that make it trivial to queue up work, control concurrency and sync threads at a higher level than can be done easily when manually managing threads.<ref>{{Cite web | url=https://doc.qt.io/qt-5/qthreadpool.html | title=QThreadPool Class &#124; Qt Core 5.13.1}}</ref><ref>{{Cite web | url=https://github.com/vit-vit/ctpl |title =GitHub - vit-vit/CTPL: Modern and efficient C++ Thread Pool Library.|website =[[GitHub]]|date = 2019-09-24}}</ref>  In these cases the performance benefits of use may be secondary.

Typically, a thread pool executes on a single computer. However, thread pools are conceptually related to [[server farm]]s in which a master process, which might be a thread pool itself, distributes tasks to worker processes on different computers, in order to increase the overall throughput. [[Embarrassingly parallel]] problems are highly amenable to this approach.{{cn|date=December 2016}}

The number of threads may be dynamically adjusted during the lifetime of an application based on the number of waiting tasks. For example, a [[web server]] can add threads if numerous [[web page]] requests come in and can remove threads when those requests taper down.{{disputed inline|reason=This sounds more like pre-spawning than like a pool pattern.|date=December 2015}} The cost of having a larger thread pool is increased resource usage. The algorithm used to determine when to create or destroy threads affects the overall performance:
* Creating too many threads wastes resources and costs time creating the unused threads.
* Destroying too many threads requires more time later when creating them again.
* Creating threads too slowly might result in poor client performance (long wait times).
* Destroying threads too slowly may starve other processes of resources.

== In languages ==
In [[Bash (Unix shell)|bash]] implemented by <code>--max-procs</code> / <code>-P</code> in [[xargs]], for example:
<syntaxhighlight lang="bash">
# Fetch 5 URLs in parallel
urls=(
    "https://example.com/file1.txt"
    "https://example.com/file2.txt"
    "https://example.com/file3.txt"
    "https://example.com/file4.txt"
    "https://example.com/file5.txt"
)

printf '%s\n' "${urls[@]}" | xargs -P 5 -I {} curl -sI {} | grep -i "content-length:"
</syntaxhighlight><ref>{{Cite web |last=Shved |first=Paul |date=2010-01-07 |title=Easy parallelization with Bash in Linux |url=http://coldattic.info/post/7/ |access-date=2025-01-26 |website=coldattic.info |language=en}}</ref><ref>{{Cite web |title=xargs(1) - Linux manual page |url=https://www.man7.org/linux/man-pages/man1/xargs.1.html |access-date=2025-01-26 |website=www.man7.org}}</ref><ref>{{Cite web |title=Controlling Parallelism (GNU Findutils 4.10.0) |url=https://www.gnu.org/software/findutils/manual/html_node/find_html/Controlling-Parallelism.html |access-date=2025-01-26 |website=www.gnu.org}}</ref>

In [[Go (programming language)|Go]], called worker pool:
<syntaxhighlight lang="go">
package main

import (
    "fmt"
    "time"
)

func worker(id int, jobs <-chan int, results chan<- int) {
    for j := range jobs {
        fmt.Println("worker", id, "started  job", j)
        time.Sleep(time.Second)
        fmt.Println("worker", id, "finished job", j)
        results <- j * 2
    }
}

func main() {
    const numJobs = 5
    jobs := make(chan int, numJobs)
    results := make(chan int, numJobs)

    for w := 1; w <= 3; w++ {
        go worker(w, jobs, results)
    }

    for j := 1; j <= numJobs; j++ {
        jobs <- j
    }
    close(jobs)

    for a := 1; a <= numJobs; a++ {
        <-results
    }
}
</syntaxhighlight>

It will print:
<syntaxhighlight lang="console">
$ time go run worker-pools.go 
worker 1 started  job 1
worker 2 started  job 2
worker 3 started  job 3
worker 1 finished job 1
worker 1 started  job 4
worker 2 finished job 2
worker 2 started  job 5
worker 3 finished job 3
worker 1 finished job 4
worker 2 finished job 5

real    0m2.358s
</syntaxhighlight><ref>{{Cite web |title=Go by Example: Worker Pools |url=https://gobyexample.com/worker-pools |access-date=2021-07-27 |website=gobyexample.com}}</ref><ref>{{Cite web |title=Effective Go - The Go Programming Language |url=https://golang.org/doc/effective_go#channels |access-date=2021-07-27 |website=golang.org |quote=another approach that manages resources well is to start a fixed number of handle goroutines all reading from the request channel. The number of goroutines limits the number of simultaneous calls to process}}</ref><ref>{{Cite web |title=The Case For A Go Worker Pool — brandur.org |url=https://brandur.org/go-worker-pool |access-date=2021-07-27 |website=brandur.org |quote=Worker pools are a model in which a fixed number of m workers (implemented in Go with goroutines) work their way through n tasks in a work queue (implemented in Go with a channel). Work stays in a queue until a worker finishes up its current task and pulls a new one off.}}</ref>

==See also==
{{Portal|Computer programming}}
* [[Asynchrony (computer programming)]]
* [[Object pool pattern]]
* [[Concurrency pattern]]
* [[Grand Central Dispatch]]
* [[Parallel Extensions]] (.NET)
* [[Parallelization]]
* [[Server farm]]
* [[Staged event-driven architecture]]

==References==
{{Reflist|30em|refs=

<ref name="ACM, Thread pool size" >{{Cite journal |journal=ACM SIGOPS Operating Systems Review | title=Analysis of optimal thread pool size | author=Yibei Ling | author2=Tracy Mullen | author3=Xiaola Lin |volume=34 |issue=2 |date=April 2000 | pages=42–55 |doi = 10.1145/346152.346320| s2cid=14048829 }}</ref>
}}

==External links==
* "[https://web.archive.org/web/20080207124322/http://today.java.net/pub/a/today/2008/01/31/query-by-slice-parallel-execute-join-thread-pool-pattern.html Query by Slice, Parallel Execute, and Join: A Thread Pool Pattern in Java]" by Binildas C. A.
* "[https://usf-cs272-spring2022.github.io/files/Thread%20Pools%20and%20Work%20Queues.pdf Thread pools and work queues]" by Brian Goetz
* "[https://www.codeproject.com/Articles/3631/A-Method-of-Worker-Thread-Pooling A Method of Worker Thread Pooling]" by Pradeep Kumar Sahu
* "[https://www.codeproject.com/Articles/3607/Work-Queue Work Queue]" by Uri Twig: C++ code demonstration of pooled threads executing a work queue.
* "[http://www.codeproject.com/Articles/6863/Windows-Thread-Pooling-and-Execution-Chaining Windows Thread Pooling and Execution Chaining]"
* "[http://www.codeproject.com/KB/threads/smartthreadpool.aspx Smart Thread Pool]" by Ami Bar
* "[http://msdn.microsoft.com/en-us/library/ms973903.aspx Programming the Thread Pool in the .NET Framework]" by David Carmona
* "[http://today.java.net/pub/a/today/2008/10/23/creating-a-notifying-blocking-thread-pool-executor.html Creating a Notifying Blocking Thread Pool in Java]" by Amir Kirsh
* "[http://www.ibm.com/developerworks/aix/library/au-threadingpython/ Practical Threaded Programming with Python:  Thread Pools and Queues]" by Noah Gift
* "[http://www.cs.wustl.edu/~schmidt/PDF/OM-01.pdf Optimizing Thread-Pool Strategies for Real-Time CORBA]" by Irfan Pyarali, Marina Spivak, [[Douglas C. Schmidt]] and Ron Cytron
* "[http://doi.acm.org/10.1145/1753196.1753218 Deferred cancellation. A behavioral pattern]" by Philipp Bachmann
* "[https://arxiv.org/abs/2105.00613 A C++17 Thread Pool for High-Performance Scientific Computing]" by Barak Shoshany

{{Design Patterns patterns}}

[[Category:Threads (computing)]]
[[Category:Software design patterns]]
[[Category:Concurrent computing]]
[[Category:Parallel computing]]