[PR #127] [MERGED] add forkrun #3819

New Issue

GiteaMirror · 2026-07-10T17:52:36-05:00

GiteaMirror commented

2026-07-10 17:52:36 -05:00

📋 Pull Request Information

Original PR: https://github.com/awesome-lists/awesome-bash/pull/127
Author: @jkool702
Created: 7/26/2024
Status: ✅ Merged
Merged: 7/26/2024
Merged by: @hyperupcall

Base: master ← Head: master

📝 Commits (3)

1fbfbef add forkrun
ac16ca7 add forkrun
595676d Update README.md

📊 Changes

1 file changed (+1 additions, -0 deletions)

View changed files

📝 README.md (+1 -0)

📄 Description

New App Submission

I've read the contribution guidelines.

Repo or homepage link: https://github.com/jkool702/forkrun

Description: A pure-bash tool for running code in parallel. Similar in syntax and speed to xargs -P but with more features and native bash function support.

Why I think it's awesome: forkrun is the fastest "loop parallelizer" ever written in bash. In many situations is is even slightly faster than the fastest invocation of xargs -P, and almost always a good bit faster than parallel. It also includes several nice features missing from xargs, such as "ordering output the same as if inputs were run sequentially" and "passing inputs to whatever you are parallelizing via its stdin". It also natively supports using bash functions, making it trivially easy to wrap complex multistep tasks in a bash function and parallelize it.

Forkrun is so fast because it doesnt fork individual calls - rather it initially forks several persistent bash coprocs and distributes stdin to them. these coprocs each run an infinite loop (until an end condition is met) that reads data and runs it through whatever you are parallelizing. Forkrun also uses a novel and extremely efficient IPC method to distribute stdin to these coprocs without needing a separate process to actively manage it via a tmpfile on a tmpfs and a shared reading file descriptor. This tmpfile is simultaneously accesses by 3 different processes, though importantly these processes all are accessing the file at different byte offsets.

In a general sense, forkrun has been optimized considerably more than any other bash function Ive ever come across. Sometimes this comes at the cost of code readability, but it allows forkrun to directly challenge and age-old conception that "bash will always run things slowly"...it is rather hard to support that viewpoint when forkrun is parallelizing stuff faster than xargs (which is compiled C) is. (see the "hyperfine benchmarks" folder in the forkrun repo for more details and extensive benchmarking)

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/awesome-lists/awesome-bash/pull/127 **Author:** [@jkool702](https://github.com/jkool702) **Created:** 7/26/2024 **Status:** ✅ Merged **Merged:** 7/26/2024 **Merged by:** [@hyperupcall](https://github.com/hyperupcall) **Base:** `master` ← **Head:** `master` --- ### 📝 Commits (3) - [`1fbfbef`](https://github.com/awesome-lists/awesome-bash/commit/1fbfbefa7633b5e7c820e4022f29233cd7fb7880) add forkrun - [`ac16ca7`](https://github.com/awesome-lists/awesome-bash/commit/ac16ca7412da0cb457d5b1da0c4efa60dbbf3046) add forkrun - [`595676d`](https://github.com/awesome-lists/awesome-bash/commit/595676dde69442e22558d9518a38ce373e53ae74) Update README.md ### 📊 Changes **1 file changed** (+1 additions, -0 deletions) <details> <summary>View changed files</summary> 📝 `README.md` (+1 -0) </details> ### 📄 Description  #### New App Submission - [x] I've read the [contribution guidelines](https://github.com/awesome-lists/awesome-bash/blob/master/contributing.md). **Repo or homepage link:** https://github.com/jkool702/forkrun **Description:** A pure-bash tool for running code in parallel. Similar in syntax and speed to `xargs -P` but with more features and native bash function support. **Why I think it's awesome:** `forkrun` is the fastest "loop parallelizer" ever written in bash. In many situations is is even slightly faster than the fastest invocation of `xargs -P`, and almost always a good bit faster than parallel. It also includes several nice features missing from xargs, such as "ordering output the same as if inputs were run sequentially" and "passing inputs to whatever you are parallelizing via its stdin". It also natively supports using bash functions, making it trivially easy to wrap complex multistep tasks in a bash function and parallelize it. Forkrun is so fast because it doesnt fork individual calls - rather it initially forks several persistent bash coprocs and distributes stdin to them. these coprocs each run an infinite loop (until an end condition is met) that reads data and runs it through whatever you are parallelizing. Forkrun also uses a novel and extremely efficient IPC method to distribute stdin to these coprocs without needing a separate process to actively manage it via a tmpfile on a tmpfs and a shared reading file descriptor. This tmpfile is simultaneously accesses by 3 different processes, though importantly these processes all are accessing the file at different byte offsets. In a general sense, forkrun has been optimized considerably more than any other bash function Ive ever come across. Sometimes this comes at the cost of code readability, but it allows forkrun to directly challenge and age-old conception that "bash will always run things slowly"...it is rather hard to support that viewpoint when forkrun is parallelizing stuff faster than xargs (which is compiled C) is. (see the "hyperfine benchmarks" folder in the forkrun repo for more details and extensive benchmarking) --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

GiteaMirror added the pull-request label 2026-07-10 17:52:36 -05:00

GiteaMirror closed this issue

2026-07-10 17:52:42 -05:00

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: github-starred/awesome-bash#3819