Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 7 additions & 2 deletions .github/workflows/website.yml
Original file line number Diff line number Diff line change
Expand Up @@ -81,8 +81,13 @@ jobs:
export CFLAGS_wasm32_wasip1="--sysroot=$WASI_SDK_PATH/share/wasi-sysroot"
cd coreutils
# Build the multicall binary for WASI target
# Use --no-default-features to avoid platform-specific dependencies
cargo build --release --target wasm32-wasip1 -p coreutils --no-default-features --features feat_wasm
# Use --no-default-features to avoid platform-specific dependencies.
# feat_diagnostics is part of the default feature set, so it has to be
# asked for explicitly here; without it uucore::diagnostics is the stub
# and the playground would print the plain one-line errors instead of
# the caret reports (browser_wasi_shim reports stderr as a TTY, which is
# what the diagnostics gate on).
cargo build --release --target wasm32-wasip1 -p coreutils --no-default-features --features feat_wasm,feat_diagnostics
test -f target/wasm32-wasip1/release/coreutils.wasm
mkdir -p ../wasm-out
cp target/wasm32-wasip1/release/coreutils.wasm ../wasm-out/uutils.wasm
Expand Down
24 changes: 10 additions & 14 deletions content/blog/2026-08-error-diagnostics.md
Original file line number Diff line number Diff line change
Expand Up @@ -133,8 +133,6 @@ After:
</div>
</div>

[Try it in the playground](https://uutils.org/playground/?cmd=chmod+%27g%2Brw%3Fx%27+fruits.txt).

`sort` keys are short enough that a stray character is easy to miss.

Before:
Expand Down Expand Up @@ -212,8 +210,6 @@ After:
</div>
</div>

[Try it in the playground](https://uutils.org/playground/?cmd=env+-S+%27echo+%24%7B1FOO%7D%27).

`test` builds its expression out of separate arguments. The report echoes the expression on its own, without the `test` in front, and marks the argument that broke it.

Before:
Expand Down Expand Up @@ -376,24 +372,24 @@ That label comes straight from the regex engine and is not translated, since it

### Where it applies

28 utilities use it in 0.11.0. Every example below runs in the playground:
28 utilities use it in 0.11.0. The linked examples run in the playground; the others are for utilities the WebAssembly build does not ship, so try those locally:

| Utility | What the caret points at | Try it |
| -------- | ------------------------ | ------ |
| `test` | the argument that made the expression fail | [`test 7 -eq zap`](https://uutils.org/playground/?cmd=test+7+-eq+zap) |
| `expr` | the argument that made the expression fail | [`expr 9 + foo`](https://uutils.org/playground/?cmd=expr+9+%2B+foo) |
| `chmod` | the failing clause (or character) of an invalid symbolic or octal mode | [`chmod 'g+rw?x' fruits.txt`](https://uutils.org/playground/?cmd=chmod+%27g%2Brw%3Fx%27+fruits.txt) |
| `chmod` | the failing clause (or character) of an invalid symbolic or octal mode | `chmod 'g+rw?x' fruits.txt` |
| `mkdir` | the failing part of the mode given to `-m`/`--mode` | [`mkdir -m u+q mydir`](https://uutils.org/playground/?cmd=mkdir+-m+u%2Bq+mydir) |
| `mkfifo` | the failing part of the mode given to `-m`/`--mode` | [`mkfifo -m u+q mypipe`](https://uutils.org/playground/?cmd=mkfifo+-m+u%2Bq+mypipe) |
| `mknod` | the failing part of the mode given to `-m`/`--mode` | [`mknod -m u+q mydev c 1 3`](https://uutils.org/playground/?cmd=mknod+-m+u%2Bq+mydev+c+1+3) |
| `install`| the failing part of the mode given to `-m`/`--mode` | [`install -m u+q fruits.txt dest`](https://uutils.org/playground/?cmd=install+-m+u%2Bq+fruits.txt+dest) |
| `mkfifo` | the failing part of the mode given to `-m`/`--mode` | `mkfifo -m u+q mypipe` |
| `mknod` | the failing part of the mode given to `-m`/`--mode` | `mknod -m u+q mydev c 1 3` |
| `install`| the failing part of the mode given to `-m`/`--mode` | `install -m u+q fruits.txt dest` |
| `tr` | the part of a set that is at fault (bad class, backwards range, bad repeat count, …) | [`tr 'qw[y-b]' x`](https://uutils.org/playground/?cmd=tr+%27qw%5By-b%5D%27+x) |
| `sort` | the failing part of a `-k`/`--key` or field specification, or of the SIZE given to `-S` | [`sort -k2.3x fruits.txt`](https://uutils.org/playground/?cmd=sort+-k2.3x+fruits.txt) |
| `numfmt` | the failing part of a `--format` or `--field` specification, the value given to `--from`, `--to`, `--from-unit`, `--to-unit`, `--padding` or `--header`, or the input number itself | [`numfmt --format=%q 1000`](https://uutils.org/playground/?cmd=numfmt+--format%3D%25q+1000) |
| `printf` | the failing conversion or escape in the format string | [`printf %5.2c q`](https://uutils.org/playground/?cmd=printf+%255.2c+q) |
| `seq` | the failing conversion in the format given to `-f`/`--format` | [`seq -f %5.2c 1 3`](https://uutils.org/playground/?cmd=seq+-f+%255.2c+1+3) |
| `stat` | the failing directive of a `-c`/`--format` or `--printf` format | [`stat -c %d%.3 fruits.txt`](https://uutils.org/playground/?cmd=stat+-c+%25d%25.3+fruits.txt) |
| `env` | the failing part of a `-S`/`--split-string` string | [`env -S 'echo ${1FOO}'`](https://uutils.org/playground/?cmd=env+-S+%27echo+%24%7B1FOO%7D%27) |
| `stat` | the failing directive of a `-c`/`--format` or `--printf` format | `stat -c %d%.3 fruits.txt` |
| `env` | the failing part of a `-S`/`--split-string` string | `env -S 'echo ${1FOO}'` |
| `dd` | the failing key, value or flag of a `KEY=VALUE` operand | [`dd conv=ucase,zap`](https://uutils.org/playground/?cmd=dd+conv%3Ducase%2Czap) |
| `join` | the failing field of the output format given to `-o` | [`join -o 1.2,2.x fruits.txt fruits.txt`](https://uutils.org/playground/?cmd=join+-o+1.2%2C2.x+fruits.txt+fruits.txt) |
| `cut` | the failing range in the list given to `-b`, `-c`, `-f` or `-F` | [`cut -f 1,4-2 fruits.txt`](https://uutils.org/playground/?cmd=cut+-f+1%2C4-2+fruits.txt) |
Expand All @@ -404,10 +400,10 @@ That label comes straight from the regex engine and is not translated, since it
| `tail` | the failing part of the SIZE given to `-c` or `-n` | [`tail -c 1fb fruits.txt`](https://uutils.org/playground/?cmd=tail+-c+1fb+fruits.txt) |
| `truncate` | the failing part of the SIZE given to `-s`/`--size` | [`truncate -s 10fb fruits.txt`](https://uutils.org/playground/?cmd=truncate+-s+10fb+fruits.txt) |
| `od` | the failing part of the SIZE given to `-j`, `-N`, `-S` or `-w` | [`od -N 3zz fruits.txt`](https://uutils.org/playground/?cmd=od+-N+3zz+fruits.txt) |
| `du` | the failing part of the SIZE given to `-B`/`--block-size` or `-t`/`--threshold` | [`du -B 1fb`](https://uutils.org/playground/?cmd=du+-B+1fb) |
| `df` | the failing part of the SIZE given to `-B`/`--block-size` | [`df -B 1fb`](https://uutils.org/playground/?cmd=df+-B+1fb) |
| `du` | the failing part of the SIZE given to `-B`/`--block-size` or `-t`/`--threshold` | `du -B 1fb` |
| `df` | the failing part of the SIZE given to `-B`/`--block-size` | `df -B 1fb` |
| `ls` | the failing part of the SIZE given to `--block-size` (also `dir` and `vdir`) | [`ls --block-size=1fb`](https://uutils.org/playground/?cmd=ls+--block-size%3D1fb) |
| `stdbuf` | the failing part of the buffering mode given to `-i`, `-o` or `-e` | [`stdbuf -o 6pq head`](https://uutils.org/playground/?cmd=stdbuf+-o+6pq+head) |
| `stdbuf` | the failing part of the buffering mode given to `-i`, `-o` or `-e` | `stdbuf -o 6pq head` |

### Compatibility first

Expand Down
14 changes: 7 additions & 7 deletions content/playground-how-it-works.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,13 +15,13 @@ flowchart LR
A["Browser"] -->|"1. Load page"| B["Zola static site"]
B -->|"2. Fetch core binary"| C["uutils.wasm<br/>(coreutils multicall)"]
A -->|"3. User types command"| D["JavaScript shell"]
D -->|"4a. On-demand fetch<br/>(grep, find, diff, sed, awk…)"| F["Standalone<br/>WASM modules"]
D -->|"4a. On-demand fetch<br/>(grep, find, diff, sed…)"| F["Standalone<br/>WASM modules"]
D -->|"4b. Execute via WASI"| E["WebAssembly runtime"]
F --> E
E -->|"5. Output"| A
</pre>

Everything runs **client-side**. The page loads only the **coreutils multicall binary** up front; the optional standalone modules (`grep`, `find`/`locate`/`updatedb`, `diff`/`cmp`, `sed`, `awk`) are fetched **on demand** the first time you use them — either by clicking their "Load" button or simply by running the command. Once a module is downloaded, it works entirely offline.
Everything runs **client-side**. The page loads only the **coreutils multicall binary** up front; the optional standalone modules (`grep`, `find`/`locate`/`updatedb`, `diff`/`cmp`, `sed`) are fetched **on demand** the first time you use them — either by clicking their "Load" button or simply by running the command. Once a module is downloaded, it works entirely offline.

## Architecture

Expand Down Expand Up @@ -53,7 +53,7 @@ flowchart TB
| **JavaScript shell** | Parses command lines, manages pipes, handles builtins (`help`, `clear`, `cd`, `locale`), and dispatches to WASM. |
| **[browser_wasi_shim](https://github.com/bjorn3/browser_wasi_shim)** | Implements the WASI (WebAssembly System Interface) in JavaScript so that uutils can perform I/O operations. |
| **uutils.wasm** | The Rust coreutils, compiled with the `feat_wasm` feature to a single multicall WASM binary containing 60+ commands. This is the only binary loaded eagerly at page load. |
| **Standalone modules** | Separate uutils projects shipped as their own WASM binaries, **loaded on demand**: `grep.wasm` ([uutils/grep](https://github.com/uutils/grep)), `find.wasm`/`locate.wasm`/`updatedb.wasm` ([uutils/findutils](https://github.com/uutils/findutils)), `diffutils.wasm` providing `diff`/`cmp` ([uutils/diffutils](https://github.com/uutils/diffutils)), `sed.wasm` ([uutils/sed](https://github.com/uutils/sed)), and `awk.wasm` ([uutils/awk](https://github.com/uutils/awk), an early work in progress: `BEGIN` blocks only). |
| **Standalone modules** | Separate uutils projects shipped as their own WASM binaries, **loaded on demand**: `grep.wasm` ([uutils/grep](https://github.com/uutils/grep)), `find.wasm`/`locate.wasm`/`updatedb.wasm` ([uutils/findutils](https://github.com/uutils/findutils)), `diffutils.wasm` providing `diff`/`cmp` ([uutils/diffutils](https://github.com/uutils/diffutils)), and `sed.wasm` ([uutils/sed](https://github.com/uutils/sed)). |
| **Virtual filesystem** | An in-memory filesystem backed by WASI shim `PreopenDirectory`, pre-populated with sample files. Persists across commands within a session. |

## Lifecycle of a Command
Expand Down Expand Up @@ -98,7 +98,7 @@ sequenceDiagram
Key details:

- **Pipeline execution**: each pipe stage is a fresh WASM instantiation. The stdout of one stage becomes the stdin of the next.
- **Command dispatch**: coreutils commands go through `["coreutils", command, ...args]` - the core WASM binary is a multicall binary, similar to BusyBox. Standalone modules (`grep`, `find`, `diff`, `sed`, `awk`…) are invoked **directly by their own name** as `argv[0]`, since each is its own binary rather than a multicall entry.
- **Command dispatch**: coreutils commands go through `["coreutils", command, ...args]` - the core WASM binary is a multicall binary, similar to BusyBox. Standalone modules (`grep`, `find`, `diff`, `sed`…) are invoked **directly by their own name** as `argv[0]`, since each is its own binary rather than a multicall entry.
- **On-demand loading**: if a command lives in a standalone module that hasn't been fetched yet, the shell loads that module first (printing a `loading <module>… done` notice), then runs the command.
- **Path resolution**: relative paths are resolved against a virtual `cwd` maintained by the JS shell.

Expand All @@ -124,7 +124,7 @@ flowchart TB
K -->|No| M["Show prompt"]
</pre>

- Only the **coreutils multicall binary** loads eagerly. The standalone modules (`grep`, `find`/`locate`/`updatedb`, `diffutils`, `sed`, `awk`) are **not** part of this startup fetch.
- Only the **coreutils multicall binary** loads eagerly. The standalone modules (`grep`, `find`/`locate`/`updatedb`, `diffutils`, `sed`) are **not** part of this startup fetch.
- WASM binaries are compiled with `WebAssembly.compileStreaming()` for best performance, with a fallback to `arrayBuffer()` if the server doesn't set the `application/wasm` content-type.
- Commands are disabled until the core binary finishes loading. The terminal shows a loading message and a prompt appears once it's ready.
- The `SharedArrayBuffer` polyfill stub prevents `ReferenceError` in browsers without cross-origin isolation headers.
Expand All @@ -133,7 +133,7 @@ flowchart TB

<pre class="mermaid">
flowchart TB
A["User runs grep/find/diff/sed/awk<br/>(or clicks its Load button)"] --> B{"Module already<br/>compiled?"}
A["User runs grep/find/diff/sed<br/>(or clicks its Load button)"] --> B{"Module already<br/>compiled?"}
B -->|Yes| F["Run command"]
B -->|No| C{"Fetch in flight?"}
C -->|Yes| D["Share the existing<br/>in-flight fetch"]
Expand Down Expand Up @@ -218,7 +218,7 @@ flowchart LR

Utilities are excluded when they depend on OS-level syscalls not available in WASI - for example, `df` needs filesystem stats, `du` needs directory traversal with metadata, and `chown`/`chcon` need permission and SELinux APIs.

> **Note:** `grep`, `find`/`locate`/`updatedb`, `diff`/`cmp`, `sed` and `awk` are **not** part of the coreutils `feat_wasm` set — they live in separate uutils projects ([grep](https://github.com/uutils/grep), [findutils](https://github.com/uutils/findutils), [diffutils](https://github.com/uutils/diffutils), [sed](https://github.com/uutils/sed), [awk](https://github.com/uutils/awk)) and are compiled to their own WASM modules, loaded on demand as described above. (`xargs` is intentionally absent: it must spawn child processes, which the browser WASI sandbox can't do.)
> **Note:** `grep`, `find`/`locate`/`updatedb`, `diff`/`cmp` and `sed` are **not** part of the coreutils `feat_wasm` set — they live in separate uutils projects ([grep](https://github.com/uutils/grep), [findutils](https://github.com/uutils/findutils), [diffutils](https://github.com/uutils/diffutils), [sed](https://github.com/uutils/sed)) and are compiled to their own WASM modules, loaded on demand as described above. (`xargs` is intentionally absent: it must spawn child processes, which the browser WASI sandbox can't do.)

### Multicall Binary: How Command Dispatch Works

Expand Down
10 changes: 2 additions & 8 deletions content/playground.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,6 @@ template = "page.html"
<span class="playground-loaders-label">Extra programs:</span>
</div>

<p class="playground-note"><strong>⚠ <code>awk</code> is a work in progress.</strong> <a href="https://github.com/uutils/awk">uutils awk</a> is at a very early stage: only <code>BEGIN</code> blocks are executed for now — reading records from files or standard input is not implemented yet, so most real awk programs will print nothing or fail. It is here so you can follow along as it grows.</p>

<div class="term term-playground">
<div class="term-bar">
Expand Down Expand Up @@ -69,9 +68,6 @@ Click an example to run it in the terminal:
<button class="playground-example">updatedb; locate names</button>
<button class="playground-example">diff -u shopping-old.txt shopping-new.txt</button>
<button class="playground-example">sed -e 's/banana/🍌/g' -e 's/date/🌴/g' fruits.txt</button>
<button class="playground-example">awk 'BEGIN { print "hello from uutils awk", 2 * 21 }'</button>
<button class="playground-example">awk 'BEGIN { for (i = 1; i &lt;= 5; i++) print i, i * i }'</button>
<button class="playground-example">awk 'BEGIN { print substr("coreutils", 1, 4), toupper("awk") }'</button>
<button class="playground-example">sort -n < numbers.txt | head -3</button>
<button class="playground-example">date</button>
<button class="playground-example">uname -a</button>
Expand Down Expand Up @@ -106,10 +102,8 @@ The following are **shell builtins** implemented in JavaScript:
- `cd` - change the current working directory
- `locale` - show or change the current locale

`grep`, `find`/`locate`/`updatedb`, `diff`/`cmp`, `sed` and `awk` come from their own uutils projects and
are downloaded on demand the first time you use them. Among these, **`awk` is an early work in progress**:
only `BEGIN` blocks are executed today, so anything reading records from a file or standard input will
produce no output.
`grep`, `find`/`locate`/`updatedb`, `diff`/`cmp` and `sed` come from their own uutils projects and
are downloaded on demand the first time you use them.

Some commands (e.g. `chcon`, `runcon`, etc.) are not yet available in the WASM build because they
depend on platform-specific syscalls not fully supported by WebAssembly/WASI.
Expand Down
12 changes: 3 additions & 9 deletions static/js/playground.js
Original file line number Diff line number Diff line change
Expand Up @@ -11,11 +11,11 @@ document.addEventListener("DOMContentLoaded", function() {
initPlayground("wasm-playground");

// Build a "Load" button per optional standalone group (grep, find,
// diffutils, sed, awk). These ship as their own WASM modules and load on
// diffutils, sed). These ship as their own WASM modules and load on
// demand to keep the initial page download light; running a command
// auto-loads its module too (e.g. diff/cmp both come from the diffutils
// module, and find loads find/locate/updatedb together). Groups that are
// still early work in progress (awk) get a "WIP" marker and a tooltip
// module, and find loads find/locate/updatedb together). A group that is
// still early work in progress gets a "WIP" marker and a tooltip
// explaining what does not work yet.
var loaderBar = document.getElementById("playground-loaders");
if (loaderBar && Array.isArray(window.uutilsPrograms)) {
Expand Down Expand Up @@ -159,12 +159,6 @@ document.addEventListener("DOMContentLoaded", function() {
parts.push('sed <a href="' + sedUrl + '"><code>' +
UUTILS_SED_VERSION.short + '</code></a> (' + sedDate + ')');
}
if (typeof UUTILS_AWK_VERSION !== "undefined") {
var awkDate = UUTILS_AWK_VERSION.date.split("T")[0];
var awkUrl = "https://github.com/uutils/awk/commit/" + UUTILS_AWK_VERSION.commit;
parts.push('awk (WIP) <a href="' + awkUrl + '"><code>' +
UUTILS_AWK_VERSION.short + '</code></a> (' + awkDate + ')');
}
if (typeof SITE_VERSION !== "undefined") {
var siteDate = SITE_VERSION.date.split("T")[0];
var siteUrl = "https://github.com/uutils/uutils.github.io/commit/" + SITE_VERSION.commit;
Expand Down
Loading
Loading