perf: reduce memory usage of Command.output()#33335
Merged
Merged
Conversation
The `Command.output()` path was routing stdout/stderr through `readableStreamForRidUnrefable` + `readableStreamCollectIntoUint8Array`, which leaked native memory from the ReadableStream/ByteStream controller machinery. Over 30k iterations, RSS grew from ~70MB to ~2GB while heap stayed flat at ~18MB. The fix bypasses the ReadableStream layer entirely for `Command.output()`. Instead of creating streams just to immediately collect all their data, `spawnInner` now reads directly from the resource IDs using `core.read` in a simple loop. This also avoids creating a ChildProcess instance, eliminating the `#status` promise chain that captured `this` and prevented GC of native resources. Before: 30k iterations -> RSS ~2GB After: 30k iterations -> RSS ~215MB (stable) Closes #33334 Closes #24674 Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
109ab6a to
7e547e7
Compare
miracatbot
approved these changes
Apr 20, 2026
miracatbot
left a comment
There was a problem hiding this comment.
LGTM.
I took a close look at the new Command.output() fast path in ext/process/40_process.js.
The change makes sense to me:
- it bypasses the
ReadableStreamwrapper machinery for the hotCommand.output()case, - reads stdout/stderr directly with
core.read()in a simple loop, - still waits for process completion in parallel,
- and preserves the existing user-facing behavior around piped vs non-piped stdio.
I didn't spot a concrete correctness issue in the new resource-lifetime or output-collection logic, and the motivation / repro is compelling.
nathanwhit
approved these changes
Apr 20, 2026
Member
nathanwhit
left a comment
There was a problem hiding this comment.
I think the PR description isn't quite right but the change is a perf improvement and seems right, so LGTM
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
bartlomieju
pushed a commit
that referenced
this pull request
May 14, 2026
`Command.output()`'s fast path (`spawnInner` in `ext/process/40_process.js`) bypasses `ChildProcess`. When that path was introduced in #33335 it silently dropped the `signal` option during destructuring, so passing an `AbortSignal` to `new Deno.Command()` and then calling `.output()` no longer killed the child when the signal fired. Fixes #34066 Co-authored-by: lunadogbot <lunadogbot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes RSS memory growth in
Deno.Command.output()when called in a loop. Over 30k iterations, RSS grew from ~70MB to ~2GB while JS heap stayed flat at ~18MB.The
Command.output()path was creatingReadableStreamwrappers (viareadableStreamForRidUnrefable+readableStreamCollectIntoUint8Array) for stdout/stderr just to immediately collect all their data. The stream/controller machinery has significant overhead that causes RSS growth in tight loops.The fix bypasses both
ChildProcessandReadableStreamfor theCommand.output()hot path.spawnInnernow:op_spawn_childdirectly instead of creating aChildProcessinstancecore.readin a simple loop (readAllRid)Before: 30k iterations -> RSS ~2GB
After: 30k iterations -> RSS ~215MB (stable)
Note:
ChildProcess.output()(via.spawn().output()) is unchanged and still uses streams.Test plan
Repro script from the issue:
Closes #33334
Closes #24674
🤖 Generated with Claude Code