feat:687 | smart agents - LocalBackend, StateBackend, CompositeBackend, Ability to Read non Text Files by jchui-wd · Pull Request #6339 · FlowiseAI/Flowise · GitHub

jchui-wd · 2026-05-04T21:53:16Z

Changes

StateBackend

updated fs tools with universal edit_file, list_file, glob_files, and grep_file capabilities, add, and write already existed
Updated systemPromptBuilder with additional fs tools
SmartAgents: synced Statebackend using runTimeState.files to sync mutations and files so data is persisted across execution.

LocalBackend

Added SANDBOX_LOCAL_PATH incase users want to edit the path
adds MAX_LIST_OBJECTS for glob and grep
pinned 20.12.0 < 21.0.0 to support file system calls
each File workspace is seperated via orgId/chatflowId/chatId

CompositeBackend

routes paths to sub-backends by longest-matching prefix (default + optional mounts like /workspace,
/scratch)
requires slash-boundary matching so /foo/ never matches /foobar/; rejects "/" and "" prefixes
translates paths and re-keys FilesUpdate across mount boundaries; ls/glob/grep stay scoped to the
resolved mount
wired SANDBOX_TYPE=composite into the factory (default StateBackend, no mounts yet) and extended
hasStatefulSmartAgent so executionData still hydrates across messages

Videos

StateBackend Videos Demo

read-write-edit.mp4

read-write-edit.mp4

ls-grep-glob.mp4

ls-grep-glob.mp4

LocalBackend Videos Demo

read-write-gr...b-edit-ls.mp4

read-write-grep-glob-edit-ls.mp4

path-traversal-limits.mp4

path-traversal-limits.mp4

create-read-workspace.mp4

create-read-workspace.mp4

CompositeBackend Videos Demo

Composite

CompositeBackend-Demo.mp4

Gaps

Allow reading image files
We currently can't delete, but is probably intentional as it is saved when we do "execute"

…s. Added a basic StateBackend for read and write and connected it with fs and smart agents for basic testing

- Replace filesystemEnabled() with getSandboxType(); default to 'state' there - createBackend throws on unknown SANDBOX_TYPE instead of silently defaulting - SmartAgent always creates a backend (filesystem is always on per design) - getMimeType: explicit no-dot guard before slicing

Rename state.ts -> StateBackend.ts and protocol.ts -> BackendProtocol.ts to match the packages/components convention (PascalCase for files whose primary export is a same-named class/interface). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

…p capabilities - updated fs tools with universal edit_file, list_file, glob_files, and grep_file capabilities, add, and write already existed - Updated systemPromptBuilder with additional fs tools - SmartAgents: synced Statebackend using runTimeState.files to sync mutations and files so data is persisted across execution.

…th mimeTypes and Uint8Array, made sure edit and grep must have minimum string in it before performing action

…for read

- Added SANDBOX_LOCAL_PATH incase users want to edit the path - adds MAX_LIST_OBJECTS for glob and grep - pinned 20.12.0 < 21.0.0 to support file system calls

gemini-code-assist

Code Review

This pull request introduces a LocalBackend implementation for the SmartAgent sandbox, enabling disk-backed file operations with built-in path traversal protections. The changes include a comprehensive test suite, integration into the backend factory, and a narrowing of the supported Node.js engine version. Feedback highlights critical performance and memory concerns in the read and grep methods, where entire files are loaded into memory instead of being processed via streams or chunks. Additionally, an unused helper method toVirtual was identified for removal.

- grep uses readline + createReadStream so peak memory is one line per file, not the whole file - JSDoc no longer claims "Dev-only" — that warning belongs to LocalShellBackend (shell exec), not disk-only access - drop unused toVirtual helper Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Concurrent SmartAgent runs previously shared one global root (`~/.flowise/sandbox` or `SANDBOX_LOCAL_PATH`), causing cross-chat file collisions and EEXIST races on `write` (wx flag). Factory now composes `<base>/<orgSeg>/<flowSeg>/<chatSeg>` via new `buildScopeSegments` helper, mirroring the orgId/chatflowid/chatId hierarchy used by `getFileFromStorage`. SmartAgent passes scope at the call site; StateBackend ignores it (already isolated per execution). LocalBackend constructor unchanged. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

jchui-wd · 2026-05-05T00:23:49Z

    },
    "engines": {
-        "node": "^20",
+        "node": ">=20.12.0 <21.0.0",


I purposely bumped this so we can use some fs functionality such as fs.readdir(..., { recursive: true }) using recursive and using Dirent.parentPath.

Let me know if we shouldn't do this and I can create a manual function to manually walk the recursive directories instead

jchui-wd · 2026-05-05T18:58:50Z


        const runtimeState = options.agentflowRuntime?.state as ICommonObject
-        if (startPersistState === true && runtimeState && Object.keys(runtimeState).length) {
+        if (runtimeState && Object.keys(runtimeState).length) {


its safe to remove startPersistState here because it was redundant.

runTimeState is NON EMPTY only if startPersisteState is true

so when Object.keys(runtimeState).length) > 0 already implies that startPersistState is true.

Routes paths to sub-backends by longest matching prefix and translates paths/FilesUpdate keys across the boundary. Wires SANDBOX_TYPE=composite into the factory and includes it under hasStatefulSmartAgent so state persistence still applies. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

… through Previously rewriteBinaryToolResults stripped audio/video/PDF blocks from tool messages into a text breadcrumb, dropping bytes that providers like Anthropic (PDFs) and Gemini (audio/video) can read natively. Now images alone are smuggled out as a synthetic user message with image_url (still required for OpenAI Chat Completion); other binaries pass through, and non-supporting providers cleanly 4xx via Flowise's existing error path. Extracts the function from a private SmartAgent method to an exported helper in agentflow/utils.ts (sibling of addImageArtifactsToMessages) and adds 6 tests. Updates the read_file tool description to set the right expectation about which models can read which file types. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Adds dev-only shell execution (opt-in via SANDBOX_TYPE=local-shell) so the SmartAgent can run real coding tasks during development. The execute tool binds and surfaces in the system prompt only when the active backend implements ShellBackendProtocol, keeping state/local backends safe by default. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

jchui-wd and others added 7 commits April 29, 2026 15:51

added utils and protocol interfaces and types for backends fs command…

b1f314e

…s. Added a basic StateBackend for read and write and connected it with fs and smart agents for basic testing

added normalize content to fix potential read grep and glob issues wi…

46ec327

…th mimeTypes and Uint8Array, made sure edit and grep must have minimum string in it before performing action

edit incorrect error message and added todo for Uint8Array for later …

60203ff

…for read

added LocalBackend support for disk sandbox.

a718f7f

- Added SANDBOX_LOCAL_PATH incase users want to edit the path - adds MAX_LIST_OBJECTS for glob and grep - pinned 20.12.0 < 21.0.0 to support file system calls

gemini-code-assist Bot reviewed May 4, 2026

View reviewed changes

jchui-wd and others added 2 commits May 4, 2026 15:43

jchui-wd commented May 5, 2026

View reviewed changes

jchui-wd marked this pull request as ready for review May 5, 2026 00:24

jchui-wd requested review from HenryHengZJ, chloebyun-wd, harshit-flowise and mmattu-wd May 5, 2026 00:26

jchui-wd changed the base branch from feat/686-StateBackend to feature/SmartAgents May 5, 2026 18:37

jchui-wd commented May 5, 2026

View reviewed changes

jchui-wd changed the title ~~feat:687 | smart agents - LocalBackend~~ feat:687 | smart agents - LocalBackend, StateBackend May 5, 2026

jchui-wd and others added 4 commits May 6, 2026 11:01

Added support for reading non text files

5d11668

sandbox: extract shared BackendProtocol conformance suite

2463ca4

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

jchui-wd changed the title ~~feat:687 | smart agents - LocalBackend, StateBackend~~ feat:687 | smart agents - LocalBackend, StateBackend, CompositeBackend, Ability to Read non Text Files May 8, 2026

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

feat:687 | smart agents - LocalBackend, StateBackend, CompositeBackend, Ability to Read non Text Files#6339

feat:687 | smart agents - LocalBackend, StateBackend, CompositeBackend, Ability to Read non Text Files#6339
jchui-wd wants to merge 14 commits into
feature/SmartAgentsfrom
feat/687-LocalBackend

jchui-wd commented May 4, 2026 •

edited

Loading

Uh oh!

gemini-code-assist Bot left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

jchui-wd May 5, 2026

Uh oh!

jchui-wd May 5, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Uh oh!

Conversation

jchui-wd commented May 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Changes

StateBackend

LocalBackend

CompositeBackend

Videos

Gaps

Uh oh!

gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

Uh oh!

Uh oh!

Uh oh!

jchui-wd May 5, 2026

Choose a reason for hiding this comment

Uh oh!

jchui-wd May 5, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

jchui-wd commented May 4, 2026 •

edited

Loading