Skip to content

fix(kb-sanitization): add sanitization for kb tags when exporting workflow#1628

Merged
waleedlatif1 merged 1 commit into
stagingfrom
sim-149
Oct 14, 2025
Merged

fix(kb-sanitization): add sanitization for kb tags when exporting workflow#1628
waleedlatif1 merged 1 commit into
stagingfrom
sim-149

Conversation

@waleedlatif1

Copy link
Copy Markdown
Collaborator

Summary

  • add sanitization for kb tags when exporting workflow

Type of Change

  • New feature

Testing

Tested manually.

Checklist

  • Code follows project style guidelines
  • Self-reviewed my changes
  • Tests added/updated and passing
  • No new warnings introduced
  • I confirm that I have read and agree to the terms outlined in the Contributor License Agreement (CLA)

@vercel

vercel Bot commented Oct 14, 2025

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Preview Comments Updated (UTC)
docs Skipped Skipped Oct 14, 2025 11:47pm

@greptile-apps greptile-apps Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Greptile Overview

Summary

Added sanitization for knowledge base tagFilters and documentTags fields when exporting workflows. These fields contain workspace-specific metadata that should not be included in exported workflows.

Changes made:

  • In sanitizeSubBlocks(): Skip tagFilters and documentTags fields when sanitizing for copilot (returns early, field is omitted from output)
  • In sanitizeForExport(): Clear tagFilters and documentTags values by setting them to empty strings (preserves field structure but removes workspace-specific data)

Impact:

  • Prevents workspace-specific knowledge base configuration from leaking when workflows are exported and shared across workspaces
  • Maintains consistency with existing sensitive data sanitization patterns (credentials, OAuth tokens, API keys)
  • Aligns with the principle that exported workflows should be portable across different workspace environments

Confidence Score: 5/5

  • This PR is safe to merge with minimal risk
  • The changes follow established patterns in the codebase for sanitizing workspace-specific data. The implementation is straightforward, consistent with existing sanitization logic, and addresses a valid use case of preventing workspace-specific knowledge base metadata from being exported.
  • No files require special attention

Important Files Changed

File Analysis

Filename Score Overview
apps/sim/lib/workflows/json-sanitizer.ts 5/5 Added sanitization for tagFilters and documentTags to prevent workspace-specific knowledge base metadata from being exported

Sequence Diagram

sequenceDiagram
    participant User
    participant WorkflowExport
    participant sanitizeForExport
    participant sanitizeForCopilot
    participant KnowledgeBlock
    
    User->>WorkflowExport: Export workflow
    WorkflowExport->>sanitizeForExport: sanitizeForExport(state)
    
    Note over sanitizeForExport: Clone workflow state
    
    sanitizeForExport->>sanitizeForExport: Iterate blocks.subBlocks
    
    alt Sensitive field (credentials/oauth/secrets)
        sanitizeForExport->>sanitizeForExport: Set value to ''
    end
    
    alt tagFilters or documentTags field
        Note over sanitizeForExport: NEW: Remove KB workspace data
        sanitizeForExport->>sanitizeForExport: Set value to ''
    end
    
    sanitizeForExport-->>WorkflowExport: Return sanitized state
    WorkflowExport-->>User: Export file (without KB metadata)
    
    Note over User,KnowledgeBlock: Copilot sanitization path
    
    User->>WorkflowExport: Send to copilot
    WorkflowExport->>sanitizeForCopilot: sanitizeForCopilot(state)
    sanitizeForCopilot->>sanitizeForCopilot: sanitizeSubBlocks()
    
    alt tagFilters or documentTags field
        Note over sanitizeForCopilot: NEW: Skip KB workspace data
        sanitizeForCopilot->>sanitizeForCopilot: return (skip field)
    end
    
    sanitizeForCopilot-->>WorkflowExport: Return sanitized state
    WorkflowExport-->>User: Send to copilot (without KB metadata)
Loading

1 file reviewed, no comments

Edit Code Review Agent Settings | Greptile

@waleedlatif1 waleedlatif1 merged commit 6723adf into staging Oct 14, 2025
9 checks passed
@waleedlatif1 waleedlatif1 deleted the sim-149 branch October 14, 2025 23:56
j15z added a commit that referenced this pull request Jul 10, 2026
… workflow view

sanitizeForCopilot dropped `tagFilters` and `documentTags` from the workflow state the
agent reads (workflows/{name}/state.json), while edit_workflow is allowed to write both.
The field was therefore write-only: on a follow-up edit the agent read back an absent
field, concluded no filter was set, and cleared the user's tag filter.

The redaction was introduced for workflow *export* (#1628) and is already enforced there
by sanitizeWorkflowForSharing's key list. The duplicate in the copilot-only
sanitizeSubBlocks was redundant for export and destructive for the agent. Removes it and
pins the contract with a regression test.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
j15z added a commit that referenced this pull request Jul 10, 2026
…top it clearing them (#5546)

* fix(copilot): persist KB tag subblocks as JSON strings from edit_workflow

The edit_workflow tool normalizes array-with-id subblocks (via
normalizeArrayWithIds) but only re-stringifies the keys listed in
JSON_STRING_SUBBLOCK_KEYS. `tagFilters` (knowledge-tag-filters) and
`documentTags` (document-tag-entry) were missing, so agent-authored tag
filters were stored as raw JSON arrays while those UI components read
their value with JSON.parse (expecting a string). The result: an agent
edit to a Knowledge block's tag filter persisted correctly but rendered
as an empty filter in the editor (JSON.parse on an array throws -> []).

- Add `tagFilters` and `documentTags` to JSON_STRING_SUBBLOCK_KEYS so
  edit_workflow stores them in the same shape the UI writes.
- Make both components' parsers tolerate an already-parsed array on read,
  self-healing values already persisted in the broken (array) shape.

Search execution was unaffected (parseTagFilters accepts arrays), so the
value was never lost — only the editor render and round-trip were broken.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(copilot): expose KB tag definitions in VFS meta.json

Surface each knowledge base's defined tags (displayName -> tagSlot) inline in
its meta.json via serializeKBMeta, loaded in one batched query
(loadKbTagDefinitions), so the agent can bind a knowledge-tag filter to a real
tag slot instead of guessing a tag name it cannot otherwise see.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(copilot): stringify KB tag subblocks on the nested-node edit path

The nested-node merge path normalized array-with-id subblocks but never
re-serialized the JSON_STRING_SUBBLOCK_KEYS, so editing a block nested in a
loop/parallel container still persisted tagFilters/documentTags (and
conditions/routes) as raw arrays -- the exact shape the subblock components
cannot JSON.parse.

Route all four write paths through a single normalizeSubblockValue helper so
the normalize and re-stringify steps cannot drift apart again, and extract the
duplicated string-or-array read logic into parseJsonArrayValue.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* refactor(copilot): tighten subblock serialization helpers

Derive KbTagDefinitionSummary from the canonical TagDefinition instead of
restating its fields, make parseJsonArrayValue generic so callers drop their
`as T[]` casts, and unexport the three builders helpers that no longer have
consumers outside the module now that normalizeSubblockValue fronts them.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(copilot): stop stripping tagFilters/documentTags from the agent's workflow view

sanitizeForCopilot dropped `tagFilters` and `documentTags` from the workflow state the
agent reads (workflows/{name}/state.json), while edit_workflow is allowed to write both.
The field was therefore write-only: on a follow-up edit the agent read back an absent
field, concluded no filter was set, and cleared the user's tag filter.

The redaction was introduced for workflow *export* (#1628) and is already enforced there
by sanitizeWorkflowForSharing's key list. The duplicate in the copilot-only
sanitizeSubBlocks was redundant for export and destructive for the agent. Removes it and
pins the contract with a regression test.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(copilot): reject malformed KB tag values instead of clearing the filter

`knowledge-tag-filters` and `document-tag-entry` had no arm in the
`edit_workflow` input validator, so they fell through to the pass-through
default. Any non-array value the agent supplied -- a double-encoded JSON
string, an object, an unparseable string -- reached `normalizeSubblockValue`,
where `normalizeArrayWithIds` coerces unparseable input to `[]`. The write
path then persisted `"[]"` over the tag filter the user had configured.

`condition-input` and `router-input` already guard against exactly this and
return an actionable error to the model. Extend that arm to cover the two KB
subblock types. It keys on subblock type, so the unrelated `tagFilters`
short-input on the Algolia block is unaffected. `null`/`undefined` and empty
arrays still clear the field, so intentional clears keep working.

Also wrap `loadKbTagDefinitions` in try/catch. Tag definitions are an optional
meta.json enrichment, but the query ran inside the top-level `Promise.all`, so
a transient failure would reject the entire workspace VFS materialize and
leave the agent unable to read any file. Now it degrades to a meta.json
without tag definitions, matching the sibling materializers.

Adds regression tests for both, plus the first tests for
`parseJsonArrayValue`, the helper that keeps pre-fix raw-array rows readable.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* refactor(copilot): collapse duplicate JSON-array parsing in edit-workflow builders

`normalizeArrayWithIds` and `normalizeConditionRouterIds` each hand-rolled the
same "accept a raw array or the JSON string these subblocks persist" parse.
Extract `parseJsonArray`, which returns null when the value is neither, so each
caller keeps its own distinct fallback: `[]` for the former, the untouched
original value for the latter.

Behavior-preserving. An empty array is truthy, so `[]` and `"[]"` still parse
through rather than hitting either fallback.

`validation.ts` has a third copy, but `builders.ts` already imports from it, so
sharing the helper across the two would introduce an import cycle. Left as is.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(copilot): specify tag name and legal operators in KB meta.json

`tagDefinitions` exposed `displayName`, but a `tagFilters` entry must carry the
key `tagName`. An entry written with `displayName` passes validation and
persists, then filters nothing -- a silent failure. Rename the field at the
serializer boundary; the DB column is untouched.

Also emit the operators legal for each tag's `fieldType`, reusing
`getOperatorsForFieldType`. `between` is valid for number and date but not for
text or boolean, and the agent has no way to infer that. An unrecognized
fieldType yields an empty list rather than throwing.

Still unspecified, and deliberately out of scope: a filter entry's value key is
`tagValue` (but `value` on documentTags), and `between` needs `valueTo`. Those
describe the subblock entry shape, not the knowledge base, so meta.json is the
wrong place for them.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(copilot): pass a nullish subblock clear through instead of serializing "[]"

`validateValueForSubBlockType` accepts null as an explicit clear, but
`normalizeSubblockValue` then ran it through `normalizeArrayWithIds`, which
coerces any non-array to `[]`, and persisted the string "[]".

No data is lost either way -- "[]" and an absent field both mean "no filters".
But it left the field present when the caller asked for it to be unset, so
`sanitizeForCopilot` showed the agent an empty filter rather than an absent
one, contradicting the absent-means-unset invariant the sanitizer documents.
It also made Algolia's `if (params.tagFilters)` see a set value, since "[]" is
truthy.

An explicitly empty array still serializes to "[]" -- clearing with a value is
distinct from clearing by omission.

Reported by Cursor Bugbot on #5546.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Sg312 pushed a commit that referenced this pull request Jul 11, 2026
…top it clearing them (#5546)

* fix(copilot): persist KB tag subblocks as JSON strings from edit_workflow

The edit_workflow tool normalizes array-with-id subblocks (via
normalizeArrayWithIds) but only re-stringifies the keys listed in
JSON_STRING_SUBBLOCK_KEYS. `tagFilters` (knowledge-tag-filters) and
`documentTags` (document-tag-entry) were missing, so agent-authored tag
filters were stored as raw JSON arrays while those UI components read
their value with JSON.parse (expecting a string). The result: an agent
edit to a Knowledge block's tag filter persisted correctly but rendered
as an empty filter in the editor (JSON.parse on an array throws -> []).

- Add `tagFilters` and `documentTags` to JSON_STRING_SUBBLOCK_KEYS so
  edit_workflow stores them in the same shape the UI writes.
- Make both components' parsers tolerate an already-parsed array on read,
  self-healing values already persisted in the broken (array) shape.

Search execution was unaffected (parseTagFilters accepts arrays), so the
value was never lost — only the editor render and round-trip were broken.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(copilot): expose KB tag definitions in VFS meta.json

Surface each knowledge base's defined tags (displayName -> tagSlot) inline in
its meta.json via serializeKBMeta, loaded in one batched query
(loadKbTagDefinitions), so the agent can bind a knowledge-tag filter to a real
tag slot instead of guessing a tag name it cannot otherwise see.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(copilot): stringify KB tag subblocks on the nested-node edit path

The nested-node merge path normalized array-with-id subblocks but never
re-serialized the JSON_STRING_SUBBLOCK_KEYS, so editing a block nested in a
loop/parallel container still persisted tagFilters/documentTags (and
conditions/routes) as raw arrays -- the exact shape the subblock components
cannot JSON.parse.

Route all four write paths through a single normalizeSubblockValue helper so
the normalize and re-stringify steps cannot drift apart again, and extract the
duplicated string-or-array read logic into parseJsonArrayValue.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* refactor(copilot): tighten subblock serialization helpers

Derive KbTagDefinitionSummary from the canonical TagDefinition instead of
restating its fields, make parseJsonArrayValue generic so callers drop their
`as T[]` casts, and unexport the three builders helpers that no longer have
consumers outside the module now that normalizeSubblockValue fronts them.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(copilot): stop stripping tagFilters/documentTags from the agent's workflow view

sanitizeForCopilot dropped `tagFilters` and `documentTags` from the workflow state the
agent reads (workflows/{name}/state.json), while edit_workflow is allowed to write both.
The field was therefore write-only: on a follow-up edit the agent read back an absent
field, concluded no filter was set, and cleared the user's tag filter.

The redaction was introduced for workflow *export* (#1628) and is already enforced there
by sanitizeWorkflowForSharing's key list. The duplicate in the copilot-only
sanitizeSubBlocks was redundant for export and destructive for the agent. Removes it and
pins the contract with a regression test.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(copilot): reject malformed KB tag values instead of clearing the filter

`knowledge-tag-filters` and `document-tag-entry` had no arm in the
`edit_workflow` input validator, so they fell through to the pass-through
default. Any non-array value the agent supplied -- a double-encoded JSON
string, an object, an unparseable string -- reached `normalizeSubblockValue`,
where `normalizeArrayWithIds` coerces unparseable input to `[]`. The write
path then persisted `"[]"` over the tag filter the user had configured.

`condition-input` and `router-input` already guard against exactly this and
return an actionable error to the model. Extend that arm to cover the two KB
subblock types. It keys on subblock type, so the unrelated `tagFilters`
short-input on the Algolia block is unaffected. `null`/`undefined` and empty
arrays still clear the field, so intentional clears keep working.

Also wrap `loadKbTagDefinitions` in try/catch. Tag definitions are an optional
meta.json enrichment, but the query ran inside the top-level `Promise.all`, so
a transient failure would reject the entire workspace VFS materialize and
leave the agent unable to read any file. Now it degrades to a meta.json
without tag definitions, matching the sibling materializers.

Adds regression tests for both, plus the first tests for
`parseJsonArrayValue`, the helper that keeps pre-fix raw-array rows readable.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* refactor(copilot): collapse duplicate JSON-array parsing in edit-workflow builders

`normalizeArrayWithIds` and `normalizeConditionRouterIds` each hand-rolled the
same "accept a raw array or the JSON string these subblocks persist" parse.
Extract `parseJsonArray`, which returns null when the value is neither, so each
caller keeps its own distinct fallback: `[]` for the former, the untouched
original value for the latter.

Behavior-preserving. An empty array is truthy, so `[]` and `"[]"` still parse
through rather than hitting either fallback.

`validation.ts` has a third copy, but `builders.ts` already imports from it, so
sharing the helper across the two would introduce an import cycle. Left as is.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(copilot): specify tag name and legal operators in KB meta.json

`tagDefinitions` exposed `displayName`, but a `tagFilters` entry must carry the
key `tagName`. An entry written with `displayName` passes validation and
persists, then filters nothing -- a silent failure. Rename the field at the
serializer boundary; the DB column is untouched.

Also emit the operators legal for each tag's `fieldType`, reusing
`getOperatorsForFieldType`. `between` is valid for number and date but not for
text or boolean, and the agent has no way to infer that. An unrecognized
fieldType yields an empty list rather than throwing.

Still unspecified, and deliberately out of scope: a filter entry's value key is
`tagValue` (but `value` on documentTags), and `between` needs `valueTo`. Those
describe the subblock entry shape, not the knowledge base, so meta.json is the
wrong place for them.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(copilot): pass a nullish subblock clear through instead of serializing "[]"

`validateValueForSubBlockType` accepts null as an explicit clear, but
`normalizeSubblockValue` then ran it through `normalizeArrayWithIds`, which
coerces any non-array to `[]`, and persisted the string "[]".

No data is lost either way -- "[]" and an absent field both mean "no filters".
But it left the field present when the caller asked for it to be unset, so
`sanitizeForCopilot` showed the agent an empty filter rather than an absent
one, contradicting the absent-means-unset invariant the sanitizer documents.
It also made Algolia's `if (params.tagFilters)` see a set value, since "[]" is
truthy.

An explicitly empty array still serializes to "[]" -- clearing with a value is
distinct from clearing by omission.

Reported by Cursor Bugbot on #5546.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Sg312 pushed a commit that referenced this pull request Jul 11, 2026
…top it clearing them (#5546)

* fix(copilot): persist KB tag subblocks as JSON strings from edit_workflow

The edit_workflow tool normalizes array-with-id subblocks (via
normalizeArrayWithIds) but only re-stringifies the keys listed in
JSON_STRING_SUBBLOCK_KEYS. `tagFilters` (knowledge-tag-filters) and
`documentTags` (document-tag-entry) were missing, so agent-authored tag
filters were stored as raw JSON arrays while those UI components read
their value with JSON.parse (expecting a string). The result: an agent
edit to a Knowledge block's tag filter persisted correctly but rendered
as an empty filter in the editor (JSON.parse on an array throws -> []).

- Add `tagFilters` and `documentTags` to JSON_STRING_SUBBLOCK_KEYS so
  edit_workflow stores them in the same shape the UI writes.
- Make both components' parsers tolerate an already-parsed array on read,
  self-healing values already persisted in the broken (array) shape.

Search execution was unaffected (parseTagFilters accepts arrays), so the
value was never lost — only the editor render and round-trip were broken.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(copilot): expose KB tag definitions in VFS meta.json

Surface each knowledge base's defined tags (displayName -> tagSlot) inline in
its meta.json via serializeKBMeta, loaded in one batched query
(loadKbTagDefinitions), so the agent can bind a knowledge-tag filter to a real
tag slot instead of guessing a tag name it cannot otherwise see.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(copilot): stringify KB tag subblocks on the nested-node edit path

The nested-node merge path normalized array-with-id subblocks but never
re-serialized the JSON_STRING_SUBBLOCK_KEYS, so editing a block nested in a
loop/parallel container still persisted tagFilters/documentTags (and
conditions/routes) as raw arrays -- the exact shape the subblock components
cannot JSON.parse.

Route all four write paths through a single normalizeSubblockValue helper so
the normalize and re-stringify steps cannot drift apart again, and extract the
duplicated string-or-array read logic into parseJsonArrayValue.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* refactor(copilot): tighten subblock serialization helpers

Derive KbTagDefinitionSummary from the canonical TagDefinition instead of
restating its fields, make parseJsonArrayValue generic so callers drop their
`as T[]` casts, and unexport the three builders helpers that no longer have
consumers outside the module now that normalizeSubblockValue fronts them.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(copilot): stop stripping tagFilters/documentTags from the agent's workflow view

sanitizeForCopilot dropped `tagFilters` and `documentTags` from the workflow state the
agent reads (workflows/{name}/state.json), while edit_workflow is allowed to write both.
The field was therefore write-only: on a follow-up edit the agent read back an absent
field, concluded no filter was set, and cleared the user's tag filter.

The redaction was introduced for workflow *export* (#1628) and is already enforced there
by sanitizeWorkflowForSharing's key list. The duplicate in the copilot-only
sanitizeSubBlocks was redundant for export and destructive for the agent. Removes it and
pins the contract with a regression test.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(copilot): reject malformed KB tag values instead of clearing the filter

`knowledge-tag-filters` and `document-tag-entry` had no arm in the
`edit_workflow` input validator, so they fell through to the pass-through
default. Any non-array value the agent supplied -- a double-encoded JSON
string, an object, an unparseable string -- reached `normalizeSubblockValue`,
where `normalizeArrayWithIds` coerces unparseable input to `[]`. The write
path then persisted `"[]"` over the tag filter the user had configured.

`condition-input` and `router-input` already guard against exactly this and
return an actionable error to the model. Extend that arm to cover the two KB
subblock types. It keys on subblock type, so the unrelated `tagFilters`
short-input on the Algolia block is unaffected. `null`/`undefined` and empty
arrays still clear the field, so intentional clears keep working.

Also wrap `loadKbTagDefinitions` in try/catch. Tag definitions are an optional
meta.json enrichment, but the query ran inside the top-level `Promise.all`, so
a transient failure would reject the entire workspace VFS materialize and
leave the agent unable to read any file. Now it degrades to a meta.json
without tag definitions, matching the sibling materializers.

Adds regression tests for both, plus the first tests for
`parseJsonArrayValue`, the helper that keeps pre-fix raw-array rows readable.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* refactor(copilot): collapse duplicate JSON-array parsing in edit-workflow builders

`normalizeArrayWithIds` and `normalizeConditionRouterIds` each hand-rolled the
same "accept a raw array or the JSON string these subblocks persist" parse.
Extract `parseJsonArray`, which returns null when the value is neither, so each
caller keeps its own distinct fallback: `[]` for the former, the untouched
original value for the latter.

Behavior-preserving. An empty array is truthy, so `[]` and `"[]"` still parse
through rather than hitting either fallback.

`validation.ts` has a third copy, but `builders.ts` already imports from it, so
sharing the helper across the two would introduce an import cycle. Left as is.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(copilot): specify tag name and legal operators in KB meta.json

`tagDefinitions` exposed `displayName`, but a `tagFilters` entry must carry the
key `tagName`. An entry written with `displayName` passes validation and
persists, then filters nothing -- a silent failure. Rename the field at the
serializer boundary; the DB column is untouched.

Also emit the operators legal for each tag's `fieldType`, reusing
`getOperatorsForFieldType`. `between` is valid for number and date but not for
text or boolean, and the agent has no way to infer that. An unrecognized
fieldType yields an empty list rather than throwing.

Still unspecified, and deliberately out of scope: a filter entry's value key is
`tagValue` (but `value` on documentTags), and `between` needs `valueTo`. Those
describe the subblock entry shape, not the knowledge base, so meta.json is the
wrong place for them.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(copilot): pass a nullish subblock clear through instead of serializing "[]"

`validateValueForSubBlockType` accepts null as an explicit clear, but
`normalizeSubblockValue` then ran it through `normalizeArrayWithIds`, which
coerces any non-array to `[]`, and persisted the string "[]".

No data is lost either way -- "[]" and an absent field both mean "no filters".
But it left the field present when the caller asked for it to be unset, so
`sanitizeForCopilot` showed the agent an empty filter rather than an absent
one, contradicting the absent-means-unset invariant the sanitizer documents.
It also made Algolia's `if (params.tagFilters)` see a set value, since "[]" is
truthy.

An explicitly empty array still serializes to "[]" -- clearing with a value is
distinct from clearing by omission.

Reported by Cursor Bugbot on #5546.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
icecrasher321 pushed a commit that referenced this pull request Jul 11, 2026
…top it clearing them (#5546)

* fix(copilot): persist KB tag subblocks as JSON strings from edit_workflow

The edit_workflow tool normalizes array-with-id subblocks (via
normalizeArrayWithIds) but only re-stringifies the keys listed in
JSON_STRING_SUBBLOCK_KEYS. `tagFilters` (knowledge-tag-filters) and
`documentTags` (document-tag-entry) were missing, so agent-authored tag
filters were stored as raw JSON arrays while those UI components read
their value with JSON.parse (expecting a string). The result: an agent
edit to a Knowledge block's tag filter persisted correctly but rendered
as an empty filter in the editor (JSON.parse on an array throws -> []).

- Add `tagFilters` and `documentTags` to JSON_STRING_SUBBLOCK_KEYS so
  edit_workflow stores them in the same shape the UI writes.
- Make both components' parsers tolerate an already-parsed array on read,
  self-healing values already persisted in the broken (array) shape.

Search execution was unaffected (parseTagFilters accepts arrays), so the
value was never lost — only the editor render and round-trip were broken.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(copilot): expose KB tag definitions in VFS meta.json

Surface each knowledge base's defined tags (displayName -> tagSlot) inline in
its meta.json via serializeKBMeta, loaded in one batched query
(loadKbTagDefinitions), so the agent can bind a knowledge-tag filter to a real
tag slot instead of guessing a tag name it cannot otherwise see.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(copilot): stringify KB tag subblocks on the nested-node edit path

The nested-node merge path normalized array-with-id subblocks but never
re-serialized the JSON_STRING_SUBBLOCK_KEYS, so editing a block nested in a
loop/parallel container still persisted tagFilters/documentTags (and
conditions/routes) as raw arrays -- the exact shape the subblock components
cannot JSON.parse.

Route all four write paths through a single normalizeSubblockValue helper so
the normalize and re-stringify steps cannot drift apart again, and extract the
duplicated string-or-array read logic into parseJsonArrayValue.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* refactor(copilot): tighten subblock serialization helpers

Derive KbTagDefinitionSummary from the canonical TagDefinition instead of
restating its fields, make parseJsonArrayValue generic so callers drop their
`as T[]` casts, and unexport the three builders helpers that no longer have
consumers outside the module now that normalizeSubblockValue fronts them.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(copilot): stop stripping tagFilters/documentTags from the agent's workflow view

sanitizeForCopilot dropped `tagFilters` and `documentTags` from the workflow state the
agent reads (workflows/{name}/state.json), while edit_workflow is allowed to write both.
The field was therefore write-only: on a follow-up edit the agent read back an absent
field, concluded no filter was set, and cleared the user's tag filter.

The redaction was introduced for workflow *export* (#1628) and is already enforced there
by sanitizeWorkflowForSharing's key list. The duplicate in the copilot-only
sanitizeSubBlocks was redundant for export and destructive for the agent. Removes it and
pins the contract with a regression test.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(copilot): reject malformed KB tag values instead of clearing the filter

`knowledge-tag-filters` and `document-tag-entry` had no arm in the
`edit_workflow` input validator, so they fell through to the pass-through
default. Any non-array value the agent supplied -- a double-encoded JSON
string, an object, an unparseable string -- reached `normalizeSubblockValue`,
where `normalizeArrayWithIds` coerces unparseable input to `[]`. The write
path then persisted `"[]"` over the tag filter the user had configured.

`condition-input` and `router-input` already guard against exactly this and
return an actionable error to the model. Extend that arm to cover the two KB
subblock types. It keys on subblock type, so the unrelated `tagFilters`
short-input on the Algolia block is unaffected. `null`/`undefined` and empty
arrays still clear the field, so intentional clears keep working.

Also wrap `loadKbTagDefinitions` in try/catch. Tag definitions are an optional
meta.json enrichment, but the query ran inside the top-level `Promise.all`, so
a transient failure would reject the entire workspace VFS materialize and
leave the agent unable to read any file. Now it degrades to a meta.json
without tag definitions, matching the sibling materializers.

Adds regression tests for both, plus the first tests for
`parseJsonArrayValue`, the helper that keeps pre-fix raw-array rows readable.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* refactor(copilot): collapse duplicate JSON-array parsing in edit-workflow builders

`normalizeArrayWithIds` and `normalizeConditionRouterIds` each hand-rolled the
same "accept a raw array or the JSON string these subblocks persist" parse.
Extract `parseJsonArray`, which returns null when the value is neither, so each
caller keeps its own distinct fallback: `[]` for the former, the untouched
original value for the latter.

Behavior-preserving. An empty array is truthy, so `[]` and `"[]"` still parse
through rather than hitting either fallback.

`validation.ts` has a third copy, but `builders.ts` already imports from it, so
sharing the helper across the two would introduce an import cycle. Left as is.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(copilot): specify tag name and legal operators in KB meta.json

`tagDefinitions` exposed `displayName`, but a `tagFilters` entry must carry the
key `tagName`. An entry written with `displayName` passes validation and
persists, then filters nothing -- a silent failure. Rename the field at the
serializer boundary; the DB column is untouched.

Also emit the operators legal for each tag's `fieldType`, reusing
`getOperatorsForFieldType`. `between` is valid for number and date but not for
text or boolean, and the agent has no way to infer that. An unrecognized
fieldType yields an empty list rather than throwing.

Still unspecified, and deliberately out of scope: a filter entry's value key is
`tagValue` (but `value` on documentTags), and `between` needs `valueTo`. Those
describe the subblock entry shape, not the knowledge base, so meta.json is the
wrong place for them.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(copilot): pass a nullish subblock clear through instead of serializing "[]"

`validateValueForSubBlockType` accepts null as an explicit clear, but
`normalizeSubblockValue` then ran it through `normalizeArrayWithIds`, which
coerces any non-array to `[]`, and persisted the string "[]".

No data is lost either way -- "[]" and an absent field both mean "no filters".
But it left the field present when the caller asked for it to be unset, so
`sanitizeForCopilot` showed the agent an empty filter rather than an absent
one, contradicting the absent-means-unset invariant the sanitizer documents.
It also made Algolia's `if (params.tagFilters)` see a set value, since "[]" is
truthy.

An explicitly empty array still serializes to "[]" -- clearing with a value is
distinct from clearing by omission.

Reported by Cursor Bugbot on #5546.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant