fix(web-security): finalize produces item schemas so report_item builds#90
Merged
GangGreenTemperTatum merged 1 commit intoJul 14, 2026
Conversation
The web-security manifest declares structured output types via `produces`
(web_vulnerability, web_endpoint plus built-in finding/asset), but the typed
`report_item` tool was never offered to the agent in production — it silently
fell back to the generic `report` tool, so no structured findings appeared.
Root cause: items.py uses `from __future__ import annotations` (PEP 563), so
field annotations are strings resolved lazily. The SDK capability loader and
OCI packager import items.py by path (spec_from_file_location + exec_module)
without registering it in sys.modules. In that state Pydantic cannot resolve
the `Severity` forward reference, and the first model_json_schema() call — made
by both the report_item tool build and the package builder — raises
`PydanticUserError: ... is not fully defined`. That error is swallowed
upstream, dropping the entire report_item/update_item/link_items tool set.
Fix: rebuild every BaseModel at module scope so forward references resolve
against the module globals regardless of how the file is imported. The loop
covers all current models and any added later — KISS and self-maintaining, no
per-model wiring.
Verified against the real installed SDK (dreadnode 2.0.33):
- runtime resolver `_resolve_produces_models` imports both custom models
- `build_capability_report_item` yields item_type enum
{finding, asset, web_vulnerability, web_endpoint}
- build-time `_resolve_produced_item_types` embeds both JSON schemas
- negative control: without the fix, model_json_schema() fails as before
Adds tests/test_items_produces.py which reproduces the loader's exact import
path (no sys.modules registration) and asserts every declared model builds a
schema, guarding against silent regression when new types are added.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The web-security manifest declares structured output types via
produces:(
web_vulnerability,web_endpoint, plus built-infinding/asset), but inproduction the typed
report_itemtool was never offered to the agent — itsilently fell back to the generic
reporttool, so no structured findingsappeared in the app.
Confirmed in session
0896781d-1f46-4726-ba02-596c3051eb39: 6reportcalls,zero
report_itemcalls, zero structured items emitted.Root cause
items.pyusesfrom __future__ import annotations(PEP 563), so fieldannotations like
severity: Severityare stored as strings and resolvedlazily. The SDK capability loader and OCI packager import
items.pyby path(
spec_from_file_location+exec_module) without registering it insys.modules. In that state Pydantic cannot resolve theSeverityforwardreference, so the first
model_json_schema()call — made by both thereport_itemtool build and the package builder — raisesPydanticUserError: ... is not fully defined. That error is swallowedupstream, dropping the entire
report_item/update_item/link_itemstool set.
Fix
Rebuild every
BaseModelat module scope so forward references resolve againstthe module globals regardless of how the file is imported. The loop covers all
current models and any added later — KISS and self-maintaining, no per-model
wiring.
Testing (against the real installed SDK, dreadnode 2.0.33)
_resolve_produces_modelsimports both custom modelsbuild_capability_report_itemyields item_type enum{finding, asset, web_vulnerability, web_endpoint}_resolve_produced_item_typesembeds both JSON schemasmodel_json_schema()fails as reportedtests/test_items_produces.py(6 tests) reproduces the loader's exactimport path and asserts every declared model builds a schema
dreadnode capability validate --strictpasses (only unrelated caido/burpbinary warnings)
Note: complementary SDK bug (tracked separately)
The SDK capability loader (
_parse_capability_file) does not threadproduces(or legacy
items) intoCapabilityManifest, and the loader/packager omitsys.modulesregistration beforeexec_module. This capability-side fix isnecessary and correct, but full publish-path coverage also needs the SDK
change. Filed internally.