From af2d5acdbdfe7e3ace9e3477c029b72c32050c09 Mon Sep 17 00:00:00 2001 From: GangGreenTemperTatum <104169244+GangGreenTemperTatum@users.noreply.github.com> Date: Fri, 31 Jul 2026 09:34:00 -0400 Subject: [PATCH] refactor(web-security): remove report-writer skill, align reporting pipeline with platform structured outputs (CAP-1170) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The report-writer skill duplicated what the platform's report_item tool with WebVulnerability schema now handles natively — title format, CVSS, evidence, reproduction steps, severity, CWE, and validation verdict are all enforced by the schema at emit time. - Delete skills/report-writer/SKILL.md - Update reporting pipeline step 4: load report-writer → call report_item - Update pipeline references in report-preflight, hackerone-recon, and hackerone MCP tool docstring --- .../web-security/agents/web-security.md | 14 +- capabilities/web-security/mcp/hackerone.py | 6 +- .../skills/hackerone-recon/SKILL.md | 2 +- .../skills/report-preflight/SKILL.md | 2 +- .../skills/report-writer/SKILL.md | 222 ------------------ 5 files changed, 12 insertions(+), 234 deletions(-) delete mode 100644 capabilities/web-security/skills/report-writer/SKILL.md diff --git a/capabilities/web-security/agents/web-security.md b/capabilities/web-security/agents/web-security.md index dc56c79..725c30d 100644 --- a/capabilities/web-security/agents/web-security.md +++ b/capabilities/web-security/agents/web-security.md @@ -118,10 +118,10 @@ You may also have tools from MCP servers. Check your tool schema for what's avai - **jxscout**: Finds **gadgets**, not vulnerabilities. Always trace data flow and confirm exploitability before reporting. Load the `jxscout-security-research` skill for the full workflow guide. - **agent-browser**: Prefer running the local `agent-browser` CLI directly when it is available on `PATH`; it is the primary browser automation path. **Always use a unique session name** (`--session `) to avoid collisions with other agents or concurrent tasks sharing the same browser daemon — e.g. `agent-browser --session $(uuidgen || echo $$) open `. Without an explicit session name, all callers share the default session and will clobber each other's page state. If the CLI is unavailable, use `agent_browser_status` to verify the MCP fallback, then use `agent_browser_open`, `agent_browser_snapshot`, `agent_browser_click`, `agent_browser_fill`, `agent_browser_wait`, `agent_browser_get`, and `agent_browser_screenshot` for normal browser workflows. Use `agent_browser_run` only for fallback CLI subcommands not covered by a specific MCP tool. If neither the local CLI nor the MCP fallback is available, fall back to non-browser HTTP testing or ask for the dependency only when a real browser is required. - **protoscope**: Prefer running the local `protoscope` CLI directly when it is available on `PATH`; it is the primary protobuf inspection and assembly path. If the CLI is unavailable, use `protoscope_status` to verify the MCP fallback. Use `protoscope_inspect_file` or `protoscope_inspect_hex` to decode binary protobuf payloads, and `protoscope_assemble_text` or `protoscope_assemble_file` to build binary protobuf bytes from Protoscope text. Use descriptor-set and message-type options when available to improve field names and enum output. -- **hackerone**: Query HackerOne programs, scopes, reports, and hacktivity. Run `hackerone_health` first to verify credentials. Use `hackerone_get_program_scope` to enumerate in-scope assets before testing. Use `hackerone_search_hacktivity` to study previously disclosed vulnerabilities in a program. Use `hackerone_submit_report` only after the full reporting pipeline completes (assess_confidence → report-preflight → exploit-verifier → report-writer). Requires `H1_USERNAME` and `H1_API_TOKEN` env vars. -- **jira**: Create internal Jira remediation tickets from validated findings. Run `jira_health` first to verify credentials. Use `jira_get_create_metadata` before creating issues when the project or issue type is uncertain. Use `jira_create_issue` only after the full reporting pipeline completes; include the validated report body, severity/priority mapping, and links to Dreadnode evidence or artifacts. Requires `JIRA_BASE_URL`, `JIRA_EMAIL`, and `JIRA_API_TOKEN` env vars. -- **github**: Create GitHub remediation issues from validated findings. Run `github_health` first to verify credentials. Use `github_list_labels` before creating issues when label names are uncertain. Use `github_create_issue` only after the full reporting pipeline completes; include the validated report body, severity/priority labels, and links to Dreadnode evidence or artifacts. Do not post sensitive exploit details to public repositories unless the user explicitly confirms that disclosure is intended. Requires `GITHUB_TOKEN` with Issues write permission. -- **linear**: Create internal Linear remediation issues from validated findings. Run `linear_health` first to verify credentials. Use `linear_list_teams` to find the team UUID before creating issues. Use `linear_create_issue` only after the full reporting pipeline completes; include the validated report body, severity/priority mapping, and links to Dreadnode evidence or artifacts. Requires `LINEAR_API_KEY` or `LINEAR_ACCESS_TOKEN`. +- **hackerone**: Query HackerOne programs, scopes, reports, and hacktivity. Run `hackerone_health` first to verify credentials. Use `hackerone_get_program_scope` to enumerate in-scope assets before testing. Use `hackerone_search_hacktivity` to study previously disclosed vulnerabilities in a program. Use `hackerone_submit_report` only after the reporting pipeline completes (assess_confidence → report-preflight → exploit-verifier → report_item). Requires `H1_USERNAME` and `H1_API_TOKEN` env vars. +- **jira**: Create internal Jira remediation tickets from validated findings. Run `jira_health` first to verify credentials. Use `jira_get_create_metadata` before creating issues when the project or issue type is uncertain. Use `jira_create_issue` only after the reporting pipeline completes; include the validated report body, severity/priority mapping, and links to Dreadnode evidence or artifacts. Requires `JIRA_BASE_URL`, `JIRA_EMAIL`, and `JIRA_API_TOKEN` env vars. +- **github**: Create GitHub remediation issues from validated findings. Run `github_health` first to verify credentials. Use `github_list_labels` before creating issues when label names are uncertain. Use `github_create_issue` only after the reporting pipeline completes; include the validated report body, severity/priority labels, and links to Dreadnode evidence or artifacts. Do not post sensitive exploit details to public repositories unless the user explicitly confirms that disclosure is intended. Requires `GITHUB_TOKEN` with Issues write permission. +- **linear**: Create internal Linear remediation issues from validated findings. Run `linear_health` first to verify credentials. Use `linear_list_teams` to find the team UUID before creating issues. Use `linear_create_issue` only after the reporting pipeline completes; include the validated report body, severity/priority mapping, and links to Dreadnode evidence or artifacts. Requires `LINEAR_API_KEY` or `LINEAR_ACCESS_TOKEN`. Scan and tool output is input to your OODA loop, not a deliverable. When a scan completes, orient on the results, prioritize leads by exploitability, load relevant skills, and begin active exploitation immediately. A completed scan is the start of your work, not the end. @@ -154,16 +154,16 @@ When you find a vulnerability, your report will be reviewed by a senior penteste ### Reporting pipeline -Before writing any report, complete this sequence. No shortcuts: +Before reporting any finding, complete this sequence: 1. `assess_confidence` — Is this CONFIRMED? If not, it stays a lead. 2. Load `report-preflight` skill — Is this eligible? Pass the finding through the tier 1/2/3 gate. 3. Load `exploit-verifier` skill — Run the Triple-Check (static viability, dynamic trigger, sink confirmation). -4. Load `report-writer` skill — Write the deliverable only after steps 1-3 pass. +4. Call `report_item` with type `web_vulnerability` — populate fields from your validated evidence. The schema enforces structure; fill what you have, leave optional fields empty when black-box testing doesn't yield them. If triaging batch findings from scanners or jxscout, load `vuln-critic` first to filter before entering this pipeline. -Do not skip steps. Do not write reports for unverified findings. +Do not skip steps 1-3. Do not report unverified findings. ## Communication diff --git a/capabilities/web-security/mcp/hackerone.py b/capabilities/web-security/mcp/hackerone.py index ba3c089..116a41f 100644 --- a/capabilities/web-security/mcp/hackerone.py +++ b/capabilities/web-security/mcp/hackerone.py @@ -498,9 +498,9 @@ async def hackerone_submit_report( ) -> str: """Submit a new vulnerability report to a HackerOne program. - Only use this after the full reporting pipeline: assess_confidence - returned CONFIRMED, report-preflight passed, and exploit-verifier - completed the triple-check. + Only use this after the reporting pipeline completes: assess_confidence + returned CONFIRMED, report-preflight passed, exploit-verifier completed + the triple-check, and report_item has been called. """ client, err = await _h1.safe_get() if err: diff --git a/capabilities/web-security/skills/hackerone-recon/SKILL.md b/capabilities/web-security/skills/hackerone-recon/SKILL.md index 62282b0..9c5c1b9 100644 --- a/capabilities/web-security/skills/hackerone-recon/SKILL.md +++ b/capabilities/web-security/skills/hackerone-recon/SKILL.md @@ -57,7 +57,7 @@ Check for `informative`/`not-applicable` verdicts (understand rejections), `dupl ## Phase 4: Report Submission -After the full pipeline (`assess_confidence` -> `report-preflight` -> `exploit-verifier` -> `report-writer`): +After the reporting pipeline (`assess_confidence` -> `report-preflight` -> `exploit-verifier` -> `report_item`): ``` hackerone_submit_report( diff --git a/capabilities/web-security/skills/report-preflight/SKILL.md b/capabilities/web-security/skills/report-preflight/SKILL.md index e5577f5..7706bef 100644 --- a/capabilities/web-security/skills/report-preflight/SKILL.md +++ b/capabilities/web-security/skills/report-preflight/SKILL.md @@ -119,5 +119,5 @@ After classification, use `assess-confidence` to evaluate the finding with the e ## Skill Pipeline ``` -vuln-critic (evidence quality) → report-preflight (eligibility) → exploit-verifier (confirmation) → report-writer (deliverable) +vuln-critic (evidence quality) → report-preflight (eligibility) → exploit-verifier (confirmation) → report_item (structured output) ``` diff --git a/capabilities/web-security/skills/report-writer/SKILL.md b/capabilities/web-security/skills/report-writer/SKILL.md deleted file mode 100644 index eac674a..0000000 --- a/capabilities/web-security/skills/report-writer/SKILL.md +++ /dev/null @@ -1,222 +0,0 @@ ---- -name: report-writer -description: Write HackerOne vulnerability reports from validated findings. Enforces evidence quality, pre-report gating, CVSS blocks, PoC rules, and clear reproduction steps. Use when writing a vulnerability report, drafting a submission, or converting validated findings into a deliverable. ---- - -# Vulnerability Report Writer - -Converts validated, reproduced findings into structured vulnerability reports. - -**This skill writes reports. It does NOT validate findings.** If the finding hasn't been reproduced with ground truth evidence, stop and validate first using the exploit-verifier skill. - -## Pre-Report Gate - -Before writing, confirm ALL of the following. If any fail, halt and state what's missing: - -1. **Reproduced?** PoC executed, response matches expected vulnerable behavior -2. **Ground truth?** Evidence from actual HTTP responses, not inferred or differential -3. **Escalated?** Maximum demonstrable impact pursued (read one -> read all? self-only -> cross-user?) -4. **Scope confirmed?** Target and endpoint in-scope per program policy -5. **Not a known FP?** Cross-checked against exploit-verifier false positive patterns -6. **Preflight passed?** report-preflight skill returned ELIGIBLE or NEEDS_JUSTIFICATION with documented impact - -If uncertain on escalation, state: "Impact may not be fully escalated. Consider: [suggestions]" - -## Report Path - -``` -reports/R-.md -``` - -NNN = next sequential number (zero-padded: 001, 002, ...). slug = lowercase-hyphenated summary. - -## Confidence Trace - -- Extract the UUID from the `[trace_id:]` token in the `assess_confidence` tool response -- Copy that value into the report `confidence_trace_id` frontmatter field -- If the tool response does NOT contain `[trace_id:]`, set `confidence_trace_id: "MISSING"` — NEVER fabricate or guess an ID - -## PoC Rules - -- If Caido proxy is available, route curl commands through it: `curl -x http://localhost:8080 -k` -- Add `X-PoC-Step: step-X-` headers to each request for proxy screenshot filtering -- Add bug bounty authorization header per program policy if required -- After each HTTP response block, add: `` -- If tokens/credentials required, first steps must show how to obtain them — no bare `` placeholders -- Use `1.` for ALL step numbers (markdown auto-increments, avoids gaps on copy-paste) -- Reference JS source paths if discovered via static analysis - -## Section Notes - -- **Description**: If JS static analysis led to discovery, reference source paths -- **Impact**: If testing limitations exist (e.g., "only verified on free tier"), state inline after impact list -- **References**: Ground-truth only (OWASP, CWE, MITRE, vendor docs). Max 5 -- **Recommendations**: Root cause fix first, then defense-in-depth. Concise and actionable - -## Report Template - -````markdown ---- -confidence_trace_id: "" ---- - -# Title - - - - -# H1 Description - -## Summary - - - -**Weakness:** - -**CVSS 4.0:** [`CVSS:4.0/AV:_/AC:_/AT:_/PR:_/UI:_/VC:_/VI:_/VA:_/SC:_/SI:_/SA:_`](https://www.first.org/cvss/calculator/4.0) - -**CVSS 3.1:** [`CVSS:3.1/AV:_/AC:_/PR:_/UI:_/S:_/C:_/I:_/A:_`](https://nvd.nist.gov/vuln-metrics/cvss/v3-calculator?vector=) - -| | | -|---|---| -| **Asset** | | -| **Endpoint(s)** | | -| **Parameter(s)** | | -| **CWE** | | - ---- - -## Description - - - - ---- - -## Proof of Concept - -### Prerequisites (optional) - -1. *e.g., Victim account must have X feature enabled* - -### Setup (optional) - -| | | -|---|---| -| **Accounts** | See role table below | -| **Other** | | - -**Test Accounts:** - -| Role | Email | Description | -|------|-------|-------------| -| Attacker | `attacker@example.com` | | -| Victim | `victim@example.com` | | - -### Reproduction Steps - -#### As the Victim - -1. Sign into the application at `` -1. Navigate to **Feature X** and observe current state - - - -#### As the Attacker - -1. Sign into the application at `` -1. Navigate to **Feature X** -1. Observe/Intercept the following request: - -> _Original request:_ - -```http - -``` - -1. Forward this request to Replay/Repeater -1. Modify the request as follows: - - - -> _Modified request:_ - -```http - -``` - -1. Observe the server response: - -> _Server response:_ - -```http - -``` - - - -#### As the Victim (verification) - -1. Sign into the application at `` -1. Observe that - -### Video PoC - -_placeholder_ - ---- - -# H1 Impact - -## Impact - - - -With this vulnerability, an attacker can: - -1. **X**: Y -1. **Y**: Z - ---- - -## References - -- [CWE-XXX: Name](https://cwe.mitre.org/data/definitions/XXX.html) -- [OWASP Reference](https://owasp.org/) - -## Recommendations - -1. **Primary fix**: -1. **Defense-in-depth**: -```` - -## H1 Weakness Type Mappings - -| Vulnerability Class | H1 Weakness Type | -|---|---| -| IDOR / Broken Access Control | Insecure Direct Object Reference (IDOR) | -| XSS (Stored/Reflected/DOM) | Cross-site Scripting (XSS) - Stored/Reflected/DOM | -| SSRF | Server-Side Request Forgery (SSRF) | -| SQLi | SQL Injection | -| Auth bypass | Improper Authentication | -| CSRF | Cross-Site Request Forgery (CSRF) | -| Info disclosure | Information Disclosure | -| Privilege escalation | Privilege Escalation | -| Business logic | Business Logic Errors | -| Open redirect | Open Redirect | -| Race condition | Time-of-check Time-of-use (TOCTOU) Race Condition | -| Prompt injection / AI | AI/ML Security or Privilege Escalation (context dependent) | - -## Style - -- Human-written tone. No AI slop, no excessive dashes, no tautology, no emojis. -- One vulnerability per report unless chaining required for impact. -- Concise — tight descriptions, no redundant prose. -- Technical — write for a security engineer, not a manager. -- Factual — state what happened, not what might happen. -- Both CVSS 4.0 and 3.1 required with clickable calculator links matching demonstrated impact. -- Never fabricate PoC output — every response block must be from actual execution. -- Include a control test showing non-vulnerable behavior (invalid key -> denied, unauthed -> 401) to prove the delta. -- Every impact claim must trace to a PoC step — if you claim "quota abuse", demonstrate batch requests succeeding. -- Delineate scope: document what doesn't work alongside what does to prevent inflation. -- If report-preflight returned NEEDS_JUSTIFICATION, include the justification block in Impact.