# Feature Specification: Verification Checklist Framework (Enterprise-Ready) **Feature Branch**: `074-verification-checklist` **Created**: 2026-02-03 **Status**: Draft **Input**: User description: "Replace binary verification UX with a structured, reusable verification checklist attached to verification runs; DB-only viewing; enterprise semantics (reason codes, audit, idempotency, RBAC)." ## Clarifications ### Session 2026-02-03 - Q: What idempotency policy do we want for “Start verification”? → A: Dedupe only while a run is active (queued/running); once completed/failed, “Start verification” creates a new run. - Q: Who should be allowed to view verification reports? → A: Any authenticated workspace member with access to the tenant scope may view reports; starting verification requires a separate capability. - Q: What policy should we use for `reason_code` taxonomy? → A: Versioned central taxonomy with a small baseline set + reserved `ext.*` namespace for feature-specific extensions. - Q: What’s the required evidence/redaction policy for `evidence` in check results? → A: Evidence is strictly structured safe pointers only (internal IDs, masked strings, hashes); never raw payloads, tokens, claims, headers, or full error bodies. - Q: Should “Next steps” CTAs be links only, or can they trigger server-side actions? → A: Links only (navigation-only) in v1. ## User Scenarios & Testing *(mandatory)* ### User Story 1 - Operator sees what’s wrong (Priority: P1) As a workspace member onboarding or operating a managed tenant, I can run “Verify access” and see a structured checklist that clearly shows which checks passed, which failed, and what to do next. **Why this priority**: This is the primary value of verification: reduce ambiguity and enable fast, correct remediation. **Independent Test**: Seed a verification run with a report containing mixed outcomes and confirm the viewer renders an accurate summary, per-check status, and next steps without making any external calls. **Acceptance Scenarios**: 1. **Given** a completed verification run with 2 failed checks, **When** I open the verification report viewer, **Then** I see an overall summary (“Needs attention” or “Blocked”), counts, and the two failed checks with actionable next steps. 2. **Given** a verification run that is still in progress, **When** I open the viewer, **Then** I see a “Running” state and partial results (if available) without errors. --- ### User Story 2 - Deterministic starts (idempotency / dedupe) (Priority: P2) As an operator, if I click “Start verification” multiple times for the same tenant + provider connection + flow, the system behaves deterministically: it does not start duplicate active runs and guides me to the already-running run. **Why this priority**: Prevents confusing duplicates, reduces load, and makes support/debugging repeatable. **Independent Test**: Attempt to start verification twice for the same identity and assert that only one active run exists and the UI returns a consistent “already running” outcome. **Acceptance Scenarios**: 1. **Given** an active verification run exists for the same identity, **When** I click “Start verification”, **Then** no duplicate run is started and I am directed to view the active run/report. 2. **Given** no active verification run exists (including when the most recent run is completed or failed), **When** I click “Start verification”, **Then** a new run starts and I can view its report as it progresses. --- ### User Story 3 - Least-privilege and safe disclosure (Priority: P3) As a workspace member with access to the tenant scope (including read-only), I can view verification reports but cannot start verification unless I have the start capability. As a non-member, I cannot discover that a tenant or report exists. **Why this priority**: Verification data can leak operational posture; access must follow least-privilege and “deny-as-not-found” for non-members. **Independent Test**: Validate both authorization paths: read-only can view but cannot start; non-member receives a not-found response for all tenant-scoped verification routes. **Acceptance Scenarios**: 1. **Given** I am a workspace member without the “start verification” capability, **When** I open the verification page, **Then** I can view past reports but the “Start verification” action is disabled and cannot be executed. 2. **Given** I am not a member of the workspace/tenant scope, **When** I attempt to access the verification report route, **Then** I receive a not-found response with no identifying hints. --- ### Edge Cases - Report missing or malformed (e.g., run exists but report is absent or partial) → viewer shows a safe “Report unavailable” state and guidance. - Unknown check keys or unknown reason codes (newer schema written by a newer verifier) → viewer degrades gracefully, still showing status/message/next steps when present. - Large reports (near upper bound, e.g., 50 checks) → viewer remains responsive and summary counts remain correct. - A run transitions from running → complete while the user is viewing → the viewer refreshes safely or the user can re-open without inconsistent states. - Evidence contains unexpected fields → redaction rules prevent sensitive values from being displayed. ## Out of Scope - Introducing a separate monitoring/observability platform beyond the existing run tracking and audit log. - Any workflow that requires client-side handling of secrets. - A full overhaul of onboarding wizards beyond replacing/embedding verification status with the checklist viewer. - Provider job orchestration redesign unrelated to running verification checks. - Server-side actions triggered directly from the checklist viewer (v1 is navigation-only). ## Requirements *(mandatory)* **Constitution alignment (required):** If this feature introduces any external provider calls, any write/change behavior, or any long-running/background work, the spec MUST describe safety gates (preview/confirmation/audit), tenant isolation, run observability (run identity, visibility, and outcomes), and tests. If security-relevant DB-only actions intentionally skip run tracking, the spec MUST describe the audit log entries. **Constitution alignment (RBAC-UX):** If this feature introduces or changes authorization behavior, the spec MUST: - state which authorization plane(s) are involved (tenant-scoped admin area vs platform/system admin area), - ensure any cross-plane access is deny-as-not-found (404), - explicitly define 404 vs 403 semantics: - non-member / not entitled to tenant scope → 404 (deny-as-not-found) - member but missing capability → 403 - describe how authorization is enforced server-side (Gates/Policies) for every mutation/operation-start/credential change, - reference the canonical capability registry (no raw capability strings; no role-string checks in feature code), - ensure global search is tenant-scoped and non-member-safe (no hints; inaccessible results treated as 404 semantics), - ensure destructive-like actions require explicit user confirmation, - include at least one positive and one negative authorization test, and note any RBAC regression tests added/updated. **Constitution alignment (OPS-EX-AUTH-001):** OIDC/SAML login handshakes may perform synchronous outbound HTTP (e.g., token exchange) on `/auth/*` endpoints without an `OperationRun`. This MUST NOT be used for Monitoring/Operations pages. **Constitution alignment (BADGE-001):** If this feature changes status-like badges (status/outcome/severity/risk/availability/boolean), the spec MUST describe how badge semantics stay centralized (no ad-hoc mappings) and which tests cover any new/changed values. ### Functional Requirements - **FR-001 — Canonical verification report contract**: The system MUST generate a versioned “Verification Report” document for each verification run, including: schema version, flow identifier, identity/scope, generated timestamp, summary counts, and a list of check results. - **FR-002 — Check result contract**: Each check result MUST include: stable key, title, status (pass/fail/warn/skip/running), severity (info/low/medium/high/critical), blocking flag, reason code, human-readable message, safe evidence pointers, and one or more “next steps” actions (where applicable). Evidence MUST be strictly limited to structured safe pointers (internal IDs, masked strings, hashes) and MUST NOT contain raw payloads, tokens, claims, headers, or full error bodies. Next steps in v1 MUST be navigation-only (links to internal pages or external documentation) and MUST NOT trigger server-side actions. - **FR-003 — Stable reason code taxonomy**: The system MUST use stable, documented reason codes for failed/warned/skipped outcomes so that support, automation, and future UI changes remain consistent. The taxonomy MUST include a small baseline set of cross-cutting codes and MUST reserve an `ext.*` namespace for flow-specific or check-specific extensions. - **FR-004 — DB-only viewing**: Viewing a verification checklist MUST be read-only and MUST NOT trigger any external calls (e.g., no provider API calls, no HTTP calls, no background jobs started as a side effect of rendering). - **FR-005 — Start verification creates a run**: Starting verification MUST create (or reuse, per dedupe policy) a new verification run record and begin executing the verification checks using existing background processing. - **FR-006 — Dedupe / idempotency**: If a verification run is already active for the same identity (tenant + provider connection + flow), the system MUST NOT start a duplicate active run; it MUST present a clear “already running” outcome and an affordance to view the active run/report. If no run is active (including when the most recent run is completed or failed), “Start verification” MUST create a new run. - **FR-007 — Capability-first authorization**: Permission checks for viewing and starting verification MUST reference the canonical capability registry (no string-literal capability checks in feature code). - **FR-008 — RBAC UX semantics**: Non-members attempting to access tenant-scoped verification pages/routes MUST receive not-found responses. Members lacking the “start” capability MUST be able to view reports but MUST NOT be able to start verification (UI disabled + server-side enforcement). Viewing reports MUST NOT require the start capability. - **FR-009 — Standardized UI semantics**: The viewer MUST render consistent status labels, a summary banner (e.g., Ready / Needs attention / Blocked), and per-check expandable details with standardized “Next steps” calls-to-action. - **FR-010 — Reuse across suite**: The framework MUST be adoptable by multiple verification flows without re-implementing viewer logic, including: managed tenant onboarding verification, provider connection verification, RBAC setup verification, consent & permission verification, and future readiness/health checks. - **FR-011 — Auditing**: Starting and completing verification MUST emit audit events with stable action identifiers and redaction rules, recording minimal metadata (workspace/tenant identifiers, run identifier, and result counts). ### Key Entities *(include if feature involves data)* - **Verification Flow**: A named verification context (e.g., managed tenant onboarding) that defines which checks run. - **Verification Identity (Scope)**: The set of identifiers that uniquely represent “what is being verified” (tenant + provider connection + flow). - **Verification Run**: A single execution attempt for a given identity that produces a report (and is auditable). - **Verification Report**: A versioned, structured document attached to a run, containing summary and check results. - **Check Definition**: A reusable definition of an atomic readiness check (key, title, expected preconditions, severity, blocking behavior). - **Check Result**: The outcome of executing a check within a report. - **Reason Code**: A stable, machine-readable classification of why a check is pass/fail/warn/skip. - **Next Step**: An actionable remediation hint (label + optional destination/action) that helps the operator resolve a failed check. - **Evidence Pointer**: Safe references that support diagnostics (IDs, masked strings, hashes), without exposing secrets. ### Assumptions - A run-tracking mechanism already exists and can store an attached, versioned verification report per run. - A canonical capability registry exists and is the source of truth for permission checks. - An audit logging mechanism exists that can record start/complete events with redaction. - Verification execution uses existing background processing patterns (no new observability platform is introduced). ### Dependencies - Workspace membership and tenant-scoped authorization boundaries are already modeled. - Run visibility rules support “deny-as-not-found” behavior for non-members. - UI surfaces exist (or can be added) where “Next steps” can route users for remediation. ## Success Criteria *(mandatory)* ### Measurable Outcomes - **SC-001 (Clarity)**: In a usability test with a pre-seeded failed report, 90% of operators can identify the top blocking failure and the recommended next step within 60 seconds. - **SC-002 (Determinism)**: When “Start verification” is triggered repeatedly for the same identity while a run is active, the system starts at most 1 active run (0 duplicates) and always provides a path to view the active run. - **SC-003 (Safety / data minimization)**: Verification reports contain no secrets or tokens; evidence is limited to safe pointers (validated by automated tests and/or static checks). - **SC-004 (Performance)**: The verification report viewer renders within 200ms server time for a typical report of up to 50 checks. - **SC-005 (Authorization)**: Non-member access to tenant-scoped verification pages results in not-found responses in 100% of tested cases; members without the start capability cannot execute start actions in 100% of tested cases.