226 lines
13 KiB
Markdown
226 lines
13 KiB
Markdown
# Feature Specification: Verification Checklist Framework V1.5 (Governance + Supportability + UI-Complete)
|
|
|
|
**Feature Branch**: `075-verification-v1_5`
|
|
**Created**: 2026-02-05
|
|
**Status**: Draft
|
|
**Input**: User description: "Extend verification checklist framework with report fingerprint + previous report change indicator, per-check acknowledgements with audit/confirmation, and issues-first operator-ready verify-step UX."
|
|
|
|
## Goal
|
|
|
|
V1.5 extends the V1 verification checklist framework with two enterprise-critical additions while keeping scope intentionally small:
|
|
|
|
1) **Supportability / determinism**: show whether results changed since the previous verification for the same identity.
|
|
2) **Governance**: allow explicit, auditable acknowledgement of known issues per failing check.
|
|
3) **Enterprise UX completeness**: make the Verify step operator-ready (issues-first, one clear primary action, technical details secondary).
|
|
|
|
## Clarifications
|
|
|
|
### Session 2026-02-05
|
|
|
|
- Q: How is “block” represented on checks? → A: No new status; a “Blocker” is `status=fail` with `blocking=true`.
|
|
- Q: Should `severity` be part of the fingerprint? → A: Yes; include `severity` always (normalize missing to empty) to keep hashing deterministic and to treat severity-only changes as “Changed”.
|
|
- Q: Should `ack_reason` be included in the audit event payload? → A: No; keep audit metadata minimal and store the reason only in the acknowledgement record.
|
|
- Q: How should `provider_connection_id` be treated when resolving `previous_report_id`? → A: Match exactly; `NULL` only matches `NULL` (no cross-connection mixing).
|
|
- **Report shape (canonical, inherited from 074)**: Persist reports as the existing V1 JSON shape (`schema_version`, `flow`, `generated_at`, `summary`, `checks[]`). V1.5 adds `fingerprint` + `previous_report_id` at the top level. No `sections[]` array is stored.
|
|
- **Idempotency (inherited)**: Deduplication applies only while a run is active (`queued` / `running`). Once `completed` / `failed`, starting verification creates a new run.
|
|
- **Viewing (inherited)**: Viewing is DB-only; rendering MUST NOT perform external calls.
|
|
- **Evidence (inherited)**: Evidence is limited to safe pointers only; no secrets (no tokens/claims/headers/raw payloads).
|
|
- **Next steps (inherited)**: Navigation-only links (no server-side “fix it” actions from the viewer).
|
|
|
|
## User Scenarios & Testing *(mandatory)*
|
|
|
|
### User Story 1 - Operator can tell “nothing changed” (Priority: P1)
|
|
|
|
As an operator, I can immediately see whether the current verification findings are unchanged compared to the previous verification for the same identity, so I can avoid unnecessary re-diagnosis.
|
|
|
|
**Why this priority**: This is the fastest path to supportability: it reduces repeated analysis and makes troubleshooting deterministic.
|
|
|
|
**Independent Test**: Create two reports for the same identity with identical normalized check outcomes; confirm the viewer indicates “No changes since previous verification”.
|
|
|
|
**Acceptance Scenarios**:
|
|
|
|
1. **Given** a report with a previous report available, **When** I open the viewer, **Then** I see a clear indicator “Changed” or “No changes”.
|
|
2. **Given** the current report has the same fingerprint as the previous report, **When** I open the viewer, **Then** I see “No changes since previous verification”.
|
|
|
|
---
|
|
|
|
### User Story 2 - Owner/Manager can acknowledge a known issue (Priority: P1)
|
|
|
|
As an owner/manager, I can acknowledge a failing/warning/blocking check with a short reason (and optionally an expiry) so the team can see that the risk is known, evaluated, and accepted.
|
|
|
|
**Why this priority**: Acknowledgements provide governance without masking risk; they improve shared context and auditability.
|
|
|
|
**Independent Test**: With and without the acknowledgement capability, attempt to acknowledge a failing check; assert correct authorization (403) and that an audit event is recorded for the successful path.
|
|
|
|
**Acceptance Scenarios**:
|
|
|
|
1. **Given** a check in status `fail` / `warn` (including failing blockers where `blocking=true`), **When** I acknowledge it with a reason, **Then** the UI shows who acknowledged it, when, and the reason.
|
|
2. **Given** I do not have the acknowledgement capability, **When** I attempt to acknowledge a check, **Then** the server returns 403 and the UI does not offer the acknowledgement action.
|
|
|
|
---
|
|
|
|
### User Story 3 - Verify step is operator-ready (issues-first) (Priority: P1)
|
|
|
|
As a workspace member, I see issues-first results with clear next steps and exactly one primary action (start or refresh), so I can remediate quickly without hunting through technical details.
|
|
|
|
**Why this priority**: The Verify step is a high-frequency operator surface; clarity and deterministic states reduce time-to-resolution.
|
|
|
|
**Independent Test**: Seed a report with blockers and a running state; confirm the default tab and “one primary CTA” rule is enforced in both completed and running scenarios.
|
|
|
|
**Acceptance Scenarios**:
|
|
|
|
1. **Given** a report with blockers, **When** I open the Verify step/viewer, **Then** the Issues tab is the default and blockers are at the top.
|
|
2. **Given** a run is active, **When** I open the Verify step/viewer, **Then** the primary action is “Refresh results” and technical links are secondary.
|
|
|
|
---
|
|
|
|
### Edge Cases
|
|
|
|
- No previous report exists for an identity → no “changed/no-change” indicator is shown.
|
|
- Run is active but no report is available yet → UI shows a clear “running, results will appear” explanation (no empty states without guidance).
|
|
- Partial report while running → partial results render with a “Partial results” label.
|
|
- Unknown check keys or reason codes → UI degrades gracefully, showing status and message without breaking.
|
|
- Acknowledgement attempted for non-acknowledgeable status (e.g., `pass`) → request is rejected and UI does not offer it.
|
|
|
|
## Out of Scope
|
|
|
|
- Diff/compare UI between reports
|
|
- Server-side fixes initiated from the viewer
|
|
- Undo / unacknowledge acknowledgements (V1.5 acknowledgements are immutable per report)
|
|
- Complex staleness/TTL semantics (fresh/stale/expired)
|
|
- Global dashboards / cross-tenant reporting
|
|
- Export features (PDF/JSON) as a product feature
|
|
- Live polling (V1.5 uses manual refresh)
|
|
|
|
## Requirements *(mandatory)*
|
|
|
|
**Constitution alignment (required):** This feature adds new tenant-scoped mutations (acknowledgements) and new report metadata. It MUST include explicit confirmation, audit logging for mutations, tenant isolation, and tests.
|
|
|
|
**Constitution alignment (RBAC-UX):** Tenant-scoped routes MUST preserve deny-as-not-found (404) for non-members, and use 403 for members missing a capability. UI visibility is not authorization; server-side enforcement is required.
|
|
|
|
**Constitution alignment (BADGE-001):** Status-like badges MUST use centralized mapping semantics; no ad-hoc UI mappings.
|
|
|
|
### Functional Requirements
|
|
|
|
- **FR-075-001 — Report fingerprint**: Each verification report MUST store a deterministic `fingerprint` derived from normalized check outcomes.
|
|
|
|
**Normalization rule (deterministic):**
|
|
- Flatten all check results across `report.checks[]`
|
|
- Sort by stable `check.key`
|
|
- For each check, contribute a stable string using: `key | status | blocking | reason_code | severity`
|
|
- `severity` MUST be included always; if the source report omits it, normalize to an empty string
|
|
- The fingerprint MUST be a stable cryptographic hash of the joined contributions, stored as a fixed-length lowercase hex string.
|
|
|
|
- **FR-075-002 — Previous report link**: Each report MUST store `previous_report_id` (nullable) that points to the most recent earlier report for the same **verification identity**.
|
|
|
|
**Identity match** MUST include:
|
|
- flow
|
|
- workspace
|
|
- tenant
|
|
- provider connection (`provider_connection_id`) matched exactly; `NULL` only matches `NULL`
|
|
|
|
- **FR-075-003 — Change indicator**: When a previous report exists, the viewer MUST show:
|
|
- “No changes since previous verification” if `fingerprint` matches
|
|
- “Changed since previous verification” otherwise
|
|
|
|
- **FR-075-004 — Per-check acknowledgements (first-class)**: The system MUST allow acknowledging checks with status `fail` / `warn`.
|
|
|
|
An acknowledgement MUST record:
|
|
- reason (max 160 characters)
|
|
- acknowledged timestamp
|
|
- acknowledged-by user
|
|
- optional expiry timestamp
|
|
|
|
Acknowledgements MUST be unique per (report, check key). Expiry, when provided, is informational only in V1.5 and MUST NOT introduce automatic staleness/TTL behavior.
|
|
|
|
- **FR-075-005 — Acknowledgement does not change outcomes**: Acknowledging MUST NOT change:
|
|
- the check status
|
|
- the report summary status/outcome
|
|
- the run outcome
|
|
|
|
- **FR-075-006 — Acknowledgement allowed conditions**: Acknowledgement MUST only be possible for checks whose status is in `{fail, warn}`. It MUST NOT be available for passing/green checks.
|
|
|
|
A check is considered a **Blocker** when `status=fail` and `blocking=true`; blockers are acknowledgeable under the same `{fail, warn}` rule (no separate `block` status exists).
|
|
|
|
- **FR-075-007 — Acknowledgement authorization (capability-first)**: Acknowledgement MUST require the capability `tenant_verification.acknowledge` as defined in the canonical capability registry.
|
|
|
|
RBAC UX semantics:
|
|
- non-member / not entitled to tenant scope → 404
|
|
- member without acknowledgement capability → 403
|
|
- members with tenant scope but without acknowledgement capability can still view reports (view remains read-only)
|
|
|
|
- **FR-075-007A — Viewing authorization semantics preserved (inherited)**: Viewing tenant-scoped verification pages (Verify step + report viewer) MUST preserve V1 semantics:
|
|
- non-member / not entitled to tenant scope → 404
|
|
- member with tenant scope → can view
|
|
- capability checks apply to mutations only (start verification, acknowledgement)
|
|
|
|
- **FR-075-008 — Confirmation + audit required**: Acknowledgement is a mutation and MUST require explicit user confirmation and MUST emit an audit event.
|
|
|
|
- **FR-075-009 — Audit event metadata (minimal)**: The audit event for acknowledgement MUST include minimally:
|
|
- workspace, tenant, run, report, flow
|
|
- check key and reason code
|
|
- acknowledged-by user
|
|
|
|
It MUST NOT include `ack_reason`, secrets, tokens, or raw payloads.
|
|
|
|
- **FR-075-010 — DB-only viewing guard (inherited)**: Rendering the viewer and the Verify step MUST NOT trigger external calls.
|
|
|
|
- **FR-075-011 — Centralized badge semantics (BADGE-001)**: All check-status badges and summary-status badges used by V1.5 MUST use the centralized badge mapping registry.
|
|
|
|
- **FR-075-012 — Verify step enterprise UX (normative)**: The Verify step/viewer MUST follow an issues-first layout and deterministic UI states:
|
|
|
|
**Structure**
|
|
- Always-visible summary card
|
|
- Tabs: Issues (default), Passed, Technical details
|
|
|
|
**DB-only hint**
|
|
- The summary surface MUST include a clear hint that viewing is read-only and performs no external calls.
|
|
|
|
**Primary action rule (strict)**
|
|
- Exactly one primary call-to-action is shown at any time
|
|
- “Start verification” and “Refresh results” MUST NOT both be primary simultaneously
|
|
|
|
**Issues tab ordering**
|
|
1) Blockers (not acknowledged)
|
|
2) Failures (not acknowledged)
|
|
3) Warnings (not acknowledged)
|
|
4) Acknowledged issues (collapsed group)
|
|
|
|
**Next steps rendering**
|
|
- Max 2 navigation-only links per issue card
|
|
- “Open run details” MUST appear only in Technical details (not in issue cards)
|
|
|
|
**Technical details**
|
|
- Secondary surface that can show identifiers (run/report IDs), fingerprint, and previous report link
|
|
- No raw payloads/tokens/full error bodies
|
|
|
|
### Key Entities *(include if feature involves data)*
|
|
|
|
- **Verification Identity**: The stable identifiers that define “what is being verified” (flow, workspace, tenant, and optional provider connection).
|
|
- **Verification Report**: A structured record of verification outcomes for a run.
|
|
- **Report Fingerprint**: A deterministic hash representing normalized check outcomes.
|
|
- **Previous Report**: The immediately preceding report for the same identity.
|
|
- **Check Acknowledgement**: A governance record that an issue is known/accepted (who/when/reason/optional expiry) without altering the check outcome.
|
|
|
|
### Assumptions
|
|
|
|
- A verification run/report concept already exists from V1.
|
|
- The system has an audit log mechanism capable of recording acknowledgement actions.
|
|
- Manual refresh is acceptable (no polling required).
|
|
|
|
### Dependencies
|
|
|
|
- Spec 074 (Verification Checklist Framework V1)
|
|
|
|
## Success Criteria *(mandatory)*
|
|
|
|
### Measurable Outcomes
|
|
|
|
- **SC-075-001 (Supportability)**: With a previous report present, operators can determine “changed vs no changes” within 10 seconds in 95% of tested sessions.
|
|
- **SC-075-002 (Governance)**: 100% of successful acknowledgements create an audit log record with minimal metadata and no sensitive content.
|
|
- **SC-075-003 (UX determinism)**: The Verify step renders exactly one primary CTA in all tested UI states (not started, running with/without report, completed).
|
|
- **SC-075-004 (Authorization correctness)**: Non-members receive 404 for tenant-scoped access routes in 100% of tests; members without acknowledgement capability receive 403 for acknowledgement attempts in 100% of tests.
|
|
- **SC-075-005 (No greenwashing)**: Acknowledging an issue never changes check status or the report summary in any tested scenario.
|
|
|
|
```
|