04d61cbad0
- Implement Spec 116 baseline capture/compare + coverage guard\n- Add UI surfaces and widgets for baseline compare\n- Add tests and research report
63 lines
3.1 KiB
Markdown
63 lines
3.1 KiB
Markdown
# Quickstart — Spec 116 Baseline Drift Engine
|
|
|
|
This quickstart is for developers validating the behavior locally.
|
|
|
|
## Prerequisites
|
|
|
|
- Sail up: `vendor/bin/sail up -d`
|
|
- Install deps (if needed): `vendor/bin/sail composer install`
|
|
- Run migrations: `vendor/bin/sail artisan migrate`
|
|
|
|
## Workflow
|
|
|
|
### 1) Run an inventory sync (establish coverage)
|
|
|
|
- Trigger inventory sync for a tenant using the existing UI/command for inventory sync.
|
|
- Verify the latest inventory sync `operation_runs.context` contains a coverage payload:
|
|
- `inventory.coverage.policy_types.{type}.status = succeeded|failed`
|
|
- (optionally) `inventory.coverage.foundation_types.{type}.status = ...`
|
|
|
|
Expected:
|
|
- If some types fail or are not processed, the coverage payload reflects that.
|
|
|
|
### 2) Capture a baseline snapshot
|
|
|
|
- Use the Baseline Profile UI to capture a snapshot.
|
|
|
|
Expected:
|
|
- Snapshot items store `baseline_hash` computed from the Inventory Meta Contract.
|
|
- Snapshot identity/dedupe follows existing snapshot identity rules, but content hashes come from the explicit contract.
|
|
|
|
### 3) Compare baseline to tenant
|
|
|
|
- Trigger “Compare now” from the Baseline Compare landing page.
|
|
|
|
Expected:
|
|
- Compare uses latest successful baseline snapshot by default (or explicit snapshot selection if provided).
|
|
- Compare uses the latest **completed** inventory sync run coverage:
|
|
- For uncovered policy types, **no findings are emitted**.
|
|
- OperationRun outcome becomes “completed with warnings” (partial) when uncovered types exist.
|
|
- `summary_counts.errors_recorded = count(uncovered_types)`.
|
|
- Edge case: if effective scope expands to zero types, outcome is still partial (warnings) and `summary_counts.errors_recorded = 1`.
|
|
- Fail-safe: if there is **no completed inventory sync run** or the coverage payload is missing/unreadable, coverage is treated as **unproven** for all effective types:
|
|
- **no findings are emitted**
|
|
- outcome is partial (warnings)
|
|
- compare run context includes `baseline_compare.coverage.proof = false`
|
|
- Findings identity:
|
|
- stable `recurrence_key` uses `baseline_snapshot_id` and does **not** include baseline/current hashes.
|
|
- `fingerprint == recurrence_key`.
|
|
- `scope_key` remains profile-scoped (`baseline_profile:{id}`).
|
|
|
|
### 4) Validate UI counts
|
|
|
|
- Verify baseline compare stats remain grouped by the profile scope (`scope_key = baseline_profile:{id}`), consistent with research.md Decision #2.
|
|
- Validate that re-capturing (new snapshot) creates a new set of findings due to snapshot-scoped identity (recurrence key includes `baseline_snapshot_id`).
|
|
|
|
## Minimal smoke test checklist
|
|
|
|
- Compare with full coverage: produces correct findings; outcome success.
|
|
- Compare with partial coverage: produces findings only for covered types; outcome partial; uncovered types listed in context.
|
|
- Compare with unproven coverage (no completed sync / missing coverage): emits zero findings; outcome partial; warning visible in UI.
|
|
- Re-run compare with no changes: no new findings; `times_seen` increments.
|
|
- Re-capture snapshot and compare: findings identity changes (snapshot-scoped).
|