147 lines
5.3 KiB
Markdown
147 lines
5.3 KiB
Markdown
# Quickstart: Humanized Diagnostic Summaries for Governance Operations
|
|
|
|
## Goal
|
|
|
|
Validate that canonical governance operation run detail now answers the first operator question with one dominant summary, one short reason, one affected-scale cue where available, and one next step, while keeping raw diagnostics secondary and preserving current authorization and navigation semantics.
|
|
|
|
## Prerequisites
|
|
|
|
1. Start Sail if it is not already running.
|
|
2. Ensure the acting user is a valid workspace member and is entitled to the target tenant where the run is tenant-bound.
|
|
3. Prepare representative runs for these cases:
|
|
- blocked baseline capture with no usable inventory basis
|
|
- baseline compare with ambiguous matches or evidence gaps
|
|
- evidence snapshot generation with stale or incomplete output
|
|
- tenant review composition with missing sections or stale evidence
|
|
- review-pack generation with internal-only or blocked outcome
|
|
- one multi-cause degraded run
|
|
- one zero-output or all-zero run that must not read as green
|
|
|
|
## Focused Automated Verification
|
|
|
|
Run formatting first:
|
|
|
|
```bash
|
|
cd apps/platform && ./vendor/bin/sail bin pint --dirty --format agent
|
|
```
|
|
|
|
Then run the smallest proving set:
|
|
|
|
```bash
|
|
cd apps/platform && ./vendor/bin/sail artisan test --compact \
|
|
tests/Feature/Monitoring/GovernanceOperationRunSummariesTest.php \
|
|
tests/Feature/Filament/OperationRunBaselineTruthSurfaceTest.php \
|
|
tests/Feature/Monitoring/ArtifactTruthRunDetailTest.php \
|
|
tests/Feature/Authorization/OperatorExplanationSurfaceAuthorizationTest.php \
|
|
tests/Feature/RunAuthorizationTenantIsolationTest.php \
|
|
tests/Unit/Support/OpsUx/GovernanceRunDiagnosticSummaryBuilderTest.php \
|
|
tests/Unit/Support/OperatorExplanation/OperatorExplanationBuilderTest.php
|
|
```
|
|
|
|
If the new focused suite is not yet isolated, run the Monitoring subset instead:
|
|
|
|
```bash
|
|
cd apps/platform && ./vendor/bin/sail artisan test --compact tests/Feature/Monitoring/
|
|
```
|
|
|
|
## Manual Validation Pass
|
|
|
|
### 1. Canonical run detail entry path
|
|
|
|
Open `/admin/operations` and drill into a governance run.
|
|
|
|
Confirm that:
|
|
|
|
- row navigation remains the inspect model,
|
|
- no new row or header action appears,
|
|
- and arriving from tenant context does not silently widen back to all-tenant semantics.
|
|
|
|
### 2. Baseline capture blocked by prerequisite
|
|
|
|
Open a blocked baseline-capture run.
|
|
|
|
Confirm that:
|
|
|
|
- the page leads with `no baseline was captured`-style meaning,
|
|
- the missing prerequisite appears before raw payloads,
|
|
- execution status and artifact usability are visible as separate facts,
|
|
- and raw diagnostics remain lower on the page.
|
|
|
|
### 3. Baseline compare with ambiguity or suppressed output
|
|
|
|
Open a baseline-compare run with evidence gaps, ambiguous matches, or suppressed output.
|
|
|
|
Confirm that:
|
|
|
|
- the first summary names the compare outcome and its trust limitation,
|
|
- the dominant cause is understandable without raw JSON,
|
|
- any affected-scale cue is visible when supported by stored counts or gap detail,
|
|
- and `0 findings` or zero-output does not read as an all-clear.
|
|
|
|
### 4. Evidence snapshot generation
|
|
|
|
Open a run that produced stale or incomplete evidence.
|
|
|
|
Confirm that:
|
|
|
|
- processing success does not imply trustworthy evidence,
|
|
- the page states the evidence limitation before technical payloads,
|
|
- and next-step guidance points to the right recovery action.
|
|
|
|
### 5. Tenant review composition and review-pack generation
|
|
|
|
Open one review-compose run and one review-pack-generation run.
|
|
|
|
Confirm that:
|
|
|
|
- review generation can explain missing sections or stale evidence without JSON,
|
|
- pack generation can explain internal-only or blocked shareability outcomes,
|
|
- and related artifact links remain available only when the actor is entitled to them.
|
|
|
|
### 6. Multi-cause degraded run
|
|
|
|
Open a run with two or more stored degraded causes.
|
|
|
|
Confirm that:
|
|
|
|
- one dominant cause is shown first,
|
|
- at least one secondary cause is still discoverable,
|
|
- and the ordering is stable across reloads.
|
|
|
|
### 7. Cross-family parity
|
|
|
|
Open two covered governance runs from different families that share the same dominant cause class.
|
|
|
|
Confirm that:
|
|
|
|
- the same cause class keeps the same primary reading direction,
|
|
- the next-step category stays consistent where the persisted truth supports the same operator action,
|
|
- and cross-family wording does not drift into conflicting operator guidance.
|
|
|
|
### 8. Authorization and tenant safety
|
|
|
|
Confirm that:
|
|
|
|
- non-members still receive deny-as-not-found behavior,
|
|
- in-scope members lacking capability still receive `403` where expected,
|
|
- summary text does not leak inaccessible tenant or artifact hints,
|
|
- and `OperationRun` remains non-searchable.
|
|
|
|
### 9. Ten-second scan check
|
|
|
|
Timebox the first visible scan of one blocked, one degraded, and one zero-output governance run detail page.
|
|
|
|
Confirm that within 10-15 seconds an operator can determine:
|
|
|
|
- what happened,
|
|
- whether the resulting artifact is trustworthy enough to act on,
|
|
- what was affected when the stored data supports that cue,
|
|
- and what the next step is,
|
|
|
|
without opening diagnostic sections.
|
|
|
|
## Final Verification Notes
|
|
|
|
- Keep diagnostics present but secondary.
|
|
- Do not add retry, cancel, force-fail, or other intervention controls as part of this slice.
|
|
- If a manual reviewer sees the same dominant-cause copy both in a banner and in the decision zone, treat that as a regression and tighten the summary ownership. |