67 lines
4.7 KiB
Markdown
67 lines
4.7 KiB
Markdown
# Quickstart: Baseline Compare Summary Trust Propagation & Compliance Claim Hardening
|
|
|
|
## Goal
|
|
|
|
Verify that compact baseline and drift summary surfaces stop issuing false compliant or all-clear claims when the underlying compare result is limited, incomplete, stale, in progress, unavailable, suppressed, or otherwise not decision-grade.
|
|
|
|
## Preconditions
|
|
|
|
1. Start Sail and ensure the tenant panel is accessible.
|
|
2. Use a tenant with an assigned baseline profile and an active baseline snapshot.
|
|
3. Prepare representative compare scenarios using existing factories or fixtures:
|
|
- trustworthy no-drift result
|
|
- limited-confidence zero-findings result
|
|
- evidence-gap-affected result with no open findings
|
|
- stale compare history with no new confirmed drift
|
|
- failed compare result
|
|
- no compare yet, compare in progress, or no snapshot result
|
|
|
|
## Manual Verification Flow
|
|
|
|
### Scenario 1: Trustworthy no-drift result
|
|
|
|
1. Open the tenant dashboard.
|
|
2. Confirm the baseline summary widget may show a positive aligned state and still links to the deeper Baseline Compare or run-detail path.
|
|
3. Confirm the same tenant's Baseline Compare landing page shows compatible no-drift semantics and preserves its findings or run-detail drilldowns.
|
|
4. Confirm the canonical run detail for the same compare is equally confident or more detailed, never less aligned.
|
|
|
|
### Scenario 2: Limited-confidence zero-findings result
|
|
|
|
1. Open the tenant dashboard for a compare result with zero visible findings but limited confidence or suppressed output.
|
|
2. Confirm the baseline summary widget does not show compliant or all-clear wording.
|
|
3. Confirm `Needs Attention` does not fall back to a blanket healthy message and does not introduce a new drilldown path if the surface is intentionally non-navigational.
|
|
4. Open the landing page and verify the primary explanation remains cautionary while its drilldowns still resolve to the expected findings or run-detail surface.
|
|
5. Open the run detail and confirm the summary was not more optimistic than the detail surface.
|
|
|
|
### Scenario 3: Evidence gaps with no open findings
|
|
|
|
1. Open a tenant with evidence gaps recorded but no open drift findings.
|
|
2. Confirm a compact summary surface visibly signals caution or review.
|
|
3. Confirm the coverage or evidence banner appears when appropriate and offers the expected drilldown path to landing or run detail.
|
|
4. Confirm the landing page still exposes deeper evidence-gap detail and diagnostics.
|
|
|
|
### Scenario 4: Missing, stale, or unusable result
|
|
|
|
1. Verify the stale-history state stays distinct from no-result and does not render as healthy.
|
|
2. Verify the compare-in-progress state is visibly in progress rather than unavailable or healthy.
|
|
3. Verify the no-snapshot or no-compare-yet state remains unavailable rather than in progress or healthy.
|
|
4. Verify the failed-compare state gives an investigation-oriented next step.
|
|
5. Verify the existing `Compare now` action remains available only where already authorized and correctly guarded.
|
|
|
|
## Automated Verification
|
|
|
|
Run the same focused verification pack referenced by `tasks.md` through Sail:
|
|
|
|
1. `vendor/bin/sail artisan test --compact tests/Feature/Baselines/BaselineCompareSummaryAssessmentTest.php tests/Feature/Baselines/BaselineCompareWhyNoFindingsReasonCodeTest.php tests/Feature/Baselines/BaselineCompareStatsTest.php tests/Feature/Baselines/BaselineCompareExplanationFallbackTest.php`
|
|
2. `vendor/bin/sail artisan test --compact tests/Feature/Filament/BaselineCompareNowWidgetTest.php tests/Feature/Filament/NeedsAttentionWidgetTest.php`
|
|
3. `vendor/bin/sail artisan test --compact tests/Feature/Filament/BaselineCompareExplanationSurfaceTest.php tests/Feature/Filament/BaselineCompareLandingWhyNoFindingsTest.php tests/Feature/Filament/BaselineCompareCoverageBannerTest.php`
|
|
4. `vendor/bin/sail artisan test --compact tests/Feature/Filament/BaselineCompareSummaryConsistencyTest.php tests/Feature/Filament/BaselineCompareLandingStartSurfaceTest.php tests/Feature/ReasonTranslation/ReasonTranslationExplanationTest.php`
|
|
5. `vendor/bin/sail artisan test --compact tests/Feature/Filament/OperationRunBaselineTruthSurfaceTest.php tests/Feature/Filament/TenantDashboardDbOnlyTest.php`
|
|
6. `vendor/bin/sail bin pint --dirty --format agent`
|
|
|
|
## Expected Outcome
|
|
|
|
- No in-scope summary surface presents compliant or equivalent all-clear copy for limited-confidence, incomplete-evidence, stale, in-progress, suppressed-result, failed, or unavailable scenarios.
|
|
- Trustworthy no-drift scenarios can still present a positive aligned state.
|
|
- Dashboard, landing, banner, and canonical detail surfaces remain semantically aligned.
|
|
- Existing compare action behavior, lazy widget behavior, and DB-only dashboard rendering remain intact. |