TenantAtlas/specs/235-baseline-capture-truth/quickstart.md
ahmido 2752515da5
Some checks failed
Main Confidence / confidence (push) Failing after 54s
Spec 235: harden baseline truth and onboarding flows (#271)
## Summary
- harden baseline capture truth, compare readiness, and monitoring explanations around latest inventory eligibility, blocked prerequisites, and zero-subject outcomes
- improve onboarding verification and bootstrap recovery handling, including admin-consent callback invalidation and queued execution legitimacy/report behavior
- align workspace findings/workspace overview signals and refresh the related spec, roadmap, and spec-candidate artifacts

## Validation
- `cd apps/platform && ./vendor/bin/sail bin pint --dirty --format agent`
- `cd apps/platform && ./vendor/bin/sail artisan test --compact tests/Feature/BaselineDriftEngine/BaselineCaptureAuditEventsTest.php tests/Feature/BaselineDriftEngine/BaselineSnapshotNoTenantIdentifiersTest.php tests/Feature/BaselineDriftEngine/CaptureBaselineContentTest.php tests/Feature/BaselineDriftEngine/CaptureBaselineFullContentOnDemandTest.php tests/Feature/BaselineDriftEngine/CaptureBaselineMetaFallbackTest.php tests/Feature/Baselines/BaselineCaptureTest.php tests/Feature/Baselines/BaselineCompareFindingsTest.php tests/Feature/Baselines/BaselineSnapshotBackfillTest.php tests/Feature/Filament/BaselineCaptureResultExplanationSurfaceTest.php tests/Feature/Filament/BaselineCompareLandingStartSurfaceTest.php tests/Feature/Filament/BaselineProfileCaptureStartSurfaceTest.php tests/Feature/Filament/OperationRunBaselineTruthSurfaceTest.php tests/Feature/Monitoring/AuditCoverageGovernanceTest.php tests/Feature/Monitoring/GovernanceOperationRunSummariesTest.php tests/Feature/Notifications/OperationRunNotificationTest.php tests/Feature/Authorization/OperatorExplanationSurfaceAuthorizationTest.php`
- `cd apps/platform && ./vendor/bin/sail artisan test --compact tests/Feature/AdminConsentCallbackTest.php tests/Feature/Filament/WorkspaceOverviewDbOnlyTest.php tests/Feature/Guards/Spec194GovernanceActionSemanticsGuardTest.php tests/Feature/ManagedTenantOnboardingWizardTest.php tests/Feature/Onboarding/OnboardingVerificationTest.php tests/Feature/Operations/QueuedExecutionAuditTrailTest.php tests/Unit/Operations/QueuedExecutionLegitimacyGateTest.php`

## Notes
- browser validation was not re-run in this pass

Co-authored-by: Ahmed Darrazi <ahmed.darrazi@live.de>
Reviewed-on: #271
2026-04-24 05:44:54 +00:00

165 lines
8.3 KiB
Markdown

# Quickstart: Baseline Capture Truthful Outcomes and Upstream Guardrails
## Purpose
Validate that baseline capture no longer reports green success when the upstream inventory basis is not credible or when the capture finds zero in-scope subjects, confirm that the previous consumable baseline remains the effective compare anchor until a new consumable snapshot exists, and verify that Monitoring, audit prose, and terminal notification copy stay aligned to the same dominant truth.
## Preconditions
1. Sail services are running.
2. The workspace has a tenant, baseline profile, and inventory fixtures available for the targeted tests.
3. Baseline resources remain Filament v5 / Livewire v4 surfaces; no extra asset build is expected beyond standard PHP/test tooling.
## Validation Flow
### 1. Start-surface preflight blocks non-credible inventory truth
**Goal**: Known upstream problems are rejected before enqueue and use the shared reason-code family.
Check:
- No latest relevant inventory sync exists.
- Latest relevant inventory sync is blocked or failed.
- Latest relevant inventory sync exists but does not provide usable in-scope coverage.
Expected result:
- Capture does not enqueue a run.
- The start surface shows the shared translated baseline-capture reason.
- No success wording appears.
## 2. Runtime recheck blocks prerequisite drift after enqueue
**Goal**: If the latest relevant inventory state changes after page load or after enqueue, the queued run still resolves truthfully.
Check:
- Enqueue capture with a credible latest inventory basis.
- Change the latest relevant inventory run to a blocked/failed/unusable-coverage/non-credible state before the job evaluates subjects.
Expected result:
- The run ends as `completed + blocked`.
- Monitoring and run-detail explanation lead with the upstream inventory reason.
- No snapshot becomes the current consumable baseline.
## 3. Consumable capture still produces succeeded baseline truth
**Goal**: A clean capture with at least one resolved in-scope subject still succeeds and advances effective baseline truth.
Check:
- Use a credible latest inventory run whose effective in-scope subject count is greater than zero.
- Ensure the capture completes without warning conditions that would intentionally downgrade the run to `partially_succeeded`.
Expected result:
- The run ends as `completed + succeeded`.
- A consumable snapshot is produced or reused consistently with the current truth contract.
- `BaselineProfile.active_snapshot_id` remains anchored to the consumable current snapshot after the run, whether that required a new pointer update or reuse of the already-current consumable snapshot.
- Run context and audit summary preserve the same metadata contract for success, including the eligibility decision, upstream inventory reference, and whether current baseline truth changed.
## 4. Zero-subject capture produces no-data truth, not a green refresh
**Goal**: A capture with zero in-scope subjects remains visible but cannot silently refresh current baseline truth.
Check:
- Use a credible latest inventory run whose effective in-scope subject count is zero.
Expected result:
- The run ends as `completed + partially_succeeded`.
- The dominant reason is the zero-subject/no-data capture code.
- Any retained snapshot artifact is non-consumable.
- `BaselineProfile.active_snapshot_id` remains on the previous consumable snapshot.
## 5. Compare readiness stays anchored to consumable baseline truth
**Goal**: Compare surfaces must reflect whether a usable baseline actually exists, not whether a capture was merely attempted.
Check:
- Trigger a blocked capture after a previously successful baseline exists.
- Trigger a zero-subject capture after a previously successful baseline exists.
- Trigger a blocked or zero-subject capture when no previous consumable baseline exists.
Expected result:
- With a previous consumable baseline, compare remains available against that prior truth and explains that the latest capture did not refresh baseline truth.
- The profile-level compare affordance reflects the same truthful availability state and guidance as compare landing.
- Without any consumable baseline, compare remains unavailable and explains why.
## 6. Monitoring summary stays explanation-first
**Goal**: Operators should immediately see whether the run was blocked upstream or completed with no usable baseline.
Check:
- Open the run detail for a blocked latest-inventory capture.
- Open the run detail for a failed latest-inventory capture.
- Open the run detail for a zero-subject capture.
Expected result:
- The summary headline differentiates blocked upstream prerequisites, failed latest inventory, and no-data capture.
- Raw numeric counts remain secondary diagnostics.
## 7. Audit prose and terminal notification stay aligned with run truth
**Goal**: Interactive runs, initiator-null runs, and audit coverage must preserve the same dominant baseline-capture reason without introducing notification drift.
Check:
- Complete an interactive blocked baseline-capture run and inspect the terminal `OperationRunCompleted` payload.
- Complete an interactive zero-subject baseline-capture run and inspect the terminal `OperationRunCompleted` payload.
- Complete an initiator-null or scheduled baseline-capture run with blocked or no-data truth.
- Inspect the governance audit coverage surface or assertions for the same runs.
Expected result:
- Interactive terminal notifications use the same dominant blocked or no-data reason vocabulary as Monitoring.
- Initiator-null runs emit no terminal DB notification while preserving Monitoring and audit truth.
- Audit prose records the same dominant cause and whether current baseline truth changed.
## Commands
Run the narrowest proof set from repository root:
```bash
export PATH="/bin:/usr/bin:/usr/local/bin:$PATH" && cd apps/platform && ./vendor/bin/sail bin pint --dirty --format agent
export PATH="/bin:/usr/bin:/usr/local/bin:$PATH" && cd apps/platform && ./vendor/bin/sail artisan test --compact tests/Feature/Baselines/BaselineCaptureTest.php
export PATH="/bin:/usr/bin:/usr/local/bin:$PATH" && cd apps/platform && ./vendor/bin/sail artisan test --compact tests/Feature/Filament/BaselineProfileCaptureStartSurfaceTest.php tests/Feature/Filament/BaselineCompareLandingStartSurfaceTest.php tests/Feature/Filament/BaselineCaptureResultExplanationSurfaceTest.php
export PATH="/bin:/usr/bin:/usr/local/bin:$PATH" && cd apps/platform && ./vendor/bin/sail artisan test --compact tests/Feature/Filament/OperationRunBaselineTruthSurfaceTest.php tests/Feature/Monitoring/GovernanceOperationRunSummariesTest.php tests/Feature/Monitoring/AuditCoverageGovernanceTest.php tests/Feature/Notifications/OperationRunNotificationTest.php tests/Feature/Authorization/OperatorExplanationSurfaceAuthorizationTest.php
```
Run the legacy edge check only if implementation touches historical empty-snapshot classification:
```bash
export PATH="/bin:/usr/bin:/usr/local/bin:$PATH" && cd apps/platform && ./vendor/bin/sail artisan test --compact tests/Feature/Baselines/BaselineSnapshotBackfillTest.php
```
## Manual Smoke Focus
If a manual UI check is needed after automated proof:
1. Open a baseline profile detail page and verify the capture header action shows the translated upstream block when inventory truth is not credible.
2. Open compare landing after a zero-subject capture and verify it explains whether the prior consumable baseline still anchors compare availability.
3. Open the Monitoring run detail and verify the headline distinguishes upstream block from no-data capture before showing counts.
4. Verify an interactive run shows aligned terminal notification wording, while an initiator-null run leaves notification delivery suppressed and keeps Monitoring as the audit surface.
## Close-out Record
Record the feature close-out outcome here and mirror it into the active PR description:
1. `Guardrail` status for the changed native Filament surfaces.
2. Whether `standard-native-filament` and `monitoring-state-page` coverage both ran successfully.
3. Whether `T022` stayed `document-in-feature` or required a follow-up for legacy empty-snapshot behavior.
### Recorded Outcome
- `Guardrail`: pass
- `standard-native-filament`: pass
- `monitoring-state-page`: pass
- `T022`: implemented in feature; legacy empty complete snapshots are now backfilled as incomplete no-data captures with `baseline.capture.zero_subjects`