Some checks failed
Main Confidence / confidence (push) Failing after 54s
## Summary - harden baseline capture truth, compare readiness, and monitoring explanations around latest inventory eligibility, blocked prerequisites, and zero-subject outcomes - improve onboarding verification and bootstrap recovery handling, including admin-consent callback invalidation and queued execution legitimacy/report behavior - align workspace findings/workspace overview signals and refresh the related spec, roadmap, and spec-candidate artifacts ## Validation - `cd apps/platform && ./vendor/bin/sail bin pint --dirty --format agent` - `cd apps/platform && ./vendor/bin/sail artisan test --compact tests/Feature/BaselineDriftEngine/BaselineCaptureAuditEventsTest.php tests/Feature/BaselineDriftEngine/BaselineSnapshotNoTenantIdentifiersTest.php tests/Feature/BaselineDriftEngine/CaptureBaselineContentTest.php tests/Feature/BaselineDriftEngine/CaptureBaselineFullContentOnDemandTest.php tests/Feature/BaselineDriftEngine/CaptureBaselineMetaFallbackTest.php tests/Feature/Baselines/BaselineCaptureTest.php tests/Feature/Baselines/BaselineCompareFindingsTest.php tests/Feature/Baselines/BaselineSnapshotBackfillTest.php tests/Feature/Filament/BaselineCaptureResultExplanationSurfaceTest.php tests/Feature/Filament/BaselineCompareLandingStartSurfaceTest.php tests/Feature/Filament/BaselineProfileCaptureStartSurfaceTest.php tests/Feature/Filament/OperationRunBaselineTruthSurfaceTest.php tests/Feature/Monitoring/AuditCoverageGovernanceTest.php tests/Feature/Monitoring/GovernanceOperationRunSummariesTest.php tests/Feature/Notifications/OperationRunNotificationTest.php tests/Feature/Authorization/OperatorExplanationSurfaceAuthorizationTest.php` - `cd apps/platform && ./vendor/bin/sail artisan test --compact tests/Feature/AdminConsentCallbackTest.php tests/Feature/Filament/WorkspaceOverviewDbOnlyTest.php tests/Feature/Guards/Spec194GovernanceActionSemanticsGuardTest.php tests/Feature/ManagedTenantOnboardingWizardTest.php tests/Feature/Onboarding/OnboardingVerificationTest.php tests/Feature/Operations/QueuedExecutionAuditTrailTest.php tests/Unit/Operations/QueuedExecutionLegitimacyGateTest.php` ## Notes - browser validation was not re-run in this pass Co-authored-by: Ahmed Darrazi <ahmed.darrazi@live.de> Reviewed-on: #271
165 lines
8.3 KiB
Markdown
165 lines
8.3 KiB
Markdown
# Quickstart: Baseline Capture Truthful Outcomes and Upstream Guardrails
|
|
|
|
## Purpose
|
|
|
|
Validate that baseline capture no longer reports green success when the upstream inventory basis is not credible or when the capture finds zero in-scope subjects, confirm that the previous consumable baseline remains the effective compare anchor until a new consumable snapshot exists, and verify that Monitoring, audit prose, and terminal notification copy stay aligned to the same dominant truth.
|
|
|
|
## Preconditions
|
|
|
|
1. Sail services are running.
|
|
2. The workspace has a tenant, baseline profile, and inventory fixtures available for the targeted tests.
|
|
3. Baseline resources remain Filament v5 / Livewire v4 surfaces; no extra asset build is expected beyond standard PHP/test tooling.
|
|
|
|
## Validation Flow
|
|
|
|
### 1. Start-surface preflight blocks non-credible inventory truth
|
|
|
|
**Goal**: Known upstream problems are rejected before enqueue and use the shared reason-code family.
|
|
|
|
Check:
|
|
|
|
- No latest relevant inventory sync exists.
|
|
- Latest relevant inventory sync is blocked or failed.
|
|
- Latest relevant inventory sync exists but does not provide usable in-scope coverage.
|
|
|
|
Expected result:
|
|
|
|
- Capture does not enqueue a run.
|
|
- The start surface shows the shared translated baseline-capture reason.
|
|
- No success wording appears.
|
|
|
|
## 2. Runtime recheck blocks prerequisite drift after enqueue
|
|
|
|
**Goal**: If the latest relevant inventory state changes after page load or after enqueue, the queued run still resolves truthfully.
|
|
|
|
Check:
|
|
|
|
- Enqueue capture with a credible latest inventory basis.
|
|
- Change the latest relevant inventory run to a blocked/failed/unusable-coverage/non-credible state before the job evaluates subjects.
|
|
|
|
Expected result:
|
|
|
|
- The run ends as `completed + blocked`.
|
|
- Monitoring and run-detail explanation lead with the upstream inventory reason.
|
|
- No snapshot becomes the current consumable baseline.
|
|
|
|
## 3. Consumable capture still produces succeeded baseline truth
|
|
|
|
**Goal**: A clean capture with at least one resolved in-scope subject still succeeds and advances effective baseline truth.
|
|
|
|
Check:
|
|
|
|
- Use a credible latest inventory run whose effective in-scope subject count is greater than zero.
|
|
- Ensure the capture completes without warning conditions that would intentionally downgrade the run to `partially_succeeded`.
|
|
|
|
Expected result:
|
|
|
|
- The run ends as `completed + succeeded`.
|
|
- A consumable snapshot is produced or reused consistently with the current truth contract.
|
|
- `BaselineProfile.active_snapshot_id` remains anchored to the consumable current snapshot after the run, whether that required a new pointer update or reuse of the already-current consumable snapshot.
|
|
- Run context and audit summary preserve the same metadata contract for success, including the eligibility decision, upstream inventory reference, and whether current baseline truth changed.
|
|
|
|
## 4. Zero-subject capture produces no-data truth, not a green refresh
|
|
|
|
**Goal**: A capture with zero in-scope subjects remains visible but cannot silently refresh current baseline truth.
|
|
|
|
Check:
|
|
|
|
- Use a credible latest inventory run whose effective in-scope subject count is zero.
|
|
|
|
Expected result:
|
|
|
|
- The run ends as `completed + partially_succeeded`.
|
|
- The dominant reason is the zero-subject/no-data capture code.
|
|
- Any retained snapshot artifact is non-consumable.
|
|
- `BaselineProfile.active_snapshot_id` remains on the previous consumable snapshot.
|
|
|
|
## 5. Compare readiness stays anchored to consumable baseline truth
|
|
|
|
**Goal**: Compare surfaces must reflect whether a usable baseline actually exists, not whether a capture was merely attempted.
|
|
|
|
Check:
|
|
|
|
- Trigger a blocked capture after a previously successful baseline exists.
|
|
- Trigger a zero-subject capture after a previously successful baseline exists.
|
|
- Trigger a blocked or zero-subject capture when no previous consumable baseline exists.
|
|
|
|
Expected result:
|
|
|
|
- With a previous consumable baseline, compare remains available against that prior truth and explains that the latest capture did not refresh baseline truth.
|
|
- The profile-level compare affordance reflects the same truthful availability state and guidance as compare landing.
|
|
- Without any consumable baseline, compare remains unavailable and explains why.
|
|
|
|
## 6. Monitoring summary stays explanation-first
|
|
|
|
**Goal**: Operators should immediately see whether the run was blocked upstream or completed with no usable baseline.
|
|
|
|
Check:
|
|
|
|
- Open the run detail for a blocked latest-inventory capture.
|
|
- Open the run detail for a failed latest-inventory capture.
|
|
- Open the run detail for a zero-subject capture.
|
|
|
|
Expected result:
|
|
|
|
- The summary headline differentiates blocked upstream prerequisites, failed latest inventory, and no-data capture.
|
|
- Raw numeric counts remain secondary diagnostics.
|
|
|
|
## 7. Audit prose and terminal notification stay aligned with run truth
|
|
|
|
**Goal**: Interactive runs, initiator-null runs, and audit coverage must preserve the same dominant baseline-capture reason without introducing notification drift.
|
|
|
|
Check:
|
|
|
|
- Complete an interactive blocked baseline-capture run and inspect the terminal `OperationRunCompleted` payload.
|
|
- Complete an interactive zero-subject baseline-capture run and inspect the terminal `OperationRunCompleted` payload.
|
|
- Complete an initiator-null or scheduled baseline-capture run with blocked or no-data truth.
|
|
- Inspect the governance audit coverage surface or assertions for the same runs.
|
|
|
|
Expected result:
|
|
|
|
- Interactive terminal notifications use the same dominant blocked or no-data reason vocabulary as Monitoring.
|
|
- Initiator-null runs emit no terminal DB notification while preserving Monitoring and audit truth.
|
|
- Audit prose records the same dominant cause and whether current baseline truth changed.
|
|
|
|
## Commands
|
|
|
|
Run the narrowest proof set from repository root:
|
|
|
|
```bash
|
|
export PATH="/bin:/usr/bin:/usr/local/bin:$PATH" && cd apps/platform && ./vendor/bin/sail bin pint --dirty --format agent
|
|
export PATH="/bin:/usr/bin:/usr/local/bin:$PATH" && cd apps/platform && ./vendor/bin/sail artisan test --compact tests/Feature/Baselines/BaselineCaptureTest.php
|
|
export PATH="/bin:/usr/bin:/usr/local/bin:$PATH" && cd apps/platform && ./vendor/bin/sail artisan test --compact tests/Feature/Filament/BaselineProfileCaptureStartSurfaceTest.php tests/Feature/Filament/BaselineCompareLandingStartSurfaceTest.php tests/Feature/Filament/BaselineCaptureResultExplanationSurfaceTest.php
|
|
export PATH="/bin:/usr/bin:/usr/local/bin:$PATH" && cd apps/platform && ./vendor/bin/sail artisan test --compact tests/Feature/Filament/OperationRunBaselineTruthSurfaceTest.php tests/Feature/Monitoring/GovernanceOperationRunSummariesTest.php tests/Feature/Monitoring/AuditCoverageGovernanceTest.php tests/Feature/Notifications/OperationRunNotificationTest.php tests/Feature/Authorization/OperatorExplanationSurfaceAuthorizationTest.php
|
|
```
|
|
|
|
Run the legacy edge check only if implementation touches historical empty-snapshot classification:
|
|
|
|
```bash
|
|
export PATH="/bin:/usr/bin:/usr/local/bin:$PATH" && cd apps/platform && ./vendor/bin/sail artisan test --compact tests/Feature/Baselines/BaselineSnapshotBackfillTest.php
|
|
```
|
|
|
|
## Manual Smoke Focus
|
|
|
|
If a manual UI check is needed after automated proof:
|
|
|
|
1. Open a baseline profile detail page and verify the capture header action shows the translated upstream block when inventory truth is not credible.
|
|
2. Open compare landing after a zero-subject capture and verify it explains whether the prior consumable baseline still anchors compare availability.
|
|
3. Open the Monitoring run detail and verify the headline distinguishes upstream block from no-data capture before showing counts.
|
|
4. Verify an interactive run shows aligned terminal notification wording, while an initiator-null run leaves notification delivery suppressed and keeps Monitoring as the audit surface.
|
|
|
|
## Close-out Record
|
|
|
|
Record the feature close-out outcome here and mirror it into the active PR description:
|
|
|
|
1. `Guardrail` status for the changed native Filament surfaces.
|
|
2. Whether `standard-native-filament` and `monitoring-state-page` coverage both ran successfully.
|
|
3. Whether `T022` stayed `document-in-feature` or required a follow-up for legacy empty-snapshot behavior.
|
|
|
|
### Recorded Outcome
|
|
|
|
- `Guardrail`: pass
|
|
- `standard-native-filament`: pass
|
|
- `monitoring-state-page`: pass
|
|
- `T022`: implemented in feature; legacy empty complete snapshots are now backfilled as incomplete no-data captures with `baseline.capture.zero_subjects`
|