spec(116): close tasks coverage gaps

This commit is contained in:
Ahmed Darrazi 2026-03-02 02:22:51 +01:00
parent add136cc3c
commit 2a0fb01569

View File

@ -16,10 +16,11 @@ ## Phase 1: Setup (Shared Infrastructure)
**Purpose**: Ensure local dev + feature artifacts are ready.
- [ ] T001 Re-run Speckit prerequisites check via `.specify/scripts/bash/check-prerequisites.sh` (references `specs/116-baseline-drift-engine/plan.md`)
- [ ] T002 Ensure Sail + migrations are up for local validation (references `docker-compose.yml` and `database/migrations/`)
- [ ] T003 [P] Re-validate Spec 116 UI Action Matrix remains accurate after planned UI tweaks in `specs/116-baseline-drift-engine/spec.md`
- [ ] T004 [P] Confirm supported policy types + foundations config sources match UI selectors (`config/tenantpilot.php` and `app/Support/Inventory/InventoryPolicyTypeMeta.php`)
- [ ] T001 Re-run Speckit prerequisites check via `.specify/scripts/bash/check-prerequisites.sh --json` (references `specs/116-baseline-drift-engine/plan.md`)
- [ ] T002 Run required agent context update via `.specify/scripts/bash/update-agent-context.sh copilot` (required by `specs/116-baseline-drift-engine/plan.md`)
- [ ] T003 Ensure Sail + migrations are up for local validation (references `vendor/bin/sail`, `docker-compose.yml`, and `database/migrations/`)
- [ ] T004 [P] Re-validate Spec 116 UI Action Matrix: confirm “no changes required” OR update the matrix table in `specs/116-baseline-drift-engine/spec.md`
- [ ] T005 [P] Confirm supported policy types + foundations config sources match UI selectors (`config/tenantpilot.php` and `app/Support/Inventory/InventoryPolicyTypeMeta.php`)
---
@ -29,20 +30,20 @@ ## Phase 2: Foundational (Blocking Prerequisites)
**Independent Test**: Baseline Profile can be created with the new scope shape, and scope defaults expand deterministically.
- [ ] T005 Update baseline scope schema + default semantics (policy_types excludes foundations by default; foundation_types defaults to none) in `app/Support/Baselines/BaselineScope.php`
- [ ] T006 [P] Update BaselineProfile default scope shape to include `foundation_types` in `database/factories/BaselineProfileFactory.php`
- [ ] T007 [P] Ensure BaselineProfile scope casting/normalization supports `foundation_types` safely in `app/Models/BaselineProfile.php`
- [ ] T008 [P] Create focused tests for scope expansion defaults (empty policy_types => supported excluding foundations; empty foundation_types => none) in `tests/Unit/Baselines/BaselineScopeTest.php`
- [ ] T009 Update BaselineProfile Create/Edit form layout to satisfy UX-001 (Main/Aside 3-column layout; no naked inputs) while preserving existing fields in `app/Filament/Resources/BaselineProfileResource.php`
- [ ] T010 Update BaselineProfile scope picker UI to include Foundations multi-select + corrected helper text semantics in `app/Filament/Resources/BaselineProfileResource.php`
- [ ] T011 Update BaselineProfile infolist to display selected foundations (and default “None”) in `app/Filament/Resources/BaselineProfileResource.php`
- [ ] T012 Run focused verification for foundational scope/UI changes with `vendor/bin/sail artisan test --compact tests/Unit/Baselines/BaselineScopeTest.php` (references `tests/Unit/Baselines/BaselineScopeTest.php`)
- [ ] T006 Update baseline scope schema + default semantics (policy_types excludes foundations by default; foundation_types defaults to none) in `app/Support/Baselines/BaselineScope.php`
- [ ] T007 [P] Update BaselineProfile default scope shape to include `foundation_types` in `database/factories/BaselineProfileFactory.php`
- [ ] T008 [P] Ensure BaselineProfile scope casting/normalization supports `foundation_types` safely in `app/Models/BaselineProfile.php`
- [ ] T009 [P] Create focused tests for scope expansion defaults (empty policy_types => supported excluding foundations; empty foundation_types => none) in `tests/Unit/Baselines/BaselineScopeTest.php`
- [ ] T010 Update BaselineProfile Create/Edit form (UX-001 Main/Aside), scope picker (Policy Types + Foundations), and infolist display semantics in `app/Filament/Resources/BaselineProfileResource.php`
- [ ] T011 [P] Update create-page scope normalization to persist both `policy_types` and `foundation_types` in `app/Filament/Resources/BaselineProfileResource/Pages/CreateBaselineProfile.php`
- [ ] T012 [P] Update edit-page scope normalization to persist both `policy_types` and `foundation_types` in `app/Filament/Resources/BaselineProfileResource/Pages/EditBaselineProfile.php`
- [ ] T013 Run focused verification for foundational scope/UI changes with `vendor/bin/sail artisan test --compact tests/Unit/Baselines/BaselineScopeTest.php` (references `tests/Unit/Baselines/BaselineScopeTest.php`)
**Checkpoint**: Scope semantics + scope UI are correct and test-covered.
---
## Phase 3: User Story 1 — Capture and compare a baseline with stable findings (Priority: P1) 🎯
## Phase 3: User Story 1 — Capture and compare a baseline with stable findings (Priority: P1)
**Goal**: Define the v1 meta-fidelity hash contract, use it for capture/compare, and make baseline-compare findings snapshot-scoped stable identities.
@ -50,22 +51,24 @@ ## Phase 3: User Story 1 — Capture and compare a baseline with stable findings
### Tests for User Story 1 (write first)
- [ ] T013 [P] [US1] Update capture tests for effective_scope recording + contract-based hashing in `tests/Feature/Baselines/BaselineCaptureTest.php`
- [ ] T014 [P] [US1] Update compare findings tests to assert recurrence-key-based identity (no hashes in fingerprint) + lifecycle idempotency per run + `run.context.findings.counts_by_change_type` is present and accurate in `tests/Feature/Baselines/BaselineCompareFindingsTest.php`
- [ ] T015 [P] [US1] Add/adjust preconditions tests for default baseline snapshot selection (latest successful snapshot) in `tests/Feature/Baselines/BaselineComparePreconditionsTest.php`
- [ ] T016 [P] [US1] Add test that re-capturing (new snapshot id) produces new finding identities (snapshot-scoped) in `tests/Feature/Baselines/BaselineCompareFindingsTest.php`
- [ ] T017 [P] [US1] Add negative auth test ensuring compare start surface is gated by capability (`tenant.sync` / `Capabilities::TENANT_SYNC`) in `tests/Feature/Filament/BaselineCompareLandingStartSurfaceTest.php`
- [ ] T014 [P] [US1] Update capture tests for effective_scope recording + contract-based hashing in `tests/Feature/Baselines/BaselineCaptureTest.php`
- [ ] T015 [P] [US1] Update compare findings tests to assert recurrence-key-based identity (no hashes in fingerprint) + lifecycle idempotency per run + `run.context.findings.counts_by_change_type` is present and accurate in `tests/Feature/Baselines/BaselineCompareFindingsTest.php`
- [ ] T016 [P] [US1] Add/adjust preconditions tests for default baseline snapshot selection (latest successful snapshot) + explicit snapshot override (per `specs/116-baseline-drift-engine/contracts/openapi.yaml`) in `tests/Feature/Baselines/BaselineComparePreconditionsTest.php`
- [ ] T017 [P] [US1] Add test that re-capturing (new snapshot id) produces new finding identities (snapshot-scoped) in `tests/Feature/Baselines/BaselineCompareFindingsTest.php`
- [ ] T018 [P] [US1] Extend compare-start auth tests to cover positive + negative (403 for member missing `tenant.sync`, 404/redirect for missing tenant access) in `tests/Feature/Filament/BaselineCompareLandingStartSurfaceTest.php`
- [ ] T019 [P] [US1] Add capture-start auth tests for Baseline Profile “Capture Snapshot” action (positive + negative manage capability, plus deny-as-not-found coverage via view route) in `tests/Feature/Filament/BaselineProfileCaptureStartSurfaceTest.php`
- [ ] T020 [P] [US1] Confirm baseline profile create/edit mutation surfaces have positive + negative authorization coverage; update for new scope shape if needed in `tests/Feature/Baselines/BaselineProfileAuthorizationTest.php`
### Implementation for User Story 1
- [ ] T018 [US1] Create + implement Inventory Meta Contract builder (normalized whitelist inputs; deterministic ordering) in `app/Services/Baselines/InventoryMetaContract.php`
- [ ] T019 [US1] Update snapshot hashing to hash ONLY the meta contract output (not entire meta_jsonb) in `app/Services/Baselines/BaselineSnapshotIdentity.php`
- [ ] T020 [US1] Update baseline capture job to compute/store baseline_hash via InventoryMetaContract + record fidelity/meta observation evidence in `app/Jobs/CaptureBaselineSnapshotJob.php`
- [ ] T021 [US1] Ensure capture OperationRun context records `effective_scope.*` (policy_types, foundation_types, all_types, foundations_included) in `app/Services/Baselines/BaselineCaptureService.php`
- [ ] T022 [US1] Update baseline compare job to compute current_hash via InventoryMetaContract consistently with capture in `app/Jobs/CompareBaselineToTenantJob.php`
- [ ] T023 [US1] Switch baseline-compare finding identity to recurrence key derived from (tenant, baseline_snapshot_id, policy_type, external_id, change_type) and set `fingerprint == recurrence_key` in `app/Jobs/CompareBaselineToTenantJob.php`
- [ ] T024 [US1] Enforce per-run idempotency by using `findings.current_operation_run_id` (and/or evidence) so `times_seen` increments at most once per run identity in `app/Jobs/CompareBaselineToTenantJob.php`
- [ ] T025 [US1] Ensure compare run context includes `baseline_profile_id`, `baseline_snapshot_id`, and `findings.counts_by_change_type` for stats/widgets/auditability in `app/Services/Baselines/BaselineCompareService.php`
- [ ] T021 [US1] Create + implement Inventory Meta Contract builder (normalized whitelist inputs; deterministic ordering) in `app/Services/Baselines/InventoryMetaContract.php`
- [ ] T022 [US1] Update snapshot hashing to hash ONLY the meta contract output (not entire meta_jsonb) in `app/Services/Baselines/BaselineSnapshotIdentity.php`
- [ ] T023 [US1] Update baseline capture job to compute/store baseline_hash via InventoryMetaContract + record fidelity/meta observation evidence in `app/Jobs/CaptureBaselineSnapshotJob.php`
- [ ] T024 [US1] Ensure capture OperationRun context records `effective_scope.*` (policy_types, foundation_types, all_types, foundations_included) in `app/Services/Baselines/BaselineCaptureService.php`
- [ ] T025 [US1] Update baseline compare job to compute current_hash via InventoryMetaContract consistently with capture in `app/Jobs/CompareBaselineToTenantJob.php`
- [ ] T026 [US1] Switch baseline-compare finding identity to recurrence key derived from (tenant, baseline_snapshot_id, policy_type, external_id, change_type) and set `fingerprint == recurrence_key` in `app/Jobs/CompareBaselineToTenantJob.php`
- [ ] T027 [US1] Enforce per-run idempotency by using `findings.current_operation_run_id` (and/or evidence) so `times_seen` increments at most once per run identity in `app/Jobs/CompareBaselineToTenantJob.php`
- [ ] T028 [US1] Write compare audit context fields (baseline ids + `findings.counts_by_change_type`) onto the compare OperationRun context in `app/Jobs/CompareBaselineToTenantJob.php`
**Checkpoint**: US1 tests pass: `vendor/bin/sail artisan test --compact tests/Feature/Baselines/BaselineCaptureTest.php tests/Feature/Baselines/BaselineCompareFindingsTest.php tests/Feature/Baselines/BaselineComparePreconditionsTest.php`.
@ -79,15 +82,15 @@ ## Phase 4: User Story 2 — Coverage warnings prevent misleading missing-policy
### Tests for User Story 2 (write first)
- [ ] T026 [P] [US2] Extend inventory sync tests to assert per-type coverage payload is written to OperationRun context in `tests/Feature/Inventory/InventorySyncStartSurfaceTest.php`
- [ ] T027 [P] [US2] Create coverage-guard regression test: (a) uncovered types => no findings of any kind; (b) no completed inventory sync run / missing coverage payload => fail-safe zero findings; (c) effective scope expands to zero types => warnings + zero findings; outcome partially_succeeded; covered types still emit findings in `tests/Feature/Baselines/BaselineCompareCoverageGuardTest.php`
- [ ] T029 [P] [US2] Extend inventory sync tests to assert per-type coverage payload is written to OperationRun context in `tests/Feature/Inventory/InventorySyncStartSurfaceTest.php`
- [ ] T030 [P] [US2] Create coverage-guard regression test: (a) uncovered types => no findings of any kind; (b) no completed inventory sync run / missing coverage payload => fail-safe zero findings; (c) effective scope expands to zero types => warnings + zero findings; outcome partially_succeeded; covered types still emit findings in `tests/Feature/Baselines/BaselineCompareCoverageGuardTest.php`
### Implementation for User Story 2
- [ ] T028 [US2] Persist inventory sync coverage payload into latest inventory sync run context (`inventory.coverage.policy_types` + `inventory.coverage.foundation_types`) in `app/Services/Inventory/InventorySyncService.php`
- [ ] T029 [P] [US2] Create a small coverage parser/helper to normalize context payload for downstream consumers in `app/Support/Inventory/InventoryCoverage.php`
- [ ] T030 [US2] Update baseline compare to read latest inventory sync run coverage, compute uncovered types, skip emission for uncovered types, and write coverage details into compare run context in `app/Jobs/CompareBaselineToTenantJob.php`
- [ ] T031 [US2] Treat missing coverage proof (no completed inventory sync run, or unreadable/missing coverage payload) as uncovered-for-all-types (fail-safe): emit zero findings and mark outcome partially_succeeded (via OperationRunService), setting numeric summary_counts (including errors_recorded) using canonical keys only in `app/Jobs/CompareBaselineToTenantJob.php`
- [ ] T031 [US2] Persist inventory sync coverage payload into latest inventory sync run context (`inventory.coverage.policy_types` + `inventory.coverage.foundation_types`) in `app/Services/Inventory/InventorySyncService.php`
- [ ] T032 [P] [US2] Create a small coverage parser/helper to normalize context payload for downstream consumers in `app/Support/Inventory/InventoryCoverage.php`
- [ ] T033 [US2] Update baseline compare to read latest inventory sync run coverage, compute uncovered types, skip emission for uncovered types, and write coverage details into compare run context in `app/Jobs/CompareBaselineToTenantJob.php`
- [ ] T034 [US2] Treat missing coverage proof (no completed inventory sync run, or unreadable/missing coverage payload) as uncovered-for-all-types (fail-safe): emit zero findings and mark outcome partially_succeeded (via OperationRunService), setting numeric summary_counts (including errors_recorded) using canonical keys only in `app/Jobs/CompareBaselineToTenantJob.php`
**Note (canonical warning magnitude)**: For (c) “effective scope expands to zero types”, the compare MUST still surface a warning and therefore MUST set `summary_counts.errors_recorded = 1` (even though uncovered-types count is 0), to keep the warning visible under the numeric-only summary_counts contract.
@ -103,16 +106,16 @@ ## Phase 5: User Story 3 — Operators can understand scope, coverage, and fidel
### Tests for User Story 3 (write first)
- [ ] T032 [P] [US3] Update Baseline Compare landing tests to cover warning/coverage state rendering inputs (stats DTO fields) in `tests/Feature/Filament/BaselineCompareLandingStartSurfaceTest.php`
- [ ] T033 [P] [US3] Update drift landing comparison-info tests to include coverage/fidelity context when source is baseline compare in `tests/Feature/Drift/DriftLandingShowsComparisonInfoTest.php`
- [ ] T035 [P] [US3] Update Baseline Compare landing tests to cover warning/coverage state rendering inputs (stats DTO fields) in `tests/Feature/Filament/BaselineCompareLandingStartSurfaceTest.php`
- [ ] T036 [P] [US3] Update drift landing comparison-info tests to include coverage/fidelity context when source is baseline compare in `tests/Feature/Drift/DriftLandingShowsComparisonInfoTest.php`
### Implementation for User Story 3
- [ ] T034 [US3] Extend BaselineCompareStats DTO to include coverage status + uncovered types summary + fidelity indicator sourced from latest compare run context in `app/Support/Baselines/BaselineCompareStats.php`
- [ ] T035 [US3] Wire new stats fields into the BaselineCompareLanding Livewire page state in `app/Filament/Pages/BaselineCompareLanding.php`
- [ ] T036 [US3] Render coverage badge + warning banner + fidelity label on the landing view in `resources/views/filament/pages/baseline-compare-landing.blade.php`
- [ ] T037 [US3] Add a findings-list banner when latest baseline compare run had uncovered types (linking to the run) in `app/Filament/Resources/FindingResource/Pages/ListFindings.php`
- [ ] T038 [US3] Ensure run detail already shows context; if needed, add baseline compare “Coverage” summary entry for readability in `app/Filament/Resources/OperationRunResource.php`
- [ ] T037 [US3] Extend BaselineCompareStats DTO to include coverage status + uncovered types summary + fidelity indicator sourced from latest compare run context in `app/Support/Baselines/BaselineCompareStats.php`
- [ ] T038 [US3] Wire new stats fields into the BaselineCompareLanding Livewire page state in `app/Filament/Pages/BaselineCompareLanding.php`
- [ ] T039 [US3] Render coverage badge + warning banner + fidelity label on the landing view in `resources/views/filament/pages/baseline-compare-landing.blade.php`
- [ ] T040 [US3] Add a findings-list banner when latest baseline compare run had uncovered types (linking to the run) in `app/Filament/Resources/FindingResource/Pages/ListFindings.php`
- [ ] T041 [US3] Ensure run detail already shows context; if needed, add baseline compare “Coverage” summary entry for readability in `app/Filament/Resources/OperationRunResource.php`
**Checkpoint**: US3 tests pass: `vendor/bin/sail artisan test --compact tests/Feature/Filament/BaselineCompareLandingStartSurfaceTest.php tests/Feature/Drift/DriftLandingShowsComparisonInfoTest.php`.
@ -122,15 +125,19 @@ ## Phase 6: Polish & Cross-Cutting Concerns
**Purpose**: Preserve operability semantics (auto-close, stats), Ops-UX compliance, and fast regression feedback.
- [ ] T039 Confirm baseline compare stats remain profile-grouped via `scope_key = baseline_profile:{id}` after identity change in `app/Support/Baselines/BaselineCompareStats.php`
- [ ] T040 Ensure baseline auto-close behavior still works with snapshot-scoped identities (no stale open findings after successful compare) in `app/Services/Baselines/BaselineAutoCloseService.php`
- [ ] T041 [P] Update/verify auto-close regression test remains valid after identity change in `tests/Feature/Baselines/BaselineOperabilityAutoCloseTest.php`
- [ ] T042 [P] Add/extend guard test asserting OperationRun summary_counts are numeric-only and keys are limited to `OperationSummaryKeys::all()` for baseline capture/compare runs in `tests/Feature/Baselines/BaselineCompareFindingsTest.php`
- [ ] T043 Run baseline-focused test pack for Spec 116: `vendor/bin/sail artisan test --compact tests/Feature/Baselines/` (references `tests/Feature/Baselines/`)
- [ ] T044 Run Pint formatter on changed files: `vendor/bin/sail bin pint --dirty --format agent` (references `app/` and `tests/`)
- [ ] T045 Validate developer quickstart still matches real behavior (update if needed) in `specs/116-baseline-drift-engine/quickstart.md`
- [ ] T046 [P] Create Baseline Profile archive action tests (confirmation required + RBAC 403/404 semantics + success path) in `tests/Feature/Baselines/BaselineProfileArchiveActionTest.php`
- [ ] T047 [P] Ensure archive action is declared in Action Surface slots and remains “More” row action only (max 2 visible row actions) in `tests/Feature/Guards/ActionSurfaceContractTest.php`
- [ ] T042 Confirm baseline compare stats remain profile-grouped via `scope_key = baseline_profile:{id}` after identity change in `app/Support/Baselines/BaselineCompareStats.php`
- [ ] T043 Ensure baseline auto-close behavior still works with snapshot-scoped identities (no stale open findings after successful compare) in `app/Services/Baselines/BaselineAutoCloseService.php`
- [ ] T044 [P] Update/verify auto-close regression test remains valid after identity change in `tests/Feature/Baselines/BaselineOperabilityAutoCloseTest.php`
- [ ] T045 [P] Add/extend guard test asserting OperationRun summary_counts are numeric-only and keys are limited to `OperationSummaryKeys::all()` for baseline capture/compare runs in `tests/Feature/Baselines/BaselineCompareFindingsTest.php`
- [ ] T046 [P] Add Spec 116 “One engine” regression guard to prevent legacy compare/hash paths (no `DriftHasher::fingerprint()` use for baseline compare; capture/compare hashing flows through `InventoryMetaContract`) in `tests/Feature/Guards/Spec116OneEngineGuardTest.php`
- [ ] T047 [P] Add Spec 116 performance regression guard (compare is DB-only; no Graph/hydration calls; compare processes snapshot + inventory via chunking) in `tests/Feature/Baselines/BaselineComparePerformanceGuardTest.php`
- [ ] T048 [P] Ensure OPS-UX-GUARD-001 coverage is enforced for this feature by running/updating constitution guards in `tests/Feature/OpsUx/Constitution/DirectStatusTransitionGuardTest.php`, `tests/Feature/OpsUx/Constitution/JobDbNotificationGuardTest.php`, and `tests/Feature/OpsUx/Constitution/LegacyNotificationGuardTest.php`
- [ ] T049 [P] Create Baseline Profile archive action tests (confirmation required + RBAC 403/404 semantics + success path) in `tests/Feature/Baselines/BaselineProfileArchiveActionTest.php`
- [ ] T050 [P] Ensure archive action is declared in Action Surface slots and remains “More” row action only (max 2 visible row actions) in `tests/Feature/Guards/ActionSurfaceContractTest.php`
- [ ] T051 Run baseline-focused test pack for Spec 116: `vendor/bin/sail artisan test --compact tests/Feature/Baselines/` (references `tests/Feature/Baselines/`)
- [ ] T052 Run Ops-UX guard test pack: `vendor/bin/sail artisan test --compact --group=ops-ux` (references `tests/Feature/OpsUx/Constitution/`)
- [ ] T053 Run Pint formatter on changed files: `vendor/bin/sail pint --dirty --format agent` (references `app/` and `tests/`)
- [ ] T054 Validate developer quickstart still matches real behavior (update if needed) in `specs/116-baseline-drift-engine/quickstart.md`
---
@ -170,10 +177,13 @@ # Tests can be updated in parallel:
Task: "Update capture tests" (tests/Feature/Baselines/BaselineCaptureTest.php)
Task: "Update compare findings tests" (tests/Feature/Baselines/BaselineCompareFindingsTest.php)
Task: "Update compare preconditions tests" (tests/Feature/Baselines/BaselineComparePreconditionsTest.php)
Task: "Update compare-start auth tests" (tests/Feature/Filament/BaselineCompareLandingStartSurfaceTest.php)
Task: "Add capture-start auth tests" (tests/Feature/Filament/BaselineProfileCaptureStartSurfaceTest.php)
# Implementation can be split with care (different files):
Task: "Implement InventoryMetaContract" (app/Services/Baselines/InventoryMetaContract.php)
Task: "Update BaselineProfileResource scope UI" (app/Filament/Resources/BaselineProfileResource.php)
Task: "Update CaptureBaselineSnapshotJob hashing" (app/Jobs/CaptureBaselineSnapshotJob.php)
Task: "Update CompareBaselineToTenantJob identity + hashing" (app/Jobs/CompareBaselineToTenantJob.php)
```