TenantAtlas/specs/117-baseline-drift-engine/tasks.md

description
Task list for implementing Spec 117 (Baseline Drift Engine)

Tasks: Golden Master Baseline Drift — Deep Settings Drift via Provider Chain

Input: Design documents from /specs/117-baseline-drift-engine/ Prerequisites: plan.md (required), spec.md (required), research.md, data-model.md, contracts/, quickstart.md

Tests: REQUIRED (Pest) — runtime behavior + persistence + Filament UI changes

Phase 1: Setup (Shared Infrastructure)

Purpose: Ensure local/dev environment can run/verify this feature end-to-end.

• T001 Confirm Sail + queue runner workflow in quickstart.md at specs/117-baseline-drift-engine/quickstart.md
• T002 Create Baseline drift engine test suite skeleton in tests/Feature/BaselineDriftEngine/ResolverTest.php

---

Phase 2: Foundational (Blocking Prerequisites)

Purpose: Core building blocks that MUST be complete before any user story work.

• T003 Define a stable evidence/provenance JSON shape constants/helpers in app/Services/Baselines (new file app/Services/Baselines/Evidence/EvidenceProvenance.php)
• T004 Implement provider-chain interfaces + DTO for resolved evidence in app/Services/Baselines (new files app/Services/Baselines/CurrentStateHashResolver.php and app/Services/Baselines/Evidence/ResolvedEvidence.php)
• T005 [P] Implement content evidence provider (PolicyVersion since-rule) in app/Services/Baselines/Evidence/ContentEvidenceProvider.php
• T006 [P] Implement meta evidence provider (Inventory meta contract) in app/Services/Baselines/Evidence/MetaEvidenceProvider.php
• T007 Implement batch-oriented resolution orchestration (first-non-null wins) in app/Services/Baselines/CurrentStateHashResolver.php
• T008 Add resolver precedence + since-rule tests in tests/Feature/BaselineDriftEngine/ResolverTest.php

Checkpoint: Resolver exists, is batch-capable, and is test-covered.

---

Phase 3: User Story 1 — Deep settings drift appears when content evidence exists (Priority: P1) 🎯 MVP

Goal: Compare uses content evidence (PolicyVersion) when available since baseline captured time, producing correct drift findings.

Independent Test: Create baseline snapshot item + newer content evidence; run compare job; assert finding created for changed content and not created for equal content.

• T009 [US1] Add factories for baseline snapshots/items and policy versions in database/factories/BaselineSnapshotFactory.php, database/factories/BaselineSnapshotItemFactory.php, database/factories/PolicyVersionFactory.php
• T010 [US1] Extend CompareBaselineToTenantJob to resolve current hashes via CurrentStateHashResolver (no direct meta hashing) in app/Jobs/CompareBaselineToTenantJob.php
• T011 [US1] Ensure compare uses baseline snapshot captured_at as since for content evidence provider in app/Jobs/CompareBaselineToTenantJob.php
• T012 [US1] Implement null-evidence handling: skip subject + record evidence gap in run context (no finding) in app/Jobs/CompareBaselineToTenantJob.php
• T013 [US1] Add integration test: content evidence produces finding when different in tests/Feature/BaselineDriftEngine/CompareContentEvidenceTest.php
• T014 [US1] Add integration test: content evidence produces no finding when equal in tests/Feature/BaselineDriftEngine/CompareContentEvidenceTest.php

Note (SC-117-01): Expand the integration coverage to a deterministic fixture matrix (at least 20 settings-only change cases across at least 3 policy types) so “≥95% success rate” is measurable in CI.

Checkpoint: US1 is shippable on its own (deep drift works when content exists; compare remains DB-only).

---

Phase 4: User Story 2 — Mixed fidelity is transparent and interpretable (Priority: P1)

Goal: Findings show fidelity badge + provenance (baseline + current) and can be filtered by fidelity; compare run detail gets coverage breakdown (content vs meta vs gaps).

Independent Test: Run compare with a mixed set of subjects (some content, some meta, some missing) and assert fidelity/provenance stored and filter works.

Data & persistence

• T015 [US2] Add migration for findings.evidence_fidelity + index + backfill to meta in database/migrations/2026_03_02_000001_add_evidence_fidelity_to_findings_table.php
• T016 [US2] Update Finding creation to persist evidence JSON with baseline+current provenance and set evidence_fidelity (weaker-of) in app/Jobs/CompareBaselineToTenantJob.php
• T017 [US2] Store compare coverage breakdown + evidence gaps in operation_runs.context (not summary_counts) in app/Jobs/CompareBaselineToTenantJob.php

Filament UI

• T018 [US2] Add fidelity badge column (content/meta) to Finding list table in app/Filament/Resources/FindingResource.php
• T019 [US2] Add fidelity filter with exactly two values (content, meta) to FindingResource table filters in app/Filament/Resources/FindingResource.php
• T020 [US2] Update Finding view/record display to show baseline+current provenance fields from evidence_jsonb in app/Filament/Resources/FindingResource.php

Tests

• T021 [P] [US2] Add DB assertion tests for evidence JSON provenance (both sides) in tests/Feature/BaselineDriftEngine/FindingProvenanceTest.php
• T022 [P] [US2] Add tests for weaker-of fidelity semantics (meta dominates) in tests/Feature/BaselineDriftEngine/FindingFidelityTest.php
• T023 [P] [US2] Add tests for fidelity filter query behavior (content vs meta) in tests/Feature/BaselineDriftEngine/FindingFidelityFilterTest.php
• T032 [US2] Prevent cross-fidelity hash comparisons (meta vs content) and add mismatch coverage tests in app/Jobs/CompareBaselineToTenantJob.php and tests/Feature/BaselineDriftEngine/CompareFidelityMismatchTest.php

Checkpoint: Mixed fidelity is visible, filterable, and provenance is complete.

---

Phase 5: User Story 3 — Baseline capture uses best available evidence (Priority: P2)

Goal: Baseline capture stores the strongest available hash per snapshot item (content if present, else meta) and stores provenance on the snapshot item.

Independent Test: Run capture job with mixed evidence availability and assert snapshot items record expected hash + provenance.

• T024 [US3] Update CaptureBaselineSnapshotJob to resolve baseline hashes via CurrentStateHashResolver (provider chain) in app/Jobs/CaptureBaselineSnapshotJob.php
• T025 [US3] Store baseline-side provenance (fidelity/source/observed_at) in baseline_snapshot_items.meta_jsonb in app/Jobs/CaptureBaselineSnapshotJob.php
• T026 [P] [US3] Add test: capture stores content fidelity when available in tests/Feature/BaselineDriftEngine/CaptureBaselineContentTest.php
• T027 [P] [US3] Add test: capture falls back to meta fidelity when content missing in tests/Feature/BaselineDriftEngine/CaptureBaselineMetaFallbackTest.php

Checkpoint: Baseline capture produces stronger baseline items without extra workflows.

---

Phase 6: Polish & Cross-Cutting Concerns

Purpose: Guardrails, performance checks, and verification.

• T028 Add batch-performance guard tests to prevent per-subject query loops in app/Services/Baselines/CurrentStateHashResolver.php and tests/Feature/BaselineDriftEngine/PerformanceGuardTest.php

Note (SC-117-04): Performance guard should enforce a fixed query upper bound for resolving a representative batch (e.g., 500 subjects resolved in ≤ 20 queries, excluding factory/seed setup), proving set-based batch behavior.

• T029 [P] Verify compare remains read-only/no upstream calls by scanning CompareBaselineToTenantJob for Graph client usage in app/Jobs/CompareBaselineToTenantJob.php
• T030 [P] Run formatter and focused tests per quickstart.md: vendor/bin/sail bin pint --dirty --format agent and vendor/bin/sail artisan test --compact --filter=Baseline (document in specs/117-baseline-drift-engine/quickstart.md if updates needed)
• T031 Verify OperationRun lifecycle transitions remain service-owned (no direct status/outcome updates) when editing jobs in app/Jobs/CompareBaselineToTenantJob.php and app/Jobs/CaptureBaselineSnapshotJob.php

---

Dependencies & Execution Order

Phase Dependencies

• Setup (Phase 1): Can start immediately.
• Foundational (Phase 2): Depends on Phase 1; BLOCKS all user stories.
• US1 (Phase 3): Depends on Phase 2.
• US2 (Phase 4): Depends on US1 (needs compare emitting findings + evidence) and Phase 2.
• US3 (Phase 5): Depends on Phase 2 (reuses resolver).
• Polish (Phase 6): Depends on completing at least US1+US2; US3 optional.

User Story completion order (dependency graph)

• US1 → US2
• US3 is independent after Foundational

---

Parallel execution examples

Foundational parallel work

• [P] T005 (ContentEvidenceProvider) and [P] T006 (MetaEvidenceProvider) can be implemented in parallel.
• T008 resolver tests can be written while providers are being implemented (same file as T002; coordinate to avoid merge conflicts).

US1 parallel work

• T013 and T014 live in the same test file; implement sequentially or split into separate files if you want true parallel work.

US2 parallel work

• [P] T021–T023 tests can be written in parallel with UI tasks T018–T020.

---

Implementation strategy (MVP first)

1. Phase 1–2 (resolver + providers) ✔︎
2. Phase 3 (US1) — ship deep drift in compare where content exists
3. Phase 4 (US2) — add fidelity/provenance UX + filtering + run coverage breakdown
4. Phase 5 (US3) — upgrade capture to best available evidence

Explicit out-of-scope (v1.5)

• v2.0 optional requirements in specs/117-baseline-drift-engine/spec.md are intentionally not implemented by this tasks list.