ahmido/TenantAtlas
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects 1 Releases Wiki Activity
dev
TenantAtlas/specs/119-baseline-drift-engine/tasks.md
ahmido da1adbdeb5 Spec 119: Drift cutover to Baseline Compare (golden master) (#144) 
Implements Spec 119 (Drift Golden Master Cutover):

- Baseline Compare is the only drift writer (`source = baseline.compare`).
- Drift findings now store diff-compatible `evidence_jsonb` (summary.kind, baseline/current policy_version_id refs, fidelity + provenance).
- Findings UI renders one-sided diffs for `missing_policy`/`unexpected_policy` when a single ref exists; otherwise shows explicit “diff unavailable”.
- Removes legacy drift generator runtime (jobs/services/UI) and related tests.
- Adds one-time migration to delete legacy drift findings (`finding_type=drift` where source is null or != baseline.compare).
- Scopes baseline capture & landing duplicate warnings to latest completed inventory sync.
- Canonicalizes compliance `scheduledActionsForRule` drift signal and keeps legacy snapshots comparable.

Tests:
- `vendor/bin/sail artisan test --compact` (full suite per tasks)
- Focused pack: BaselinePolicyVersionResolverTest, BaselineCompareDriftEvidenceContractTest, DriftFindingDiffUnavailableTest, LegacyDriftFindingsCleanupMigrationTest, ComplianceNoncomplianceActionsDriftTest

Notes:
- Livewire v4+ / Filament v5 compatible (no legacy APIs).
- No new external dependencies.

Co-authored-by: Ahmed Darrazi <ahmed.darrazi@live.de>
Reviewed-on: #144
2026-03-06 14:30:49 +00:00

21 KiB
Raw Permalink Blame History

description
Task list for Spec 119 implementation

Tasks: Drift Golden Master Cutover (Baseline Compare) (Spec 119)

Input: Design documents from /Users/ahmeddarrazi/Documents/projects/TenantAtlas/specs/119-baseline-drift-engine/ Prerequisites:

  • /Users/ahmeddarrazi/Documents/projects/TenantAtlas/specs/119-baseline-drift-engine/plan.md (required)
  • /Users/ahmeddarrazi/Documents/projects/TenantAtlas/specs/119-baseline-drift-engine/spec.md (required)
  • /Users/ahmeddarrazi/Documents/projects/TenantAtlas/specs/119-baseline-drift-engine/research.md
  • /Users/ahmeddarrazi/Documents/projects/TenantAtlas/specs/119-baseline-drift-engine/data-model.md
  • /Users/ahmeddarrazi/Documents/projects/TenantAtlas/specs/119-baseline-drift-engine/contracts/drift.openapi.yaml
  • /Users/ahmeddarrazi/Documents/projects/TenantAtlas/specs/119-baseline-drift-engine/quickstart.md

Tests: REQUIRED (Pest) because this feature changes runtime behavior.

Organization: Tasks are grouped by user story to enable independent implementation and testing of each story.

Phase 1: Setup (Shared Infrastructure)

Purpose: Local readiness + baseline validation before changing runtime behavior

  • T001 Start local stack and run migrations using /Users/ahmeddarrazi/Documents/projects/TenantAtlas/vendor/bin/sail (up -d, artisan migrate)
  • T002 Run a baseline test subset using /Users/ahmeddarrazi/Documents/projects/TenantAtlas/vendor/bin/sail to confirm a green starting point (artisan test --compact --filter=BaselineCompare)

Phase 2: Foundational (Blocking Prerequisites)

Purpose: Shared building blocks used across user stories (evidence contract + resolvers)

⚠️ CRITICAL: Complete this phase before starting any user story work.

  • T003 Create baseline policy-version resolver in /Users/ahmeddarrazi/Documents/projects/TenantAtlas/app/Services/Baselines/Evidence/BaselinePolicyVersionResolver.php (resolve baseline policy_version_id deterministically from baseline snapshot item identity policy_type + subject_key and baseline evidence provenance observed_at; return null when no match)
  • T004 [P] Add resolver unit tests in /Users/ahmeddarrazi/Documents/projects/TenantAtlas/tests/Unit/Baselines/BaselinePolicyVersionResolverTest.php (covers: found, not found, invalid observed_at, deterministic tie-breaker when multiple candidates exist)
  • T005 [P] Add drift evidence contract assertion helper in /Users/ahmeddarrazi/Documents/projects/TenantAtlas/tests/Support/AssertsDriftEvidenceContract.php (required keys, allowed summary.kind, provenance keys, fidelity algorithm, and diff-renderability rule)

Checkpoint: Contract helper + resolver exist; US1 tests can be written against them.

Phase 3: User Story 1 — Understand drift with consistent diffs (Priority: P1) 🎯 MVP

Goal: Baseline Compare drift findings carry diff-compatible evidence so operators get consistent diffs when content evidence exists, and a clear “diff unavailable” explanation when it does not.

Independent Test: Run Baseline Compare to produce (a) a different_version drift finding with both refs, (b) a missing_policy or unexpected_policy finding with a single required ref, and (c) a meta-only finding, then verify the finding detail view renders the correct diff or an explicit “diff unavailable” explanation.

Tests for User Story 1 (write first) ⚠️

  • T006 [P] [US1] Add Baseline Compare evidence-contract tests in /Users/ahmeddarrazi/Documents/projects/TenantAtlas/tests/Feature/Baselines/BaselineCompareDriftEvidenceContractTest.php (different_version + missing_policy; asserts source, change_type semantics unchanged, summary.kind, baseline/current policy_version_id, fidelity algorithm, and evidence_jsonb.provenance.* keys including compare_operation_run_id)
  • T007 [P] [US1] Add FindingResource “diff unavailable” regression test in /Users/ahmeddarrazi/Documents/projects/TenantAtlas/tests/Feature/Drift/DriftFindingDiffUnavailableTest.php (missing baseline/current refs → explicit message)
  • T008 [P] [US1] Update Baseline Compare findings tests for new evidence shape in /Users/ahmeddarrazi/Documents/projects/TenantAtlas/tests/Feature/Baselines/BaselineCompareFindingsTest.php (replace current_hash/baseline_hash assertions with nested baseline/current evidence + summary.kind)

Implementation for User Story 1

  • T009 [US1] Upgrade Baseline Compare drift evidence schema in /Users/ahmeddarrazi/Documents/projects/TenantAtlas/app/Jobs/CompareBaselineToTenantJob.php (write evidence_jsonb.summary.kind, evidence_jsonb.baseline.policy_version_id, evidence_jsonb.current.policy_version_id, compute evidence_jsonb.fidelity, and write evidence_jsonb.provenance.{baseline_profile_id,baseline_snapshot_id,compare_operation_run_id,inventory_sync_run_id})
  • T010 [US1] Populate evidence_jsonb.current.policy_version_id from content evidence meta in /Users/ahmeddarrazi/Documents/projects/TenantAtlas/app/Jobs/CompareBaselineToTenantJob.php (use ResolvedEvidence.meta.policy_version_id when available; else null)
  • T011 [US1] Populate evidence_jsonb.baseline.policy_version_id via /Users/ahmeddarrazi/Documents/projects/TenantAtlas/app/Services/Baselines/Evidence/BaselinePolicyVersionResolver.php and integrate into /Users/ahmeddarrazi/Documents/projects/TenantAtlas/app/Jobs/CompareBaselineToTenantJob.php (when baseline snapshot provenance indicates content evidence + has observed_at, attempt resolve; otherwise null)
  • T012 [US1] Implement summary.kind selection in /Users/ahmeddarrazi/Documents/projects/TenantAtlas/app/Jobs/CompareBaselineToTenantJob.php by “stealing” dimension detection logic from /Users/ahmeddarrazi/Documents/projects/TenantAtlas/app/Services/Drift/DriftFindingGenerator.php (prefer settings snapshot changes; else assignments; else scope tags; fallback policy_snapshot)
  • T013 [US1] Ensure findings.evidence_fidelity and evidence_jsonb.fidelity stay aligned in /Users/ahmeddarrazi/Documents/projects/TenantAtlas/app/Jobs/CompareBaselineToTenantJob.php (compute deterministically from policy-version refs: both present = content, exactly one = mixed, none = meta)
  • T014 [US1] Enforce diff-renderability rule in /Users/ahmeddarrazi/Documents/projects/TenantAtlas/app/Filament/Resources/FindingResource.php (different_version requires both refs; missing_policy/unexpected_policy render against an empty side when their single required ref exists; otherwise show explicit “diff unavailable” explanation in the Diff section)
  • T015 [US1] Run US1-focused tests via /Users/ahmeddarrazi/Documents/projects/TenantAtlas/vendor/bin/sail (artisan test --compact --filter=BaselineCompareDriftEvidenceContract; artisan test --compact --filter=DriftFindingDiffUnavailable)

Checkpoint: Baseline Compare findings show consistent diffs for content evidence; meta-only shows “diff unavailable” without errors.

Phase 4: User Story 2 — Eliminate “two truths” for drift (Priority: P2)

Goal: Operators only see Baseline Compare as the drift engine; no legacy “Generate drift” UI or source switching remains.

Independent Test: After the cutover, verify all drift findings shown in the UI are Baseline Compare-origin and no UI surface offers a legacy “Generate drift” action or “source” switching.

Tests for User Story 2 (write first) ⚠️

  • T016 [P] [US2] Remove/replace DriftLanding enforcement tests in /Users/ahmeddarrazi/Documents/projects/TenantAtlas/tests/Feature/Rbac/DriftLandingUiEnforcementTest.php (assert drift entry point is Baseline Compare landing instead)

Implementation for User Story 2

  • T017 [US2] Remove legacy Drift landing surface by deleting /Users/ahmeddarrazi/Documents/projects/TenantAtlas/app/Filament/Pages/DriftLanding.php and /Users/ahmeddarrazi/Documents/projects/TenantAtlas/resources/views/filament/pages/drift-landing.blade.php
  • T018 [US2] Update related-run links to point drift to Baseline Compare landing in /Users/ahmeddarrazi/Documents/projects/TenantAtlas/app/Support/OperationRunLinks.php (remove DriftLanding import; add BaselineCompareLanding link for baseline_compare runs)
  • T019 [US2] Update dashboard attention widget to use Baseline Compare runs (not drift_generate) in /Users/ahmeddarrazi/Documents/projects/TenantAtlas/app/Filament/Widgets/Dashboard/NeedsAttention.php (stale/failed checks + URLs → Baseline Compare landing / Operations)
  • T020 [US2] Remove DriftLanding action-surface exemption in /Users/ahmeddarrazi/Documents/projects/TenantAtlas/app/Support/Ui/ActionSurface/ActionSurfaceExemptions.php
  • T021 [US2] Update Filament auth allowlist to remove DriftLanding in /Users/ahmeddarrazi/Documents/projects/TenantAtlas/tests/Feature/Guards/NoAdHocFilamentAuthPatternsTest.php
  • T022 [US2] Run US2-focused tests via /Users/ahmeddarrazi/Documents/projects/TenantAtlas/vendor/bin/sail (artisan test --compact --filter=BaselineCompareLanding; artisan test --compact --filter=NeedsAttention)

Checkpoint: No DriftLanding surface exists; “Drift” entry point routes to Baseline Compare landing.

Phase 5: User Story 3 — Clean cutover & legacy removal (Priority: P3)

Goal: Hard cut: legacy run-to-run drift generator is removed end-to-end and legacy drift findings are deleted so operators never encounter mixed states.

Independent Test: After deployment and the one-time cleanup step, verify legacy drift findings no longer exist, legacy drift generation cannot be started, and Baseline Compare drift continues to work.

Tests for User Story 3 (write first) ⚠️

  • T023 [P] [US3] Add cleanup migration test in /Users/ahmeddarrazi/Documents/projects/TenantAtlas/tests/Feature/Drift/LegacyDriftFindingsCleanupMigrationTest.php (deletes finding_type=drift where source is null or != baseline.compare; keeps baseline.compare)
  • T024 [P] [US3] Update alert + monitoring tests to remove drift_generate_findings references in /Users/ahmeddarrazi/Documents/projects/TenantAtlas/tests/Feature/Alerts/BaselineCompareFailedAlertTest.php and /Users/ahmeddarrazi/Documents/projects/TenantAtlas/tests/Feature/RunAuthorizationTenantIsolationTest.php
  • T025 [P] [US3] Remove legacy drift generator tests tied to DriftLanding/GenerateDriftFindingsJob/DriftFindingGenerator in /Users/ahmeddarrazi/Documents/projects/TenantAtlas/tests/Feature/Drift/ (delete or rewrite: DriftGenerationDispatchTest.php, DriftLandingCopyTest.php, DriftLandingShowsComparisonInfoTest.php, GenerateDriftFindingsJobNotificationTest.php, Drift*DriftDetectionTest.php, DriftGenerationDeterminismTest.php, DriftTenantIsolationTest.php)

Implementation for User Story 3

  • T026 [US3] Delete legacy drift generation runtime code: /Users/ahmeddarrazi/Documents/projects/TenantAtlas/app/Jobs/GenerateDriftFindingsJob.php, /Users/ahmeddarrazi/Documents/projects/TenantAtlas/app/Services/Drift/DriftFindingGenerator.php, /Users/ahmeddarrazi/Documents/projects/TenantAtlas/app/Services/Drift/DriftRunSelector.php, /Users/ahmeddarrazi/Documents/projects/TenantAtlas/app/Services/Drift/DriftScopeKey.php
  • T027 [US3] Remove legacy drift operation type from catalogs + triage: /Users/ahmeddarrazi/Documents/projects/TenantAtlas/app/Support/OperationRunType.php, /Users/ahmeddarrazi/Documents/projects/TenantAtlas/app/Support/OperationCatalog.php, /Users/ahmeddarrazi/Documents/projects/TenantAtlas/app/Services/SystemConsole/OperationRunTriageService.php (no drift_generate_findings label/duration/retry/cancel)
  • T028 [US3] Remove legacy drift-generate alert event producer in /Users/ahmeddarrazi/Documents/projects/TenantAtlas/app/Jobs/Alerts/EvaluateAlertsJob.php (drop compareFailedEvents() and its call site)
  • T029 [US3] Add one-time cleanup migration in /Users/ahmeddarrazi/Documents/projects/TenantAtlas/database/migrations/2026_03_05_000001_delete_legacy_drift_findings.php (delete finding_type=drift where source is null or <> baseline.compare)
  • T030 [US3] Remove remaining legacy drift references discovered by search in /Users/ahmeddarrazi/Documents/projects/TenantAtlas/app/ and /Users/ahmeddarrazi/Documents/projects/TenantAtlas/resources/ (target: no DriftLanding / drift_generate_findings strings outside historical migrations/specs; explicitly audit Findings UI for legacy-source badges/filters/labels or “source switching” states)
  • T031 [US3] Run US3-focused tests via /Users/ahmeddarrazi/Documents/projects/TenantAtlas/vendor/bin/sail (artisan test --compact --filter=Alerts; artisan test --compact --filter=OperationRun; artisan test --compact --filter=Drift)

Checkpoint: Legacy generator is gone; DB cleanup migration removes legacy drift findings; all tests green.

Phase N: Polish & Cross-Cutting Concerns

Purpose: Final quality pass across all user stories

  • T032 [P] Format changed files using /Users/ahmeddarrazi/Documents/projects/TenantAtlas/vendor/bin/sail (php vendor/bin/pint)
  • T033 Run full test suite using /Users/ahmeddarrazi/Documents/projects/TenantAtlas/vendor/bin/sail (artisan test)
  • T034 [P] Validate manual smoke steps remain accurate in /Users/ahmeddarrazi/Documents/projects/TenantAtlas/specs/119-baseline-drift-engine/quickstart.md (update only if behavior/screens changed)
  • T035 [P] Update Spec 119 docs for one-sided drift diff rendering in /Users/ahmeddarrazi/Documents/projects/TenantAtlas/specs/119-baseline-drift-engine/ (spec, plan, research, data-model, quickstart)
  • T036 [US1] Extend FindingResource one-sided diff rendering in /Users/ahmeddarrazi/Documents/projects/TenantAtlas/app/Filament/Resources/FindingResource.php (unexpected_policy => added against empty baseline, missing_policy => removed against empty current)
  • T037 [P] Extend drift view regressions in /Users/ahmeddarrazi/Documents/projects/TenantAtlas/tests/Feature/Drift/DriftFindingDiffUnavailableTest.php (cover one-sided empty-side rendering and explicit unavailable messaging)
  • T038 [US1] Scope baseline snapshot capture to the latest completed Inventory Sync in /Users/ahmeddarrazi/Documents/projects/TenantAtlas/app/Jobs/CaptureBaselineSnapshotJob.php (ignore stale inventory_items rows from older sync runs when deriving subject-key matches; record baseline_capture.inventory_sync_run_id for operability)
  • T039 [P] Add baseline-capture stale-inventory regression coverage in /Users/ahmeddarrazi/Documents/projects/TenantAtlas/tests/Feature/Baselines/BaselineCaptureAmbiguousMatchGapTest.php (duplicates in the same latest sync still gap; stale duplicates from older syncs no longer do)
  • T040 [P] Update Spec 119 docs for baseline-capture latest-sync scoping in /Users/ahmeddarrazi/Documents/projects/TenantAtlas/specs/119-baseline-drift-engine/ (spec, plan, research, data-model, quickstart)
  • T041 [US1] Reuse same-run full-content capture evidence in /Users/ahmeddarrazi/Documents/projects/TenantAtlas/app/Jobs/CompareBaselineToTenantJob.php (overlay compare-time captured/reused policy_versions before the since-based resolver so unchanged policies do not become missing_current)
  • T042 [P] Add full-content compare reuse regression in /Users/ahmeddarrazi/Documents/projects/TenantAtlas/tests/Feature/Baselines/BaselineCompareWhyNoFindingsReasonCodeTest.php (reused identical compare-purpose version still yields no_drift_detected, not evidence_capture_incomplete)
  • T043 [P] Update Spec 119 docs for reused compare-evidence handling in /Users/ahmeddarrazi/Documents/projects/TenantAtlas/specs/119-baseline-drift-engine/ (spec, plan, research, quickstart)
  • T044 [US1] Scope Baseline Compare landing duplicate-name warnings to the latest completed Inventory Sync in /Users/ahmeddarrazi/Documents/projects/TenantAtlas/app/Support/Baselines/BaselineCompareStats.php (historical stale duplicates must not keep the warning banner visible)
  • T045 [P] Add landing-page stale-duplicate regression in /Users/ahmeddarrazi/Documents/projects/TenantAtlas/tests/Feature/Filament/BaselineCompareLandingDuplicateNamesBannerTest.php (latest sync clean ⇒ no warning, latest sync duplicate ⇒ warning remains)
  • T046 [P] Update Spec 119 docs for landing-page duplicate warning scoping in /Users/ahmeddarrazi/Documents/projects/TenantAtlas/specs/119-baseline-drift-engine/ (spec, research)
  • T047 [US1] Canonicalize compliance scheduledActionsForRule in /Users/ahmeddarrazi/Documents/projects/TenantAtlas/app/Services/Intune/CompliancePolicyNormalizer.php (include semantic noncompliance action fields in the drift signal; ignore opaque IDs/order-only noise)
  • T048 [P] Add unit coverage for canonical compliance action drift normalization in /Users/ahmeddarrazi/Documents/projects/TenantAtlas/tests/Unit/CompliancePolicyNormalizerTest.php (grace-period/template signal, stable ordering, ignored internal IDs)
  • T049 [US1] Recompute effective baseline content hashes from resolved baseline policy_versions in /Users/ahmeddarrazi/Documents/projects/TenantAtlas/app/Jobs/CompareBaselineToTenantJob.php (keep existing content-backed baseline snapshots comparable when drift-signal semantics expand)
  • T050 [P] Add compare regressions + docs for compliance noncompliance action drift in /Users/ahmeddarrazi/Documents/projects/TenantAtlas/tests/Feature/BaselineDriftEngine/ComplianceNoncomplianceActionsDriftTest.php and /Users/ahmeddarrazi/Documents/projects/TenantAtlas/specs/119-baseline-drift-engine/ (unchanged legacy-hash snapshot stays quiet; changed grace/action semantics create drift)

Dependencies & Execution Order

Phase Dependencies

  • Setup (Phase 1): No dependencies - can start immediately
  • Foundational (Phase 2): Depends on Setup completion - BLOCKS all user stories
  • User Stories (Phase 3+): All depend on Foundational phase completion
    • Stories can proceed in priority order (P1 → P2 → P3)
    • Or in parallel after Phase 2 if staffed (be mindful of overlapping files)
  • Polish (Final Phase): Depends on all desired user stories being complete

User Story Dependencies

  • US1 (P1): No dependencies after Phase 2; delivers the MVP (diff-compatible evidence + diff guardrails)
  • US2 (P2): Depends on US1 being stable in UI terms (so removing DriftLanding doesn’t remove drift visibility)
  • US3 (P3): Can start after Phase 2, but recommended after US1/US2 so the cutover is clean and operators still have a drift workflow

Parallel Opportunities (examples)

  • Tests marked [P] can be written in parallel with implementation (and should fail before the fix lands).
  • Deletions of legacy drift files in US3 can be parallelized with catalog/triage cleanup (different files).

Parallel Example: US1

# In parallel (different files):
Task: "Add Baseline Compare evidence-contract tests in /Users/ahmeddarrazi/Documents/projects/TenantAtlas/tests/Feature/Baselines/BaselineCompareDriftEvidenceContractTest.php"
Task: "Implement evidence schema upgrade in /Users/ahmeddarrazi/Documents/projects/TenantAtlas/app/Jobs/CompareBaselineToTenantJob.php"
Task: "Add diff-unavailable regression test in /Users/ahmeddarrazi/Documents/projects/TenantAtlas/tests/Feature/Drift/DriftFindingDiffUnavailableTest.php"

Parallel Example: US2

# In parallel (different files):
Task: "Delete Drift landing surface in /Users/ahmeddarrazi/Documents/projects/TenantAtlas/app/Filament/Pages/DriftLanding.php"
Task: "Update dashboard widget links in /Users/ahmeddarrazi/Documents/projects/TenantAtlas/app/Filament/Widgets/Dashboard/NeedsAttention.php"
Task: "Update run links in /Users/ahmeddarrazi/Documents/projects/TenantAtlas/app/Support/OperationRunLinks.php"

Parallel Example: US3

# In parallel (different files):
Task: "Add cleanup migration in /Users/ahmeddarrazi/Documents/projects/TenantAtlas/database/migrations/2026_03_05_000001_delete_legacy_drift_findings.php"
Task: "Remove legacy drift code in /Users/ahmeddarrazi/Documents/projects/TenantAtlas/app/Jobs/GenerateDriftFindingsJob.php"
Task: "Update triage/catalog in /Users/ahmeddarrazi/Documents/projects/TenantAtlas/app/Support/OperationCatalog.php"

Implementation Strategy

MVP First (User Story 1 Only)

  1. Complete Phase 1: Setup
  2. Complete Phase 2: Foundational (CRITICAL - blocks all stories)
  3. Complete Phase 3: User Story 1
  4. STOP and VALIDATE: Test User Story 1 independently
  5. Deploy/demo if ready

Incremental Delivery

  1. Complete Setup + Foundational → Foundation ready
  2. Add User Story 1 → Test independently → Deploy/Demo (MVP!)
  3. Add User Story 2 → Test independently → Deploy/Demo
  4. Add User Story 3 → Test independently → Deploy/Demo
  5. Each story adds value without breaking previous stories

Notes

  • [P] tasks = different files, no dependencies
  • [US#] label maps task to specific user story for traceability
  • Each user story should be independently completable and testable
  • Prefer Sail for all local commands: /Users/ahmeddarrazi/Documents/projects/TenantAtlas/vendor/bin/sail