TenantAtlas/specs/118-baseline-drift-engine/research.md

# Research — Spec 118 Golden Master Deep Drift v2

This document resolves planning unknowns for implementing `/Users/ahmeddarrazi/Documents/projects/TenantAtlas/specs/118-baseline-drift-engine/spec.md` in the existing Laravel + Filament codebase.

## Decision 1 — Full-content evidence capture orchestration

**Decision**: Introduce a dedicated “baseline content capture” phase that can be invoked from both baseline capture and baseline compare:

- Baseline capture (`baseline_capture` run): capture evidence needed to build a content-fidelity baseline snapshot (as budget allows).
- Baseline compare (`baseline_compare` run): refresh current evidence before drift evaluation (as budget allows).

The phase reuses the existing Intune capture orchestration (`/Users/ahmeddarrazi/Documents/projects/TenantAtlas/app/Services/Intune/PolicyCaptureOrchestrator.php`) so we do not introduce a second capture implementation.

**Rationale**:
- Aligns with Spec 118 goal: deep drift by default, without per-policy manual capture.
- Keeps a single source of truth for content capture (policy payload + assignments + scope tags).
- Makes quota management, retries, and resumability explicit at the operation level.

**Alternatives considered**:
- Opportunistic only (rejected: repeats Spec 117 fragility; “no drift” can still be a silent failure).
- UI-driven per-policy capture (rejected: explicitly out of UX goals).

## Decision 2 — PolicyVersion purpose tagging + run traceability

**Decision**: Extend `policy_versions` with baseline-purpose attribution:

- `capture_purpose`: `backup | baseline_capture | baseline_compare`
- `operation_run_id` (nullable): link to the run that captured the version
- `baseline_profile_id` (nullable): link for baseline_* captures

**Rationale**:
- Enables audit/debug (“which run produced this evidence, for what purpose?”) without introducing a separate evidence table.
- Supports idempotency and “resume capture” semantics (skip already-captured subjects for the same run/purpose).

**Alternatives considered**:
- Store purpose only in `policy_versions.metadata` (rejected: harder to index/query; weaker guardrails).
- Create an EvidenceItems model now (rejected: explicitly not required in Spec 118).

## Decision 3 — Golden Master subject matching across tenants

**Decision**: Treat the Golden Master “subject identity” as a cross-tenant match key derived from policy display name:

- Subject match key: `policy_type + normalized_display_name`
- `normalized_display_name` rules: trim leading/trailing whitespace, collapse internal whitespace to single spaces, lowercase.

Implementation uses a dedicated snapshot-item field (e.g., `baseline_snapshot_items.subject_key`) for matching, while preserving tenant-specific external IDs separately for evidence resolution.

Ambiguous/missing match handling:
- Missing match in current tenant → eligible for “missing policy” (only with coverage proof).
- Multiple matches for the same key within a tenant/type → record evidence gap and suppress drift evaluation for that subject key (no finding).

**Rationale**:
- Baselines are workspace-owned and can be assigned to multiple tenants; external IDs are tenant-specific and cannot be used for cross-tenant matching.
- The match key keeps snapshot items free of tenant identifiers while enabling consistent comparisons.

**Alternatives considered**:
- Match by tenant external ID (rejected: breaks cross-tenant baseline assignment).
- Require per-tenant baseline snapshots (rejected for Spec 118: changes product semantics and assignment UX).
- Introduce an explicit mapping table (rejected for R1: higher effort and requires operational UX not described in spec).

## Decision 4 — Quota-aware capture + resumable token

**Decision**: Evidence capture is bounded and resumable:

- Enforce per-run limits (max items, max concurrency, max retry attempts).
- Store an opaque “resume token” in `operation_runs.context` when a run cannot complete within budget.
- Provide a “Resume capture” UI action that starts a follow-up run continuing from that token.

**Rationale**:
- Large tenants/scopes must not create uncontrolled queue storms or long-running jobs.
- Operators need explicit visibility into “what was captured vs skipped” and a safe path to completion.

**Alternatives considered**:
- “Always finish no matter what” (rejected: risks rate limiting and operational instability).
- Mark run failed on any capture failure (rejected: Spec 118 allows partial failure with warnings).

## Decision 5 — Ops-UX + run context contract (“Why no findings?”)

**Decision**: Baseline runs explicitly populate:

- `context.target_scope` (required for Monitoring run detail; avoids “No target scope details…”)
- `context.effective_scope` + `context.capture_mode`
- evidence capture stats + gaps + reason codes when subjects processed = 0 or findings = 0

Keep `summary_counts` numeric-only and limited to keys from `/Users/ahmeddarrazi/Documents/projects/TenantAtlas/app/Support/OpsUx/OperationSummaryKeys.php`; store richer detail in `context`.

**Rationale**:
- Eliminates ambiguous “0 findings” outcomes and improves operator trust.
- Conforms to Ops-UX 3-surface feedback contract and Monitoring expectations.

**Alternatives considered**:
- Put details into `summary_counts` (rejected: key whitelist contract).
- Only log details (rejected: operators need UI visibility).

## Notes on current codebase (facts observed)

- Baseline capture run creation: `/Users/ahmeddarrazi/Documents/projects/TenantAtlas/app/Services/Baselines/BaselineCaptureService.php`
- Baseline compare run creation: `/Users/ahmeddarrazi/Documents/projects/TenantAtlas/app/Services/Baselines/BaselineCompareService.php`
- Capture job (currently opportunistic content): `/Users/ahmeddarrazi/Documents/projects/TenantAtlas/app/Jobs/CaptureBaselineSnapshotJob.php`
- Compare job (provider-chain evidence): `/Users/ahmeddarrazi/Documents/projects/TenantAtlas/app/Jobs/CompareBaselineToTenantJob.php`
- Evidence providers + resolver: `/Users/ahmeddarrazi/Documents/projects/TenantAtlas/app/Services/Baselines/CurrentStateHashResolver.php` and `/Users/ahmeddarrazi/Documents/projects/TenantAtlas/app/Services/Baselines/Evidence/*`
- Monitoring target scope rendering expects `context.target_scope`: `/Users/ahmeddarrazi/Documents/projects/TenantAtlas/app/Filament/Resources/OperationRunResource.php`