TenantAtlas/specs/115-baseline-operability-alerts/spec.md

# Feature Specification: Baseline Operability & Alert Integration (R1.1–R1.4 Extension)

**Feature Branch**: `115-baseline-operability-alerts`  
**Created**: 2026-02-28  
**Status**: Ready (Design complete; implementation pending)  
**Input**: User description: "115 — Baseline Operability & Alert Integration (R1.1–R1.4 Extension)"

## Spec Scope Fields *(mandatory)*

- **Scope**: workspace (management) + tenant (monitoring)
- **Primary Routes**:
  - Workspace (admin): Baselines management (Baseline Profiles) and Workspace Settings
  - Tenant-context (admin): Baseline Compare monitoring and Baseline Compare “run now” surface
- **Data Ownership**:
  - Workspace-owned: Baseline profiles, baseline-to-tenant assignments, workspace settings, alert rules
  - Tenant-scoped (within a workspace): Findings produced by baseline compare; operation runs for tenant compares
- **RBAC**:
  - Workspace Baselines (view/manage): workspace members must be granted the workspace baselines view/manage capabilities (from the canonical capability registry)
  - Workspace Settings (view/manage): workspace members must be granted the workspace settings view/manage capabilities (from the canonical capability registry)
  - Alerts (view/manage): workspace members must be granted the alerts view/manage capabilities (from the canonical capability registry)
  - Tenant monitoring surfaces require tenant access (tenant view) in addition to workspace membership

For canonical-view specs: not applicable (this is not a canonical-view feature).

## Clarifications

### Session 2026-02-28

- Q: What should `baseline.severity_mapping` map from? → A: Baseline drift `change_type` only (keys: `missing_policy`, `different_version`, `unexpected_policy`).
- Q: What is the canonical “fully successful compare” gate for auto-close? → A: Outcome `succeeded` AND Ops-UX canonical `OperationRun.summary_counts` gate: `summary_counts.processed == summary_counts.total` AND `summary_counts.failed == 0`.
- Q: When a previously resolved baseline finding reappears, what status should it transition to? → A: `reopened`.
- Q: For `baseline_compare_failed` alerts, what cooldown behavior applies? → A: Use the existing dispatcher cooldown (no baseline-specific cooldown setting).
- Q: Should `baseline.auto_close_enabled` exist as a kill-switch? → A: Yes; keep it with default `true`.

### Definitions

- **Baseline finding**: A drift finding produced by comparing a tenant against a baseline.
- **Fingerprint**: A stable identifier for “the same underlying issue” across runs, used for idempotency and alert deduplication.
- **Fully successful compare**: A compare run that succeeded and is complete (no failed items and all expected items processed).
  - Completeness is proven via Ops-UX canonical `OperationRun.summary_counts` counters: `summary_counts.processed == summary_counts.total` and `summary_counts.failed == 0`.

## User Scenarios & Testing *(mandatory)*

### User Story 1 - Safe auto-close removes stale baseline drift (Priority: P1)

As an MSP operator, I want baseline drift findings to automatically resolve when drift is no longer present, so the findings list remains actionable and doesn’t accumulate noise.

**Why this priority**: This is the main “operability” gap; without auto-close, drift remediation cannot be reliably observed, and alerting becomes noisy.

**Independent Test**: Can be fully tested by running a baseline compare that produces a “seen set” of fingerprints and verifying that previously-open baseline findings not present in the seen set are resolved only when the compare is fully successful.

**Acceptance Scenarios**:

1. **Given** an open baseline finding with `source = baseline.compare`, **When** a baseline compare completes fully successfully and that finding’s fingerprint is not present in the current compare result, **Then** the finding becomes `resolved` with reason `no_longer_drifting`.
2. **Given** an open baseline finding with `source = baseline.compare`, **When** a baseline compare is `partially_succeeded` or failed (or incomplete), **Then** no baseline findings are auto-resolved.

---

### User Story 2 - Baseline alerts are precise and deduplicated (Priority: P1)

As an on-call operator, I want alerts for baseline drift and baseline compare failures to trigger only when there is new actionable work (new or reopened findings) or when a compare fails, so I don’t get spammed on every run.

**Why this priority**: MSP sellability depends on trust in alerts; repeated “same problem” alerts make alerting unusable.

**Independent Test**: Can be fully tested by creating findings with controlled timestamps and statuses (new/reopened/open) and verifying that only new/reopened findings generate baseline drift alert events, while repeated compares do not.

**Acceptance Scenarios**:

1. **Given** a new high-severity baseline drift finding (deduped by fingerprint) created after the alert window start, **When** alerts are evaluated, **Then** a `baseline_high_drift` event is produced exactly once.
2. **Given** a resolved baseline drift finding that later reappears and transitions to `reopened`, **When** alerts are evaluated, **Then** a `baseline_high_drift` event is produced again exactly once.
3. **Given** a baseline compare run that completes with outcome failed or `partially_succeeded`, **When** alerts are evaluated, **Then** a `baseline_compare_failed` event is produced (deduped by run identity + cooldown).

---

### User Story 3 - Workspace-controlled severity mapping and alert threshold (Priority: P2)

As a workspace admin, I want to configure how baseline drift categories map to severity, and optionally the minimum severity that triggers baseline drift alerts, so the system matches the MSP’s operational standards.

**Why this priority**: Settings are required for enterprise adoption; hardcoded severity and alert thresholds don’t fit different environments.

**Independent Test**: Can be fully tested by setting workspace overrides and verifying that newly created baseline findings inherit the configured severity and that alert generation respects the configured minimum severity.

**Acceptance Scenarios**:

1. **Given** a workspace-level baseline severity mapping override, **When** a new baseline drift finding is created, **Then** it uses the mapped severity (and rejects invalid severity values).
2. **Given** a workspace-level baseline alert minimum severity override, **When** baseline findings are evaluated for alerts, **Then** only findings meeting or exceeding that threshold emit `baseline_high_drift` events.

### Edge Cases

- Compare completes but is not “fully successful” (e.g., `summary_counts.failed > 0`, incomplete processing where `summary_counts.processed != summary_counts.total`, or compare preconditions prevent the run from being created): auto-close MUST NOT occur.
- Compare does not evaluate all assigned items (e.g., missing baseline snapshot or assignment changes mid-run): auto-close MUST NOT resolve findings for items not evaluated.
- A baseline finding was resolved previously and reappears later: it must transition to an actionable open state (e.g., `reopened`) and be eligible for alerting once.
- Workspace settings payload is malformed (unknown drift categories or invalid severity values): save MUST be rejected and effective values MUST remain unchanged.

## Requirements *(mandatory)*

**Constitution alignment (required):** If this feature introduces any Microsoft Graph calls, any write/change behavior,
or any long-running/queued/scheduled work, the spec MUST describe contract registry updates, safety gates
(preview/confirmation/audit), tenant isolation, run observability (`OperationRun` type/identity/visibility), and tests.
If security-relevant DB-only actions intentionally skip `OperationRun`, the spec MUST describe `AuditLog` entries.

**Constitution alignment (OPS-UX):** If this feature creates/reuses an `OperationRun`, the spec MUST:
- explicitly state compliance with the Ops-UX 3-surface feedback contract (toast intent-only, progress surfaces, terminal DB notification),
- state that `OperationRun.status` / `OperationRun.outcome` transitions are service-owned (only via `OperationRunService`),
- describe how `summary_counts` keys/values comply with `OperationSummaryKeys::all()` and numeric-only rules,
- clarify scheduled/system-run behavior (initiator null → no terminal DB notification; audit is via Monitoring),
- list which regression guard tests are added/updated to keep these rules enforceable in CI.

**Constitution alignment (RBAC-UX):** If this feature introduces or changes authorization behavior, the spec MUST:
- state which authorization plane(s) are involved (tenant/admin `/admin` + tenant-context `/admin/t/{tenant}/...` vs platform `/system`),
- ensure any cross-plane access is deny-as-not-found (404),
- explicitly define 404 vs 403 semantics:
  - non-member / not entitled to workspace scope OR tenant scope → 404 (deny-as-not-found)
  - member but missing capability → 403
- describe how authorization is enforced server-side (Gates/Policies) for every mutation/operation-start/credential change,
- reference the canonical capability registry (no raw capability strings; no role-string checks in feature code),
- ensure global search is tenant-scoped and non-member-safe (no hints; inaccessible results treated as 404 semantics),
- ensure destructive-like actions require confirmation (`->requiresConfirmation()`),
- include at least one positive and one negative authorization test, and note any RBAC regression tests added/updated.

**Constitution alignment (OPS-EX-AUTH-001):** OIDC/SAML login handshakes may perform synchronous outbound HTTP (e.g., token exchange)
on `/auth/*` endpoints without an `OperationRun`. This MUST NOT be used for Monitoring/Operations pages.

**Constitution alignment (BADGE-001):** If this feature changes status-like badges (status/outcome/severity/risk/availability/boolean),
the spec MUST describe how badge semantics stay centralized (no ad-hoc mappings) and which tests cover any new/changed values.

**Constitution alignment (Filament Action Surfaces):** If this feature adds or modifies any Filament Resource / RelationManager / Page,
the spec MUST include a “UI Action Matrix” (see below) and explicitly state whether the Action Surface Contract is satisfied.
If the contract is not satisfied, the spec MUST include an explicit exemption with rationale.
**Constitution alignment (UX-001 — Layout & Information Architecture):** If this feature adds or modifies any Filament screen,
the spec MUST describe compliance with UX-001: Create/Edit uses Main/Aside layout (3-col grid), all fields inside Sections/Cards
(no naked inputs), View pages use Infolists (not disabled edit forms), status badges use BADGE-001, empty states have a specific
title + explanation + exactly 1 CTA, and tables provide search/sort/filters for core dimensions.
If UX-001 is not fully satisfied, the spec MUST include an explicit exemption with documented rationale.

### Functional Requirements

- **FR-001 (Finding source contract)**: Findings created by baseline compare MUST be identifiable as baseline compare findings via a stable source identifier (`source = baseline.compare`).

- **FR-002 (Fully successful guardrail)**: Auto-close MUST run only after a fully successful baseline compare run. “Fully successful” means all of the following:
  - The compare run outcome is `succeeded`.
  - The compare run emits Ops-UX canonical completeness counters in `OperationRun.summary_counts`, and they indicate:
    - `summary_counts.failed == 0`
    - `summary_counts.processed == summary_counts.total`
  - Compare preconditions (e.g., missing active baseline snapshot, missing assignment) are enforced before enqueue and MUST prevent a compare run from being created; therefore auto-close cannot run when preconditions fail.
  - **Implementation note (required invariant)**: precondition failures are returned as stable reason codes and MUST result in **no `OperationRun` being created**.

- **FR-003 (Safe auto-close behavior)**: After a fully successful compare, the system MUST resolve open baseline compare findings that are not present in the current run’s seen set.
- **FR-004 (No partial resolution)**: The system MUST NOT resolve findings for any items that were not evaluated in the run.
- **FR-005 (Resolution reason)**: Auto-resolved findings MUST record resolution reason `no_longer_drifting`.

- **FR-006 (Reopen semantics)**: If a previously resolved baseline compare finding reappears in a later compare, it MUST transition to an actionable open state (e.g., `reopened`) and be treated as “new actionable work” for alerting.
  - The required open state for reappearance is `reopened`.

- **FR-007 (Alert event type: baseline drift)**: The system MUST support an alert event type `baseline_high_drift` for baseline compare findings.
- **FR-008 (Alert producer: baseline drift)**: Baseline drift alert events MUST be produced only for baseline compare findings that are actionable (open states) AND are either newly created or newly reopened within the evaluation window.
- **FR-009 (Baseline drift deduplication)**: Baseline drift alert events MUST be deduplicated by a stable key derived from the finding fingerprint. The same open finding MUST NOT emit repeated events on subsequent compares.

- **FR-010 (Alert event type: compare failed)**: The system MUST support an alert event type `baseline_compare_failed`.
- **FR-011 (Alert producer: compare failed)**: A `baseline_compare_failed` event MUST be produced when a baseline compare run completes with outcome failed or `partially_succeeded`.
- **FR-012 (Compare-failed dedup + cooldown)**: Compare-failed events MUST be deduplicated per run identity and MUST respect existing cooldown/quiet-hours behavior.
  - This feature MUST NOT introduce a baseline-specific cooldown interval; it reuses the existing dispatcher cooldown behavior.

- **FR-013 (Canonical run types)**: Baseline capture and baseline compare MUST use centrally defined canonical run types (`baseline_capture`, `baseline_compare`) and MUST NOT rely on ad-hoc string literals.

- **FR-014 (Workspace settings: severity mapping)**: The system MUST support a workspace setting `baseline.severity_mapping` that maps baseline drift categories to severity.
- **FR-015 (Workspace settings: validation)**: The severity mapping MUST:
  - accept only the baseline drift `change_type` keys `missing_policy`, `different_version`, and `unexpected_policy` (no other keys),
  - reject invalid severity values,
  - and expose “effective value” behavior (system defaults + workspace overrides).
- **FR-016 (Workspace settings: alert threshold)**: The system MUST support a workspace setting `baseline.alert_min_severity` with allowed values low/medium/high/critical and default high.
  - Severity threshold comparison MUST use the canonical severity ordering: `low < medium < high < critical` (inclusive).
- **FR-017 (Workspace settings: auto-close toggle)**: The system MUST support a workspace setting `baseline.auto_close_enabled` defaulting to true.
  - When set to `false`, auto-close MUST be skipped even if the compare is fully successful.

- **FR-018 (Information architecture / ownership)**: Baseline Profile CRUD MUST remain workspace-owned. It MUST NOT appear tenant-scoped, must not show tenant scope banners, and must not be reachable from tenant-only navigation.

#### Assumptions & Dependencies

- Baseline compare already produces stable finding fingerprints and a per-run Ops-UX `summary_counts` payload that can express completeness (`processed` vs `total`) and failures (`failed`).
- Findings support lifecycle transitions including resolve with a reason and reopen semantics for a recurring fingerprint.
- Alert dispatch already supports deduplication, cooldown, and quiet hours; this feature reuses that behavior for new baseline-specific event types.

#### Constitution Alignment Notes (non-functional but mandatory)

- This feature adds no new Microsoft Graph calls.
- Baseline compare and alert evaluation are long-running operations; any new auto-close and alert integration MUST preserve tenant isolation and run observability.

- **Ops-UX (3-surface feedback)**: baseline compare/capture and alerts evaluation must continue to provide:
  - an intent-only toast on start,
  - progress surfaces (Operations pages),
  - and terminal DB notifications where applicable.

- **Operation run ownership**: Operation status/outcome transitions are owned by the operations subsystem and must not be mutated directly by UI code.
- **Summary counts contract**: Any summary counters produced/updated by this feature MUST use the canonical summary key registry and numeric-only values.
- **Scheduled/system runs**: Runs initiated without a human initiator MUST not produce terminal DB notifications; monitoring remains via Operations/Alerts.

- **RBAC-UX**: Authorization planes involved:
  - Workspace management plane (admin, workspace-owned baselines + workspace settings)
  - Tenant-context plane (baseline compare monitoring)
  Cross-plane access MUST be deny-as-not-found (404).
  - Non-member or not entitled to the workspace scope or tenant scope → 404 (deny-as-not-found)
  - Member but missing capability for the surface → 403

- **BADGE-001**: Any new baseline severity mapping must remain centralized (single mapping source) and covered by tests.

## UI Action Matrix *(mandatory when Filament is changed)*

| Surface | Location | Header Actions | Inspect Affordance (List/Table) | Row Actions (max 2 visible) | Bulk Actions (grouped) | Empty-State CTA(s) | View Header Actions | Create/Edit Save+Cancel | Audit log? | Notes / Exemptions |
|---|---|---|---|---|---|---|---|---|---|---|
| Resource | Admin → Governance → Baselines (workspace management) | Create baseline profile (capability-gated) | View action | Edit (capability-gated), Delete/Archive (if present) | None | Create baseline profile | Capture / Compare shortcuts (if present), Edit | Save + Cancel | Yes | Workspace-owned; MUST NOT show tenant scope banner; must not appear in tenant nav. |
| Page | Admin → Settings → Workspace settings | Save (capability-gated) | N/A | N/A | N/A | N/A | N/A | Save + Cancel (or equivalent) | Yes | Adds baseline settings fields; validation must reject malformed mapping. |
| Page | Admin → Tenant context → Governance → Baseline Compare | Compare now (capability-gated) | Link to findings / operation run details | N/A | N/A | N/A | N/A | N/A | Yes | Tenant-context monitoring surface; must not expose workspace management actions. |

### Key Entities *(include if feature involves data)*

- **Baseline Compare Finding**: A finding produced by a baseline compare run, identified by `source = baseline.compare` and a stable fingerprint.
- **Baseline Compare Run**: A run that evaluates tenant configuration against a baseline profile and produces a compare summary that can indicate completeness.
- **Alert Event**: A deduplicated, rule-dispatchable representation of actionable baseline drift or baseline compare failure.
- **Workspace Baseline Settings**: Workspace-specific overrides for severity mapping, alert threshold, and auto-close enablement.

## Success Criteria *(mandatory)*

### Measurable Outcomes

- **SC-001 (Noise reduction)**: In a controlled test scenario where drift disappears, 100% of baseline drift findings created by baseline compare auto-resolve after the first fully successful compare.
- **SC-002 (Safety)**: In scenarios where compare is failed/`partially_succeeded`/incomplete, 0 baseline findings are auto-resolved.
- **SC-003 (Alert dedupe)**: The same open baseline drift finding does not generate more than 1 `baseline_high_drift` alert event per open/reopen cycle.
- **SC-004 (Timeliness)**: Baseline compare failures generate a `baseline_compare_failed` alert event within the next alert evaluation cycle.
- **SC-005 (Configurability)**: A workspace admin can change baseline severity mapping and minimum alert severity in under 2 minutes, and newly generated findings reflect the change.