TenantAtlas/specs/040-inventory-core/spec.md

Feature Specification: Inventory Core (Sync + Catalog)

Feature Branch: feat/040-inventory-core
Created: 2026-01-07
Status: Draft

Overview

TenantPilot needs a reliable, tenant-scoped inventory catalog that represents what the system last observed in Microsoft Intune. This inventory is used as the primary substrate for analysis, reporting, monitoring, and UI visibility.

Key intent: Inventory is a “last observed” catalog (TenantPilot’s truth), not an absolute truth about Intune completeness.

Non-goal: A sync MUST NOT create snapshots or backups automatically.

User Scenarios & Testing (mandatory)

User Story 1 — Run Inventory Sync for a Tenant (Priority: P1)

A tenant admin (or scheduled automation) runs an inventory sync for a tenant to populate/update the inventory catalog.

Why this priority: Everything else depends on having a stable, queryable inventory catalog.

Independent Test: Run a sync for a tenant and verify inventory items are upserted, tenant-scoped, and last-observed fields update without producing snapshots/backups.

Acceptance Scenarios:

1. Given a tenant and a configured selection of policy types/categories, When a sync completes, Then inventory items are upserted for each observed object with correct tenant_id, policy_type, external_id, and last_seen_at.
2. Given an existing inventory item, When the same object is observed again, Then the existing record is updated (not duplicated) and last_seen_at and last_seen_run_id are updated.
3. Given a sync selection that excludes some policy types/categories, When the sync completes, Then only objects within that selection are observed/updated.
4. Given a successful sync, When the sync finishes, Then no policy snapshots/backups are created as a side effect.

---

User Story 2 — Observe Completeness/Confidence of a Sync (Priority: P1)

A tenant admin views whether missing items are likely “not seen” due to partial/failed sync vs confidently missing in a clean run.

Why this priority: Prevents misleading conclusions (e.g., “deleted”) when Graph errors or permissions issues occur.

Independent Test: Mark a run as partial/failed and verify missing items are presented as low confidence (derived at query/UI time) and do not imply deletion.

Acceptance Scenarios:

1. Given a latestRun for a tenant+selection that has status != success or had_errors = true, When inventory is queried for missing items relative to that run, Then missing is presented as low confidence (and no stronger claim is made).
2. Given a latestRun for a tenant+selection that is status = success and had_errors = false, When an item was not observed in that run, Then the UI can show it as “not seen in latest run” (higher confidence) without implying deletion.

---

User Story 3 — Monitor Sync Runs (Priority: P2)

A tenant admin (and platform admin) can see sync run history and quickly diagnose failures using stable error codes and counts.

Why this priority: Makes automation observable and supportable at MSP scale.

Independent Test: Create sync runs with different statuses and verify run records include counts and stable error codes.

Acceptance Scenarios:

1. Given multiple sync runs, When a user views run history, Then each run shows status, started/finished timestamps, and counts (observed/updated/errors).
2. Given a throttling event, When a sync run records it, Then the run captures a stable error code (e.g., “graph_throttled”) and does not fail silently.

---

Edge Cases

• Sync is triggered twice for the same tenant+selection while the first is still running.
• Sync completes with partial results due to transient Graph errors.
• A tenant’s permissions change between runs causing objects to be invisible.
• Selection payload is equivalent but arrays are ordered differently.

Requirements (mandatory)

Functional Requirements

• FR-001: System MUST maintain an Inventory Catalog that represents TenantPilot’s last observed state of Intune objects.
• FR-002: System MUST upsert inventory items by a stable identity key that prevents duplicates.
• FR-003: System MUST record Sync Runs with status, timestamps, counts, and stable error codes.
• FR-004: System MUST ensure tenant isolation for all inventory and run queries.
• FR-005: System MUST support deterministic selection scoping via selection_hash for sync runs.
• FR-006: System MUST NOT create snapshots/backups during inventory sync (sync is not backup).
• FR-007: System MUST derive “missing” as a computed state relative to the latest completed run for the same tenant+selection.
• FR-008: System MUST enforce meta_jsonb key whitelisting by dropping unknown keys without failing the sync.
• FR-009: System MUST implement safe automation behavior: locking, idempotency, and observable failures.

Non-Functional Requirements

• NFR-001 (Concurrency limits): Sync automation MUST enforce two limits: a global concurrency limit (across tenants) and a per-tenant concurrency limit.
• NFR-002 (Throttling resilience): Sync MUST handle throttling/transient failures (e.g., 429/503) using backoff + jitter.
• NFR-003 (Deterministic behavior): Selection hashing and capability derivation MUST be deterministic and testable.
• NFR-004 (Data minimization): Inventory MUST store metadata and whitelisted meta only; payload-heavy content belongs to snapshots/backups.
• NFR-005 (Safe logging): Logs MUST not contain secrets/tokens; monitoring MUST rely on run records + error codes.

Key Entities (include if feature involves data)

• Inventory Item: A tenant-scoped record representing a single Intune object as last observed (type, external identity, display name/metadata, last observed fields, whitelisted meta).
• Sync Run: A tenant-scoped record representing an inventory sync execution for a specific selection (selection_hash, status, timestamps, counts, stable error codes).
• Selection Payload: The normalized representation of the run scope used to compute selection_hash.

Success Criteria (mandatory)

Measurable Outcomes

• SC-001: For a given tenant, inventory sync can be executed repeatedly without creating duplicate inventory items.
• SC-002: A sync run always produces a run record with status, timestamps, and counts.
• SC-003: Missing is computed relative to latest completed run for the same tenant+selection; runs with different selection hashes do not affect each other.
• SC-004: Unknown meta keys never break sync and are not persisted.
• SC-005: Operators can distinguish “not seen” from “deleted” (deleted is reserved and not produced in this feature).

Spec Appendix: Deterministic Selection + Missing Semantics (copy/paste-ready)

Definition: “completed” and “latestRun”

• Definition: completed means status ∈ {success, partial, failed, skipped} and finished_at != null (or the equivalent field used by the run model).
• Definition: latestRun is the latest completed Sync Run for (tenant_id, selection_hash).

Selection Hash

• selection_payload includes only fields that influence run scope:
  • policy_types[], categories[], include_foundations (bool), include_dependencies (bool)
• canonical_json(payload) is a canonical JSON serialization with:
  • sorted object keys
  • sorted arrays for policy_types and categories
  • no whitespace / pretty formatting
• selection_hash = sha256(canonical_json(selection_payload))
• AC: Identical selection payload ⇒ identical selection_hash (independent of array ordering).

Missing is derived (not persisted)

• Definition: Missing is a derived state computed at query/UI time relative to latestRun(tenant_id, selection_hash).
• AC: Runs with different selection_hash do not affect missing computation for other selections.
• If latestRun.status != success or latestRun.had_errors = true, items not observed in that run are presented as missing (low confidence).

Deleted is reserved

• deleted is reserved and MUST NOT be produced by this feature.
• Only a later lifecycle feature may set deleted with strict verification rules.

Meta Whitelist (Fail-safe)

• meta_jsonb has a documented whitelist of allowed keys.
• AC: Unknown meta_jsonb keys are dropped (not persisted) and MUST NOT cause sync to fail.

Initial meta_jsonb whitelist (v1)

Allowed keys (all optional; if not applicable for a type, omit):

• odata_type: string (copied from Graph @odata.type)
• etag: string|null (Graph etag if available; never treated as a secret)
• scope_tag_ids: array (IDs only; no display names required)
• assignment_target_count: int|null (count only; no target details)
• warnings: array (bounded, human-readable, no secrets)

AC: Any other key is dropped silently (not persisted) and MUST NOT fail sync.

Observed Run

• inventory_items.last_seen_run_id and inventory_items.last_seen_at are updated when an item is observed.
• last_seen_run_id implies the selection via sync_runs.selection_hash; no per-item selection hash is required for core.

Run Error Codes (taxonomy)

Sync runs record:

• status: one of success|partial|failed|skipped
• had_errors: bool (true if any non-ideal condition occurred)
• error_codes[]: array of stable machine-readable codes (no secrets)

Minimal taxonomy (3–8 codes):

• lock_contended (a run could not start because the per-tenant+selection lock is held)
• concurrency_limit_global (global concurrency limit reached; run skipped)
• concurrency_limit_tenant (per-tenant concurrency limit reached; run skipped)
• graph_throttled (429 encountered; run partial/failed depending on recovery)
• graph_transient (503/timeout/other transient errors)
• graph_forbidden (403/insufficient permission)
• unexpected_exception (unexpected failure; message must be safe/redacted)

Rule: Run records MUST store codes (and safe, bounded context) rather than raw exception dumps or tokens.

Concurrency Limits (source, defaults, behavior)

Source: Config (recommended keys):

• tenantpilot.inventory_sync.concurrency.global_max
• tenantpilot.inventory_sync.concurrency.per_tenant_max

Defaults (if not configured):

• global_max = 2
• per_tenant_max = 1

Behavior when limits are hit:

• The system MUST create a Sync Run record with:
  • status = skipped
  • had_errors = true (so missing stays low-confidence for that selection)
  • error_codes[] includes concurrency_limit_global or concurrency_limit_tenant
  • started_at/finished_at set (observable)
• No inventory items are mutated in a skipped run.

Testing Guidance (non-implementation)

These are test cases expressed in behavior terms (not code).

Test Cases — Sync and Upsert

• TC-001: Sync creates or updates inventory items and sets last_seen_at.
• TC-002: Re-running sync for the same tenant+selection updates existing records and does not create duplicates.
• TC-003: Inventory queries scoped to Tenant A never return Tenant B’s items.
• TC-004: Inventory sync does not create or modify snapshot/backup records (e.g., no new rows in policy_versions, backup_sets, backup_items, backup_schedules, backup_schedule_runs).

Test Cases — Selection Hash Determinism

• TC-010: Same selection payload with arrays in different order yields the same selection_hash.
• TC-011: Different selection payload yields a different selection_hash.

Test Cases — Missing Semantics

• TC-020: Missing is derived relative to latest completed run for the same tenant+selection.
• TC-021: A run for selection Y does not affect missing computation for selection X.
• TC-022: If latestRun is partial/failed or had_errors, missing is shown as low confidence.

Test Cases — Meta Whitelist

• TC-030: Unknown meta keys are not persisted and do not fail sync.

Test Cases — Automation Safety

• TC-040: Concurrent sync triggers for the same tenant+selection do not result in overlapping runs (lock behavior).
• TC-041: A throttling event results in a visible, stable error code and a non-silent failure signal.