ahmido
8ae7a7234e
Summary Implements Inventory Core (Spec 040): a tenant-scoped, mutable “last observed” inventory catalog + sync run logging, with deterministic selection hashing and safe derived “missing” semantics. This establishes the foundation for Inventory UI (041), Dependencies Graph (042), Compare/Promotion (043), and Drift (044). What’s included • DB schema • inventory_items (unique: tenant_id + policy_type + external_id; indexes; last_seen_at, last_seen_run_id) • inventory_sync_runs (tenant_id, selection_hash/payload, status, started/finished, counts, error_codes, correlation_id) • Selection hashing • Deterministic selection_hash via canonical JSON (sorted keys + sorted arrays) + sha256 • Sync semantics • Idempotent upsert (no duplicates) • Updates last_seen_* when observed • Enforces tenant scoping for all reads/writes • Guardrail: inventory sync does not create snapshots/backups • Missing semantics (derived) • “missing” computed relative to latest completed run for same (tenant_id, selection_hash) • Low confidence when latest run is partial/failed or had_errors=true • Selection isolation (runs for other selections don’t affect missing) • deleted is reserved (not produced here) • Safety • meta_jsonb whitelist enforced (unknown keys dropped; never fail sync) • Safe error persistence (no bearer tokens / secrets) • Locking to prevent overlapping runs for same tenant+selection • Concurrency limiter (global + per-tenant) and throttling resilience (429/503 backoff + jitter) Tests Added Pest coverage for: • selection_hash determinism (array order invariant) • upsert idempotency + last_seen updates • missing derived semantics + selection isolation • low confidence missing on partial/had_errors • meta whitelist drop (no exception) • lock prevents overlapping runs • no snapshots/backups side effects • safe error persistence (no bearer tokens) Non-goals • Inventory UI pages/resources (Spec 041) • Dependency graph hydration (Spec 042) • Cross-tenant compare/promotion flows (Spec 043) • Drift analysis dashboards (Spec 044) Review focus • Data model correctness + indexes/constraints • Selection hash canonicalization (determinism) • Missing semantics (latest completed run + confidence rule) • Guardrails (no snapshot/backups side effects) • Safety: error_code taxonomy + safe persistence/logging Co-authored-by: Ahmed Darrazi <ahmeddarrazi@adsmac.local> Reviewed-on: #43
127 lines
4.8 KiB
Markdown
127 lines
4.8 KiB
Markdown
# Implementation Plan: Inventory Core (040)
|
|
|
|
**Branch**: `spec/040-inventory-core` | **Date**: 2026-01-07 | **Spec**: `specs/040-inventory-core/spec.md`
|
|
**Scope (this step)**: Produce a clean, implementable `plan.md` + consistent `tasks.md` for Spec 040 only.
|
|
|
|
## Summary
|
|
|
|
Implement tenant-scoped Inventory + Sync Run tracking as the foundational substrate for later Inventory UI and higher-order features.
|
|
|
|
Key outcomes:
|
|
|
|
- Inventory is “last observed” (not backup), stored as metadata + whitelisted `meta_jsonb`.
|
|
- Sync runs are observable, selection-scoped via deterministic `selection_hash`.
|
|
- “Missing” is derived relative to latest completed run for the same `(tenant_id, selection_hash)`.
|
|
- Automation is safe: locks, idempotency, throttling handling, global+per-tenant concurrency limits.
|
|
|
|
## Technical Context
|
|
|
|
- **Language/Version**: PHP 8.4
|
|
- **Framework**: Laravel 12
|
|
- **Admin UI**: Filament v4 + Livewire v3
|
|
- **Storage**: PostgreSQL (JSONB available)
|
|
- **Queue/Locks**: Laravel queue + cache/Redis locks (as configured)
|
|
- **Testing**: Pest v4 (`php artisan test`)
|
|
- **Target Platform**: Sail-first local dev; container deploy (Dokploy)
|
|
|
|
## Constitution Check
|
|
|
|
- Inventory-first: inventory stores last observed state only (no snapshot/backup side effects).
|
|
- Read/write separation: this feature introduces no Intune write paths.
|
|
- Single contract path to Graph: Graph reads (if needed) go via Graph abstraction and contracts.
|
|
- Tenant isolation: all reads/writes tenant-scoped; no cross-tenant shortcuts.
|
|
- Automation: locked + idempotent + observable; handle 429/503 with backoff+jitter.
|
|
- Data minimization: no payload-heavy storage; safe logs.
|
|
|
|
No constitution violations expected.
|
|
|
|
## Project Structure (Impacted Areas)
|
|
|
|
```text
|
|
specs/040-inventory-core/
|
|
├── spec.md
|
|
├── plan.md
|
|
└── tasks.md
|
|
|
|
app/
|
|
├── Models/
|
|
├── Jobs/
|
|
├── Services/
|
|
└── Support/
|
|
|
|
database/migrations/
|
|
tests/Feature/
|
|
tests/Unit/
|
|
```
|
|
|
|
## Implementation Approach
|
|
|
|
### Phase A — Data Model + Migrations
|
|
|
|
1. Add `inventory_items` table
|
|
- Identity: unique constraint to prevent duplicates, recommended:
|
|
- `(tenant_id, policy_type, external_id)`
|
|
- Fields: `display_name`, `platform`/`category` (if applicable), `meta_jsonb`, `last_seen_at`, `last_seen_run_id`.
|
|
- Indexing: indexes supporting tenant/type listing; consider partials as needed.
|
|
|
|
2. Add `inventory_sync_runs` table
|
|
- Identity: `tenant_id`, `selection_hash`
|
|
- Status fields: `status`, `started_at`, `finished_at`, `had_errors`
|
|
- Counters: `items_observed_count`, `items_upserted_count`, `errors_count`
|
|
- Error reporting: stable error code(s) list or summary field.
|
|
|
|
### Phase B — Selection Hash (Deterministic)
|
|
|
|
Implement canonicalization exactly as Spec Appendix:
|
|
|
|
- Only include scope-affecting keys in `selection_payload`.
|
|
- Sort object keys; sort `policy_types[]` and `categories[]` arrays.
|
|
- Compute `selection_hash = sha256(canonical_json(selection_payload))`.
|
|
|
|
### Phase C — Sync Run Lifecycle + Upsert
|
|
|
|
- Create a service that:
|
|
- acquires a lock for `(tenant_id, selection_hash)`
|
|
- creates a run record
|
|
- enumerates selected policy types
|
|
- upserts inventory items by identity key
|
|
- updates `last_seen_at` and `last_seen_run_id` per observed item
|
|
- finalizes run status + counters
|
|
- never creates/modifies snapshot/backup records (`policy_versions`, `backup_*`)
|
|
|
|
### Phase D — Derived “Missing” Semantics
|
|
|
|
- Implement “missing” as a computed state relative to `latestRun(tenant_id, selection_hash)`.
|
|
- Do not persist “missing” or “deleted”.
|
|
- Mark missing as low-confidence when `latestRun.status != success` or `latestRun.had_errors = true`.
|
|
|
|
### Phase E — Meta Whitelist
|
|
|
|
- Define a whitelist of allowed `meta_jsonb` keys.
|
|
- Enforce by dropping unknown keys (never fail sync).
|
|
|
|
### Phase F — Concurrency Limits
|
|
|
|
- Enforce global concurrency (across tenants) and per-tenant concurrency.
|
|
- The implementation may be via queue worker limits, semaphore/lock strategy, or both; the behavior must be testable.
|
|
- When limits are hit, create an observable run record with `status=skipped`, `had_errors=true`, and stable error code(s).
|
|
|
|
## Test Plan (Pest)
|
|
|
|
Minimum required coverage aligned to Spec test cases:
|
|
|
|
- Upsert identity prevents duplicates; `last_seen_*` updates.
|
|
- `selection_hash` determinism (array ordering invariant).
|
|
- Missing derived per latest completed run for same `(tenant_id, selection_hash)`.
|
|
- Low-confidence missing when latest run is partial/failed or had_errors.
|
|
- Meta whitelist drops unknown keys.
|
|
- Lock prevents overlapping runs per tenant+selection.
|
|
- No snapshot/backup rows are created/modified by inventory sync.
|
|
- Error reporting uses stable `error_codes` and stores no secrets/tokens.
|
|
|
|
## Out of Scope (Explicit)
|
|
|
|
- Any UI (covered by Spec 041)
|
|
- Any snapshot/backup creation
|
|
- Any restore/promotion/remediation write paths
|