TenantAtlas/specs/096-ops-polish-assignment-dedupe-system-tracking/spec.md

# Feature Specification: 096 — Ops Polish Bundle (Assignment job summaries + job dedupe + system job tracking + seeder DX)

**Feature Branch**: `096-ops-polish-assignment-dedupe-system-tracking`  
**Created**: 2026-02-15  
**Status**: Draft  
**Input**: User description: "096 — Ops Polish Bundle (Assignment job summaries + job dedupe + system job tracking + seeder DX)"

## Spec Scope Fields *(mandatory)*

- **Scope**: tenant
- **Primary Routes**: None (background operations only)
- **Data Ownership**: tenant-owned operational records (run tracking + run outcomes) and tenant seed data
- **RBAC**: No new permissions; no changes to user-facing authorization behavior (existing gates/policies remain the source of truth for starting operations)

## Clarifications

### Session 2026-02-15

- Q: What is the dedupe identity rule for assignment jobs? → A: Use `operation_run_id` when available; otherwise dedupe by `tenant_id + job_type + stable input fingerprint` (no secrets).
- Q: If an assignment job retries, how should `OperationRun` summary counters be persisted? → A: On completion (success or terminal failure), write/overwrite the run’s counters so they reflect the final attempt.
- Q: What deduplication duration should we enforce for assignment jobs? → A: 15 minutes.
- Q: For seeded tenants, what format should `tenants.external_id` use? → A: UUID string (v4).
- Q: What `OperationRun.type` should we use for `ReconcileAdapterRunsJob` tracking? → A: `ops.reconcile_adapter_runs`.

## User Scenarios & Testing *(mandatory)*

### User Story 1 - Assignment runs show durable summaries (Priority: P2)

As an operator, I want assignment-related background runs to persist consistent summary counters so that run outcomes can be audited reliably (especially under retries).

**Why this priority**: These jobs already run in production; missing summaries reduce observability and make incident triage slower.

**Independent Test**: Dispatch one assignment job run, then verify the recorded run summary includes total/processed/failed and remains correct after retry simulation.

**Acceptance Scenarios**:

1. **Given** an assignment fetch run completes, **When** its run record is inspected, **Then** it includes non-null summary counters for total, processed, and failed.
2. **Given** an assignment restore run completes, **When** its run record is inspected, **Then** it includes non-null summary counters for total, processed, and failed.
3. **Given** an assignment run retries due to transient failure, **When** it ultimately completes, **Then** its summary counters do not double-count work across attempts.

---

### User Story 2 - Duplicate dispatches do not overlap (Priority: P2)

As an operator, I want assignment jobs to be deduplicated by identity so accidental double dispatch (or queue redelivery) does not cause concurrent overlapping work.

**Why this priority**: Duplicate concurrency increases the risk of conflicting writes, rate limiting, and confusing operational outcomes.

**Independent Test**: Attempt to dispatch the same job identity twice and assert only one execution proceeds while the other is deduped/skipped.

**Acceptance Scenarios**:

1. **Given** an assignment job with a stable identity is dispatched twice in a short window, **When** workers attempt to execute both, **Then** only one execution proceeds (no concurrent overlap) for that identity.
2. **Given** an assignment job with a stable identity completed recently, **When** the same identity is dispatched again within the deduplication duration, **Then** the new execution is skipped/deduped.
3. **Given** the dedupe duration has elapsed since the last terminal completion for an identity, **When** the same identity is legitimately run again, **Then** the new execution is allowed to proceed.

---

### User Story 3 - Housekeeping runs are tracked like everything else (Priority: P3)

As an operator, I want housekeeping/system jobs to produce the same run tracking as other operations so that the operations ledger is complete.

**Why this priority**: This is operational completeness. It is not ship-blocking but improves consistency and reduces blind spots.

**Independent Test**: Execute a housekeeping run and verify a run record is created/updated with success/failure outcome details.

**Acceptance Scenarios**:

1. **Given** the reconcile-adapter-runs job executes successfully, **When** the operations ledger is inspected, **Then** a run record exists with a success outcome.
2. **Given** the reconcile-adapter-runs job fails, **When** the operations ledger is inspected, **Then** the run record includes a stable reason code and a sanitized error message suitable for operators.

---

### User Story 4 - Fresh seed flows work without manual intervention (Priority: P3)

As a developer, I want local and CI seed flows to run successfully so that onboarding and test environments are reproducible.

**Why this priority**: Broken seeding slows development and increases setup variability.

**Independent Test**: Run a clean database reset with seeding and confirm it completes without constraint errors.

**Acceptance Scenarios**:

1. **Given** a clean database, **When** a full reset-and-seed workflow is executed, **Then** it succeeds without requiring manual edits.
2. **Given** seeded tenants exist, **When** their records are validated, **Then** required identifiers are populated, and `external_id` is a UUID v4 string.

### Edge Cases

- Retries: If a job attempt fails and retries, summary counters must remain consistent (no double counting).
- Partial failure: If some items fail, the run must record failures without obscuring successful processing.
- Deduplication window: Dedupe should block concurrent overlap but must not prevent legitimate future runs after the window.
- Error reporting: Failure messages must be sanitized and stable enough to support searching and alerting.

## Requirements *(mandatory)*

**Constitution alignment (required):** This feature touches queued/background work and run observability. It must:
- maintain tenant isolation (no cross-tenant leakage in run tracking),
- ensure run observability is complete for the jobs in scope,
- and add automated tests for the changed operational behavior.

**Constitution alignment (RBAC-UX):** No new UI surfaces are added and no authorization behavior is changed. Any existing authorization checks for starting operations remain server-side and unchanged.

**Constitution alignment (Filament Action Surfaces):** Not applicable (no Filament Resources/Pages/RelationManagers are added or modified).

### Functional Requirements

- **FR-001**: The system MUST persist summary counters (total, processed, failed) for assignment fetch runs upon completion.
- **FR-002**: The system MUST persist summary counters (total, processed, failed) for assignment restore runs upon completion.
- **FR-003**: Summary counters MUST be idempotent-friendly: on completion (success or terminal failure), the run summary counters MUST reflect the final attempt and retries MUST NOT double-count totals for the same run identity.
- **FR-004**: Assignment jobs MUST enforce server-side deduplication by a stable, non-secret job identity to prevent concurrent overlap.
- **FR-004a**: The job identity MUST be derived as: `operation_run_id` when available; otherwise `tenant_id + job_type + stable input fingerprint`.
- **FR-004b**: The stable input fingerprint MUST be deterministic and MUST NOT include secrets.
- **FR-005**: The deduplication duration MUST cover expected worst-case runtime while allowing legitimate future executions after it expires.
- **FR-005a**: The deduplication duration MUST be 15 minutes.
- **FR-006**: Deduplication MUST be enforced at execution time (not solely by UI gating or caller-side checks).
- **FR-007**: The reconcile-adapter-runs housekeeping job MUST create/update an operational run record each time it executes.
- **FR-007a**: The reconcile-adapter-runs housekeeping job MUST use `OperationRun.type = ops.reconcile_adapter_runs`.
- **FR-008**: On failure, the housekeeping run record MUST include a stable reason code and a sanitized operator-facing error message.
- **FR-009**: The seed workflow MUST populate required tenant identifiers so database constraints are satisfied.
- **FR-009a**: Seeded tenants MUST have `external_id` populated as a UUID string (v4).
- **FR-010**: A clean reset-and-seed workflow MUST succeed without manual intervention.

### Assumptions & Dependencies

- The system already has a durable operations ledger concept (run tracking) that can store outcomes and summary counters.
- Assignment fetch/restore jobs already produce item-level totals (or can derive them) without changing business behavior.
- Dedupe is evaluated based on a stable identity that is safe to store and log (no secrets).
- No new UI, routes, or end-user workflows are introduced by this work.

### Key Entities *(include if feature involves data)*

- **Operation Run**: A durable record of a background operation execution, its outcome, and its summary counters.
- **Job Identity**: A stable identifier used to deduplicate concurrent executions of the same logical work.
- **Tenant**: The scope boundary for operational records and seeded data.

## Success Criteria *(mandatory)*

### Measurable Outcomes

- **SC-001**: 100% of completed assignment fetch and restore runs show persisted summary counters (total/processed/failed) in the operations ledger.
- **SC-002**: Duplicate dispatch attempts for the same assignment job identity result in at most one concurrent execution within the deduplication window.
- **SC-003**: 100% of reconcile-adapter-runs executions produce an operational run record with a success or failure outcome.
- **SC-004**: A clean reset-and-seed workflow completes successfully in CI and locally without database constraint failures.