260 lines
15 KiB
Markdown
260 lines
15 KiB
Markdown
# Tasks: Remove Legacy BulkOperationRun & Canonicalize Operations (v1.0)
|
||
|
||
**Input**: Design documents from `/specs/056-remove-legacy-bulkops/`
|
||
**Prerequisites**: plan.md (required), spec.md (required), research.md, data-model.md, contracts/, quickstart.md
|
||
|
||
**Tests**: Required (Pest)
|
||
**Operations**: This feature consolidates queued/bulk work onto canonical `OperationRun` and removes the legacy `BulkOperationRun` system.
|
||
|
||
## Phase 1: Setup (Shared Infrastructure)
|
||
|
||
**Purpose**: Ensure feature docs/paths are ready for implementation and review
|
||
|
||
- [X] T001 Review and update quickstart commands in specs/056-remove-legacy-bulkops/quickstart.md
|
||
- [X] T002 [P] Create discovery report scaffold in specs/056-remove-legacy-bulkops/discovery.md
|
||
- [X] T003 [P] Add a “legacy history decision” section to specs/056-remove-legacy-bulkops/discovery.md
|
||
|
||
---
|
||
|
||
## Phase 2: Foundational (Blocking Prerequisites)
|
||
|
||
**Purpose**: Shared primitives required for all bulk migrations and hardening
|
||
|
||
- [X] T004 Populate the full repo-wide discovery sweep in specs/056-remove-legacy-bulkops/discovery.md (app/, resources/, database/, tests/)
|
||
- [X] T005 [P] Add config keys for per-target scope concurrency (default=1) in config/tenantpilot.php
|
||
- [X] T006 [P] Register new bulk operation types/labels/durations in app/Support/OperationCatalog.php
|
||
- [X] T007 [P] Implement hybrid selection identity hasher in app/Services/Operations/BulkSelectionIdentity.php
|
||
- [X] T008 [P] Implement idempotency fingerprint builder in app/Services/Operations/BulkIdempotencyFingerprint.php
|
||
- [X] T009 [P] Implement per-target scope concurrency limiter (Cache locks) in app/Services/Operations/TargetScopeConcurrencyLimiter.php
|
||
- [X] T010 Extend OperationRun identity inputs to include target scope + selection identity in app/Services/OperationRunService.php
|
||
- [X] T011 Add a bulk enqueue helper that standardizes ensureRun + dispatchOrFail usage in app/Services/OperationRunService.php
|
||
|
||
**Checkpoint**: Shared primitives exist; bulk migrations can proceed consistently
|
||
|
||
---
|
||
|
||
## Phase 3: User Story 1 - Run-backed bulk actions are always observable (Priority: P1) 🎯 MVP
|
||
|
||
**Goal**: All bulk actions are OperationRun-backed and observable end-to-end
|
||
|
||
**Independent Test**: Trigger one migrated bulk action and verify OperationRun is created/reused, queued toast occurs, and Monitoring → Operations shows it.
|
||
|
||
### Tests for User Story 1
|
||
|
||
- [X] T012 [P] [US1] Add bulk enqueue idempotency test in tests/Feature/OpsUx/BulkEnqueueIdempotencyTest.php
|
||
- [X] T013 [P] [US1] Add per-target concurrency default=1 test in tests/Feature/OpsUx/TargetScopeConcurrencyLimiterTest.php
|
||
- [X] T014 [P] [US1] Add hybrid selection identity hashing test in tests/Unit/Operations/BulkSelectionIdentityTest.php
|
||
|
||
### Implementation for User Story 1
|
||
|
||
- [X] T015 [P] [US1] Add OperationRun context shape helpers for bulk runs in app/Support/OpsUx/BulkRunContext.php
|
||
- [X] T016 [P] [US1] Implement orchestrator job skeleton for bulk runs in app/Jobs/Operations/BulkOperationOrchestratorJob.php
|
||
- [X] T017 [P] [US1] Implement worker job skeleton for bulk items in app/Jobs/Operations/BulkOperationWorkerJob.php
|
||
- [X] T018 [US1] Ensure worker jobs update summary_counts via canonical whitelist in app/Services/OperationRunService.php
|
||
|
||
**Decision (applies to all US1 migrations)**: Use the standard orchestrator + item worker pattern.
|
||
- Keep T016/T017 as the canonical implementation.
|
||
- Per-domain bulk logic MUST be implemented as item worker(s) invoked by the orchestrator.
|
||
- Avoid parallel legacy bulk job systems.
|
||
|
||
- [X] T019 [US1] Migrate policy bulk delete start action to OperationRun-backed flow in app/Filament/Resources/PolicyResource.php
|
||
- [X] T020 [US1] Refactor policy bulk delete execution into orchestrator/item worker pattern (replace legacy bulk job semantics) in app/Jobs/BulkPolicyDeleteJob.php
|
||
|
||
- [X] T021 [US1] Migrate backup set bulk delete start action to OperationRun-backed flow in app/Filament/Resources/BackupSetResource.php
|
||
- [X] T022 [US1] Refactor backup set bulk delete execution into orchestrator/item worker pattern (replace legacy bulk job semantics) in app/Jobs/BulkBackupSetDeleteJob.php
|
||
|
||
- [X] T023 [US1] Migrate policy version prune start action to OperationRun-backed flow in app/Filament/Resources/PolicyVersionResource.php
|
||
- [X] T024 [US1] Refactor policy version prune execution into orchestrator/item worker pattern (replace legacy bulk job semantics) in app/Jobs/BulkPolicyVersionPruneJob.php
|
||
|
||
- [X] T025 [US1] Migrate policy version force delete start action to OperationRun-backed flow in app/Filament/Resources/PolicyVersionResource.php
|
||
- [X] T026 [US1] Refactor policy version force delete execution into orchestrator/item worker pattern (replace legacy bulk job semantics) in app/Jobs/BulkPolicyVersionForceDeleteJob.php
|
||
|
||
- [X] T027 [US1] Migrate restore run bulk delete start action to OperationRun-backed flow in app/Filament/Resources/RestoreRunResource.php
|
||
- [X] T028 [US1] Refactor restore run bulk delete execution into orchestrator/item worker pattern (replace legacy bulk job semantics) in app/Jobs/BulkRestoreRunDeleteJob.php
|
||
|
||
- [X] T029 [US1] Migrate tenant bulk sync start action to OperationRun-backed flow in app/Filament/Resources/TenantResource.php
|
||
- [X] T030 [US1] Refactor tenant bulk sync execution into orchestrator/item worker pattern (replace legacy bulk job semantics) in app/Jobs/BulkTenantSyncJob.php
|
||
|
||
- [X] T031 [US1] Migrate policy snapshot capture action to OperationRun-backed flow in app/Filament/Resources/PolicyResource/Pages/ViewPolicy.php
|
||
- [X] T032 [US1] Refactor snapshot capture execution into orchestrator/item worker pattern (replace legacy bulk job semantics) in app/Jobs/CapturePolicySnapshotJob.php
|
||
|
||
- [X] T033 [US1] Migrate drift generation flow to OperationRun-backed flow in app/Filament/Pages/DriftLanding.php
|
||
- [X] T034 [US1] Remove legacy BulkOperationRun coupling from drift generator job in app/Jobs/GenerateDriftFindingsJob.php
|
||
|
||
- [X] T035 [US1] Remove legacy “fallback link” usage and standardize view-run URLs to canonical OperationRun links in app/Jobs/AddPoliciesToBackupSetJob.php
|
||
|
||
**Checkpoint**: At this point, at least one representative bulk action is fully run-backed and visible in Monitoring
|
||
|
||
---
|
||
|
||
## Phase 4: User Story 2 - Monitoring is the single source of run history (Priority: P2)
|
||
|
||
**Goal**: No legacy “Bulk Operation Runs” surfaces exist; all view-run links route to Monitoring’s canonical run detail
|
||
|
||
**Independent Test**: From any bulk action, “View run” navigates to OperationRun detail and there is no legacy BulkOperationRun resource.
|
||
|
||
### Tests for User Story 2
|
||
|
||
- [X] T036 [P] [US2] Add regression test that notifications do not link to BulkOperationRun resources in tests/Feature/OpsUx/NotificationViewRunLinkTest.php
|
||
|
||
### Implementation for User Story 2
|
||
|
||
- [X] T037 [US2] Replace BulkOperationRun resource links with OperationRun links in app/Notifications/RunStatusChangedNotification.php
|
||
- [X] T038 [US2] Replace BulkOperationRun resource links with OperationRun links in app/Filament/Resources/BackupSetResource.php
|
||
- [X] T039 [US2] Replace BulkOperationRun resource links with OperationRun links in app/Filament/Resources/PolicyVersionResource.php
|
||
- [X] T040 [US2] Replace BulkOperationRun resource links/redirects with OperationRun links in app/Filament/Resources/PolicyResource/Pages/ViewPolicy.php
|
||
|
||
- [X] T041 [US2] Remove legacy resource and pages: app/Filament/Resources/BulkOperationRunResource.php
|
||
- [X] T042 [US2] Remove legacy resource pages: app/Filament/Resources/BulkOperationRunResource/Pages/ListBulkOperationRuns.php
|
||
- [X] T043 [US2] Remove legacy resource pages: app/Filament/Resources/BulkOperationRunResource/Pages/ViewBulkOperationRun.php
|
||
|
||
- [X] T044 [US2] Remove legacy authorization policy in app/Policies/BulkOperationRunPolicy.php
|
||
- [X] T045 [US2] Remove BulkOperationRun gate registration in app/Providers/AppServiceProvider.php
|
||
|
||
**Checkpoint**: No navigation/link surfaces reference legacy bulk runs; Monitoring is the sole run history surface
|
||
|
||
---
|
||
|
||
## Phase 5: User Story 3 - Developers can’t accidentally reintroduce legacy patterns (Priority: P3)
|
||
|
||
**Goal**: Guardrails enforce single-run model and prevent UX drift / legacy reintroduction
|
||
|
||
**Independent Test**: Introduce a forbidden legacy reference or bulk start surface without OperationRun and confirm automated tests fail.
|
||
|
||
### Tests for User Story 3
|
||
|
||
- [X] T046 [P] [US3] Add “no legacy references” test guard in tests/Feature/Guards/NoLegacyBulkOperationsTest.php
|
||
- [X] T047 [P] [US3] Add tenant isolation guard for bulk enqueue inputs in tests/Feature/OpsUx/BulkTenantIsolationTest.php
|
||
- [X] T048 [P] [US3] Extend summary_counts whitelist coverage for bulk updates in tests/Feature/OpsUx/SummaryCountsWhitelistTest.php
|
||
|
||
### Implementation for User Story 3
|
||
|
||
- [X] T049 [US3] Remove legacy BulkOperationRun unit tests in tests/Unit/BulkOperationRunStatusBucketTest.php
|
||
- [X] T050 [US3] Remove legacy BulkOperationRun unit tests in tests/Unit/BulkOperationRunProgressTest.php
|
||
- [X] T051 [US3] Remove legacy factory and update any dependent tests in database/factories/BulkOperationRunFactory.php
|
||
- [X] T052 [US3] Remove legacy test seeder and update any dependent docs/tests in database/seeders/BulkOperationsTestSeeder.php
|
||
|
||
**Checkpoint**: Guardrails prevent reintroduction; test suite enforces taxonomy and canonical run surfaces
|
||
|
||
---
|
||
|
||
## Phase 6: Polish & Cross-Cutting Concerns
|
||
|
||
**Purpose**: Final removal of legacy DB artifacts and cleanup
|
||
|
||
- [X] T053 Create migration to drop legacy bulk_operation_runs table in database/migrations/2026_01_18_000001_drop_bulk_operation_runs_table.php
|
||
- [X] T054 Do NOT delete historical migrations; add forward drop migrations only
|
||
- Keep old migrations to support fresh installs and CI rebuilds
|
||
- Add a new forward migration: drop legacy bulk tables after cutover
|
||
- Document the cutover precondition (no new legacy writes)
|
||
|
||
- [ ] T055 (optional) If schema history cleanup is required, use a documented squash/snapshot process
|
||
- Only via explicit procedure (not ad-hoc deletes)
|
||
- Must keep a reproducible schema for new environments
|
||
- [X] T056 Validate feature via targeted test run list and update notes in specs/056-remove-legacy-bulkops/quickstart.md
|
||
|
||
---
|
||
|
||
## Monitoring DB-only render guard (NFR-01)
|
||
|
||
- [X] T061 Add a regression test ensuring Monitoring → Operations pages do not invoke Graph/remote calls during render
|
||
- Approach:
|
||
- Mock/spy Graph client (or equivalent remote client)
|
||
- Render Operations index and OperationRun detail pages
|
||
- Assert no remote calls were made
|
||
- DoD: test fails if any Graph client is called from Monitoring render paths
|
||
|
||
## Legacy removal (FR-006)
|
||
|
||
- [X] T062 Remove BulkOperationRun model + related database artifacts (after cutover)
|
||
- Delete app/Models/BulkOperationRun.php (and any related models)
|
||
- Ensure no runtime references remain
|
||
|
||
- [X] T063 Remove BulkOperationService and migrate all call sites to OperationRunService patterns
|
||
- Replace all uses of BulkOperationService::createRun(...) / dispatch flows
|
||
- Ensure all bulk actions create OperationRun and dispatch orchestrator/worker jobs
|
||
|
||
- [X] T064 Add CI guard to prevent reintroduction of BulkOperationRun/BulkOperationService
|
||
- Grep/arch test: fail if repo contains BulkOperationRun or BulkOperationService
|
||
|
||
## Target scope display (FR-008)
|
||
|
||
- [X] T065 Update OperationRun run detail view to display target scope consistently
|
||
- Show entra_tenant_name if present, else show entra_tenant_id
|
||
- If directory_context_id exists, optionally show it as secondary info
|
||
- Ensure this is visible in Monitoring → Operations → Run Detail
|
||
- DoD: reviewers can start a run for a specific Entra tenant and see the target clearly on Run Detail
|
||
|
||
## Failure semantics hardening (NFR-02)
|
||
|
||
- [X] T066 Define/standardize reason codes for migrated bulk operations and enforce message sanitization bounds
|
||
- Baseline reason_code set: graph_throttled, graph_timeout, permission_denied, validation_error, conflict_detected, unknown_error
|
||
- Ensure reason_code is stable and machine-readable
|
||
- Ensure failure message is sanitized + bounded (no secrets/tokens/PII/raw payload dumps)
|
||
- DoD: for each new/migrated bulk operation type, expected reason_code usage is clear and consistent
|
||
|
||
- [X] T067 Add a regression test asserting failures/notifications never persist secrets/PII
|
||
- Approach: create a run failure with sensitive-looking strings and assert persisted failures/notifications are sanitized
|
||
- DoD: test fails if sensitive patterns appear in stored failures/notifications
|
||
|
||
## Remote retry/backoff/jitter policy (NFR-03)
|
||
|
||
- [X] T068 Ensure migrated remote calls use the shared retry/backoff policy (429/503) and forbid ad-hoc retry loops
|
||
- Use bounded retries + exponential backoff with jitter
|
||
- DoD: no hand-rolled sleep/random retry logic in bulk workers; one test or assertion proves shared policy is used
|
||
|
||
## Canonical “View run” sweep and guard (FR-005)
|
||
|
||
- [X] T069 Perform a repo-wide sweep to ensure all “View run” links route to Monitoring → Operations → Run Detail
|
||
- Grep (or ripgrep if available) for legacy routes/resource URLs and legacy BulkOperationRun links
|
||
- Ensure links go through a canonical OperationRun URL helper (or equivalent single source)
|
||
- Optional: add a CI grep/guard forbidding known legacy route names/URLs
|
||
- DoD: documented check/list shows no legacy “View run” links remain
|
||
|
||
---
|
||
|
||
## Adapter run reconciliation (NFR-04)
|
||
|
||
- [X] T070 Add DB-only adapter run reconciler in app/Services/AdapterRunReconciler.php
|
||
- [X] T071 Implement ops:reconcile-adapter-runs command in app/Console/Commands/OpsReconcileAdapterRuns.php
|
||
- [X] T072 Implement scheduled reconciliation job + scheduler wiring in app/Jobs/ReconcileAdapterRunsJob.php and routes/console.php
|
||
- [X] T073 Add AdapterRunReconciler tests in tests/Feature/OpsUx/AdapterRunReconcilerTest.php
|
||
|
||
---
|
||
|
||
## Dependencies & Execution Order
|
||
|
||
### User Story Dependencies
|
||
|
||
- **US1 (P1)**: Foundation for cutover; must complete before deleting legacy UI/DB.
|
||
- **US2 (P2)**: Depends on US1 (cutover) so links can be fully canonicalized.
|
||
- **US3 (P3)**: Can start after Foundational and run in parallel with US2, but should be finalized after US1 cutover.
|
||
|
||
### Parallel Opportunities
|
||
|
||
- Tasks marked **[P]** are safe to do in parallel (new files or isolated edits).
|
||
- Within US1, jobs and Filament start-surface migrations can be split by resource (Policies vs BackupSets vs PolicyVersions vs RestoreRuns).
|
||
|
||
---
|
||
|
||
## Parallel Example: User Story 1
|
||
|
||
- Task: T012 [US1] tests/Feature/OpsUx/BulkEnqueueIdempotencyTest.php
|
||
- Task: T013 [US1] tests/Feature/OpsUx/TargetScopeConcurrencyLimiterTest.php
|
||
- Task: T014 [US1] tests/Unit/Operations/BulkSelectionIdentityTest.php
|
||
|
||
---
|
||
|
||
## Implementation Strategy
|
||
|
||
### MVP First (User Story 1 Only)
|
||
|
||
1. Complete Setup + Foundational
|
||
2. Complete US1 migrations for at least one representative bulk action (end-to-end)
|
||
3. Validate Monitoring visibility + queued toast + terminal notification
|
||
|
||
### Incremental Delivery
|
||
|
||
- Migrate bulk workflows in small, independently testable slices (one resource at a time) while keeping Monitoring canonical.
|
||
- Remove legacy surfaces only after all start surfaces are migrated.
|