TenantAtlas/specs/005-bulk-operations/plan.md

Implementation Plan: Feature 005 - Bulk Operations

Branch: feat/005-bulk-operations | Date: 2025-12-22 | Spec: spec.md

Summary

Enable efficient bulk operations (delete, export, prune) across TenantPilot's main resources (Policies, Policy Versions, Backup Sets, Restore Runs) with safety gates, progress tracking, and comprehensive audit logging. Technical approach: Filament bulk actions + Laravel Queue jobs with chunked processing + BulkOperationRun tracking model + Livewire polling for progress updates.

Technical Context

Language/Version: PHP 8.4.15
Framework: Laravel 12
Primary Dependencies:

• Filament v4 (admin panel + bulk actions)
• Livewire v3 (reactive UI + polling)
• Laravel Queue (async job processing)
• PostgreSQL (JSONB for tracking)

Storage: PostgreSQL with JSONB fields for:

• bulk_operation_runs.item_ids (array of resource IDs)
• bulk_operation_runs.failures (per-item error details)
• Existing audit logs (metadata column)

Testing: Pest v4 (unit, feature, browser tests)
Target Platform: Web (Dokploy deployment)
Project Type: Web application (Filament admin panel)

Performance Goals:

• Process 100 items in <2 minutes (queued)
• Handle up to 500 items per operation without timeout
• Progress notifications update every 5-10 seconds

Constraints:

• Queue jobs MUST process in chunks of 10-20 items (memory efficiency)
• Progress tracking requires explicit polling (not automatic in Filament)
• Type-to-confirm required for ≥20 destructive items
• Tenant isolation enforced at job level

Scale/Scope:

• 4 primary resources (Policies, PolicyVersions, BackupSets, RestoreRuns)
• 8-12 bulk actions (P1/P2 priority)
• Estimated 26-34 hours implementation (3 phases for P1/P2)

Constitution Check

GATE: Must pass before Phase 0 research. Re-check after Phase 1 design.

Note: Project constitution is template-only (not populated). Using Laravel/TenantPilot conventions instead.

Architecture Principles

✅ Library-First: N/A (feature extends existing app, no new libraries)
✅ Test-First: TDD enforced - Pest tests required before implementation
✅ Simplicity: Uses existing patterns (Jobs, Filament bulk actions, Livewire polling)
✅ Sail-First: Local development uses Laravel Sail (Docker)
✅ Dokploy Deployment: Production/staging via Dokploy (VPS containers)

Laravel Conventions

✅ PSR-12: Code formatting enforced via Laravel Pint
✅ Eloquent-First: No raw DB queries, use Model::query() patterns
✅ Permission Gates: Leverage existing RBAC (Feature 001)
✅ Queue Jobs: Use ShouldQueue interface, chunked processing
✅ Audit Logging: Extend existing AuditLog model/service

Safety Requirements

✅ Tenant Isolation: Job constructor accepts explicit tenantId
✅ Audit Trail: One audit log entry per bulk operation + per-item outcomes
✅ Confirmation: Type-to-confirm for ≥20 destructive items
✅ Fail-Soft: Continue processing on individual failures, abort if >50% fail
✅ Immutability: Policy Versions check eligibility before prune (referenced, current, age)

Gates

🔒 GATE-01: Bulk operations MUST use existing permission model (policies.delete, etc.)
🔒 GATE-02: Progress tracking MUST use BulkOperationRun model (not fire-and-forget)
🔒 GATE-03: Type-to-confirm MUST be case-sensitive "DELETE" for ≥20 items
🔒 GATE-04: Policies bulk delete = local only (ignored_at flag, NO Graph DELETE)

Project Structure

Documentation (this feature)

specs/005-bulk-operations/
├── plan.md              # This file
├── research.md          # Phase 0 output (see below)
├── data-model.md        # Phase 1 output (see below)
├── quickstart.md        # Phase 1 output (see below)
└── tasks.md             # Phase 2 output (/speckit.tasks command - NOT YET CREATED)

Source Code (repository root)

app/
├── Models/
│   ├── BulkOperationRun.php         # NEW: Tracks progress/outcomes
│   ├── Policy.php                   # EXTEND: Add markIgnored() scope
│   ├── PolicyVersion.php            # EXTEND: Add pruneEligible() scope
│   ├── BackupSet.php                # EXTEND: Cascade delete logic
│   └── RestoreRun.php               # EXTEND: Skip running status
│
├── Jobs/
│   ├── BulkPolicyDeleteJob.php      # NEW: Async bulk delete (local)
│   ├── BulkPolicyExportJob.php      # NEW: Export to backup set
│   ├── BulkPolicyVersionPruneJob.php # NEW: Prune old versions
│   ├── BulkBackupSetDeleteJob.php   # NEW: Delete backup sets
│   └── BulkRestoreRunDeleteJob.php  # NEW: Delete restore runs
│
├── Services/
│   ├── BulkOperationService.php     # NEW: Orchestrates bulk ops + tracking
│   └── Audit/
│       └── AuditLogger.php          # EXTEND: Add bulk operation events
│
├── Filament/
│   └── Resources/
│       ├── PolicyResource.php       # EXTEND: Add bulk actions
│       ├── PolicyVersionResource.php # EXTEND: Add bulk prune
│       ├── BackupSetResource.php    # EXTEND: Add bulk delete
│       └── RestoreRunResource.php   # EXTEND: Add bulk delete
│
└── Livewire/
    └── BulkOperationProgress.php    # NEW: Progress polling component

database/
└── migrations/
    └── YYYY_MM_DD_create_bulk_operation_runs_table.php # NEW

tests/
├── Unit/
│   ├── BulkPolicyDeleteJobTest.php
│   ├── BulkActionPermissionTest.php
│   └── BulkEligibilityCheckTest.php
│
└── Feature/
    ├── BulkDeletePoliciesTest.php
    ├── BulkExportToBackupTest.php
    ├── BulkProgressNotificationTest.php
    └── BulkTypeToConfirmTest.php

Structure Decision: Single web application structure (Laravel + Filament). New bulk operations extend existing Resources with BulkAction definitions. New BulkOperationRun model tracks async job progress. No separate API layer needed (Livewire polling uses Filament infolists/resource pages).

Complexity Tracking

No constitution violations requiring justification.

---

Phase 0: Research & Technology Decisions

See research.md for detailed research findings.

Key Decisions Summary

Decision	Chosen	Rationale
Progress tracking	BulkOperationRun model + Livewire polling	Explicit state, survives page refresh, queryable outcomes
Job chunking	collect()->chunk(10)	Simple, memory-efficient, easy to test
Type-to-confirm	Filament form + validation rule	Built-in UI, reusable pattern
Tenant isolation	Explicit tenantId param	Fail-safe, auditable, no reliance on global scopes
Policy deletion	ignored_at flag	Prevents re-sync, restorable, doesn't touch Intune
Eligibility checks	Eloquent scopes	Reusable, testable, composable

---

Phase 1: Data Model & Contracts

See data-model.md for detailed schemas and entity diagrams.

Core Entities

BulkOperationRun (NEW):

• Tracks progress, outcomes, failures for bulk operations
• Fields: resource, action, status, total_items, processed_items, succeeded, failed, skipped
• JSONB: item_ids, failures
• Relationships: tenant, user, auditLog

Policy (EXTEND):

• Add ignored_at timestamp (prevents re-sync)
• Add markIgnored() method and notIgnored() scope

PolicyVersion (EXTEND):

• Add pruneEligible() scope (checks age, references, current status)

RestoreRun (EXTEND):

• Add deletable() scope (filters by completed/failed status)

---

Phase 2: Implementation Tasks

Detailed tasks will be generated via /speckit.tasks command. High-level phases:

Phase 2.1: Foundation (P1 - Policies) - 8-12 hours

• BulkOperationRun migration + model
• Policies: ignored_at column, bulk delete/export jobs
• Filament bulk actions + type-to-confirm
• BulkOperationService orchestration
• Tests (unit, feature)

Phase 2.2: Progress Tracking (P1) - 8-10 hours

• Livewire progress component
• Job progress updates (chunked)
• Circuit breaker (>50% fail abort)
• Audit logging integration
• Tests (progress, polling, audit)

Phase 2.3: Additional Resources (P2) - 6-8 hours

• PolicyVersion prune (eligibility scope)
• BackupSet bulk delete
• RestoreRun bulk delete
• Resource extensions
• Tests for each resource

Phase 2.4: Polish & Deployment - 4-6 hours

• Manual QA (type-to-confirm, progress UI)
• Load testing (500 items)
• Documentation updates
• Staging → Production deployment

---

Risk Mitigation

Risk	Mitigation
Queue timeouts	Chunk processing (10-20 items), timeout config (300s), circuit breaker
Progress polling overhead	Limit interval (5s), index queries, cache recent runs
Accidental deletes	Type-to-confirm ≥20 items, ignored_at flag (restorable), audit trail
Job crashes	Fail-soft, BulkOperationRun status tracking, Laravel retry
Eligibility misses	Conservative JSONB queries, manual review before hard delete
Sync re-adds policies	ignored_at filter in SyncPoliciesJob

---

Success Criteria

• ✅ Bulk delete 100 policies in <2 minutes
• ✅ Type-to-confirm prevents accidents (≥20 items)
• ✅ Progress updates every 5-10s
• ✅ Audit log captures per-item outcomes
• ✅ 95%+ operation success rate
• ✅ All P1/P2 tests pass

---

Next Steps

1. ✅ Generate plan.md (this file)
2. → Generate research.md (detailed technology findings)
3. → Generate data-model.md (schemas + diagrams)
4. → Generate quickstart.md (developer onboarding)
5. → Run /speckit.tasks to create task breakdown
6. → Begin Phase 2.1 implementation

---

Status: Plan Complete - Ready for Research
Created: 2025-12-22
Last Updated: 2025-12-22