TenantAtlas/specs/005-bulk-operations/plan.md

# Implementation Plan: Feature 005 - Bulk Operations

**Branch**: `feat/005-bulk-operations` | **Date**: 2025-12-22 | **Spec**: [spec.md](./spec.md)

## Summary

Enable efficient bulk operations (delete, export, prune) across TenantPilot's main resources (Policies, Policy Versions, Backup Sets, Restore Runs) with safety gates, progress tracking, and comprehensive audit logging. Technical approach: Filament bulk actions + Laravel Queue jobs with chunked processing + BulkOperationRun tracking model + Livewire polling for progress updates.

## Technical Context

**Language/Version**: PHP 8.4.15  
**Framework**: Laravel 12  
**Primary Dependencies**: 
- Filament v4 (admin panel + bulk actions)
- Livewire v3 (reactive UI + polling)
- Laravel Queue (async job processing)
- PostgreSQL (JSONB for tracking)

**Storage**: PostgreSQL with JSONB fields for:
- `bulk_operation_runs.item_ids` (array of resource IDs)
- `bulk_operation_runs.failures` (per-item error details)
- Existing audit logs (metadata column)

**Testing**: Pest v4 (unit, feature, browser tests)  
**Target Platform**: Web (Dokploy deployment)  
**Project Type**: Web application (Filament admin panel)

**Performance Goals**: 
- Process 100 items in <2 minutes (queued)
- Handle up to 500 items per operation without timeout
- Progress notifications update every 5-10 seconds

**Constraints**: 
- Queue jobs MUST process in chunks of 10-20 items (memory efficiency)
- Progress tracking requires explicit polling (not automatic in Filament)
- Type-to-confirm required for ≥20 destructive items
- Tenant isolation enforced at job level

**Scale/Scope**: 
- 4 primary resources (Policies, PolicyVersions, BackupSets, RestoreRuns)
- 8-12 bulk actions (P1/P2 priority)
- Estimated 26-34 hours implementation (3 phases for P1/P2)

## Constitution Check

*GATE: Must pass before Phase 0 research. Re-check after Phase 1 design.*

**Note**: Project constitution is template-only (not populated). Using Laravel/TenantPilot conventions instead.

### Architecture Principles

✅ **Library-First**: N/A (feature extends existing app, no new libraries)  
✅ **Test-First**: TDD enforced - Pest tests required before implementation  
✅ **Simplicity**: Uses existing patterns (Jobs, Filament bulk actions, Livewire polling)  
✅ **Sail-First**: Local development uses Laravel Sail (Docker)  
✅ **Dokploy Deployment**: Production/staging via Dokploy (VPS containers)

### Laravel Conventions

✅ **PSR-12**: Code formatting enforced via Laravel Pint  
✅ **Eloquent-First**: No raw DB queries, use Model::query() patterns  
✅ **Permission Gates**: Leverage existing RBAC (Feature 001)  
✅ **Queue Jobs**: Use ShouldQueue interface, chunked processing  
✅ **Audit Logging**: Extend existing AuditLog model/service

### Safety Requirements

✅ **Tenant Isolation**: Job constructor accepts explicit `tenantId`  
✅ **Audit Trail**: One audit log entry per bulk operation + per-item outcomes  
✅ **Confirmation**: Type-to-confirm for ≥20 destructive items  
✅ **Fail-Soft**: Continue processing on individual failures, abort if >50% fail  
✅ **Immutability**: Policy Versions check eligibility before prune (referenced, current, age)

### Gates

🔒 **GATE-01**: Bulk operations MUST use existing permission model (policies.delete, etc.)  
🔒 **GATE-02**: Progress tracking MUST use BulkOperationRun model (not fire-and-forget)  
🔒 **GATE-03**: Type-to-confirm MUST be case-sensitive "DELETE" for ≥20 items  
🔒 **GATE-04**: Policies bulk delete = local only (ignored_at flag, NO Graph DELETE)

## Project Structure

### Documentation (this feature)

```text
specs/005-bulk-operations/
├── plan.md              # This file
├── research.md          # Phase 0 output (see below)
├── data-model.md        # Phase 1 output (see below)
├── quickstart.md        # Phase 1 output (see below)
└── tasks.md             # Phase 2 output (/speckit.tasks command - NOT YET CREATED)
```

### Source Code (repository root)

```text
app/
├── Models/
│   ├── BulkOperationRun.php         # NEW: Tracks progress/outcomes
│   ├── Policy.php                   # EXTEND: Add markIgnored() scope
│   ├── PolicyVersion.php            # EXTEND: Add pruneEligible() scope
│   ├── BackupSet.php                # EXTEND: Cascade delete logic
│   └── RestoreRun.php               # EXTEND: Skip running status
│
├── Jobs/
│   ├── BulkPolicyDeleteJob.php      # NEW: Async bulk delete (local)
│   ├── BulkPolicyExportJob.php      # NEW: Export to backup set
│   ├── BulkPolicyVersionPruneJob.php # NEW: Prune old versions
│   ├── BulkBackupSetDeleteJob.php   # NEW: Delete backup sets
│   └── BulkRestoreRunDeleteJob.php  # NEW: Delete restore runs
│
├── Services/
│   ├── BulkOperationService.php     # NEW: Orchestrates bulk ops + tracking
│   └── Audit/
│       └── AuditLogger.php          # EXTEND: Add bulk operation events
│
├── Filament/
│   └── Resources/
│       ├── PolicyResource.php       # EXTEND: Add bulk actions
│       ├── PolicyVersionResource.php # EXTEND: Add bulk prune
│       ├── BackupSetResource.php    # EXTEND: Add bulk delete
│       └── RestoreRunResource.php   # EXTEND: Add bulk delete
│
└── Livewire/
    └── BulkOperationProgress.php    # NEW: Progress polling component

database/
└── migrations/
    └── YYYY_MM_DD_create_bulk_operation_runs_table.php # NEW

tests/
├── Unit/
│   ├── BulkPolicyDeleteJobTest.php
│   ├── BulkActionPermissionTest.php
│   └── BulkEligibilityCheckTest.php
│
└── Feature/
    ├── BulkDeletePoliciesTest.php
    ├── BulkExportToBackupTest.php
    ├── BulkProgressNotificationTest.php
    └── BulkTypeToConfirmTest.php
```

**Structure Decision**: Single web application structure (Laravel + Filament). New bulk operations extend existing Resources with BulkAction definitions. New BulkOperationRun model tracks async job progress. No separate API layer needed (Livewire polling uses Filament infolists/resource pages).

## Complexity Tracking

> No constitution violations requiring justification.

---

## Phase 0: Research & Technology Decisions

See [research.md](./research.md) for detailed research findings.

### Key Decisions Summary

| Decision | Chosen | Rationale |
|----------|--------|-----------|
| Progress tracking | BulkOperationRun model + Livewire polling | Explicit state, survives page refresh, queryable outcomes |
| Job chunking | collect()->chunk(10) | Simple, memory-efficient, easy to test |
| Type-to-confirm | Filament form + validation rule | Built-in UI, reusable pattern |
| Tenant isolation | Explicit tenantId param | Fail-safe, auditable, no reliance on global scopes |
| Policy deletion | ignored_at flag | Prevents re-sync, restorable, doesn't touch Intune |
| Eligibility checks | Eloquent scopes | Reusable, testable, composable |

---

## Phase 1: Data Model & Contracts

See [data-model.md](./data-model.md) for detailed schemas and entity diagrams.

### Core Entities

**BulkOperationRun** (NEW):
- Tracks progress, outcomes, failures for bulk operations
- Fields: resource, action, status, total_items, processed_items, succeeded, failed, skipped
- JSONB: item_ids, failures
- Relationships: tenant, user, auditLog

**Policy** (EXTEND):
- Add `ignored_at` timestamp (prevents re-sync)
- Add `markIgnored()` method and `notIgnored()` scope

**PolicyVersion** (EXTEND):
- Add `pruneEligible()` scope (checks age, references, current status)

**RestoreRun** (EXTEND):
- Add `deletable()` scope (filters by completed/failed status)

---

## Phase 2: Implementation Tasks

Detailed tasks will be generated via `/speckit.tasks` command. High-level phases:

### Phase 2.1: Foundation (P1 - Policies) - 8-12 hours
- BulkOperationRun migration + model
- Policies: ignored_at column, bulk delete/export jobs
- Filament bulk actions + type-to-confirm
- BulkOperationService orchestration
- Tests (unit, feature)

### Phase 2.2: Progress Tracking (P1) - 8-10 hours
- Livewire progress component
- Job progress updates (chunked)
- Circuit breaker (>50% fail abort)
- Audit logging integration
- Tests (progress, polling, audit)

### Phase 2.3: Additional Resources (P2) - 6-8 hours
- PolicyVersion prune (eligibility scope)
- BackupSet bulk delete
- RestoreRun bulk delete
- Resource extensions
- Tests for each resource

### Phase 2.4: Polish & Deployment - 4-6 hours
- Manual QA (type-to-confirm, progress UI)
- Load testing (500 items)
- Documentation updates
- Staging → Production deployment

---

## Risk Mitigation

| Risk | Mitigation |
|------|------------|
| Queue timeouts | Chunk processing (10-20 items), timeout config (300s), circuit breaker |
| Progress polling overhead | Limit interval (5s), index queries, cache recent runs |
| Accidental deletes | Type-to-confirm ≥20 items, `ignored_at` flag (restorable), audit trail |
| Job crashes | Fail-soft, BulkOperationRun status tracking, Laravel retry |
| Eligibility misses | Conservative JSONB queries, manual review before hard delete |
| Sync re-adds policies | `ignored_at` filter in SyncPoliciesJob |

---

## Success Criteria

- ✅ Bulk delete 100 policies in <2 minutes
- ✅ Type-to-confirm prevents accidents (≥20 items)
- ✅ Progress updates every 5-10s
- ✅ Audit log captures per-item outcomes
- ✅ 95%+ operation success rate
- ✅ All P1/P2 tests pass

---

## Next Steps

1. ✅ Generate plan.md (this file)
2. → Generate research.md (detailed technology findings)
3. → Generate data-model.md (schemas + diagrams)
4. → Generate quickstart.md (developer onboarding)
5. → Run `/speckit.tasks` to create task breakdown
6. → Begin Phase 2.1 implementation

---

**Status**: Plan Complete - Ready for Research  
**Created**: 2025-12-22  
**Last Updated**: 2025-12-22