feat/004-assignments-scope-tags #4
29
.github/agents/copilot-instructions.md
vendored
Normal file
29
.github/agents/copilot-instructions.md
vendored
Normal file
@ -0,0 +1,29 @@
|
||||
# TenantAtlas Development Guidelines
|
||||
|
||||
Auto-generated from all feature plans. Last updated: 2025-12-22
|
||||
|
||||
## Active Technologies
|
||||
|
||||
- PHP 8.4.15 (feat/005-bulk-operations)
|
||||
|
||||
## Project Structure
|
||||
|
||||
```text
|
||||
src/
|
||||
tests/
|
||||
```
|
||||
|
||||
## Commands
|
||||
|
||||
# Add commands for PHP 8.4.15
|
||||
|
||||
## Code Style
|
||||
|
||||
PHP 8.4.15: Follow standard conventions
|
||||
|
||||
## Recent Changes
|
||||
|
||||
- feat/005-bulk-operations: Added PHP 8.4.15
|
||||
|
||||
<!-- MANUAL ADDITIONS START -->
|
||||
<!-- MANUAL ADDITIONS END -->
|
||||
711
specs/005-bulk-operations/data-model.md
Normal file
711
specs/005-bulk-operations/data-model.md
Normal file
@ -0,0 +1,711 @@
|
||||
# Data Model: Feature 005 - Bulk Operations
|
||||
|
||||
**Feature**: Bulk Operations for Resource Management
|
||||
**Date**: 2025-12-22
|
||||
|
||||
---
|
||||
|
||||
## Overview
|
||||
|
||||
This document describes the data model for bulk operations, including new entities, schema changes, relationships, and query patterns.
|
||||
|
||||
---
|
||||
|
||||
## Entity Diagram
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ BulkOperationRun │
|
||||
├─────────────────────────────────────────────────────────────┤
|
||||
│ id: bigint PK │
|
||||
│ tenant_id: bigint FK → Tenant │
|
||||
│ user_id: bigint FK → User │
|
||||
│ resource: string (policies, policy_versions, etc.) │
|
||||
│ action: string (delete, export, prune, etc.) │
|
||||
│ status: enum (pending, running, completed, failed, aborted) │
|
||||
│ total_items: int │
|
||||
│ processed_items: int │
|
||||
│ succeeded: int │
|
||||
│ failed: int │
|
||||
│ skipped: int │
|
||||
│ item_ids: jsonb (array of IDs) │
|
||||
│ failures: jsonb (array of {id, reason}) │
|
||||
│ audit_log_id: bigint FK → AuditLog (nullable) │
|
||||
│ created_at: timestamp │
|
||||
│ updated_at: timestamp │
|
||||
└─────────────────────────────────────────────────────────────┘
|
||||
│ │ │
|
||||
│ │ └─────────────┐
|
||||
│ │ │
|
||||
▼ ▼ ▼
|
||||
┌──────────────┐ ┌──────────────┐ ┌──────────────┐
|
||||
│ Tenant │ │ User │ │ AuditLog │
|
||||
└──────────────┘ └──────────────┘ └──────────────┘
|
||||
|
||||
|
||||
┌─────────────────────────────────────────┐
|
||||
│ Policy (Extended) │
|
||||
├─────────────────────────────────────────┤
|
||||
│ id: bigint PK │
|
||||
│ tenant_id: bigint FK │
|
||||
│ graph_id: string │
|
||||
│ name: string │
|
||||
│ ... (existing columns) │
|
||||
│ deleted_at: timestamp (nullable) │
|
||||
│ ignored_at: timestamp (nullable) ← NEW │
|
||||
└─────────────────────────────────────────┘
|
||||
|
||||
|
||||
┌─────────────────────────────────────────┐
|
||||
│ PolicyVersion (Extended) │
|
||||
├─────────────────────────────────────────┤
|
||||
│ id: bigint PK │
|
||||
│ policy_id: bigint FK → Policy │
|
||||
│ is_current: boolean │
|
||||
│ created_at: timestamp │
|
||||
│ ... (existing columns) │
|
||||
├─────────────────────────────────────────┤
|
||||
│ Scope: pruneEligible() ← NEW │
|
||||
│ WHERE is_current = false │
|
||||
│ AND created_at < NOW() - 90 days │
|
||||
│ AND NOT IN (backup_items) │
|
||||
│ AND NOT IN (restore_runs.metadata) │
|
||||
└─────────────────────────────────────────┘
|
||||
|
||||
|
||||
┌─────────────────────────────────────────┐
|
||||
│ RestoreRun (Extended) │
|
||||
├─────────────────────────────────────────┤
|
||||
│ id: bigint PK │
|
||||
│ status: enum (pending, running, ...) │
|
||||
│ ... (existing columns) │
|
||||
├─────────────────────────────────────────┤
|
||||
│ Scope: deletable() ← NEW │
|
||||
│ WHERE status IN (completed, failed) │
|
||||
└─────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Database Schema
|
||||
|
||||
### New Table: bulk_operation_runs
|
||||
|
||||
```sql
|
||||
CREATE TABLE bulk_operation_runs (
|
||||
id BIGSERIAL PRIMARY KEY,
|
||||
tenant_id BIGINT NOT NULL REFERENCES tenants(id) ON DELETE CASCADE,
|
||||
user_id BIGINT NOT NULL REFERENCES users(id) ON DELETE CASCADE,
|
||||
resource VARCHAR(50) NOT NULL, -- 'policies', 'policy_versions', 'backup_sets', 'restore_runs'
|
||||
action VARCHAR(50) NOT NULL, -- 'delete', 'export', 'prune', 'archive'
|
||||
status VARCHAR(20) NOT NULL, -- 'pending', 'running', 'completed', 'failed', 'aborted'
|
||||
total_items INT NOT NULL,
|
||||
processed_items INT NOT NULL DEFAULT 0,
|
||||
succeeded INT NOT NULL DEFAULT 0,
|
||||
failed INT NOT NULL DEFAULT 0,
|
||||
skipped INT NOT NULL DEFAULT 0,
|
||||
item_ids JSONB NOT NULL, -- [1, 2, 3, ...]
|
||||
failures JSONB, -- [{"id": 1, "reason": "Graph error"}, ...]
|
||||
audit_log_id BIGINT REFERENCES audit_logs(id) ON DELETE SET NULL,
|
||||
created_at TIMESTAMP NOT NULL DEFAULT NOW(),
|
||||
updated_at TIMESTAMP NOT NULL DEFAULT NOW()
|
||||
);
|
||||
|
||||
CREATE INDEX idx_bulk_operation_runs_tenant_resource_status
|
||||
ON bulk_operation_runs(tenant_id, resource, status);
|
||||
|
||||
CREATE INDEX idx_bulk_operation_runs_user_created
|
||||
ON bulk_operation_runs(user_id, created_at DESC);
|
||||
|
||||
CREATE INDEX idx_bulk_operation_runs_status
|
||||
ON bulk_operation_runs(status)
|
||||
WHERE status IN ('pending', 'running');
|
||||
```
|
||||
|
||||
**Indexes Rationale**:
|
||||
- Composite (tenant_id, resource, status): Filter by tenant + resource type + status (UI queries)
|
||||
- (user_id, created_at): User's recent bulk operations
|
||||
- Partial index on status: Only index running/pending (99% of queries check these)
|
||||
|
||||
---
|
||||
|
||||
### Schema Change: policies table
|
||||
|
||||
```sql
|
||||
ALTER TABLE policies
|
||||
ADD COLUMN ignored_at TIMESTAMP NULL;
|
||||
|
||||
CREATE INDEX idx_policies_ignored_at
|
||||
ON policies(ignored_at)
|
||||
WHERE ignored_at IS NOT NULL;
|
||||
```
|
||||
|
||||
**Purpose**: Prevent SyncPoliciesJob from re-importing locally deleted policies.
|
||||
|
||||
**Query Pattern**:
|
||||
```sql
|
||||
-- Sync job filters
|
||||
SELECT * FROM policies WHERE ignored_at IS NULL;
|
||||
|
||||
-- Bulk delete sets
|
||||
UPDATE policies SET ignored_at = NOW() WHERE id IN (...);
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Schema Change: policy_versions table
|
||||
|
||||
**No schema changes needed.** Eligibility scope uses existing columns:
|
||||
- `is_current` (boolean)
|
||||
- `created_at` (timestamp)
|
||||
- Foreign keys checked via relationships
|
||||
|
||||
---
|
||||
|
||||
## Eloquent Models
|
||||
|
||||
### New Model: BulkOperationRun
|
||||
|
||||
```php
|
||||
<?php
|
||||
|
||||
namespace App\Models;
|
||||
|
||||
use Illuminate\Database\Eloquent\Model;
|
||||
use Illuminate\Database\Eloquent\Relations\BelongsTo;
|
||||
|
||||
class BulkOperationRun extends Model
|
||||
{
|
||||
protected $guarded = [];
|
||||
|
||||
protected $casts = [
|
||||
'item_ids' => 'array',
|
||||
'failures' => 'array',
|
||||
'created_at' => 'datetime',
|
||||
'updated_at' => 'datetime',
|
||||
];
|
||||
|
||||
// Relationships
|
||||
|
||||
public function tenant(): BelongsTo
|
||||
{
|
||||
return $this->belongsTo(Tenant::class);
|
||||
}
|
||||
|
||||
public function user(): BelongsTo
|
||||
{
|
||||
return $this->belongsTo(User::class);
|
||||
}
|
||||
|
||||
public function auditLog(): BelongsTo
|
||||
{
|
||||
return $this->belongsTo(AuditLog::class);
|
||||
}
|
||||
|
||||
// Status Helpers
|
||||
|
||||
public function isRunning(): bool
|
||||
{
|
||||
return $this->status === 'running';
|
||||
}
|
||||
|
||||
public function isComplete(): bool
|
||||
{
|
||||
return in_array($this->status, ['completed', 'failed', 'aborted']);
|
||||
}
|
||||
|
||||
public function isPending(): bool
|
||||
{
|
||||
return $this->status === 'pending';
|
||||
}
|
||||
|
||||
// Progress Helpers
|
||||
|
||||
public function progressPercentage(): int
|
||||
{
|
||||
if ($this->total_items === 0) {
|
||||
return 0;
|
||||
}
|
||||
return (int) round(($this->processed_items / $this->total_items) * 100);
|
||||
}
|
||||
|
||||
public function summaryText(): string
|
||||
{
|
||||
return match ($this->status) {
|
||||
'pending' => "Pending: {$this->total_items} items",
|
||||
'running' => "Processing... {$this->processed_items}/{$this->total_items}",
|
||||
'completed' => "Completed: {$this->succeeded} succeeded, {$this->failed} failed, {$this->skipped} skipped",
|
||||
'failed' => "Failed: {$this->failed}/{$this->total_items} errors",
|
||||
'aborted' => "Aborted: Too many failures ({$this->failed}/{$this->total_items})",
|
||||
default => "Unknown status",
|
||||
};
|
||||
}
|
||||
|
||||
public function hasFailures(): bool
|
||||
{
|
||||
return $this->failed > 0;
|
||||
}
|
||||
|
||||
// Scopes
|
||||
|
||||
public function scopeForResource($query, string $resource)
|
||||
{
|
||||
return $query->where('resource', $resource);
|
||||
}
|
||||
|
||||
public function scopeForUser($query, int $userId)
|
||||
{
|
||||
return $query->where('user_id', $userId);
|
||||
}
|
||||
|
||||
public function scopeRecent($query)
|
||||
{
|
||||
return $query->orderBy('created_at', 'desc');
|
||||
}
|
||||
|
||||
public function scopeInProgress($query)
|
||||
{
|
||||
return $query->whereIn('status', ['pending', 'running']);
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Extended Model: Policy
|
||||
|
||||
```php
|
||||
<?php
|
||||
|
||||
namespace App\Models;
|
||||
|
||||
use Illuminate\Database\Eloquent\Model;
|
||||
use Illuminate\Database\Eloquent\SoftDeletes;
|
||||
|
||||
class Policy extends Model
|
||||
{
|
||||
use SoftDeletes;
|
||||
|
||||
// ... existing code ...
|
||||
|
||||
// NEW: Scopes for bulk operations
|
||||
|
||||
public function scopeNotIgnored($query)
|
||||
{
|
||||
return $query->whereNull('ignored_at');
|
||||
}
|
||||
|
||||
public function scopeIgnored($query)
|
||||
{
|
||||
return $query->whereNotNull('ignored_at');
|
||||
}
|
||||
|
||||
// NEW: Methods for bulk operations
|
||||
|
||||
public function markIgnored(): void
|
||||
{
|
||||
$this->update(['ignored_at' => now()]);
|
||||
}
|
||||
|
||||
public function unmarkIgnored(): void
|
||||
{
|
||||
$this->update(['ignored_at' => null]);
|
||||
}
|
||||
|
||||
public function isIgnored(): bool
|
||||
{
|
||||
return $this->ignored_at !== null;
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Extended Model: PolicyVersion
|
||||
|
||||
```php
|
||||
<?php
|
||||
|
||||
namespace App\Models;
|
||||
|
||||
use Illuminate\Database\Eloquent\Model;
|
||||
use Illuminate\Support\Facades\DB;
|
||||
|
||||
class PolicyVersion extends Model
|
||||
{
|
||||
// ... existing code ...
|
||||
|
||||
// Relationships (if not already defined)
|
||||
|
||||
public function backupItems()
|
||||
{
|
||||
return $this->hasMany(BackupItem::class, 'policy_version_id');
|
||||
}
|
||||
|
||||
// NEW: Scope for eligibility check
|
||||
|
||||
public function scopePruneEligible($query, int $retentionDays = 90)
|
||||
{
|
||||
return $query
|
||||
->where('is_current', false)
|
||||
->where('created_at', '<', now()->subDays($retentionDays))
|
||||
->whereDoesntHave('backupItems')
|
||||
->whereNotIn('id', function ($subquery) {
|
||||
$subquery->select(DB::raw("CAST(metadata->>'policy_version_id' AS INTEGER)"))
|
||||
->from('restore_runs')
|
||||
->whereNotNull(DB::raw("metadata->>'policy_version_id'"));
|
||||
});
|
||||
}
|
||||
|
||||
// NEW: Check if version is eligible for pruning
|
||||
|
||||
public function isPruneEligible(int $retentionDays = 90): bool
|
||||
{
|
||||
if ($this->is_current) {
|
||||
return false;
|
||||
}
|
||||
|
||||
if ($this->created_at->diffInDays(now()) < $retentionDays) {
|
||||
return false;
|
||||
}
|
||||
|
||||
if ($this->backupItems()->exists()) {
|
||||
return false;
|
||||
}
|
||||
|
||||
$referencedInRestoreRuns = DB::table('restore_runs')
|
||||
->whereNotNull(DB::raw("metadata->>'policy_version_id'"))
|
||||
->where(DB::raw("CAST(metadata->>'policy_version_id' AS INTEGER)"), $this->id)
|
||||
->exists();
|
||||
|
||||
return !$referencedInRestoreRuns;
|
||||
}
|
||||
|
||||
// NEW: Get reason why version is not eligible
|
||||
|
||||
public function getIneligibilityReason(int $retentionDays = 90): ?string
|
||||
{
|
||||
if ($this->is_current) {
|
||||
return 'Current version';
|
||||
}
|
||||
|
||||
if ($this->created_at->diffInDays(now()) < $retentionDays) {
|
||||
return "Too recent (< {$retentionDays} days)";
|
||||
}
|
||||
|
||||
if ($this->backupItems()->exists()) {
|
||||
return 'Referenced in backup items';
|
||||
}
|
||||
|
||||
$referencedInRestoreRuns = DB::table('restore_runs')
|
||||
->whereNotNull(DB::raw("metadata->>'policy_version_id'"))
|
||||
->where(DB::raw("CAST(metadata->>'policy_version_id' AS INTEGER)"), $this->id)
|
||||
->exists();
|
||||
|
||||
if ($referencedInRestoreRuns) {
|
||||
return 'Referenced in restore runs';
|
||||
}
|
||||
|
||||
return null; // Eligible
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Extended Model: RestoreRun
|
||||
|
||||
```php
|
||||
<?php
|
||||
|
||||
namespace App\Models;
|
||||
|
||||
use Illuminate\Database\Eloquent\Model;
|
||||
|
||||
class RestoreRun extends Model
|
||||
{
|
||||
// ... existing code ...
|
||||
|
||||
// NEW: Scope for deletable restore runs
|
||||
|
||||
public function scopeDeletable($query)
|
||||
{
|
||||
return $query->whereIn('status', ['completed', 'failed', 'aborted']);
|
||||
}
|
||||
|
||||
public function scopeNotDeletable($query)
|
||||
{
|
||||
return $query->whereNotIn('status', ['completed', 'failed', 'aborted']);
|
||||
}
|
||||
|
||||
// NEW: Check if restore run can be deleted
|
||||
|
||||
public function isDeletable(): bool
|
||||
{
|
||||
return in_array($this->status, ['completed', 'failed', 'aborted']);
|
||||
}
|
||||
|
||||
public function getNotDeletableReason(): ?string
|
||||
{
|
||||
if ($this->isDeletable()) {
|
||||
return null;
|
||||
}
|
||||
|
||||
return match ($this->status) {
|
||||
'pending' => 'Restore run is pending',
|
||||
'running' => 'Restore run is currently running',
|
||||
default => "Restore run has status: {$this->status}",
|
||||
};
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Query Patterns
|
||||
|
||||
### 1. Create Bulk Operation Run
|
||||
|
||||
```php
|
||||
$run = BulkOperationRun::create([
|
||||
'tenant_id' => $tenantId,
|
||||
'user_id' => $userId,
|
||||
'resource' => 'policies',
|
||||
'action' => 'delete',
|
||||
'status' => 'pending',
|
||||
'total_items' => count($policyIds),
|
||||
'item_ids' => $policyIds,
|
||||
]);
|
||||
|
||||
// Dispatch job
|
||||
BulkPolicyDeleteJob::dispatch($policyIds, $tenantId, $userId, $run->id);
|
||||
|
||||
$run->update(['status' => 'running']);
|
||||
```
|
||||
|
||||
### 2. Update Progress (in job)
|
||||
|
||||
```php
|
||||
$run->update([
|
||||
'processed_items' => $run->processed_items + $chunkSize,
|
||||
'succeeded' => $run->succeeded + $successCount,
|
||||
'failed' => $run->failed + $failCount,
|
||||
'failures' => array_merge($run->failures ?? [], $newFailures),
|
||||
]);
|
||||
```
|
||||
|
||||
### 3. Query Recent Bulk Operations (UI)
|
||||
|
||||
```php
|
||||
// User's recent operations
|
||||
$runs = BulkOperationRun::forUser(auth()->id())
|
||||
->recent()
|
||||
->limit(10)
|
||||
->get();
|
||||
|
||||
// Tenant's in-progress operations
|
||||
$inProgress = BulkOperationRun::where('tenant_id', $tenantId)
|
||||
->inProgress()
|
||||
->get();
|
||||
```
|
||||
|
||||
### 4. Filter Policies (exclude ignored)
|
||||
|
||||
```php
|
||||
// Sync job
|
||||
$policies = Policy::where('tenant_id', $tenantId)
|
||||
->notIgnored()
|
||||
->get();
|
||||
|
||||
// Bulk delete (mark as ignored)
|
||||
Policy::whereIn('id', $policyIds)
|
||||
->update(['ignored_at' => now()]);
|
||||
```
|
||||
|
||||
### 5. Check Policy Version Eligibility
|
||||
|
||||
```php
|
||||
// Get all eligible versions for tenant
|
||||
$eligibleVersions = PolicyVersion::whereHas('policy', function ($query) use ($tenantId) {
|
||||
$query->where('tenant_id', $tenantId);
|
||||
})
|
||||
->pruneEligible(90)
|
||||
->get();
|
||||
|
||||
// Check single version
|
||||
$version = PolicyVersion::find($id);
|
||||
if (!$version->isPruneEligible()) {
|
||||
$reason = $version->getIneligibilityReason();
|
||||
// Skip with reason: "Referenced in backup items"
|
||||
}
|
||||
```
|
||||
|
||||
### 6. Filter Deletable Restore Runs
|
||||
|
||||
```php
|
||||
// Get deletable runs
|
||||
$deletableRuns = RestoreRun::where('tenant_id', $tenantId)
|
||||
->deletable()
|
||||
->get();
|
||||
|
||||
// Check individual run
|
||||
$run = RestoreRun::find($id);
|
||||
if (!$run->isDeletable()) {
|
||||
$reason = $run->getNotDeletableReason();
|
||||
// Skip: "Restore run is currently running"
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## JSONB Structure
|
||||
|
||||
### bulk_operation_runs.item_ids
|
||||
|
||||
```json
|
||||
[1, 2, 3, 4, 5, ...]
|
||||
```
|
||||
|
||||
Simple array of integer IDs.
|
||||
|
||||
### bulk_operation_runs.failures
|
||||
|
||||
```json
|
||||
[
|
||||
{
|
||||
"id": 123,
|
||||
"reason": "Graph API error: 503 Service Unavailable"
|
||||
},
|
||||
{
|
||||
"id": 456,
|
||||
"reason": "Policy not found"
|
||||
},
|
||||
{
|
||||
"id": 789,
|
||||
"reason": "Permission denied"
|
||||
}
|
||||
]
|
||||
```
|
||||
|
||||
Array of objects with `id` (resource ID) and `reason` (error message).
|
||||
|
||||
### restore_runs.metadata (existing)
|
||||
|
||||
```json
|
||||
{
|
||||
"policy_version_id": 42,
|
||||
"backup_set_id": 15,
|
||||
"items_count": 10,
|
||||
...
|
||||
}
|
||||
```
|
||||
|
||||
When checking eligibility, query:
|
||||
```sql
|
||||
SELECT * FROM restore_runs
|
||||
WHERE metadata->>'policy_version_id' = '42';
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Migration Files
|
||||
|
||||
### Migration 1: Create bulk_operation_runs table
|
||||
|
||||
```php
|
||||
<?php
|
||||
|
||||
use Illuminate\Database\Migrations\Migration;
|
||||
use Illuminate\Database\Schema\Blueprint;
|
||||
use Illuminate\Support\Facades\Schema;
|
||||
|
||||
return new class extends Migration
|
||||
{
|
||||
public function up(): void
|
||||
{
|
||||
Schema::create('bulk_operation_runs', function (Blueprint $table) {
|
||||
$table->id();
|
||||
$table->foreignId('tenant_id')->constrained()->cascadeOnDelete();
|
||||
$table->foreignId('user_id')->constrained()->cascadeOnDelete();
|
||||
$table->string('resource', 50);
|
||||
$table->string('action', 50);
|
||||
$table->string('status', 20);
|
||||
$table->integer('total_items');
|
||||
$table->integer('processed_items')->default(0);
|
||||
$table->integer('succeeded')->default(0);
|
||||
$table->integer('failed')->default(0);
|
||||
$table->integer('skipped')->default(0);
|
||||
$table->json('item_ids');
|
||||
$table->json('failures')->nullable();
|
||||
$table->foreignId('audit_log_id')->nullable()->constrained()->nullOnDelete();
|
||||
$table->timestamps();
|
||||
|
||||
$table->index(['tenant_id', 'resource', 'status']);
|
||||
$table->index(['user_id', 'created_at']);
|
||||
$table->index('status')->where('status', 'in', ['pending', 'running']);
|
||||
});
|
||||
}
|
||||
|
||||
public function down(): void
|
||||
{
|
||||
Schema::dropIfExists('bulk_operation_runs');
|
||||
}
|
||||
};
|
||||
```
|
||||
|
||||
### Migration 2: Add ignored_at to policies
|
||||
|
||||
```php
|
||||
<?php
|
||||
|
||||
use Illuminate\Database\Migrations\Migration;
|
||||
use Illuminate\Database\Schema\Blueprint;
|
||||
use Illuminate\Support\Facades\Schema;
|
||||
|
||||
return new class extends Migration
|
||||
{
|
||||
public function up(): void
|
||||
{
|
||||
Schema::table('policies', function (Blueprint $table) {
|
||||
$table->timestamp('ignored_at')->nullable()->after('deleted_at');
|
||||
$table->index('ignored_at');
|
||||
});
|
||||
}
|
||||
|
||||
public function down(): void
|
||||
{
|
||||
Schema::table('policies', function (Blueprint $table) {
|
||||
$table->dropIndex(['ignored_at']);
|
||||
$table->dropColumn('ignored_at');
|
||||
});
|
||||
}
|
||||
};
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Data Retention & Cleanup
|
||||
|
||||
### BulkOperationRun Retention
|
||||
|
||||
Recommended: Keep for 90 days, then archive or delete completed runs.
|
||||
|
||||
```php
|
||||
// Scheduled command
|
||||
Artisan::command('bulk-operations:prune', function () {
|
||||
$deleted = BulkOperationRun::where('created_at', '<', now()->subDays(90))
|
||||
->whereIn('status', ['completed', 'failed', 'aborted'])
|
||||
->delete();
|
||||
|
||||
$this->info("Pruned {$deleted} old bulk operation runs");
|
||||
})->daily();
|
||||
```
|
||||
|
||||
### PolicyVersion Retention
|
||||
|
||||
Handled by bulk prune feature (user-initiated, not automatic).
|
||||
|
||||
---
|
||||
|
||||
**Status**: Data Model Complete
|
||||
**Next Step**: Generate quickstart.md
|
||||
263
specs/005-bulk-operations/plan.md
Normal file
263
specs/005-bulk-operations/plan.md
Normal file
@ -0,0 +1,263 @@
|
||||
# Implementation Plan: Feature 005 - Bulk Operations
|
||||
|
||||
**Branch**: `feat/005-bulk-operations` | **Date**: 2025-12-22 | **Spec**: [spec.md](./spec.md)
|
||||
|
||||
## Summary
|
||||
|
||||
Enable efficient bulk operations (delete, export, prune) across TenantPilot's main resources (Policies, Policy Versions, Backup Sets, Restore Runs) with safety gates, progress tracking, and comprehensive audit logging. Technical approach: Filament bulk actions + Laravel Queue jobs with chunked processing + BulkOperationRun tracking model + Livewire polling for progress updates.
|
||||
|
||||
## Technical Context
|
||||
|
||||
**Language/Version**: PHP 8.4.15
|
||||
**Framework**: Laravel 12
|
||||
**Primary Dependencies**:
|
||||
- Filament v4 (admin panel + bulk actions)
|
||||
- Livewire v3 (reactive UI + polling)
|
||||
- Laravel Queue (async job processing)
|
||||
- PostgreSQL (JSONB for tracking)
|
||||
|
||||
**Storage**: PostgreSQL with JSONB fields for:
|
||||
- `bulk_operation_runs.item_ids` (array of resource IDs)
|
||||
- `bulk_operation_runs.failures` (per-item error details)
|
||||
- Existing audit logs (metadata column)
|
||||
|
||||
**Testing**: Pest v4 (unit, feature, browser tests)
|
||||
**Target Platform**: Web (Dokploy deployment)
|
||||
**Project Type**: Web application (Filament admin panel)
|
||||
|
||||
**Performance Goals**:
|
||||
- Process 100 items in <2 minutes (queued)
|
||||
- Handle up to 500 items per operation without timeout
|
||||
- Progress notifications update every 5-10 seconds
|
||||
|
||||
**Constraints**:
|
||||
- Queue jobs MUST process in chunks of 10-20 items (memory efficiency)
|
||||
- Progress tracking requires explicit polling (not automatic in Filament)
|
||||
- Type-to-confirm required for ≥20 destructive items
|
||||
- Tenant isolation enforced at job level
|
||||
|
||||
**Scale/Scope**:
|
||||
- 4 primary resources (Policies, PolicyVersions, BackupSets, RestoreRuns)
|
||||
- 8-12 bulk actions (P1/P2 priority)
|
||||
- Estimated 26-34 hours implementation (3 phases for P1/P2)
|
||||
|
||||
## Constitution Check
|
||||
|
||||
*GATE: Must pass before Phase 0 research. Re-check after Phase 1 design.*
|
||||
|
||||
**Note**: Project constitution is template-only (not populated). Using Laravel/TenantPilot conventions instead.
|
||||
|
||||
### Architecture Principles
|
||||
|
||||
✅ **Library-First**: N/A (feature extends existing app, no new libraries)
|
||||
✅ **Test-First**: TDD enforced - Pest tests required before implementation
|
||||
✅ **Simplicity**: Uses existing patterns (Jobs, Filament bulk actions, Livewire polling)
|
||||
✅ **Sail-First**: Local development uses Laravel Sail (Docker)
|
||||
✅ **Dokploy Deployment**: Production/staging via Dokploy (VPS containers)
|
||||
|
||||
### Laravel Conventions
|
||||
|
||||
✅ **PSR-12**: Code formatting enforced via Laravel Pint
|
||||
✅ **Eloquent-First**: No raw DB queries, use Model::query() patterns
|
||||
✅ **Permission Gates**: Leverage existing RBAC (Feature 001)
|
||||
✅ **Queue Jobs**: Use ShouldQueue interface, chunked processing
|
||||
✅ **Audit Logging**: Extend existing AuditLog model/service
|
||||
|
||||
### Safety Requirements
|
||||
|
||||
✅ **Tenant Isolation**: Job constructor accepts explicit `tenantId`
|
||||
✅ **Audit Trail**: One audit log entry per bulk operation + per-item outcomes
|
||||
✅ **Confirmation**: Type-to-confirm for ≥20 destructive items
|
||||
✅ **Fail-Soft**: Continue processing on individual failures, abort if >50% fail
|
||||
✅ **Immutability**: Policy Versions check eligibility before prune (referenced, current, age)
|
||||
|
||||
### Gates
|
||||
|
||||
🔒 **GATE-01**: Bulk operations MUST use existing permission model (policies.delete, etc.)
|
||||
🔒 **GATE-02**: Progress tracking MUST use BulkOperationRun model (not fire-and-forget)
|
||||
🔒 **GATE-03**: Type-to-confirm MUST be case-sensitive "DELETE" for ≥20 items
|
||||
🔒 **GATE-04**: Policies bulk delete = local only (ignored_at flag, NO Graph DELETE)
|
||||
|
||||
## Project Structure
|
||||
|
||||
### Documentation (this feature)
|
||||
|
||||
```text
|
||||
specs/005-bulk-operations/
|
||||
├── plan.md # This file
|
||||
├── research.md # Phase 0 output (see below)
|
||||
├── data-model.md # Phase 1 output (see below)
|
||||
├── quickstart.md # Phase 1 output (see below)
|
||||
└── tasks.md # Phase 2 output (/speckit.tasks command - NOT YET CREATED)
|
||||
```
|
||||
|
||||
### Source Code (repository root)
|
||||
|
||||
```text
|
||||
app/
|
||||
├── Models/
|
||||
│ ├── BulkOperationRun.php # NEW: Tracks progress/outcomes
|
||||
│ ├── Policy.php # EXTEND: Add markIgnored() scope
|
||||
│ ├── PolicyVersion.php # EXTEND: Add pruneEligible() scope
|
||||
│ ├── BackupSet.php # EXTEND: Cascade delete logic
|
||||
│ └── RestoreRun.php # EXTEND: Skip running status
|
||||
│
|
||||
├── Jobs/
|
||||
│ ├── BulkPolicyDeleteJob.php # NEW: Async bulk delete (local)
|
||||
│ ├── BulkPolicyExportJob.php # NEW: Export to backup set
|
||||
│ ├── BulkPolicyVersionPruneJob.php # NEW: Prune old versions
|
||||
│ ├── BulkBackupSetDeleteJob.php # NEW: Delete backup sets
|
||||
│ └── BulkRestoreRunDeleteJob.php # NEW: Delete restore runs
|
||||
│
|
||||
├── Services/
|
||||
│ ├── BulkOperationService.php # NEW: Orchestrates bulk ops + tracking
|
||||
│ └── Audit/
|
||||
│ └── AuditLogger.php # EXTEND: Add bulk operation events
|
||||
│
|
||||
├── Filament/
|
||||
│ └── Resources/
|
||||
│ ├── PolicyResource.php # EXTEND: Add bulk actions
|
||||
│ ├── PolicyVersionResource.php # EXTEND: Add bulk prune
|
||||
│ ├── BackupSetResource.php # EXTEND: Add bulk delete
|
||||
│ └── RestoreRunResource.php # EXTEND: Add bulk delete
|
||||
│
|
||||
└── Livewire/
|
||||
└── BulkOperationProgress.php # NEW: Progress polling component
|
||||
|
||||
database/
|
||||
└── migrations/
|
||||
└── YYYY_MM_DD_create_bulk_operation_runs_table.php # NEW
|
||||
|
||||
tests/
|
||||
├── Unit/
|
||||
│ ├── BulkPolicyDeleteJobTest.php
|
||||
│ ├── BulkActionPermissionTest.php
|
||||
│ └── BulkEligibilityCheckTest.php
|
||||
│
|
||||
└── Feature/
|
||||
├── BulkDeletePoliciesTest.php
|
||||
├── BulkExportToBackupTest.php
|
||||
├── BulkProgressNotificationTest.php
|
||||
└── BulkTypeToConfirmTest.php
|
||||
```
|
||||
|
||||
**Structure Decision**: Single web application structure (Laravel + Filament). New bulk operations extend existing Resources with BulkAction definitions. New BulkOperationRun model tracks async job progress. No separate API layer needed (Livewire polling uses Filament infolists/resource pages).
|
||||
|
||||
## Complexity Tracking
|
||||
|
||||
> No constitution violations requiring justification.
|
||||
|
||||
---
|
||||
|
||||
## Phase 0: Research & Technology Decisions
|
||||
|
||||
See [research.md](./research.md) for detailed research findings.
|
||||
|
||||
### Key Decisions Summary
|
||||
|
||||
| Decision | Chosen | Rationale |
|
||||
|----------|--------|-----------|
|
||||
| Progress tracking | BulkOperationRun model + Livewire polling | Explicit state, survives page refresh, queryable outcomes |
|
||||
| Job chunking | collect()->chunk(10) | Simple, memory-efficient, easy to test |
|
||||
| Type-to-confirm | Filament form + validation rule | Built-in UI, reusable pattern |
|
||||
| Tenant isolation | Explicit tenantId param | Fail-safe, auditable, no reliance on global scopes |
|
||||
| Policy deletion | ignored_at flag | Prevents re-sync, restorable, doesn't touch Intune |
|
||||
| Eligibility checks | Eloquent scopes | Reusable, testable, composable |
|
||||
|
||||
---
|
||||
|
||||
## Phase 1: Data Model & Contracts
|
||||
|
||||
See [data-model.md](./data-model.md) for detailed schemas and entity diagrams.
|
||||
|
||||
### Core Entities
|
||||
|
||||
**BulkOperationRun** (NEW):
|
||||
- Tracks progress, outcomes, failures for bulk operations
|
||||
- Fields: resource, action, status, total_items, processed_items, succeeded, failed, skipped
|
||||
- JSONB: item_ids, failures
|
||||
- Relationships: tenant, user, auditLog
|
||||
|
||||
**Policy** (EXTEND):
|
||||
- Add `ignored_at` timestamp (prevents re-sync)
|
||||
- Add `markIgnored()` method and `notIgnored()` scope
|
||||
|
||||
**PolicyVersion** (EXTEND):
|
||||
- Add `pruneEligible()` scope (checks age, references, current status)
|
||||
|
||||
**RestoreRun** (EXTEND):
|
||||
- Add `deletable()` scope (filters by completed/failed status)
|
||||
|
||||
---
|
||||
|
||||
## Phase 2: Implementation Tasks
|
||||
|
||||
Detailed tasks will be generated via `/speckit.tasks` command. High-level phases:
|
||||
|
||||
### Phase 2.1: Foundation (P1 - Policies) - 8-12 hours
|
||||
- BulkOperationRun migration + model
|
||||
- Policies: ignored_at column, bulk delete/export jobs
|
||||
- Filament bulk actions + type-to-confirm
|
||||
- BulkOperationService orchestration
|
||||
- Tests (unit, feature)
|
||||
|
||||
### Phase 2.2: Progress Tracking (P1) - 8-10 hours
|
||||
- Livewire progress component
|
||||
- Job progress updates (chunked)
|
||||
- Circuit breaker (>50% fail abort)
|
||||
- Audit logging integration
|
||||
- Tests (progress, polling, audit)
|
||||
|
||||
### Phase 2.3: Additional Resources (P2) - 6-8 hours
|
||||
- PolicyVersion prune (eligibility scope)
|
||||
- BackupSet bulk delete
|
||||
- RestoreRun bulk delete
|
||||
- Resource extensions
|
||||
- Tests for each resource
|
||||
|
||||
### Phase 2.4: Polish & Deployment - 4-6 hours
|
||||
- Manual QA (type-to-confirm, progress UI)
|
||||
- Load testing (500 items)
|
||||
- Documentation updates
|
||||
- Staging → Production deployment
|
||||
|
||||
---
|
||||
|
||||
## Risk Mitigation
|
||||
|
||||
| Risk | Mitigation |
|
||||
|------|------------|
|
||||
| Queue timeouts | Chunk processing (10-20 items), timeout config (300s), circuit breaker |
|
||||
| Progress polling overhead | Limit interval (5s), index queries, cache recent runs |
|
||||
| Accidental deletes | Type-to-confirm ≥20 items, `ignored_at` flag (restorable), audit trail |
|
||||
| Job crashes | Fail-soft, BulkOperationRun status tracking, Laravel retry |
|
||||
| Eligibility misses | Conservative JSONB queries, manual review before hard delete |
|
||||
| Sync re-adds policies | `ignored_at` filter in SyncPoliciesJob |
|
||||
|
||||
---
|
||||
|
||||
## Success Criteria
|
||||
|
||||
- ✅ Bulk delete 100 policies in <2 minutes
|
||||
- ✅ Type-to-confirm prevents accidents (≥20 items)
|
||||
- ✅ Progress updates every 5-10s
|
||||
- ✅ Audit log captures per-item outcomes
|
||||
- ✅ 95%+ operation success rate
|
||||
- ✅ All P1/P2 tests pass
|
||||
|
||||
---
|
||||
|
||||
## Next Steps
|
||||
|
||||
1. ✅ Generate plan.md (this file)
|
||||
2. → Generate research.md (detailed technology findings)
|
||||
3. → Generate data-model.md (schemas + diagrams)
|
||||
4. → Generate quickstart.md (developer onboarding)
|
||||
5. → Run `/speckit.tasks` to create task breakdown
|
||||
6. → Begin Phase 2.1 implementation
|
||||
|
||||
---
|
||||
|
||||
**Status**: Plan Complete - Ready for Research
|
||||
**Created**: 2025-12-22
|
||||
**Last Updated**: 2025-12-22
|
||||
425
specs/005-bulk-operations/quickstart.md
Normal file
425
specs/005-bulk-operations/quickstart.md
Normal file
@ -0,0 +1,425 @@
|
||||
# Quickstart: Feature 005 - Bulk Operations
|
||||
|
||||
**Feature**: Bulk Operations for Resource Management
|
||||
**Date**: 2025-12-22
|
||||
|
||||
---
|
||||
|
||||
## Overview
|
||||
|
||||
This quickstart guide helps developers get up and running with Feature 005 (Bulk Operations) for local development, testing, and debugging.
|
||||
|
||||
---
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- Laravel Sail installed and running
|
||||
- Composer dependencies installed
|
||||
- NPM dependencies installed
|
||||
- Database migrated
|
||||
- At least one Tenant configured
|
||||
- Sample Policies, PolicyVersions, BackupSets, RestoreRuns seeded
|
||||
|
||||
---
|
||||
|
||||
## Local Development Setup
|
||||
|
||||
### 1. Start Sail
|
||||
|
||||
```bash
|
||||
cd /path/to/TenantAtlas
|
||||
./vendor/bin/sail up -d
|
||||
```
|
||||
|
||||
### 2. Run Migrations
|
||||
|
||||
```bash
|
||||
./vendor/bin/sail artisan migrate
|
||||
```
|
||||
|
||||
This creates:
|
||||
- `bulk_operation_runs` table
|
||||
- `ignored_at` column on `policies` table
|
||||
|
||||
### 3. Seed Test Data (Optional)
|
||||
|
||||
```bash
|
||||
./vendor/bin/sail artisan db:seed --class=BulkOperationsTestSeeder
|
||||
```
|
||||
|
||||
Creates:
|
||||
- 100 test policies
|
||||
- 50 policy versions (some old, some referenced)
|
||||
- 10 backup sets
|
||||
- 20 restore runs (mixed statuses)
|
||||
|
||||
### 4. Start Queue Worker
|
||||
|
||||
Bulk operations require queue processing:
|
||||
|
||||
```bash
|
||||
./vendor/bin/sail artisan queue:work --tries=3 --timeout=300
|
||||
```
|
||||
|
||||
Or run in background with Supervisor (production):
|
||||
|
||||
```bash
|
||||
./vendor/bin/sail artisan queue:restart
|
||||
```
|
||||
|
||||
### 5. Access Filament Panel
|
||||
|
||||
```bash
|
||||
open http://localhost/admin
|
||||
```
|
||||
|
||||
Navigate to:
|
||||
- **Policies** → Select multiple → Bulk Actions dropdown
|
||||
- **Policy Versions** → Bulk Prune
|
||||
- **Backup Sets** → Bulk Delete
|
||||
- **Restore Runs** → Bulk Delete
|
||||
|
||||
---
|
||||
|
||||
## Running Tests
|
||||
|
||||
### Unit Tests
|
||||
|
||||
Test individual components (jobs, scopes, helpers):
|
||||
|
||||
```bash
|
||||
./vendor/bin/sail artisan test tests/Unit/BulkPolicyDeleteJobTest.php
|
||||
./vendor/bin/sail artisan test tests/Unit/BulkActionPermissionTest.php
|
||||
./vendor/bin/sail artisan test tests/Unit/BulkEligibilityCheckTest.php
|
||||
```
|
||||
|
||||
### Feature Tests
|
||||
|
||||
Test E2E flows (UI → job → database):
|
||||
|
||||
```bash
|
||||
./vendor/bin/sail artisan test tests/Feature/BulkDeletePoliciesTest.php
|
||||
./vendor/bin/sail artisan test tests/Feature/BulkExportToBackupTest.php
|
||||
./vendor/bin/sail artisan test tests/Feature/BulkProgressNotificationTest.php
|
||||
./vendor/bin/sail artisan test tests/Feature/BulkTypeToConfirmTest.php
|
||||
```
|
||||
|
||||
### All Tests
|
||||
|
||||
```bash
|
||||
./vendor/bin/sail artisan test --filter=Bulk
|
||||
```
|
||||
|
||||
### Browser Tests (Pest v4)
|
||||
|
||||
Test UI interactions:
|
||||
|
||||
```bash
|
||||
./vendor/bin/sail artisan test tests/Browser/BulkOperationsTest.php
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Manual Testing Workflow
|
||||
|
||||
### Scenario 1: Bulk Delete Policies (< 20 items)
|
||||
|
||||
1. **Navigate**: Admin → Policies
|
||||
2. **Select**: Check 10 policies
|
||||
3. **Action**: Click "Delete" in bulk actions dropdown
|
||||
4. **Confirm**: Modal appears: "Delete 10 policies?"
|
||||
5. **Submit**: Click "Confirm"
|
||||
6. **Verify**:
|
||||
- Success notification: "Deleted 10 policies"
|
||||
- Policies have `ignored_at` timestamp set
|
||||
- Policies still exist in Intune (no Graph DELETE call)
|
||||
- Audit log entry created
|
||||
|
||||
### Scenario 2: Bulk Delete Policies (≥ 20 items, queued)
|
||||
|
||||
1. **Navigate**: Admin → Policies
|
||||
2. **Select**: Check 25 policies
|
||||
3. **Action**: Click "Delete"
|
||||
4. **Confirm**: Modal requires typing "DELETE"
|
||||
5. **Type**: Enter "DELETE" (case-sensitive)
|
||||
6. **Submit**: Click "Confirm"
|
||||
7. **Verify**:
|
||||
- Job dispatched to queue
|
||||
- Progress notification: "Deleting policies... 0/25"
|
||||
- Notification updates every 5s: "Deleting... 10/25", "20/25"
|
||||
- Final notification: "Deleted 25 policies"
|
||||
- `BulkOperationRun` record created with status `completed`
|
||||
- Audit log entry
|
||||
|
||||
### Scenario 3: Bulk Export to Backup
|
||||
|
||||
1. **Navigate**: Admin → Policies
|
||||
2. **Select**: Check 30 policies
|
||||
3. **Action**: Click "Export to Backup"
|
||||
4. **Form**:
|
||||
- Backup Set Name: "Production Snapshot"
|
||||
- Include Assignments: ☑ (if Feature 004 implemented)
|
||||
5. **Submit**: Click "Confirm"
|
||||
6. **Verify**:
|
||||
- Job dispatched
|
||||
- Progress: "Backing up... 10/30"
|
||||
- Final: "Backup Set 'Production Snapshot' created (30 items)"
|
||||
- New `BackupSet` record
|
||||
- 30 `BackupItem` records
|
||||
- Audit log entry
|
||||
|
||||
### Scenario 4: Bulk Prune Policy Versions
|
||||
|
||||
1. **Navigate**: Admin → Policy Versions
|
||||
2. **Filter**: Show only non-current versions older than 90 days
|
||||
3. **Select**: Check 15 versions
|
||||
4. **Action**: Click "Delete"
|
||||
5. **Confirm**: Type "DELETE"
|
||||
6. **Submit**: Click "Confirm"
|
||||
7. **Verify**:
|
||||
- Eligibility check runs
|
||||
- Eligible versions deleted (hard delete)
|
||||
- Ineligible versions skipped
|
||||
- Notification: "Deleted 12 policy versions (3 skipped)"
|
||||
- Failures array shows skip reasons:
|
||||
- "Referenced in Backup Set #5"
|
||||
- "Current version"
|
||||
- "Too recent (< 90 days)"
|
||||
|
||||
### Scenario 5: Circuit Breaker (abort on >50% fail)
|
||||
|
||||
1. **Setup**: Mock Graph API to fail for 60% of items
|
||||
2. **Navigate**: Admin → Policies
|
||||
3. **Select**: Check 100 policies
|
||||
4. **Action**: Bulk Delete
|
||||
5. **Verify**:
|
||||
- Job processes ~50 items
|
||||
- Detects >50% failure rate
|
||||
- Aborts remaining items
|
||||
- Notification: "Bulk operation aborted: 55/100 failures exceeded threshold"
|
||||
- `BulkOperationRun.status` = `aborted`
|
||||
|
||||
---
|
||||
|
||||
## Debugging
|
||||
|
||||
### View Queue Jobs
|
||||
|
||||
```bash
|
||||
# List failed jobs
|
||||
./vendor/bin/sail artisan queue:failed
|
||||
|
||||
# Retry failed job
|
||||
./vendor/bin/sail artisan queue:retry <job-id>
|
||||
|
||||
# Flush failed jobs
|
||||
./vendor/bin/sail artisan queue:flush
|
||||
```
|
||||
|
||||
### Inspect BulkOperationRun Records
|
||||
|
||||
```bash
|
||||
./vendor/bin/sail tinker
|
||||
```
|
||||
|
||||
```php
|
||||
// Get recent runs
|
||||
$runs = \App\Models\BulkOperationRun::recent()->limit(10)->get();
|
||||
|
||||
// View failures
|
||||
$run = \App\Models\BulkOperationRun::find(1);
|
||||
dd($run->failures);
|
||||
|
||||
// Check progress
|
||||
echo "{$run->processed_items}/{$run->total_items} ({$run->progressPercentage()}%)";
|
||||
```
|
||||
|
||||
### View Audit Logs
|
||||
|
||||
```bash
|
||||
# Via Filament UI
|
||||
# Navigate to: Admin → Audit Logs → Filter by event: "policies.bulk_deleted"
|
||||
|
||||
# Via Tinker
|
||||
$logs = \App\Models\AuditLog::where('event', 'policies.bulk_deleted')->get();
|
||||
```
|
||||
|
||||
### Database Queries
|
||||
|
||||
```bash
|
||||
./vendor/bin/sail artisan tinker
|
||||
```
|
||||
|
||||
```php
|
||||
// Policies marked as ignored
|
||||
$ignored = \App\Models\Policy::ignored()->count();
|
||||
|
||||
// Policy versions eligible for pruning
|
||||
$eligible = \App\Models\PolicyVersion::pruneEligible(90)->count();
|
||||
|
||||
// Deletable restore runs
|
||||
$deletable = \App\Models\RestoreRun::deletable()->count();
|
||||
```
|
||||
|
||||
### Test Queue Job Manually
|
||||
|
||||
```bash
|
||||
./vendor/bin/sail artisan tinker
|
||||
```
|
||||
|
||||
```php
|
||||
use App\Jobs\BulkPolicyDeleteJob;
|
||||
use App\Models\BulkOperationRun;
|
||||
|
||||
$policyIds = [1, 2, 3];
|
||||
$tenantId = 1;
|
||||
$userId = 1;
|
||||
|
||||
$run = BulkOperationRun::create([
|
||||
'tenant_id' => $tenantId,
|
||||
'user_id' => $userId,
|
||||
'resource' => 'policies',
|
||||
'action' => 'delete',
|
||||
'status' => 'pending',
|
||||
'total_items' => count($policyIds),
|
||||
'item_ids' => $policyIds,
|
||||
]);
|
||||
|
||||
// Dispatch synchronously for debugging
|
||||
BulkPolicyDeleteJob::dispatchSync($policyIds, $tenantId, $userId, $run->id);
|
||||
|
||||
// Check result
|
||||
$run->refresh();
|
||||
echo $run->summaryText();
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Common Issues & Solutions
|
||||
|
||||
### Issue 1: Type-to-confirm not working
|
||||
|
||||
**Symptom**: Confirm button remains enabled without typing "DELETE"
|
||||
|
||||
**Solution**: Check Filament form validation rule:
|
||||
```php
|
||||
->rule('in:DELETE') // Case-sensitive
|
||||
```
|
||||
|
||||
### Issue 2: Progress notifications don't update
|
||||
|
||||
**Symptom**: Progress stuck at "0/100"
|
||||
|
||||
**Solution**:
|
||||
- Ensure queue worker is running: `./vendor/bin/sail artisan queue:work`
|
||||
- Check Livewire polling: `wire:poll.5s="refresh"`
|
||||
- Verify BulkOperationRun is updated in job
|
||||
|
||||
### Issue 3: Policies reappear after deletion
|
||||
|
||||
**Symptom**: Deleted policies show up again after sync
|
||||
|
||||
**Solution**:
|
||||
- Check `ignored_at` is set: `Policy::find($id)->ignored_at`
|
||||
- Verify SyncPoliciesJob filters: `->whereNull('ignored_at')`
|
||||
|
||||
### Issue 4: Circuit breaker not aborting
|
||||
|
||||
**Symptom**: Job continues despite >50% failures
|
||||
|
||||
**Solution**: Check circuit breaker logic in job:
|
||||
```php
|
||||
if ($results['failed'] > count($this->policyIds) * 0.5) {
|
||||
$run->update(['status' => 'aborted']);
|
||||
throw new \Exception('Bulk operation aborted: >50% failure rate');
|
||||
}
|
||||
```
|
||||
|
||||
### Issue 5: Policy versions deleted despite references
|
||||
|
||||
**Symptom**: Referenced versions are deleted
|
||||
|
||||
**Solution**: Verify eligibility scope includes:
|
||||
```php
|
||||
->whereDoesntHave('backupItems')
|
||||
->whereNotIn('id', function ($subquery) {
|
||||
$subquery->select(DB::raw("CAST(metadata->>'policy_version_id' AS INTEGER)"))
|
||||
->from('restore_runs')
|
||||
->whereNotNull(DB::raw("metadata->>'policy_version_id'"));
|
||||
});
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Performance Benchmarks
|
||||
|
||||
Expected performance (local Sail environment):
|
||||
|
||||
| Operation | Item Count | Duration | Notes |
|
||||
|-----------|------------|----------|-------|
|
||||
| Bulk Delete (sync) | 10 | <1s | Immediate feedback |
|
||||
| Bulk Delete (queued) | 100 | <2min | Chunked, progress updates |
|
||||
| Bulk Export | 50 | <3min | Includes Graph API calls |
|
||||
| Bulk Prune | 30 | <30s | Eligibility checks |
|
||||
| Progress Update | - | 5s | Polling interval |
|
||||
|
||||
---
|
||||
|
||||
## Code Formatting
|
||||
|
||||
Before committing:
|
||||
|
||||
```bash
|
||||
./vendor/bin/sail composer pint
|
||||
```
|
||||
|
||||
Formats all PHP files per PSR-12.
|
||||
|
||||
---
|
||||
|
||||
## Next Steps
|
||||
|
||||
1. ✅ Complete Phase 2.1 (Foundation) tasks
|
||||
2. ✅ Run all tests: `./vendor/bin/sail artisan test --filter=Bulk`
|
||||
3. ✅ Manual QA: Follow scenarios above
|
||||
4. ✅ Code review: Check PSR-12, permissions, audit logs
|
||||
5. ✅ Load testing: Bulk delete 500 items
|
||||
6. → Deploy to staging
|
||||
7. → Manual QA on staging
|
||||
8. → Deploy to production
|
||||
|
||||
---
|
||||
|
||||
## Useful Commands
|
||||
|
||||
```bash
|
||||
# Watch queue jobs in real-time
|
||||
./vendor/bin/sail artisan queue:work --verbose
|
||||
|
||||
# Monitor bulk operations
|
||||
./vendor/bin/sail artisan tinker
|
||||
>>> BulkOperationRun::inProgress()->get()
|
||||
|
||||
# Seed more test data
|
||||
./vendor/bin/sail artisan db:seed --class=BulkOperationsTestSeeder
|
||||
|
||||
# Clear cache
|
||||
./vendor/bin/sail artisan optimize:clear
|
||||
|
||||
# Restart queue workers (after code changes)
|
||||
./vendor/bin/sail artisan queue:restart
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Resources
|
||||
|
||||
- [Laravel Queue Documentation](https://laravel.com/docs/12.x/queues)
|
||||
- [Filament Bulk Actions](https://filamentphp.com/docs/4.x/tables/actions#bulk-actions)
|
||||
- [Livewire Polling](https://livewire.laravel.com/docs/polling)
|
||||
- [Pest Testing](https://pestphp.com/docs)
|
||||
|
||||
---
|
||||
|
||||
**Status**: Quickstart Complete
|
||||
**Next Step**: Update agent context with new learnings
|
||||
547
specs/005-bulk-operations/research.md
Normal file
547
specs/005-bulk-operations/research.md
Normal file
@ -0,0 +1,547 @@
|
||||
# Research: Feature 005 - Bulk Operations
|
||||
|
||||
**Feature**: Bulk Operations for Resource Management
|
||||
**Date**: 2025-12-22
|
||||
**Research Phase**: Technology Decisions & Best Practices
|
||||
|
||||
---
|
||||
|
||||
## Research Questions & Findings
|
||||
|
||||
### Q1: How to implement type-to-confirm in Filament bulk actions?
|
||||
|
||||
**Research Goal**: Find a Laravel/Filament-idiomatic way to require explicit confirmation for destructive bulk operations (≥20 items).
|
||||
|
||||
**Findings**:
|
||||
|
||||
Filament BulkActions support conditional forms via `->form()` method:
|
||||
|
||||
```php
|
||||
Tables\Actions\DeleteBulkAction::make()
|
||||
->requiresConfirmation()
|
||||
->modalHeading(fn (Collection $records) =>
|
||||
$records->count() >= 20
|
||||
? "⚠️ Delete {$records->count()} policies?"
|
||||
: "Delete {$records->count()} policies?"
|
||||
)
|
||||
->form(fn (Collection $records) =>
|
||||
$records->count() >= 20
|
||||
? [
|
||||
Forms\Components\TextInput::make('confirm_delete')
|
||||
->label('Type DELETE to confirm')
|
||||
->rule('in:DELETE')
|
||||
->required()
|
||||
->helperText('This action cannot be undone.')
|
||||
]
|
||||
: []
|
||||
)
|
||||
->action(fn (Collection $records, array $data) => {
|
||||
// Validation ensures $data['confirm_delete'] === 'DELETE'
|
||||
// Proceed with bulk delete
|
||||
});
|
||||
```
|
||||
|
||||
**Key Insight**: Filament's form validation automatically prevents submission if `confirm_delete` doesn't match "DELETE" (case-sensitive).
|
||||
|
||||
**Alternatives Considered**:
|
||||
- Custom modal component (more code, less reusable)
|
||||
- JavaScript validation (client-side only, less secure)
|
||||
- Laravel form request (breaks Filament UX flow)
|
||||
|
||||
**Decision**: Use Filament `->form()` with validation rule.
|
||||
|
||||
---
|
||||
|
||||
### Q2: How to track progress for queued bulk jobs?
|
||||
|
||||
**Research Goal**: Enable real-time progress tracking for async bulk operations (≥20 items) without blocking UI.
|
||||
|
||||
**Findings**:
|
||||
|
||||
Filament notifications are not reactive by default. Must implement custom progress tracking:
|
||||
|
||||
1. **Create BulkOperationRun model** to persist state:
|
||||
```php
|
||||
Schema::create('bulk_operation_runs', function (Blueprint $table) {
|
||||
$table->id();
|
||||
$table->string('status'); // 'pending', 'running', 'completed', 'failed', 'aborted'
|
||||
$table->integer('total_items');
|
||||
$table->integer('processed_items')->default(0);
|
||||
$table->integer('succeeded')->default(0);
|
||||
$table->integer('failed')->default(0);
|
||||
$table->json('item_ids');
|
||||
$table->json('failures')->nullable();
|
||||
// ... tenant_id, user_id, resource, action
|
||||
});
|
||||
```
|
||||
|
||||
2. **Job updates model after each chunk**:
|
||||
```php
|
||||
collect($this->policyIds)->chunk(10)->each(function ($chunk) use ($run) {
|
||||
foreach ($chunk as $id) {
|
||||
// Process item
|
||||
}
|
||||
$run->update([
|
||||
'processed_items' => $run->processed_items + $chunk->count(),
|
||||
// ... succeeded, failed counts
|
||||
]);
|
||||
});
|
||||
```
|
||||
|
||||
3. **UI polls for updates** via Livewire:
|
||||
```blade
|
||||
<div wire:poll.5s="refreshProgress">
|
||||
Processing... {{ $run->processed_items }}/{{ $run->total_items }}
|
||||
</div>
|
||||
```
|
||||
|
||||
**Alternatives Considered**:
|
||||
- **Bus::batch()**: Laravel's batch system tracks job progress, but adds complexity:
|
||||
- Requires job_batches table (already exists in Laravel)
|
||||
- Each item becomes separate job (overhead for small batches)
|
||||
- Good for parallelization, overkill for sequential processing
|
||||
- Decision: **Not needed** - our jobs process items sequentially with chunking
|
||||
|
||||
- **Filament Pulse**: Real-time application monitoring tool
|
||||
- Too heavy for single-feature progress tracking
|
||||
- Requires separate service
|
||||
- Decision: **Rejected** - use custom BulkOperationRun model
|
||||
|
||||
- **Pusher/WebSockets**: Real-time push notifications
|
||||
- Infrastructure overhead (Pusher subscription or custom WS server)
|
||||
- Not needed for 5-10s polling interval
|
||||
- Decision: **Rejected** - Livewire polling sufficient
|
||||
|
||||
**Decision**: BulkOperationRun model + Livewire polling (5s interval).
|
||||
|
||||
---
|
||||
|
||||
### Q3: How to handle chunked processing in queue jobs?
|
||||
|
||||
**Research Goal**: Process large batches (up to 500 items) without memory exhaustion or timeout.
|
||||
|
||||
**Findings**:
|
||||
|
||||
Laravel Collections provide `->chunk()` method for memory-efficient iteration:
|
||||
|
||||
```php
|
||||
collect($this->policyIds)->chunk(10)->each(function ($chunk) use (&$results, $run) {
|
||||
foreach ($chunk as $id) {
|
||||
try {
|
||||
// Process item
|
||||
$results['succeeded']++;
|
||||
} catch (\Exception $e) {
|
||||
$results['failed']++;
|
||||
$results['failures'][] = ['id' => $id, 'reason' => $e->getMessage()];
|
||||
}
|
||||
}
|
||||
|
||||
// Update progress after each chunk (not per-item)
|
||||
$run->update([
|
||||
'processed_items' => $results['succeeded'] + $results['failed'],
|
||||
'succeeded' => $results['succeeded'],
|
||||
'failed' => $results['failed'],
|
||||
'failures' => $results['failures'],
|
||||
]);
|
||||
|
||||
// Circuit breaker: abort if >50% failed
|
||||
if ($results['failed'] > count($this->policyIds) * 0.5) {
|
||||
$run->update(['status' => 'aborted']);
|
||||
throw new \Exception('Bulk operation aborted: >50% failure rate');
|
||||
}
|
||||
});
|
||||
```
|
||||
|
||||
**Key Insights**:
|
||||
- Chunk size: 10-20 items (balance between DB updates and progress granularity)
|
||||
- Update BulkOperationRun **after each chunk**, not per-item (reduces DB load)
|
||||
- Circuit breaker: abort if >50% failures detected mid-process
|
||||
- Fail-soft: continue processing remaining items on individual failures
|
||||
|
||||
**Alternatives Considered**:
|
||||
- **Cursor-based chunking**: `Model::chunk(100, function)`
|
||||
- Good for processing entire tables
|
||||
- Not needed - we have explicit ID list
|
||||
|
||||
- **Bus::batch()**: Parallel job processing
|
||||
- Good for independent tasks (e.g., sending emails)
|
||||
- Our tasks are sequential (delete one, then next)
|
||||
- Adds complexity without benefit
|
||||
|
||||
- **Database transactions per chunk**:
|
||||
- Risk: partial failure leaves incomplete state
|
||||
- Decision: **No transactions** - each item is atomic, fail-soft is intentional
|
||||
|
||||
**Decision**: `collect()->chunk(10)` with after-chunk progress updates.
|
||||
|
||||
---
|
||||
|
||||
### Q4: How to enforce tenant isolation in bulk jobs?
|
||||
|
||||
**Research Goal**: Ensure bulk operations cannot cross tenant boundaries (critical security requirement).
|
||||
|
||||
**Findings**:
|
||||
|
||||
Laravel Queue jobs serialize model instances poorly (especially Collections). Best practice:
|
||||
|
||||
```php
|
||||
class BulkPolicyDeleteJob implements ShouldQueue
|
||||
{
|
||||
public function __construct(
|
||||
public array $policyIds, // array, NOT Collection
|
||||
public int $tenantId, // explicit tenant ID
|
||||
public int $actorId, // user ID for audit
|
||||
public int $bulkOperationRunId // FK to tracking model
|
||||
) {}
|
||||
|
||||
public function handle(PolicyRepository $policies): void
|
||||
{
|
||||
// Verify all policies belong to tenant (defensive check)
|
||||
$count = Policy::whereIn('id', $this->policyIds)
|
||||
->where('tenant_id', $this->tenantId)
|
||||
->count();
|
||||
|
||||
if ($count !== count($this->policyIds)) {
|
||||
throw new \Exception('Tenant isolation violation detected');
|
||||
}
|
||||
|
||||
// Proceed with bulk operation...
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Key Insights**:
|
||||
- Serialize IDs as `array`, not `Collection` (Collections don't serialize well)
|
||||
- Pass explicit `tenantId` parameter (don't rely on global scopes)
|
||||
- Defensive check in job: verify all IDs belong to tenant before processing
|
||||
- Audit log records `tenantId` and `actorId` for compliance
|
||||
|
||||
**Alternatives Considered**:
|
||||
- **Global tenant scope**: Rely on Laravel's global scope filtering
|
||||
- Risk: scope could be disabled/bypassed in job context
|
||||
- Less explicit, harder to debug
|
||||
- Decision: **Rejected** - explicit is safer
|
||||
|
||||
- **Pass User model**: `public User $user`
|
||||
- Serializes entire user object (inefficient)
|
||||
- User could be deleted before job runs
|
||||
- Decision: **Rejected** - use `actorId` integer
|
||||
|
||||
**Decision**: Explicit `tenantId` + defensive validation in job.
|
||||
|
||||
---
|
||||
|
||||
### Q5: How to prevent sync from re-adding "deleted" policies?
|
||||
|
||||
**Research Goal**: User bulk-deletes 50 policies locally, but doesn't want to delete them in Intune. How to prevent SyncPoliciesJob from re-importing them?
|
||||
|
||||
**Findings**:
|
||||
|
||||
Add `ignored_at` timestamp column to policies table:
|
||||
|
||||
```php
|
||||
// Migration
|
||||
Schema::table('policies', function (Blueprint $table) {
|
||||
$table->timestamp('ignored_at')->nullable()->after('deleted_at');
|
||||
$table->index('ignored_at'); // query optimization
|
||||
});
|
||||
|
||||
// Policy model
|
||||
public function scopeNotIgnored($query)
|
||||
{
|
||||
return $query->whereNull('ignored_at');
|
||||
}
|
||||
|
||||
public function markIgnored(): void
|
||||
{
|
||||
$this->update(['ignored_at' => now()]);
|
||||
}
|
||||
```
|
||||
|
||||
**Modify SyncPoliciesJob**:
|
||||
|
||||
```php
|
||||
// Before: fetched all policies from Graph, upserted to DB
|
||||
// After: skip policies where ignored_at IS NOT NULL
|
||||
|
||||
public function handle(PolicySyncService $service): void
|
||||
{
|
||||
$graphPolicies = $service->fetchFromGraph($this->types);
|
||||
|
||||
foreach ($graphPolicies as $graphPolicy) {
|
||||
$existing = Policy::where('graph_id', $graphPolicy['id'])
|
||||
->where('tenant_id', $this->tenantId)
|
||||
->first();
|
||||
|
||||
// Skip if locally ignored
|
||||
if ($existing && $existing->ignored_at !== null) {
|
||||
continue;
|
||||
}
|
||||
|
||||
// Upsert policy...
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Key Insight**: `ignored_at` decouples local tracking from Intune state. User can:
|
||||
- Keep policy in Intune (not deleted remotely)
|
||||
- Hide policy in TenantPilot (ignored_at set)
|
||||
- Restore policy later (clear ignored_at)
|
||||
|
||||
**Alternatives Considered**:
|
||||
- **Soft delete only** (`deleted_at`):
|
||||
- Problem: Sync doesn't know if user deleted locally or Intune deleted remotely
|
||||
- Would need separate "deletion source" column
|
||||
- Decision: **Rejected** - `ignored_at` is clearer intent
|
||||
|
||||
- **Separate "sync_ignore" column**:
|
||||
- Same outcome as `ignored_at`, but less semantic
|
||||
- Decision: **Accepted as alias** - `ignored_at` is more descriptive
|
||||
|
||||
**Decision**: Add `ignored_at` timestamp, filter in SyncPoliciesJob.
|
||||
|
||||
---
|
||||
|
||||
### Q6: How to determine eligibility for Policy Version pruning?
|
||||
|
||||
**Research Goal**: Implement safe "bulk delete old policy versions" that won't break backups/restores.
|
||||
|
||||
**Findings**:
|
||||
|
||||
Eligibility criteria (all must be true):
|
||||
1. `is_current = false` (not the latest version)
|
||||
2. `created_at < NOW() - 90 days` (configurable retention period)
|
||||
3. NOT referenced in `backup_items.policy_version_id` (foreign key check)
|
||||
4. NOT referenced in `restore_runs.metadata->policy_version_id` (JSONB check)
|
||||
|
||||
Implementation via Eloquent scope:
|
||||
|
||||
```php
|
||||
// app/Models/PolicyVersion.php
|
||||
public function scopePruneEligible($query, int $retentionDays = 90)
|
||||
{
|
||||
return $query
|
||||
->where('is_current', false)
|
||||
->where('created_at', '<', now()->subDays($retentionDays))
|
||||
->whereDoesntHave('backupItems') // FK relationship
|
||||
->whereNotIn('id', function ($subquery) {
|
||||
$subquery->select(DB::raw("CAST(metadata->>'policy_version_id' AS INTEGER)"))
|
||||
->from('restore_runs')
|
||||
->whereNotNull(DB::raw("metadata->>'policy_version_id'"));
|
||||
});
|
||||
}
|
||||
```
|
||||
|
||||
**Bulk prune job**:
|
||||
|
||||
```php
|
||||
public function handle(): void
|
||||
{
|
||||
foreach ($this->versionIds as $id) {
|
||||
$version = PolicyVersion::find($id);
|
||||
|
||||
if (!$version) {
|
||||
$this->failures[] = ['id' => $id, 'reason' => 'Not found'];
|
||||
continue;
|
||||
}
|
||||
|
||||
// Check eligibility
|
||||
$eligible = PolicyVersion::pruneEligible()
|
||||
->where('id', $id)
|
||||
->exists();
|
||||
|
||||
if (!$eligible) {
|
||||
$this->skipped++;
|
||||
$this->failures[] = ['id' => $id, 'reason' => 'Referenced or too recent'];
|
||||
continue;
|
||||
}
|
||||
|
||||
$version->delete(); // hard delete
|
||||
$this->succeeded++;
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Key Insight**: Conservative eligibility check prevents accidental data loss. User sees which versions were skipped and why.
|
||||
|
||||
**Alternatives Considered**:
|
||||
- **Soft delete first, hard delete later**: Adds complexity, no clear benefit
|
||||
- **Skip JSONB check**: Risk of breaking restore runs that reference version
|
||||
- **Admin override**: Allow force-delete even if referenced
|
||||
- Too dangerous, conflicts with immutability principle
|
||||
- Decision: **Rejected**
|
||||
|
||||
**Decision**: Eloquent scope `pruneEligible()` with strict checks.
|
||||
|
||||
---
|
||||
|
||||
### Q7: How to display progress notifications in Filament?
|
||||
|
||||
**Research Goal**: Show real-time progress for bulk operations without blocking UI.
|
||||
|
||||
**Findings**:
|
||||
|
||||
Filament notifications are sent once and don't auto-update. For progress tracking:
|
||||
|
||||
**Option 1: Custom Livewire Component**
|
||||
|
||||
```blade
|
||||
{{-- resources/views/livewire/bulk-operation-progress.blade.php --}}
|
||||
<div wire:poll.5s="refresh">
|
||||
@if($run && !$run->isComplete())
|
||||
<div class="bg-blue-50 p-4 rounded">
|
||||
<h3>{{ $run->action }} in progress...</h3>
|
||||
<div class="w-full bg-gray-200 rounded">
|
||||
<div class="bg-blue-600 h-2 rounded" style="width: {{ $run->progressPercentage() }}%"></div>
|
||||
</div>
|
||||
<p>{{ $run->processed_items }}/{{ $run->total_items }} items processed</p>
|
||||
</div>
|
||||
@elseif($run && $run->isComplete())
|
||||
<div class="bg-green-50 p-4 rounded">
|
||||
<h3>✅ {{ $run->summaryText() }}</h3>
|
||||
@if($run->failed > 0)
|
||||
<a href="{{ route('filament.admin.resources.audit-logs.view', $run->audit_log_id) }}">View details</a>
|
||||
@endif
|
||||
</div>
|
||||
@endif
|
||||
</div>
|
||||
```
|
||||
|
||||
```php
|
||||
// app/Livewire/BulkOperationProgress.php
|
||||
class BulkOperationProgress extends Component
|
||||
{
|
||||
public int $bulkOperationRunId;
|
||||
public ?BulkOperationRun $run = null;
|
||||
|
||||
public function mount(int $bulkOperationRunId): void
|
||||
{
|
||||
$this->bulkOperationRunId = $bulkOperationRunId;
|
||||
$this->refresh();
|
||||
}
|
||||
|
||||
public function refresh(): void
|
||||
{
|
||||
$this->run = BulkOperationRun::find($this->bulkOperationRunId);
|
||||
|
||||
// Stop polling if complete
|
||||
if ($this->run && $this->run->isComplete()) {
|
||||
$this->dispatch('bulkOperationComplete', runId: $this->run->id);
|
||||
}
|
||||
}
|
||||
|
||||
public function render(): View
|
||||
{
|
||||
return view('livewire.bulk-operation-progress');
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Option 2: Filament Infolist Widget** (simpler, more integrated)
|
||||
|
||||
```php
|
||||
// Display in BulkOperationRun resource ViewRecord page
|
||||
public static function form(Form $form): Form
|
||||
{
|
||||
return $form
|
||||
->schema([
|
||||
Infolists\Components\Section::make('Progress')
|
||||
->schema([
|
||||
Infolists\Components\TextEntry::make('summaryText')
|
||||
->label('Status'),
|
||||
Infolists\Components\ViewEntry::make('progress')
|
||||
->view('filament.components.progress-bar')
|
||||
->state(fn ($record) => [
|
||||
'percentage' => $record->progressPercentage(),
|
||||
'processed' => $record->processed_items,
|
||||
'total' => $record->total_items,
|
||||
]),
|
||||
])
|
||||
->poll('5s') // Filament's built-in polling
|
||||
->hidden(fn ($record) => $record->isComplete()),
|
||||
]);
|
||||
}
|
||||
```
|
||||
|
||||
**Decision**: Use **Option 1** (custom Livewire component) for flexibility. Embed in:
|
||||
- Filament notification body (custom view)
|
||||
- Resource page sidebar
|
||||
- Dashboard widget (if user wants to monitor all bulk operations)
|
||||
|
||||
**Alternatives Considered**:
|
||||
- **Pusher/WebSockets**: Too complex for 5s polling
|
||||
- **JavaScript polling**: Less Laravel-way, harder to test
|
||||
- **Filament Pulse**: Overkill for single feature
|
||||
|
||||
---
|
||||
|
||||
## Technology Stack Summary
|
||||
|
||||
| Component | Technology | Justification |
|
||||
|-----------|------------|---------------|
|
||||
| Admin Panel | Filament v4 | Built-in bulk actions, forms, notifications |
|
||||
| Reactive UI | Livewire v3 | Polling, state management, no JS framework needed |
|
||||
| Queue System | Laravel Queue | Async job processing, retry, failure handling |
|
||||
| Progress Tracking | BulkOperationRun model + Livewire polling | Persistent state, survives refresh, queryable |
|
||||
| Type-to-Confirm | Filament form validation | Built-in UI, secure, reusable |
|
||||
| Tenant Isolation | Explicit tenantId param | Fail-safe, auditable, no implicit scopes |
|
||||
| Job Chunking | Collection::chunk(10) | Memory-efficient, simple, testable |
|
||||
| Eligibility Checks | Eloquent scopes | Reusable, composable, database-level filtering |
|
||||
| Database | PostgreSQL + JSONB | Native JSON support for item_ids, failures |
|
||||
|
||||
---
|
||||
|
||||
## Best Practices Applied
|
||||
|
||||
### Laravel Conventions
|
||||
- ✅ Queue jobs implement `ShouldQueue` interface
|
||||
- ✅ Use Eloquent relationships, not raw queries
|
||||
- ✅ Form validation via Filament rules
|
||||
- ✅ PSR-12 code formatting (Laravel Pint)
|
||||
|
||||
### Safety & Security
|
||||
- ✅ Tenant isolation enforced at job level
|
||||
- ✅ Type-to-confirm for ≥20 destructive items
|
||||
- ✅ Fail-soft: continue on individual failures
|
||||
- ✅ Circuit breaker: abort if >50% fail
|
||||
- ✅ Audit logging for compliance
|
||||
|
||||
### Performance
|
||||
- ✅ Chunked processing (10-20 items)
|
||||
- ✅ Indexed queries (tenant_id, ignored_at)
|
||||
- ✅ Polling interval: 5s (not 1s spam)
|
||||
- ✅ JSONB for flexible metadata storage
|
||||
|
||||
### Testing
|
||||
- ✅ Unit tests for jobs, scopes, eligibility
|
||||
- ✅ Feature tests for E2E flows
|
||||
- ✅ Pest assertions for progress tracking
|
||||
- ✅ Manual QA checklist for UI flows
|
||||
|
||||
---
|
||||
|
||||
## Rejected Alternatives
|
||||
|
||||
| Alternative | Why Rejected |
|
||||
|-------------|--------------|
|
||||
| Bus::batch() | Adds complexity, not needed for sequential processing |
|
||||
| Filament Pulse | Overkill for single-feature progress tracking |
|
||||
| Pusher/WebSockets | Infrastructure overhead, 5s polling sufficient |
|
||||
| Global tenant scopes | Less explicit, harder to debug, security risk |
|
||||
| Custom modal component | More code, less reusable than Filament form |
|
||||
| Hard delete without checks | Too risky, violates immutability principle |
|
||||
|
||||
---
|
||||
|
||||
## Open Questions for Implementation
|
||||
|
||||
1. **Chunk size**: Start with 10, benchmark if needed
|
||||
2. **Polling interval**: 5s default, make configurable?
|
||||
3. **Retention period**: 90 days for versions, make configurable?
|
||||
4. **Max bulk items**: Hard limit at 500? 1000?
|
||||
5. **Retry failed items**: Future enhancement or MVP?
|
||||
|
||||
---
|
||||
|
||||
**Status**: Research Complete
|
||||
**Next Step**: Generate data-model.md
|
||||
Loading…
Reference in New Issue
Block a user