feat: configurable bulk ops polling + chunking

- Add tenantpilot.bulk_operations config (chunk size, poll interval) - Use config chunk size across all bulk jobs - Make progress widget polling interval configurable - Document settings in README + feature quickstart; mark tasks done
2025-12-25 03:18:12 +01:00 · 2025-12-25 03:18:12 +01:00 · eef9618889
commit eef9618889
parent 239a2e6af9
19 changed files with 49 additions and 19 deletions
--- a/README.md
+++ b/README.md
@ -27,6 +27,17 @@ ## TenantPilot setup
  - Ensure queue workers are running for jobs (e.g., policy sync) after deploy.
  - Keep secrets/env in Dokploy, never in code.

+## Bulk operations (Feature 005)
+
+- Bulk actions are available in Filament resource tables (Policies, Policy Versions, Backup Sets, Restore Runs).
+- Destructive operations require type-to-confirm at higher thresholds (e.g. `DELETE`).
+- Long-running bulk ops are queued; the bottom-right progress widget polls for active runs.
+
+### Configuration
+
+- `TENANTPILOT_BULK_CHUNK_SIZE` (default `10`): job refresh/progress chunk size.
+- `TENANTPILOT_BULK_POLL_INTERVAL_SECONDS` (default `3`): Livewire polling interval for the progress widget (clamped to 1–10s).
+
 ## Intune RBAC Onboarding Wizard

 - Entry point: Tenant detail in Filament (`Setup Intune RBAC` in the ⋯ ActionGroup). Visible only for active tenants with `app_client_id`.
--- a/app/Jobs/BulkBackupSetDeleteJob.php
+++ b/app/Jobs/BulkBackupSetDeleteJob.php
@ -37,7 +37,7 @@ public function handle(BulkOperationService $service): void
        $skipped = 0;
        $skipReasons = [];

-        $chunkSize = 10;
+        $chunkSize = max(1, (int) config('tenantpilot.bulk_operations.chunk_size', 10));
        $totalItems = $run->total_items ?: count($run->item_ids ?? []);
        $failureThreshold = (int) floor($totalItems / 2);

--- a/app/Jobs/BulkBackupSetForceDeleteJob.php
+++ b/app/Jobs/BulkBackupSetForceDeleteJob.php
@ -37,7 +37,7 @@ public function handle(BulkOperationService $service): void
        $skipped = 0;
        $skipReasons = [];

-        $chunkSize = 10;
+        $chunkSize = max(1, (int) config('tenantpilot.bulk_operations.chunk_size', 10));
        $totalItems = $run->total_items ?: count($run->item_ids ?? []);
        $failureThreshold = (int) floor($totalItems / 2);

--- a/app/Jobs/BulkBackupSetRestoreJob.php
+++ b/app/Jobs/BulkBackupSetRestoreJob.php
@ -37,7 +37,7 @@ public function handle(BulkOperationService $service): void
        $skipped = 0;
        $skipReasons = [];

-        $chunkSize = 10;
+        $chunkSize = max(1, (int) config('tenantpilot.bulk_operations.chunk_size', 10));
        $totalItems = $run->total_items ?: count($run->item_ids ?? []);
        $failureThreshold = (int) floor($totalItems / 2);

--- a/app/Jobs/BulkPolicyDeleteJob.php
+++ b/app/Jobs/BulkPolicyDeleteJob.php
@ -44,7 +44,7 @@ public function handle(BulkOperationService $service): void
            $skipped = 0;
            $failures = [];

-            $chunkSize = 10;
+            $chunkSize = max(1, (int) config('tenantpilot.bulk_operations.chunk_size', 10));

            $totalItems = $run->total_items ?: count($run->item_ids ?? []);
            $failureThreshold = (int) floor($totalItems / 2);
@ -129,7 +129,7 @@ public function handle(BulkOperationService $service): void

                }

-                // Refresh the run from database every 10 items to avoid stale data
+                // Refresh the run from database every $chunkSize items to avoid stale data

                if ($itemCount % $chunkSize === 0) {

--- a/app/Jobs/BulkPolicyExportJob.php
+++ b/app/Jobs/BulkPolicyExportJob.php
@ -51,7 +51,7 @@ public function handle(BulkOperationService $service): void
            $succeeded = 0;
            $failed = 0;
            $failures = [];
-            $chunkSize = 10;
+            $chunkSize = max(1, (int) config('tenantpilot.bulk_operations.chunk_size', 10));

            $totalItems = $run->total_items ?: count($run->item_ids ?? []);
            $failureThreshold = (int) floor($totalItems / 2);
--- a/app/Jobs/BulkPolicySyncJob.php
+++ b/app/Jobs/BulkPolicySyncJob.php
@ -31,7 +31,7 @@ public function handle(BulkOperationService $service, PolicySyncService $syncSer
        $service->start($run);

        try {
-            $chunkSize = 10;
+            $chunkSize = max(1, (int) config('tenantpilot.bulk_operations.chunk_size', 10));
            $itemCount = 0;

            $totalItems = $run->total_items ?: count($run->item_ids ?? []);
--- a/app/Jobs/BulkPolicyUnignoreJob.php
+++ b/app/Jobs/BulkPolicyUnignoreJob.php
@ -35,7 +35,7 @@ public function handle(BulkOperationService $service): void
            $failed = 0;
            $skipped = 0;

-            $chunkSize = 10;
+            $chunkSize = max(1, (int) config('tenantpilot.bulk_operations.chunk_size', 10));

            foreach ($run->item_ids as $policyId) {
                $itemCount++;
--- a/app/Jobs/BulkPolicyVersionForceDeleteJob.php
+++ b/app/Jobs/BulkPolicyVersionForceDeleteJob.php
@ -37,7 +37,7 @@ public function handle(BulkOperationService $service): void
        $skipped = 0;
        $skipReasons = [];

-        $chunkSize = 10;
+        $chunkSize = max(1, (int) config('tenantpilot.bulk_operations.chunk_size', 10));
        $totalItems = $run->total_items ?: count($run->item_ids ?? []);
        $failureThreshold = (int) floor($totalItems / 2);

--- a/app/Jobs/BulkPolicyVersionPruneJob.php
+++ b/app/Jobs/BulkPolicyVersionPruneJob.php
@ -38,7 +38,7 @@ public function handle(BulkOperationService $service): void
        $skipped = 0;
        $skipReasons = [];

-        $chunkSize = 10;
+        $chunkSize = max(1, (int) config('tenantpilot.bulk_operations.chunk_size', 10));
        $totalItems = $run->total_items ?: count($run->item_ids ?? []);
        $failureThreshold = (int) floor($totalItems / 2);

--- a/app/Jobs/BulkPolicyVersionRestoreJob.php
+++ b/app/Jobs/BulkPolicyVersionRestoreJob.php
@ -37,7 +37,7 @@ public function handle(BulkOperationService $service): void
        $skipped = 0;
        $skipReasons = [];

-        $chunkSize = 10;
+        $chunkSize = max(1, (int) config('tenantpilot.bulk_operations.chunk_size', 10));
        $totalItems = $run->total_items ?: count($run->item_ids ?? []);
        $failureThreshold = (int) floor($totalItems / 2);

--- a/app/Jobs/BulkRestoreRunDeleteJob.php
+++ b/app/Jobs/BulkRestoreRunDeleteJob.php
@ -38,7 +38,7 @@ public function handle(BulkOperationService $service): void
            $skipped = 0;
            $skipReasons = [];

-            $chunkSize = 10;
+            $chunkSize = max(1, (int) config('tenantpilot.bulk_operations.chunk_size', 10));
            $totalItems = $run->total_items ?: count($run->item_ids ?? []);
            $failureThreshold = (int) floor($totalItems / 2);

--- a/app/Jobs/BulkRestoreRunForceDeleteJob.php
+++ b/app/Jobs/BulkRestoreRunForceDeleteJob.php
@ -37,7 +37,7 @@ public function handle(BulkOperationService $service): void
        $skipped = 0;
        $skipReasons = [];

-        $chunkSize = 10;
+        $chunkSize = max(1, (int) config('tenantpilot.bulk_operations.chunk_size', 10));
        $totalItems = $run->total_items ?: count($run->item_ids ?? []);
        $failureThreshold = (int) floor($totalItems / 2);

--- a/app/Jobs/BulkRestoreRunRestoreJob.php
+++ b/app/Jobs/BulkRestoreRunRestoreJob.php
@ -37,7 +37,7 @@ public function handle(BulkOperationService $service): void
        $skipped = 0;
        $skipReasons = [];

-        $chunkSize = 10;
+        $chunkSize = max(1, (int) config('tenantpilot.bulk_operations.chunk_size', 10));
        $totalItems = $run->total_items ?: count($run->item_ids ?? []);
        $failureThreshold = (int) floor($totalItems / 2);

--- a/app/Livewire/BulkOperationProgress.php
+++ b/app/Livewire/BulkOperationProgress.php
@ -11,8 +11,11 @@ class BulkOperationProgress extends Component
 {
    public $runs;

+    public int $pollSeconds = 3;
+
    public function mount()
    {
+        $this->pollSeconds = max(1, min(10, (int) config('tenantpilot.bulk_operations.poll_interval_seconds', 3)));
        $this->loadRuns();
    }

--- a/config/tenantpilot.php
+++ b/config/tenantpilot.php
@ -118,4 +118,9 @@
    'features' => [
        'conditional_access' => true,
    ],
+
+    'bulk_operations' => [
+        'chunk_size' => (int) env('TENANTPILOT_BULK_CHUNK_SIZE', 10),
+        'poll_interval_seconds' => (int) env('TENANTPILOT_BULK_POLL_INTERVAL_SECONDS', 3),
+    ],
 ];
--- a/resources/views/livewire/bulk-operation-progress.blade.php
+++ b/resources/views/livewire/bulk-operation-progress.blade.php
@ -1,4 +1,4 @@
-<div wire:poll.3s="loadRuns">
+<div wire:poll.{{ $pollSeconds }}s="loadRuns">
    <!-- Bulk Operation Progress Component: {{ $runs->count() }} active runs -->
    @if($runs->isNotEmpty())
        <div class="fixed bottom-4 right-4 z-[999999] w-96 space-y-2" style="pointer-events: auto;">
--- a/specs/005-bulk-operations/quickstart.md
+++ b/specs/005-bulk-operations/quickstart.md
@ -120,6 +120,17 @@ ### Browser Tests (Pest v4)

 ---

+## Configuration
+
+These defaults are safe for staging/production, but can be tuned per environment.
+
+- **Chunk size** (job refresh/progress cadence):
+   - `TENANTPILOT_BULK_CHUNK_SIZE` (default `10`)
+- **Progress polling interval** (UI updates):
+   - `TENANTPILOT_BULK_POLL_INTERVAL_SECONDS` (default `3`, clamped to 1–10 seconds)
+- **Policy version prune retention window**:
+   - Default `90` days (editable in the prune modal as “Retention Days”)
+
 ## Manual Testing Workflow

 ### Scenario 1: Bulk Delete Policies (< 20 items)
--- a/specs/005-bulk-operations/tasks.md
+++ b/specs/005-bulk-operations/tasks.md
@ -166,7 +166,7 @@ ### Implementation for User Story 6
 - [ ] T053 [US6] Test circuit breaker with mock failures (manual QA)
 - [x] T054 [US6] Run tests: `./vendor/bin/sail artisan test tests/Feature/BulkProgressNotificationTest.php`

- [ ] T054a [US6] Define/verify polling interval meets NFR-005.3 (≤10s updates) and document where configured
+- [x] T054a [US6] Define/verify polling interval meets NFR-005.3 (≤10s updates) and document where configured
 - [ ] T054b [US6] Manual QA: force a catastrophic job failure and verify FR-005.13 notification behavior
 - [ ] T054c [US6] Manual QA: verify UI remains responsive during a 100-item queued run (NFR-005.4)

@ -288,8 +288,8 @@ ## Phase 10: Polish & Cross-Cutting Concerns

 **Purpose**: Documentation, cleanup, performance optimization

- [ ] T086 [P] Update README.md with bulk operations feature description
- [ ] T087 [P] Update quickstart.md with manual testing scenarios (already done in planning)
+- [x] T086 [P] Update README.md with bulk operations feature description
+- [x] T087 [P] Update quickstart.md with manual testing scenarios (already done in planning)
 - [ ] T088 Code cleanup: Remove debug statements, refactor duplicated logic
 - [ ] T089 Performance test: Bulk delete 500 policies (should complete <5 minutes)
 - [ ] T090 Load test: Concurrent bulk operations (2-3 admins, different resources)
@ -298,7 +298,7 @@ ## Phase 10: Polish & Cross-Cutting Concerns
 - [ ] T093 Run full test suite: `./vendor/bin/sail artisan test`
 - [ ] T094 Run Pint formatting: `./vendor/bin/sail composer pint`
 - [ ] T095 Manual QA checklist: Complete all scenarios from quickstart.md
- [ ] T096 Document configuration options (chunk size, polling interval, retention days)
+- [x] T096 Document configuration options (chunk size, polling interval, retention days)
 - [ ] T097 Create BulkOperationRun resource page in Filament (view progress, retry failed)
 - [ ] T098 Add bulk operation metrics to dashboard (total runs, success rate)