ahmido/TenantAtlas
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects 1 Releases Wiki Activity
86505483bf
TenantAtlas/specs/243-product-usage-adoption-telemetry/tasks.md
ahmido 6053d87b99
Some checks failed
Main Confidence / confidence (push) Failing after 48s
Details
feat: implement product usage adoption telemetry (#281) 
## Summary
- implement spec 243 product usage adoption telemetry end-to-end
- add bounded product usage event capture, aggregation, retention pruning, and system dashboard KPIs
- add unit and feature coverage for telemetry capture, authorization, retention, privacy, and dashboard window behavior

## Validation
- ran focused Pest test suites for telemetry and system dashboard behavior
- ran Laravel Pint formatting
- verified the system dashboard telemetry widget in the integrated browser

## Notes
- branch: `243-product-usage-adoption-telemetry`
- target: `dev`

Co-authored-by: Ahmed Darrazi <ahmed.darrazi@live.de>
Reviewed-on: #281
2026-04-26 20:52:38 +00:00

186 lines
16 KiB
Markdown
Raw Blame History

---
description: "Task list for Product Usage & Adoption Telemetry"
---
# Tasks: Product Usage & Adoption Telemetry
**Input**: Design documents from `/Users/ahmeddarrazi/Documents/projects/wt-plattform/specs/243-product-usage-adoption-telemetry/`
**Prerequisites**: `/Users/ahmeddarrazi/Documents/projects/wt-plattform/specs/243-product-usage-adoption-telemetry/plan.md` (required), `/Users/ahmeddarrazi/Documents/projects/wt-plattform/specs/243-product-usage-adoption-telemetry/spec.md` (required), `/Users/ahmeddarrazi/Documents/projects/wt-plattform/specs/243-product-usage-adoption-telemetry/checklists/requirements.md` (required)
**Tests**: REQUIRED (Pest) for all runtime behavior changes in this slice. Keep proof in Unit + Feature lanes only.
**Operations**: This slice must not alter existing `OperationRun` UX or lifecycle. It may observe successful user-initiated tenant-bound operation starts as telemetry sources only after the current start contract succeeds.
**RBAC**: Telemetry writes occur only after the originating admin-plane action has already passed workspace membership, tenant entitlement, and capability checks. Aggregate reads remain system-plane only through the existing dashboard access rules.
**Organization**: Tasks are grouped by user story so the write path, read path, and privacy or isolation guardrails remain independently testable.
## Phase 1: Setup (Shared Infrastructure)
**Purpose**: Prepare the bounded product-telemetry namespace and the local validation surfaces without widening scope.
- [X] T001 Start the local Sail environment with `cd apps/platform && ./vendor/bin/sail up -d` (script: `apps/platform/vendor/bin/sail`)
- [X] T002 Create the bounded feature-local directories under `apps/platform/app/Support/ProductTelemetry/`, `apps/platform/tests/Unit/Support/ProductTelemetry/`, `apps/platform/tests/Feature/System/ProductTelemetry/`, `apps/platform/tests/Feature/Onboarding/`, `apps/platform/tests/Feature/SupportDiagnostics/`, `apps/platform/tests/Feature/Operations/`, and `apps/platform/tests/Feature/Reports/`
---
## Phase 2: Foundational (Blocking Prerequisites)
**Purpose**: Add the single new tenant-owned telemetry ledger, the bounded event catalog and recorder, retention scaffolding, and the aggregate summary query before any source seam is instrumented.
**Checkpoint**: The repo has one tenant-owned telemetry truth, one bounded recorder, one aggregate summary query, and one retention scaffold before story-specific source capture begins.
- [X] T003 Create the telemetry migration in `apps/platform/database/migrations/*_create_product_usage_events_table.php` with non-null `workspace_id` and `tenant_id`, actor reference, event name, feature area, subject reference, safe metadata JSON, `occurred_at`, and bounded indexes for dashboard windows and source lookups
- [X] T004 Create the telemetry model in `apps/platform/app/Models/ProductUsageEvent.php`
- [X] T005 [P] Create the telemetry factory in `apps/platform/database/factories/ProductUsageEventFactory.php`
- [X] T006 [P] Create the bounded event catalog in `apps/platform/app/Support/ProductTelemetry/ProductUsageEventCatalog.php`
- [X] T007 Create the shared recorder and safe-metadata normalizer in `apps/platform/app/Support/ProductTelemetry/ProductTelemetryRecorder.php`
- [X] T008 Create the aggregate summary query for the system dashboard in `apps/platform/app/Support/ProductTelemetry/ProductTelemetrySummaryQuery.php`, including the five visible event-family counts plus active-workspace participation for the selected window
- [X] T009 [P] Add a config-backed 90-day retention rule in `apps/platform/config/tenantpilot.php`, scaffold the `tenantpilot:product-usage:prune` command in `apps/platform/app/Console/Commands/PruneProductUsageEventsCommand.php`, and register its daily scheduler entry in `apps/platform/routes/console.php`
- [X] T010 [P] Add unit coverage for the event catalog, recorder legality, safe metadata rules, and summary query in `apps/platform/tests/Unit/Support/ProductTelemetry/ProductUsageEventCatalogTest.php`, `apps/platform/tests/Unit/Support/ProductTelemetry/ProductTelemetryRecorderTest.php`, `apps/platform/tests/Unit/Support/ProductTelemetry/ProductTelemetrySafeMetadataTest.php`, and `apps/platform/tests/Unit/Support/ProductTelemetry/ProductTelemetrySummaryQueryTest.php`
- [X] T011 Run the foundational unit suite with `cd apps/platform && ./vendor/bin/sail artisan test --compact tests/Unit/Support/ProductTelemetry/ProductUsageEventCatalogTest.php tests/Unit/Support/ProductTelemetry/ProductTelemetryRecorderTest.php tests/Unit/Support/ProductTelemetry/ProductTelemetrySafeMetadataTest.php tests/Unit/Support/ProductTelemetry/ProductTelemetrySummaryQueryTest.php` (tests: `apps/platform/tests/Unit/Support/ProductTelemetry/ProductUsageEventCatalogTest.php`)
---
## Phase 3: User Story 1 — Record High-Signal Tenant Usage Centrally (Priority: P1) 🎯 MVP
**Goal**: Record a bounded set of user-initiated tenant-bound milestones through one central contract instead of relying on scraped domain tables or local counters.
**Independent Test**: Trigger each supported source milestone once and verify that exactly one tenant-bound telemetry row is written with the expected bounded event name, subject reference, and safe metadata.
### Tests for User Story 1
- [X] T012 [P] [US1] Add onboarding capture coverage for checkpoint completion and the explicit non-capture rule for pre-tenant drafts in `apps/platform/tests/Feature/Onboarding/ProductTelemetryOnboardingCaptureTest.php`
- [X] T013 [P] [US1] Add support-diagnostics capture coverage for both tenant-dashboard and canonical run-detail entry points in `apps/platform/tests/Feature/SupportDiagnostics/ProductTelemetrySupportDiagnosticsCaptureTest.php`
- [X] T014 [P] [US1] Add operation-start capture coverage that records only user-initiated tenant-bound starts and ignores initiator-null or system-driven runs in `apps/platform/tests/Feature/Operations/ProductTelemetryOperationStartCaptureTest.php`
- [X] T015 [P] [US1] Add stored-report and review-pack capture coverage in `apps/platform/tests/Feature/Reports/ProductTelemetryReportCaptureTest.php`
### Implementation for User Story 1
- [X] T016 [US1] Instrument onboarding checkpoint completion in `apps/platform/app/Services/Onboarding/OnboardingLifecycleService.php` so telemetry writes only after a real tenant-linked checkpoint transition succeeds
- [X] T017 [US1] Instrument the support-diagnostics open action on `apps/platform/app/Filament/Pages/TenantDashboard.php` and `apps/platform/app/Filament/Pages/Operations/TenantlessOperationRunViewer.php` using the shared recorder after the authorized action succeeds
- [X] T018 [US1] Instrument user-initiated tenant-bound operation starts in `apps/platform/app/Services/OperationRunService.php` without changing the existing run-start UX contract
- [X] T019 [US1] Instrument stored-report creation and review-pack generation request seams in `apps/platform/app/Services/EntraAdminRoles/EntraAdminRolesReportService.php`, `apps/platform/app/Services/PermissionPosture/PermissionPostureFindingGenerator.php`, and `apps/platform/app/Services/ReviewPackService.php`
- [X] T020 [US1] Run the US1 suite with `cd apps/platform && ./vendor/bin/sail artisan test --compact tests/Feature/Onboarding/ProductTelemetryOnboardingCaptureTest.php tests/Feature/SupportDiagnostics/ProductTelemetrySupportDiagnosticsCaptureTest.php tests/Feature/Operations/ProductTelemetryOperationStartCaptureTest.php tests/Feature/Reports/ProductTelemetryReportCaptureTest.php` (tests: `apps/platform/tests/Feature/Onboarding/ProductTelemetryOnboardingCaptureTest.php`)
---
## Phase 4: User Story 2 — See Bounded Adoption Signals On The System Dashboard (Priority: P1)
**Goal**: Expose a read-only telemetry summary on the existing system dashboard so a platform operator can inspect recent adoption without a raw event browser.
**Independent Test**: Seed telemetry rows across a bounded time window and verify that the system dashboard widget renders the expected aggregate counts, zero state, and access behavior.
### Tests for User Story 2
- [X] T021 [P] [US2] Add widget coverage for the five visible event families, active-workspace participation, aggregate counts, zero state, and time-window behavior in `apps/platform/tests/Feature/System/ProductTelemetry/ProductTelemetryDashboardWidgetTest.php`
- [X] T022 [P] [US2] Add system-plane authorization coverage proving that only existing dashboard-eligible platform users can view aggregate telemetry in `apps/platform/tests/Feature/System/ProductTelemetry/ProductTelemetryAuthorizationTest.php`
### Implementation for User Story 2
- [X] T023 [US2] Create the read-only telemetry widget in `apps/platform/app/Filament/System/Widgets/ProductTelemetryKpis.php` with five visible family counters, active-workspace participation, and an explicit zero-state summary
- [X] T024 [US2] Register the widget on `apps/platform/app/Filament/System/Pages/Dashboard.php` and reuse the existing `SystemConsoleWindow` selection for telemetry windows
- [X] T025 [US2] Keep the widget aggregate-only and avoid any raw row list, tenant drilldown, or new dashboard action group in the first slice
- [X] T026 [US2] Run the US2 suite with `cd apps/platform && ./vendor/bin/sail artisan test --compact tests/Feature/System/ProductTelemetry/ProductTelemetryDashboardWidgetTest.php tests/Feature/System/ProductTelemetry/ProductTelemetryAuthorizationTest.php` (tests: `apps/platform/tests/Feature/System/ProductTelemetry/ProductTelemetryDashboardWidgetTest.php`)
---
## Phase 5: User Story 3 — Keep Telemetry Private, Tenant-Bound, And Cheap (Priority: P2)
**Goal**: Prevent the feature from becoming a second audit log, a page-view tracker, or a tenant-leaking analytics system.
**Independent Test**: Inspect stored telemetry rows and the prune path to confirm only safe tenant-bound metadata is stored and that old rows can be deleted without touching domain truth.
### Tests for User Story 3
- [X] T027 [P] [US3] Extend the source-capture feature tests in `apps/platform/tests/Feature/Onboarding/ProductTelemetryOnboardingCaptureTest.php`, `apps/platform/tests/Feature/SupportDiagnostics/ProductTelemetrySupportDiagnosticsCaptureTest.php`, `apps/platform/tests/Feature/Operations/ProductTelemetryOperationStartCaptureTest.php`, and `apps/platform/tests/Feature/Reports/ProductTelemetryReportCaptureTest.php` to assert emitted rows never contain forbidden free text, payload content, or email storage
- [X] T028 [P] [US3] Add retention and prune coverage in `apps/platform/tests/Feature/System/ProductTelemetry/ProductTelemetryRetentionTest.php`
- [X] T029 [P] [US3] Add a regression guard proving that passive render paths do not emit telemetry, disallowed stores such as `AuditLog` remain untouched, and no raw telemetry viewer exposure is introduced in `apps/platform/tests/Feature/System/ProductTelemetry/NoAdHocTelemetryBypassTest.php`
### Implementation for User Story 3
- [X] T030 [US3] Harden the source instrumentation in `apps/platform/app/Services/Onboarding/OnboardingLifecycleService.php`, `apps/platform/app/Filament/Pages/TenantDashboard.php`, `apps/platform/app/Filament/Pages/Operations/TenantlessOperationRunViewer.php`, `apps/platform/app/Services/OperationRunService.php`, `apps/platform/app/Services/EntraAdminRoles/EntraAdminRolesReportService.php`, `apps/platform/app/Services/PermissionPosture/PermissionPostureFindingGenerator.php`, and `apps/platform/app/Services/ReviewPackService.php` so each seam passes only bounded IDs, enums, booleans, and timestamps into the foundational recorder
- [X] T031 [US3] Implement the prune semantics inside `apps/platform/app/Console/Commands/PruneProductUsageEventsCommand.php` so the foundational command deletes only `ProductUsageEvent` rows older than the configured window and leaves source-of-truth tables untouched
- [X] T032 [US3] Add any necessary model or factory support for retention and safe-metadata assertions in `apps/platform/app/Models/ProductUsageEvent.php` and `apps/platform/database/factories/ProductUsageEventFactory.php`
- [X] T033 [US3] Run the US3 suite with `cd apps/platform && ./vendor/bin/sail artisan test --compact tests/Unit/Support/ProductTelemetry/ProductTelemetrySafeMetadataTest.php tests/Feature/System/ProductTelemetry/ProductTelemetryRetentionTest.php tests/Feature/System/ProductTelemetry/NoAdHocTelemetryBypassTest.php` (tests: `apps/platform/tests/Feature/System/ProductTelemetry/ProductTelemetryRetentionTest.php`)
---
## Phase 6: Polish & Cross-Cutting Concerns
**Purpose**: Lock down the bounded catalog, formatting, and the narrow validation suite before implementation close-out.
- [X] T034 [P] Confirm the bounded first-slice event catalog, the five visible dashboard labels, and the active-workspace participation metric stay aligned in `apps/platform/app/Support/ProductTelemetry/ProductUsageEventCatalog.php` and `apps/platform/app/Filament/System/Widgets/ProductTelemetryKpis.php`
- [X] T035 Run formatting on touched platform files with `cd apps/platform && ./vendor/bin/sail bin pint --dirty --format agent` (target: `apps/platform/`)
- [X] T036 Run the full narrow validation suite with `cd apps/platform && ./vendor/bin/sail artisan test --compact tests/Unit/Support/ProductTelemetry tests/Feature/System/ProductTelemetry tests/Feature/Onboarding/ProductTelemetryOnboardingCaptureTest.php tests/Feature/SupportDiagnostics/ProductTelemetrySupportDiagnosticsCaptureTest.php tests/Feature/Operations/ProductTelemetryOperationStartCaptureTest.php tests/Feature/Reports/ProductTelemetryReportCaptureTest.php`
---
## Dependencies & Execution Order
### User Story Dependency Graph
```text
Phase 1 (Setup)
↓
Phase 2 (Foundation: telemetry ledger + recorder + summary query + retention)
↓
US1 (source capture at onboarding / diagnostics / operations / reports) ─┐
├─→ US3 (privacy / retention / anti-bypass guards)
US2 (system dashboard aggregate visibility) ──────────────────────────────┘
```
### Parallel Opportunities
- Foundational tasks marked `[P]` can run in parallel once the event-table shape is agreed.
- US1 source-capture tests can be authored in parallel because onboarding, support diagnostics, operations, and reports touch different seams.
- US2 widget and authorization tests can run in parallel while the widget implementation is isolated to the system dashboard.
- US3 privacy, retention, and anti-bypass guard tasks can parallelize after the recorder contract is fixed.
---
## Parallel Example: User Story 1
```bash
Task: "Add onboarding capture coverage in apps/platform/tests/Feature/Onboarding/ProductTelemetryOnboardingCaptureTest.php"
Task: "Add support-diagnostics capture coverage in apps/platform/tests/Feature/SupportDiagnostics/ProductTelemetrySupportDiagnosticsCaptureTest.php"
Task: "Add operation-start capture coverage in apps/platform/tests/Feature/Operations/ProductTelemetryOperationStartCaptureTest.php"
Task: "Add stored-report and review-pack capture coverage in apps/platform/tests/Feature/Reports/ProductTelemetryReportCaptureTest.php"
```
---
## Parallel Example: User Story 2
```bash
Task: "Add widget coverage in apps/platform/tests/Feature/System/ProductTelemetry/ProductTelemetryDashboardWidgetTest.php"
Task: "Add dashboard authorization coverage in apps/platform/tests/Feature/System/ProductTelemetry/ProductTelemetryAuthorizationTest.php"
Task: "Create the widget in apps/platform/app/Filament/System/Widgets/ProductTelemetryKpis.php"
```
---
## Parallel Example: User Story 3
```bash
Task: "Add metadata guard coverage in apps/platform/tests/Unit/Support/ProductTelemetry/ProductTelemetrySafeMetadataTest.php"
Task: "Add retention coverage in apps/platform/tests/Feature/System/ProductTelemetry/ProductTelemetryRetentionTest.php"
Task: "Add anti-bypass guard coverage in apps/platform/tests/Feature/System/ProductTelemetry/NoAdHocTelemetryBypassTest.php"
Task: "Implement the prune command in apps/platform/app/Console/Commands/PruneProductUsageEventsCommand.php"
```
---
## Implementation Strategy
### MVP First (User Story 1)
1. Complete Phase 1 and Phase 2.
2. Deliver the bounded write path for real source milestones in US1.
3. Validate that the ledger remains tenant-bound and safe before surfacing it anywhere.
### Incremental Delivery
1. US1 introduces the central telemetry ledger and real source capture from existing services and actions.
2. US2 surfaces the new truth through one read-only system dashboard widget.
3. US3 adds retention, privacy guardrails, and anti-bypass regression protection.
4. Phase 6 runs formatting and the narrow validation suite only.
Reference in New Issue View Git Blame Copy Permalink