ahmido/TenantAtlas
1
0
Fork 0
Code Issues Pull Requests 1 Actions Packages Projects 1 Releases Wiki Activity
f38b8884ff
TenantAtlas/specs/243-product-usage-adoption-telemetry/tasks.md
Ahmed Darrazi f38b8884ff
Some checks failed
PR Fast Feedback / fast-feedback (pull_request) Failing after 52s
Details
feat(product-telemetry): implement spec 243 - product usage adoption telemetry
2026-04-26 22:46:32 +02:00

16 KiB
Raw Blame History

description
Task list for Product Usage & Adoption Telemetry

Tasks: Product Usage & Adoption Telemetry

Input: Design documents from /Users/ahmeddarrazi/Documents/projects/wt-plattform/specs/243-product-usage-adoption-telemetry/
Prerequisites: /Users/ahmeddarrazi/Documents/projects/wt-plattform/specs/243-product-usage-adoption-telemetry/plan.md (required), /Users/ahmeddarrazi/Documents/projects/wt-plattform/specs/243-product-usage-adoption-telemetry/spec.md (required), /Users/ahmeddarrazi/Documents/projects/wt-plattform/specs/243-product-usage-adoption-telemetry/checklists/requirements.md (required)

Tests: REQUIRED (Pest) for all runtime behavior changes in this slice. Keep proof in Unit + Feature lanes only.
Operations: This slice must not alter existing OperationRun UX or lifecycle. It may observe successful user-initiated tenant-bound operation starts as telemetry sources only after the current start contract succeeds.
RBAC: Telemetry writes occur only after the originating admin-plane action has already passed workspace membership, tenant entitlement, and capability checks. Aggregate reads remain system-plane only through the existing dashboard access rules.
Organization: Tasks are grouped by user story so the write path, read path, and privacy or isolation guardrails remain independently testable.

Phase 1: Setup (Shared Infrastructure)

Purpose: Prepare the bounded product-telemetry namespace and the local validation surfaces without widening scope.

  • T001 Start the local Sail environment with cd apps/platform && ./vendor/bin/sail up -d (script: apps/platform/vendor/bin/sail)
  • T002 Create the bounded feature-local directories under apps/platform/app/Support/ProductTelemetry/, apps/platform/tests/Unit/Support/ProductTelemetry/, apps/platform/tests/Feature/System/ProductTelemetry/, apps/platform/tests/Feature/Onboarding/, apps/platform/tests/Feature/SupportDiagnostics/, apps/platform/tests/Feature/Operations/, and apps/platform/tests/Feature/Reports/

Phase 2: Foundational (Blocking Prerequisites)

Purpose: Add the single new tenant-owned telemetry ledger, the bounded event catalog and recorder, retention scaffolding, and the aggregate summary query before any source seam is instrumented.

Checkpoint: The repo has one tenant-owned telemetry truth, one bounded recorder, one aggregate summary query, and one retention scaffold before story-specific source capture begins.

  • T003 Create the telemetry migration in apps/platform/database/migrations/*_create_product_usage_events_table.php with non-null workspace_id and tenant_id, actor reference, event name, feature area, subject reference, safe metadata JSON, occurred_at, and bounded indexes for dashboard windows and source lookups
  • T004 Create the telemetry model in apps/platform/app/Models/ProductUsageEvent.php
  • T005 [P] Create the telemetry factory in apps/platform/database/factories/ProductUsageEventFactory.php
  • T006 [P] Create the bounded event catalog in apps/platform/app/Support/ProductTelemetry/ProductUsageEventCatalog.php
  • T007 Create the shared recorder and safe-metadata normalizer in apps/platform/app/Support/ProductTelemetry/ProductTelemetryRecorder.php
  • T008 Create the aggregate summary query for the system dashboard in apps/platform/app/Support/ProductTelemetry/ProductTelemetrySummaryQuery.php, including the five visible event-family counts plus active-workspace participation for the selected window
  • T009 [P] Add a config-backed 90-day retention rule in apps/platform/config/tenantpilot.php, scaffold the tenantpilot:product-usage:prune command in apps/platform/app/Console/Commands/PruneProductUsageEventsCommand.php, and register its daily scheduler entry in apps/platform/routes/console.php
  • T010 [P] Add unit coverage for the event catalog, recorder legality, safe metadata rules, and summary query in apps/platform/tests/Unit/Support/ProductTelemetry/ProductUsageEventCatalogTest.php, apps/platform/tests/Unit/Support/ProductTelemetry/ProductTelemetryRecorderTest.php, apps/platform/tests/Unit/Support/ProductTelemetry/ProductTelemetrySafeMetadataTest.php, and apps/platform/tests/Unit/Support/ProductTelemetry/ProductTelemetrySummaryQueryTest.php
  • T011 Run the foundational unit suite with cd apps/platform && ./vendor/bin/sail artisan test --compact tests/Unit/Support/ProductTelemetry/ProductUsageEventCatalogTest.php tests/Unit/Support/ProductTelemetry/ProductTelemetryRecorderTest.php tests/Unit/Support/ProductTelemetry/ProductTelemetrySafeMetadataTest.php tests/Unit/Support/ProductTelemetry/ProductTelemetrySummaryQueryTest.php (tests: apps/platform/tests/Unit/Support/ProductTelemetry/ProductUsageEventCatalogTest.php)

Phase 3: User Story 1 — Record High-Signal Tenant Usage Centrally (Priority: P1) 🎯 MVP

Goal: Record a bounded set of user-initiated tenant-bound milestones through one central contract instead of relying on scraped domain tables or local counters.

Independent Test: Trigger each supported source milestone once and verify that exactly one tenant-bound telemetry row is written with the expected bounded event name, subject reference, and safe metadata.

Tests for User Story 1

  • T012 [P] [US1] Add onboarding capture coverage for checkpoint completion and the explicit non-capture rule for pre-tenant drafts in apps/platform/tests/Feature/Onboarding/ProductTelemetryOnboardingCaptureTest.php
  • T013 [P] [US1] Add support-diagnostics capture coverage for both tenant-dashboard and canonical run-detail entry points in apps/platform/tests/Feature/SupportDiagnostics/ProductTelemetrySupportDiagnosticsCaptureTest.php
  • T014 [P] [US1] Add operation-start capture coverage that records only user-initiated tenant-bound starts and ignores initiator-null or system-driven runs in apps/platform/tests/Feature/Operations/ProductTelemetryOperationStartCaptureTest.php
  • T015 [P] [US1] Add stored-report and review-pack capture coverage in apps/platform/tests/Feature/Reports/ProductTelemetryReportCaptureTest.php

Implementation for User Story 1

  • T016 [US1] Instrument onboarding checkpoint completion in apps/platform/app/Services/Onboarding/OnboardingLifecycleService.php so telemetry writes only after a real tenant-linked checkpoint transition succeeds
  • T017 [US1] Instrument the support-diagnostics open action on apps/platform/app/Filament/Pages/TenantDashboard.php and apps/platform/app/Filament/Pages/Operations/TenantlessOperationRunViewer.php using the shared recorder after the authorized action succeeds
  • T018 [US1] Instrument user-initiated tenant-bound operation starts in apps/platform/app/Services/OperationRunService.php without changing the existing run-start UX contract
  • T019 [US1] Instrument stored-report creation and review-pack generation request seams in apps/platform/app/Services/EntraAdminRoles/EntraAdminRolesReportService.php, apps/platform/app/Services/PermissionPosture/PermissionPostureFindingGenerator.php, and apps/platform/app/Services/ReviewPackService.php
  • T020 [US1] Run the US1 suite with cd apps/platform && ./vendor/bin/sail artisan test --compact tests/Feature/Onboarding/ProductTelemetryOnboardingCaptureTest.php tests/Feature/SupportDiagnostics/ProductTelemetrySupportDiagnosticsCaptureTest.php tests/Feature/Operations/ProductTelemetryOperationStartCaptureTest.php tests/Feature/Reports/ProductTelemetryReportCaptureTest.php (tests: apps/platform/tests/Feature/Onboarding/ProductTelemetryOnboardingCaptureTest.php)

Phase 4: User Story 2 — See Bounded Adoption Signals On The System Dashboard (Priority: P1)

Goal: Expose a read-only telemetry summary on the existing system dashboard so a platform operator can inspect recent adoption without a raw event browser.

Independent Test: Seed telemetry rows across a bounded time window and verify that the system dashboard widget renders the expected aggregate counts, zero state, and access behavior.

Tests for User Story 2

  • T021 [P] [US2] Add widget coverage for the five visible event families, active-workspace participation, aggregate counts, zero state, and time-window behavior in apps/platform/tests/Feature/System/ProductTelemetry/ProductTelemetryDashboardWidgetTest.php
  • T022 [P] [US2] Add system-plane authorization coverage proving that only existing dashboard-eligible platform users can view aggregate telemetry in apps/platform/tests/Feature/System/ProductTelemetry/ProductTelemetryAuthorizationTest.php

Implementation for User Story 2

  • T023 [US2] Create the read-only telemetry widget in apps/platform/app/Filament/System/Widgets/ProductTelemetryKpis.php with five visible family counters, active-workspace participation, and an explicit zero-state summary
  • T024 [US2] Register the widget on apps/platform/app/Filament/System/Pages/Dashboard.php and reuse the existing SystemConsoleWindow selection for telemetry windows
  • T025 [US2] Keep the widget aggregate-only and avoid any raw row list, tenant drilldown, or new dashboard action group in the first slice
  • T026 [US2] Run the US2 suite with cd apps/platform && ./vendor/bin/sail artisan test --compact tests/Feature/System/ProductTelemetry/ProductTelemetryDashboardWidgetTest.php tests/Feature/System/ProductTelemetry/ProductTelemetryAuthorizationTest.php (tests: apps/platform/tests/Feature/System/ProductTelemetry/ProductTelemetryDashboardWidgetTest.php)

Phase 5: User Story 3 — Keep Telemetry Private, Tenant-Bound, And Cheap (Priority: P2)

Goal: Prevent the feature from becoming a second audit log, a page-view tracker, or a tenant-leaking analytics system.

Independent Test: Inspect stored telemetry rows and the prune path to confirm only safe tenant-bound metadata is stored and that old rows can be deleted without touching domain truth.

Tests for User Story 3

  • T027 [P] [US3] Extend the source-capture feature tests in apps/platform/tests/Feature/Onboarding/ProductTelemetryOnboardingCaptureTest.php, apps/platform/tests/Feature/SupportDiagnostics/ProductTelemetrySupportDiagnosticsCaptureTest.php, apps/platform/tests/Feature/Operations/ProductTelemetryOperationStartCaptureTest.php, and apps/platform/tests/Feature/Reports/ProductTelemetryReportCaptureTest.php to assert emitted rows never contain forbidden free text, payload content, or email storage
  • T028 [P] [US3] Add retention and prune coverage in apps/platform/tests/Feature/System/ProductTelemetry/ProductTelemetryRetentionTest.php
  • T029 [P] [US3] Add a regression guard proving that passive render paths do not emit telemetry, disallowed stores such as AuditLog remain untouched, and no raw telemetry viewer exposure is introduced in apps/platform/tests/Feature/System/ProductTelemetry/NoAdHocTelemetryBypassTest.php

Implementation for User Story 3

  • T030 [US3] Harden the source instrumentation in apps/platform/app/Services/Onboarding/OnboardingLifecycleService.php, apps/platform/app/Filament/Pages/TenantDashboard.php, apps/platform/app/Filament/Pages/Operations/TenantlessOperationRunViewer.php, apps/platform/app/Services/OperationRunService.php, apps/platform/app/Services/EntraAdminRoles/EntraAdminRolesReportService.php, apps/platform/app/Services/PermissionPosture/PermissionPostureFindingGenerator.php, and apps/platform/app/Services/ReviewPackService.php so each seam passes only bounded IDs, enums, booleans, and timestamps into the foundational recorder
  • T031 [US3] Implement the prune semantics inside apps/platform/app/Console/Commands/PruneProductUsageEventsCommand.php so the foundational command deletes only ProductUsageEvent rows older than the configured window and leaves source-of-truth tables untouched
  • T032 [US3] Add any necessary model or factory support for retention and safe-metadata assertions in apps/platform/app/Models/ProductUsageEvent.php and apps/platform/database/factories/ProductUsageEventFactory.php
  • T033 [US3] Run the US3 suite with cd apps/platform && ./vendor/bin/sail artisan test --compact tests/Unit/Support/ProductTelemetry/ProductTelemetrySafeMetadataTest.php tests/Feature/System/ProductTelemetry/ProductTelemetryRetentionTest.php tests/Feature/System/ProductTelemetry/NoAdHocTelemetryBypassTest.php (tests: apps/platform/tests/Feature/System/ProductTelemetry/ProductTelemetryRetentionTest.php)

Phase 6: Polish & Cross-Cutting Concerns

Purpose: Lock down the bounded catalog, formatting, and the narrow validation suite before implementation close-out.

  • T034 [P] Confirm the bounded first-slice event catalog, the five visible dashboard labels, and the active-workspace participation metric stay aligned in apps/platform/app/Support/ProductTelemetry/ProductUsageEventCatalog.php and apps/platform/app/Filament/System/Widgets/ProductTelemetryKpis.php
  • T035 Run formatting on touched platform files with cd apps/platform && ./vendor/bin/sail bin pint --dirty --format agent (target: apps/platform/)
  • T036 Run the full narrow validation suite with cd apps/platform && ./vendor/bin/sail artisan test --compact tests/Unit/Support/ProductTelemetry tests/Feature/System/ProductTelemetry tests/Feature/Onboarding/ProductTelemetryOnboardingCaptureTest.php tests/Feature/SupportDiagnostics/ProductTelemetrySupportDiagnosticsCaptureTest.php tests/Feature/Operations/ProductTelemetryOperationStartCaptureTest.php tests/Feature/Reports/ProductTelemetryReportCaptureTest.php

Dependencies & Execution Order

User Story Dependency Graph

Phase 1 (Setup)
  ↓
Phase 2 (Foundation: telemetry ledger + recorder + summary query + retention)
  ↓
US1 (source capture at onboarding / diagnostics / operations / reports) ─┐
                                                                          ├─→ US3 (privacy / retention / anti-bypass guards)
US2 (system dashboard aggregate visibility) ──────────────────────────────┘

Parallel Opportunities

  • Foundational tasks marked [P] can run in parallel once the event-table shape is agreed.
  • US1 source-capture tests can be authored in parallel because onboarding, support diagnostics, operations, and reports touch different seams.
  • US2 widget and authorization tests can run in parallel while the widget implementation is isolated to the system dashboard.
  • US3 privacy, retention, and anti-bypass guard tasks can parallelize after the recorder contract is fixed.

Parallel Example: User Story 1

Task: "Add onboarding capture coverage in apps/platform/tests/Feature/Onboarding/ProductTelemetryOnboardingCaptureTest.php"
Task: "Add support-diagnostics capture coverage in apps/platform/tests/Feature/SupportDiagnostics/ProductTelemetrySupportDiagnosticsCaptureTest.php"
Task: "Add operation-start capture coverage in apps/platform/tests/Feature/Operations/ProductTelemetryOperationStartCaptureTest.php"
Task: "Add stored-report and review-pack capture coverage in apps/platform/tests/Feature/Reports/ProductTelemetryReportCaptureTest.php"

Parallel Example: User Story 2

Task: "Add widget coverage in apps/platform/tests/Feature/System/ProductTelemetry/ProductTelemetryDashboardWidgetTest.php"
Task: "Add dashboard authorization coverage in apps/platform/tests/Feature/System/ProductTelemetry/ProductTelemetryAuthorizationTest.php"
Task: "Create the widget in apps/platform/app/Filament/System/Widgets/ProductTelemetryKpis.php"

Parallel Example: User Story 3

Task: "Add metadata guard coverage in apps/platform/tests/Unit/Support/ProductTelemetry/ProductTelemetrySafeMetadataTest.php"
Task: "Add retention coverage in apps/platform/tests/Feature/System/ProductTelemetry/ProductTelemetryRetentionTest.php"
Task: "Add anti-bypass guard coverage in apps/platform/tests/Feature/System/ProductTelemetry/NoAdHocTelemetryBypassTest.php"
Task: "Implement the prune command in apps/platform/app/Console/Commands/PruneProductUsageEventsCommand.php"

Implementation Strategy

MVP First (User Story 1)

  1. Complete Phase 1 and Phase 2.
  2. Deliver the bounded write path for real source milestones in US1.
  3. Validate that the ledger remains tenant-bound and safe before surfacing it anywhere.

Incremental Delivery

  1. US1 introduces the central telemetry ledger and real source capture from existing services and actions.
  2. US2 surfaces the new truth through one read-only system dashboard widget.
  3. US3 adds retention, privacy guardrails, and anti-bypass regression protection.
  4. Phase 6 runs formatting and the narrow validation suite only.