Adds browser fixture coverage for evidence system surfaces as described in Spec 376. Co-authored-by: Ahmed Darrazi <ahmed.darrazi@live.de> Reviewed-on: #447
128 lines
14 KiB
Markdown
128 lines
14 KiB
Markdown
# Tasks: Spec 376 - Browser Audit Fixture Coverage for Evidence/System Surfaces v1
|
|
|
|
**Input**: `specs/376-browser-audit-fixture-coverage-evidence-system-surfaces/spec.md`, `plan.md`, Spec 368 audit artifacts, Spec 353/372 browser evidence, Spec 375 follow-up recommendation, existing admin/system auth and browser fixture patterns.
|
|
|
|
**Tests**: Required for later implementation. Use Pest 4 Feature tests and Pest Browser. This spec does not authorize product UI redesign.
|
|
|
|
## Test Governance Checklist
|
|
|
|
- [x] Lane assignment is named and narrow: Feature tests for fixture/auth safety, Browser lane for five-surface smoke.
|
|
- [x] New or changed tests stay in the smallest honest family; broad platform browser audit is not added.
|
|
- [x] Workspace, environment, provider, evidence, platform-user, and session fixtures remain explicit and opt-in.
|
|
- [x] Planned validation commands cover fixture safety and target surfaces without pulling in unrelated suites.
|
|
- [x] Browser screenshots are required only for the five in-scope surfaces.
|
|
- [x] Any material browser runtime, timeout, or lane-cost note is recorded in `artifacts/validation-report.md`.
|
|
|
|
## Phase 1: Preparation And Repo Truth
|
|
|
|
**Purpose**: Confirm the current blockers and existing fixture/auth patterns before code changes.
|
|
|
|
- [x] T001 Re-read `specs/376-browser-audit-fixture-coverage-evidence-system-surfaces/spec.md`, `plan.md`, `tasks.md`, and `checklists/requirements.md`.
|
|
- [x] T002 Re-read Spec 368 inputs:
|
|
- `specs/368-platform-ui-signal-to-noise-browser-audit/audit.md`
|
|
- `specs/368-platform-ui-signal-to-noise-browser-audit/findings.md`
|
|
- `specs/368-platform-ui-signal-to-noise-browser-audit/page-scorecard.csv`
|
|
- `specs/368-platform-ui-signal-to-noise-browser-audit/artifacts/raw/browser-notes.md`
|
|
- `specs/368-platform-ui-signal-to-noise-browser-audit/artifacts/raw/routes.txt`
|
|
- [x] T003 Re-read current completed evidence for the in-scope admin surfaces, without modifying completed specs:
|
|
- `specs/353-provider-connections-resolution-guidance-v1/spec.md`
|
|
- `specs/353-provider-connections-resolution-guidance-v1/tasks.md`
|
|
- `specs/353-provider-connections-resolution-guidance-v1/artifacts/screenshots/...`
|
|
- `specs/372-customer-auditor-surface-safety-pass/artifacts/browser-verification-report.md`
|
|
- `specs/372-customer-auditor-surface-safety-pass/artifacts/screenshots/...`
|
|
- [x] T004 Re-read relevant Spec 370-375 artifacts and record unavailable artifacts as `not available`, without modifying completed specs.
|
|
- [x] T005 Inspect route truth with `cd apps/platform && ./vendor/bin/sail artisan route:list` or the closest available route-list command.
|
|
- [x] T006 Inspect existing admin smoke-login and fixture patterns in `apps/platform/routes/web.php`, `SeedReviewOutputBrowserFixture`, direct Browser `actingAs()`/workspace-session harnesses, Spec 353/372 browser tests, and their Feature tests.
|
|
- [x] T007 Inspect system auth separation in `SystemPanelProvider`, `PlatformUser`, `PlatformCapabilities`, `UseSystemSessionCookie`, `EnsurePlatformCapability`, existing system Browser `actingAs(..., 'platform')` usage, and system auth/access tests.
|
|
- [x] T008 Inspect existing Evidence Snapshot, Required Permissions, and Provider Connection route helpers/tests needed to resolve scoped URLs safely.
|
|
- [x] T009 Create `artifacts/source-audit-summary.md` with Spec 368 blockers, Spec 353/372 current browser evidence, related specs, route/auth/data status, fixture gaps, and selected implementation option.
|
|
- [x] T010 Create initial `artifacts/route-reachability-report.md` with route names, paths, middleware/auth guard, required parameters, current evidence source, and known redirect/login behavior.
|
|
|
|
## Phase 2: Fixture Design Before Implementation
|
|
|
|
**Purpose**: Make the fixture contract reviewable before adding code.
|
|
|
|
- [x] T011 Create `artifacts/fixture-design.md` with per-surface route, auth, scope, data, existing fixture source, new fixture source if needed, local/testing guarantee, browser URL, and failure mode.
|
|
- [x] T012 Create `artifacts/fixture-coverage-matrix.md` with rows for Evidence Snapshot View, Required Permissions, System Dashboard, System Operations, and Provider Connection Detail.
|
|
- [x] T013 Create `artifacts/affected-files.md` with planned files, purpose, change type, runtime/test/spec classification, surface, verification level, risk, and production impact.
|
|
- [x] T014 Create `artifacts/validation-report.md` with branch, HEAD, dirty state before implementation, planned commands, and no-product-UI-refactor assertion.
|
|
- [x] T015 Create `artifacts/screenshots/` and `artifacts/follow-up-recommendations.md` with placeholder sections for blocked surfaces, fixture gaps, UI productization follow-ups, final closeout audit readiness, and recommended next spec.
|
|
|
|
## Phase 3: Tests First - Fixture Safety
|
|
|
|
**Purpose**: Prove fixture routes and resolver behavior before relying on browser screenshots.
|
|
|
|
- [x] T016 Add or update focused Feature coverage proving existing admin smoke-login or direct Browser session fixtures can target the Evidence Snapshot View and Required Permissions fixture URLs while preserving workspace/environment context.
|
|
- [x] T017 If a system local/testing smoke fixture route is added, first add Feature coverage proving it returns 404 outside local/testing. *(N/A: no system smoke fixture route was added; existing platform-guard Pest Browser auth was sufficient.)*
|
|
- [x] T018 If a system local/testing smoke fixture route is added, add Feature coverage proving it authenticates only `PlatformUser` on the `platform` guard and never authenticates a tenant/admin `User` into `/system`. *(N/A: no system smoke fixture route was added; `SystemPanelAuthTest` and Spec 376 browser smoke cover platform guard access.)*
|
|
- [x] T019 If a system local/testing smoke fixture route is added, add Feature coverage proving `platform.access_system_panel` is required and missing capability remains forbidden for `/system`. *(N/A: no system smoke fixture route was added; existing `SystemPanelAuthTest` covers the capability boundary.)*
|
|
- [x] T020 If a system local/testing smoke fixture route is added, add Feature coverage proving redirect targets are limited to safe local app paths and external or cross-plane targets are rejected. *(N/A: no system smoke fixture route was added, so no new redirect surface exists.)*
|
|
- [x] T021 If an Artisan fixture command is added or extended, add coverage proving it fails closed outside local/testing and does not mutate data there. *(N/A: no Artisan fixture command was added or extended.)*
|
|
- [x] T022 Add or update focused fixture/resolver tests proving Evidence Snapshot fixture data belongs to the selected workspace/environment and unauthorized cross-scope data is not used, unless existing Spec 372 evidence is recorded as sufficient with no new fixture code. *(Existing `EvidenceSnapshotResourceTest` plus Spec 376 browser fixture prove scoped access; no product fixture resolver was added.)*
|
|
- [x] T023 Add or update focused fixture/resolver tests proving Required Permissions fixture data belongs to the selected workspace/environment/provider context, unless existing Spec 353/283 evidence is recorded as sufficient with no new fixture code. *(Existing `RequiredPermissionsAccessTest` plus Spec 376 seeded permission rows prove scoped route access; no product fixture resolver was added.)*
|
|
- [x] T024 Add or update focused fixture/resolver tests proving Provider Connection Detail fixture data is scoped to the selected workspace/provider authority, uses explicit `environment_id` or record-derived managed-environment authority, and does not depend on stale hidden environment state. *(Existing Spec 353 provider guidance tests plus Spec 376 browser URL with explicit `environment_id` prove the selected path.)*
|
|
|
|
## Phase 4: Minimal Fixture Implementation
|
|
|
|
**Purpose**: Implement only the narrow local/testing support required for browser reachability.
|
|
|
|
- [x] T025 Reuse existing `/admin/local/smoke-login` or direct Browser `actingAs()`/workspace-session patterns for admin-plane surfaces before adding new admin fixture routes.
|
|
- [x] T026 Reuse or extend existing review-output browser fixture setup or Spec 372-compatible Evidence Snapshot browser fixture setup if it already produces the required snapshot and route context.
|
|
- [x] T027 Reuse existing Required Permissions factories/provider readiness setup and Spec 353/283 evidence before adding new fixture config.
|
|
- [x] T028 Reuse existing Provider Connection factories/fixture setup and explicit `environment_id` URL helpers before adding new fixture config.
|
|
- [x] T029 If existing system Browser `actingAs(..., 'platform')` patterns cannot produce the required evidence and system browser ergonomics require it, add a local/testing-only system smoke fixture route guarded by `app()->environment(['local', 'testing'])`. *(N/A: existing system Browser `actingAs(..., 'platform')` produced the required evidence.)*
|
|
- [x] T030 If system fixture route is added, ensure it uses `auth('platform')`, `UseSystemSessionCookie`/system session semantics where needed, `PlatformUser`, and platform capabilities only. *(N/A: no system fixture route was added.)*
|
|
- [x] T031 Validate fixture route redirects centrally or locally with an allowlist of relative/admin/system paths; reject arbitrary URLs. *(N/A for system route; existing admin smoke-login redirect behavior was reused unchanged and remains covered by existing Feature tests.)*
|
|
- [x] T032 Ensure no production route, provider, policy, model, migration, Graph contract, OperationRun behavior, or product UI copy/layout changes are included.
|
|
|
|
## Phase 5: Browser Smoke And Screenshots
|
|
|
|
**Purpose**: Produce reproducible browser evidence or exact blockers.
|
|
|
|
- [x] T033 Add a bounded Pest Browser test under `apps/platform/tests/Browser/` for Spec 376 fixture coverage or document which existing browser tests provide current evidence for a surface.
|
|
- [x] T034 Browser-open Evidence Snapshot View through the local/testing fixture and save `artifacts/screenshots/001-evidence-snapshot-view.png`, or record existing Spec 372 screenshot evidence / blocked screenshot / note.
|
|
- [x] T035 Browser-open Required Permissions through the local/testing fixture and save `artifacts/screenshots/002-required-permissions.png`, or record existing Spec 353/283 screenshot evidence / blocked screenshot / note.
|
|
- [x] T036 Browser-open System Dashboard through platform-plane fixture auth and save `artifacts/screenshots/003-system-dashboard.png`, or blocked screenshot/note.
|
|
- [x] T037 Browser-open System Operations through platform-plane fixture auth and save `artifacts/screenshots/004-system-operations.png`, or blocked screenshot/note.
|
|
- [x] T038 Browser-open Provider Connection Detail through scoped admin fixture auth with explicit `environment_id` or record-derived managed-environment authority and save `artifacts/screenshots/005-provider-connection-detail.png`, or record existing Spec 353 screenshot evidence / blocked screenshot / note.
|
|
- [x] T039 For every reachable page, assert no JavaScript errors and no console logs using Pest Browser assertions.
|
|
- [x] T040 For every blocked page, record final URL, redirect/login path, blocker class, and whether a screenshot was captured. *(N/A: no in-scope page remained blocked after the passing browser smoke.)*
|
|
|
|
## Phase 6: Reports And Validation
|
|
|
|
**Purpose**: Close the feature with evidence, limitations, and no hidden implementation drift.
|
|
|
|
- [x] T041 Complete `artifacts/browser-verification-report.md` with URL, panel, fixture used, current evidence source, expected outcome, actual outcome, final URL, screenshot path, explicit FR-376-014 verification/blocker label, and notes for all surfaces.
|
|
- [x] T042 Complete `artifacts/screenshot-index.md` with screenshot path, reachable yes/no, blocked reason, and notes.
|
|
- [x] T043 Complete `artifacts/fixture-coverage-matrix.md` with implemented fixture, reachability result, verification level, and remaining limitation.
|
|
- [x] T044 Complete `artifacts/route-reachability-report.md` with final HTTP/browser outcomes.
|
|
- [x] T045 Complete `artifacts/affected-files.md` with actual changed files and production impact.
|
|
- [x] T046 Complete `artifacts/validation-report.md` with commands run, test/browser results, screenshots, limitations, dirty state after implementation, and whether closeout audit can proceed.
|
|
- [x] T047 Complete `artifacts/follow-up-recommendations.md` with any remaining blockers and recommended next spec, defaulting to Spec 377 - Post-Productization Browser Re-Audit & Closeout Gate v1 if no narrower blocker remains.
|
|
- [x] T048 Run `git diff --check`.
|
|
- [x] T049 If PHP files changed, run `cd apps/platform && ./vendor/bin/pint --dirty`.
|
|
- [x] T050 Run targeted Feature tests selected by changed files, including auth/system/evidence/required-permissions/provider coverage.
|
|
- [x] T051 Run the Spec 376 Pest Browser smoke or document the exact reason it cannot run locally.
|
|
|
|
## Non-Goals Checklist
|
|
|
|
- [x] NT001 Do not redesign Evidence Snapshot, Required Permissions, System Dashboard, System Operations, or Provider Connection Detail.
|
|
- [x] NT002 Do not add migrations, models, product persisted truth, product enum/status families, Graph contracts, jobs, policies, or OperationRun behavior.
|
|
- [x] NT003 Do not weaken production auth, add production smoke login, or authenticate tenant users into the system panel.
|
|
- [x] NT004 Do not create real customer data or depend on fragile hardcoded local IDs.
|
|
- [x] NT005 Do not change UI bloat guard rules or run a full platform-wide browser audit.
|
|
- [x] NT006 Do not rewrite completed historical specs or remove implementation close-out/validation/browser evidence.
|
|
|
|
## Dependencies And Execution Order
|
|
|
|
- Phase 1 must complete before fixture design.
|
|
- Phase 2 must complete before code edits.
|
|
- Phase 3 tests must precede or accompany fixture implementation.
|
|
- Phase 4 keeps implementation minimal and local/testing-only.
|
|
- Phase 5 produces browser evidence.
|
|
- Phase 6 validates and closes artifacts.
|
|
|
|
## Recommended Implementation Strategy
|
|
|
|
Start by inventorying current Spec 353, Spec 372, and Spec 283 browser evidence, then prove only remaining admin-plane gaps through existing Browser `actingAs()` / workspace-session patterns, `/admin/local/smoke-login`, and fixture seed commands. Handle system panel separately with platform guard semantics; only add a system smoke-login fixture if Pest Browser cannot reliably use existing `actingAs($platformUser, 'platform')` patterns for screenshots. Treat any UI/product findings discovered during screenshots as follow-up recommendations, not in-scope fixes.
|