ahmido
3c38192405
## Summary This PR implements Spec 206 end to end and establishes the first checked-in test suite governance foundation for the platform app. Key changes: - add manifest-backed test lanes for fast-feedback, confidence, browser, heavy-governance, profiling, and junit - add budget and report helpers plus app-local artifact generation under `apps/platform/storage/logs/test-lanes` - add repo-root Sail-friendly lane/report wrappers - switch the default contributor test path to the fast-feedback lane - introduce explicit fixture profiles and cheaper defaults for shared tenant/provider test setup - add minimal/heavy factory states for tenant and provider connection setup - migrate the first high-usage and provider-sensitive tests to explicit fixture profiles - document budgets, taxonomy rules, DB reset guidance, and the full Spec 206 plan/contracts/tasks set ## Validation Executed during implementation: - focused Spec 206 guard/support/factory validation pack: 31 passed - provider-sensitive regression pack: 29 passed - first high-usage caller migration pack: 120 passed - lane routing and wrapper validation succeeded - pint completed successfully Measured lane baselines captured in docs: - fast-feedback: 176.74s - confidence: 394.38s - heavy-governance: 83.66s - browser: 128.87s - junit: 380.14s - profiling: 2701.51s - full-suite baseline anchor: 2624.60s ## Notes - Livewire v4 / Filament v5 runtime behavior is unchanged by this PR. - No new runtime routes, product UI flows, or database migrations are introduced. - Panel provider registration remains unchanged in `bootstrap/providers.php`. Co-authored-by: Ahmed Darrazi <ahmed.darrazi@live.de> Reviewed-on: #239
127 lines
8.8 KiB
Markdown
127 lines
8.8 KiB
Markdown
# Quickstart: Test Suite Governance & Performance Foundation
|
|
|
|
## Goal
|
|
|
|
Establish a fast default contributor run, a broader confidence run that owns the non-browser feature or integration middle path, dedicated browser and heavy-governance lanes, visible slow-test reporting, and cheap shared defaults for the most widely used fixture paths.
|
|
|
|
## Implementation Order
|
|
|
|
1. Introduce the checked-in lane manifest or equivalent selector map that combines suites, directories, groups, and explicit files.
|
|
2. Add the checked-in lane command entry points for fast-feedback, confidence, browser, heavy-governance, profiling, and JUnit reporting.
|
|
3. Align the default contributor run with fast-feedback and define the confidence lane as the Unit suite plus the manifest-defined non-browser Feature or Integration selectors outside explicit heavy-governance exclusions.
|
|
4. Seed the heavy-governance lane only with the obvious initial families, such as browser-adjacent governance scans, architecture, deprecation, and the first obvious guard-heavy clusters.
|
|
5. Wire artifact output to `storage/logs/test-lanes` and emit a budget-aware summary for each reporting path.
|
|
6. Add focused guard tests that verify browser exclusion, valid lane declarations, confidence-lane scope, and stable report semantics.
|
|
7. Run the first serial profiling pass and capture the current full-suite baseline plus the first measured lane baselines.
|
|
8. Refine the heavy-governance selectors from the profiling evidence and publish the initial runtime budgets.
|
|
9. Split `createUserWithTenant()` into a cheap default profile and an explicit heavy or provider-enabled profile.
|
|
10. Introduce minimal and heavy states for the next most expensive factory or fixture cluster touched in this slice.
|
|
11. Audit the obvious taxonomy misfits surfaced during rollout and reclassify or regroup the first batch.
|
|
12. Document the honest taxonomy rules, DB reset strategy, seed restrictions, and deferred schema-baseline evaluation rules.
|
|
|
|
## Suggested Code Touches
|
|
|
|
```text
|
|
apps/platform/composer.json
|
|
apps/platform/phpunit.xml
|
|
apps/platform/phpunit.pgsql.xml
|
|
apps/platform/tests/Pest.php
|
|
apps/platform/tests/TestCase.php
|
|
apps/platform/tests/Feature/Guards/*
|
|
apps/platform/tests/Architecture/*
|
|
apps/platform/tests/Deprecation/*
|
|
apps/platform/tests/Feature/OpsUx/*
|
|
apps/platform/tests/Browser/*
|
|
apps/platform/tests/Support/*
|
|
apps/platform/database/factories/*
|
|
scripts/platform-test-lane
|
|
scripts/platform-test-report
|
|
```
|
|
|
|
## Validation Flow
|
|
|
|
After the checked-in commands exist, validate the end-state workflow with the canonical repo-root wrappers:
|
|
|
|
```bash
|
|
./scripts/platform-test-lane fast-feedback
|
|
./scripts/platform-test-lane confidence
|
|
./scripts/platform-test-lane heavy-governance
|
|
./scripts/platform-test-lane browser
|
|
./scripts/platform-test-lane profiling
|
|
./scripts/platform-test-lane junit
|
|
./scripts/platform-test-report fast-feedback
|
|
./scripts/platform-test-report confidence
|
|
./scripts/platform-test-report heavy-governance
|
|
./scripts/platform-test-report browser
|
|
./scripts/platform-test-report profiling
|
|
./scripts/platform-test-report junit
|
|
cd apps/platform && ./vendor/bin/sail bin pint --dirty --format agent
|
|
```
|
|
|
|
The app-local Sail Composer commands remain valid, but the repo-root wrapper is the safer default for long lanes because it forces Composer to run with `--timeout=0`.
|
|
|
|
During implementation, keep the feedback loop tight by running the most relevant focused suites first:
|
|
|
|
```bash
|
|
cd apps/platform && ./vendor/bin/sail artisan test --compact tests/Feature/Guards
|
|
cd apps/platform && ./vendor/bin/sail artisan test --compact tests/Unit
|
|
cd apps/platform && ./vendor/bin/sail artisan test --compact tests/Browser
|
|
cd apps/platform && ./vendor/bin/sail artisan test --compact -c phpunit.pgsql.xml
|
|
```
|
|
|
|
## Recorded Baselines
|
|
|
|
These measurements were captured from the checked-in lane wrappers on the standard local Sail environment used for Spec 206 rollout.
|
|
|
|
| Scope | Wall clock | Budget | Status |
|
|
|-------|------------|--------|--------|
|
|
| Full suite baseline | `2624.60s` | reference only | Baseline anchor for lane targets |
|
|
| `fast-feedback` | `176.74s` | `200s` | Within budget and more than 50% below the full-suite baseline |
|
|
| `confidence` | `394.38s` | `450s` | Within budget and below the full-suite baseline |
|
|
| `heavy-governance` | `83.66s` | `120s` | Within budget |
|
|
| `browser` | `128.87s` | `150s` | Within budget |
|
|
| `junit` | `380.14s` | `450s` | Within budget |
|
|
| `profiling` | `2701.51s` | `3000s` | Within budget for the intentional serial drift run |
|
|
|
|
The first budget-backed family thresholds are:
|
|
|
|
- `ops-ux-governance`: `120s`
|
|
- `browser-smoke`: `150s`
|
|
|
|
## Taxonomy and Fixture Rules
|
|
|
|
- `Unit` owns isolated logic and helper coverage.
|
|
- `Feature` owns HTTP, Livewire, Filament, jobs, and non-browser integration slices.
|
|
- `Browser` owns only end-to-end browser smoke and workflow coverage.
|
|
- `heavy-governance` owns the first intentionally expensive families: `tests/Architecture`, `tests/Deprecation`, `tests/Feature/078`, `tests/Feature/090`, `tests/Feature/144`, `tests/Feature/OpsUx`, `tests/Feature/Filament/Alerts/AlertsKpiHeaderTest.php`, `tests/Feature/Guards/ActionSurfaceContractTest.php`, `tests/Feature/Guards/OperationLifecycleOpsUxGuardTest.php`, and `tests/Feature/ProviderConnections/CredentialLeakGuardTest.php`.
|
|
- `createUserWithTenant()` defaults to the cheap `minimal` profile.
|
|
- Use `createMinimalUserWithTenant()` for high-usage membership-focused callers.
|
|
- Use `provider-enabled`, `credential-enabled`, `ui-context`, or `heavy` only when the test truly needs those side effects.
|
|
- Default lanes stay on SQLite `:memory:` plus `RefreshDatabase`.
|
|
- The `Pgsql` suite stays isolated for schema and foreign-key assertions.
|
|
- Seeds stay opt-in. Do not make seeded state part of the default lane contract.
|
|
- Schema-baseline or dump acceleration remains follow-up work, not part of the first-slice lane contract.
|
|
|
|
## Manual Review Checklist
|
|
|
|
1. Confirm the default contributor command excludes browser and explicitly heavy governance families.
|
|
2. Confirm the confidence command covers the Unit suite plus the manifest-defined non-browser Feature or Integration selectors while still excluding the browser lane and explicit heavy-governance exclusions, and that its measured runtime remains below the current full-suite baseline.
|
|
3. Confirm each lane declaration records its purpose, intended audience, included families, excluded families, and ownership expectations.
|
|
4. Confirm the profiling support-lane entry runs serially and emits a ranked view with at least the top 10 slowest entries through the shared reporting envelope.
|
|
5. Confirm the JUnit support-lane entry writes machine-readable output under `storage/logs/test-lanes` through the same manifest and report contract shape.
|
|
6. Confirm the cheap default shared helper path no longer provisions workspace, membership, credentials, provider connection state, session context, cache-clearing side effects, or UI context unless explicitly requested.
|
|
7. Confirm at least one additional factory or fixture cluster exposes minimal versus heavy behavior explicitly.
|
|
8. Confirm the documented lane budgets and family thresholds are visible in report output, are anchored to the recorded measurements above, preserve the fast-feedback target of at least 50% below the current full-suite baseline, and keep the confidence lane below the current full-suite baseline.
|
|
9. Confirm the initial heavy-governance selector set was refined after the first profiling pass rather than locked purely from intuition.
|
|
10. Confirm the first obvious taxonomy misfits were reclassified or regrouped instead of only being documented.
|
|
|
|
## Exit Criteria
|
|
|
|
1. The repo has a single fast default contributor run and separate confidence, browser, heavy, profile, and JUnit entry points, with profile and JUnit represented as support-lane entries in the shared manifest and report contract.
|
|
2. Browser tests are not silently part of the fast path.
|
|
3. The confidence lane is defined as the Unit suite plus the manifest-defined non-browser Feature or Integration selectors that remain outside explicit heavy-governance exclusions.
|
|
4. Every lane declaration carries purpose, intended audience, included families, excluded families, and ownership expectations.
|
|
5. Slow-test visibility and machine-readable reports are available through checked-in commands, including a report with at least the top 10 slowest entries and family-threshold evaluation.
|
|
6. The dominant shared helper and at least one additional factory or fixture cluster have explicit cheap defaults.
|
|
7. Runtime budgets and family thresholds are documented from the recorded baseline measurements above, including the fast-feedback target of at least 50% below the current full-suite baseline and the confidence target of staying below the current full-suite baseline, and DB strategy guidance is tied to the lane model.
|
|
8. The first obvious taxonomy misfits have been reclassified or regrouped as part of the rollout. |