ahmido/TenantAtlas
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects 1 Releases Wiki Activity
platform-dev
TenantAtlas/specs/295-full-suite-ci-baseline/quickstart.md
ahmido f03555eae1 Spec 295: full suite CI lane baseline (#350) 
## Summary
- add the Spec 295 artifacts for full-suite failure classification and CI lane baseline work
- fix `scripts/platform-test-artifacts` so Sail passes artifact staging inputs into the embedded PHP script via argv
- add a guard test covering the artifact staging input contract

## Scope guards
- no browser screenshot baselines included
- no generated test artifacts included
- no runtime application code changes included

## Notes
- classification evidence and follow-up ownership are documented in `specs/295-full-suite-ci-baseline/failure-classification.md`
- this PR is intentionally limited to the CI/lane/artifact contract slice for Spec 295

Co-authored-by: Ahmed Darrazi <ahmed.darrazi@live.de>
Reviewed-on: #350
2026-05-11 11:14:56 +00:00

4.0 KiB
Raw Permalink Blame History

Quickstart: Full Suite Failure Classification & CI Lane Baseline

Purpose

Use this package to classify whether the complete platform test suite is a reliable CI signal after Specs 293 and 294.

Before Implementation

  1. Review:
    • /Users/ahmeddarrazi/Documents/projects/wt-plattform/specs/295-full-suite-ci-baseline/spec.md
    • /Users/ahmeddarrazi/Documents/projects/wt-plattform/specs/295-full-suite-ci-baseline/plan.md
    • /Users/ahmeddarrazi/Documents/projects/wt-plattform/specs/295-full-suite-ci-baseline/data-model.md
    • /Users/ahmeddarrazi/Documents/projects/wt-plattform/specs/295-full-suite-ci-baseline/failure-classification.md
  2. Confirm the branch is clean.
  3. Confirm no implementation step is about restoring TenantPanelProvider, /admin/t/..., or tenant-scoped legacy fallbacks.

Primary Classification Flow

Use only the pinned categories and seams from failure-classification.md: ci-signal-restored, ci-wrapper-or-manifest-regression, artifact-publication-regression, budget-or-trend-baseline-drift, product-runtime-or-test-regression, browser-lane-regression, flaky-or-environment, follow-up-spec-required, resolved-or-not-needed; and raw-full-suite, fast-feedback-lane, confidence-lane, heavy-governance-lane, browser-lane, profiling-or-junit-support, lane-reporting, artifact-publication, budget-trend-baseline, legacy-cutover-regression-guard, provider-verification-regression-guard.

Run the raw full suite when feasible:

export PATH="/bin:/usr/bin:/usr/local/bin:$PATH" && (cd apps/platform && ./vendor/bin/sail artisan test --compact)

Record the outcome in failure-classification.md.

If the raw full suite is too slow, noisy, or environment-blocked to classify, run the explicit lane split:

export PATH="/bin:/usr/bin:/usr/local/bin:$PATH" && ./scripts/platform-test-lane fast-feedback
export PATH="/bin:/usr/bin:/usr/local/bin:$PATH" && ./scripts/platform-test-lane confidence
export PATH="/bin:/usr/bin:/usr/local/bin:$PATH" && ./scripts/platform-test-lane heavy-governance
export PATH="/bin:/usr/bin:/usr/local/bin:$PATH" && ./scripts/platform-test-lane browser

CI Report and Artifact Flow

After lane runs, generate lane reports when needed:

export PATH="/bin:/usr/bin:/usr/local/bin:$PATH" && ./scripts/platform-test-report fast-feedback
export PATH="/bin:/usr/bin:/usr/local/bin:$PATH" && ./scripts/platform-test-report confidence
export PATH="/bin:/usr/bin:/usr/local/bin:$PATH" && ./scripts/platform-test-report heavy-governance
export PATH="/bin:/usr/bin:/usr/local/bin:$PATH" && ./scripts/platform-test-report browser

Use artifact staging only if artifact publication itself is being validated:

export PATH="/bin:/usr/bin:/usr/local/bin:$PATH" && ./scripts/platform-test-artifacts fast-feedback /tmp/tenantpilot-fast-feedback-artifacts

Fix Rules

Fix in 295 only when the failure is directly and narrowly caused by:

  • scripts/platform-test-lane
  • scripts/platform-test-report
  • scripts/platform-test-artifacts
  • apps/platform/tests/Support/TestLaneManifest.php
  • apps/platform/tests/Support/TestLaneReport.php
  • apps/platform/tests/Support/TestLaneBudget.php
  • directly related CI guard tests under apps/platform/tests/Feature/Guards/

Do not fix in 295 when the failure requires:

  • application runtime behavior changes
  • Filament page/resource changes
  • routes, middleware, policies, services, jobs, migrations, views, or models
  • provider/verification runtime changes beyond the completed Spec 294
  • browser UI repair
  • tenant-cutover compatibility restoration

Classify those as follow-up work instead.

Expected Close-Out

Close out with exactly one final readiness decision:

  • restored-ci-signal
  • classified-follow-up-required
  • blocked-by-environment

Then run formatting for any changed PHP files:

export PATH="/bin:/usr/bin:/usr/local/bin:$PATH" && (cd apps/platform && ./vendor/bin/sail bin pint --dirty --format agent)