TenantAtlas/specs/210-ci-matrix-budget-enforcement/quickstart.md
Ahmed Darrazi abdec60d7a
Some checks failed
PR Fast Feedback / fast-feedback (pull_request) Failing after 50s
Spec 210: implement CI test matrix budget enforcement
2026-04-17 19:59:25 +02:00

108 lines
7.1 KiB
Markdown

# Quickstart: CI Test Matrix & Runtime Budget Enforcement
## Preconditions
- Repository Actions are enabled in Gitea for this repository.
- At least one trusted `act_runner` is registered with a single stable runner label.
- The runner can reach the Gitea instance and the action source used by the workflows.
- `dev` remains the integration branch for broader shared validation.
- The existing local lane wrappers still pass before CI wiring begins.
## Required Validation Evidence
- One representative `pull_request` Fast Feedback run.
- One representative `push` to `dev` Confidence run.
- One manual Heavy Governance workflow run.
- One scheduled Heavy Governance workflow run after schedules are enabled.
- One manual Browser workflow run.
- One scheduled Browser workflow run after schedules are enabled.
- Each evidence record must capture the trigger, executed lane, artifact bundle names, budget outcome class, and primary failure class or explicit clean-success result. The Fast Feedback evidence must reference the chosen CI variance tolerance, and any material runtime recalibration must be recorded in `specs/210-ci-matrix-budget-enforcement/spec.md` or the active PR and may be linked from the affected evidence records.
## Frozen Trigger Matrix
| Trigger | Workflow | Lane | Expected artifact bundle |
|---|---|---|---|
| `pull_request` (`opened`, `reopened`, `synchronize`) | `test-pr-fast-feedback.yml` | `fast-feedback` | summary, report, budget, JUnit |
| `push` to `dev` | `test-main-confidence.yml` | `confidence` | summary, report, budget, JUnit |
| `workflow_dispatch` heavy | `test-heavy-governance.yml` | `heavy-governance` | summary, report, budget, JUnit |
| scheduled heavy | `test-heavy-governance.yml` | `heavy-governance` | summary, report, budget, JUnit |
| `workflow_dispatch` browser | `test-browser.yml` | `browser` | summary, report, budget, JUnit |
| scheduled browser | `test-browser.yml` | `browser` | summary, report, budget, JUnit |
## No-New-Fixture-Cost Rule
- Keep CI-governance additions limited to manifest or budget metadata, lightweight guard tests, workflow files, and the artifact staging helper.
- Do not add shared product fixtures, broaden default guard setup, or move CI-governance checks into Heavy Governance or Browser by default.
- Treat the Fast Feedback CI variance allowance as `15s`; a pull request lane run between `200s` and `215s` is warning-only, while a run above `215s` becomes a blocking budget breach.
## Artifact Locations
- Lane-local machine-readable outputs remain under `apps/platform/storage/logs/test-lanes`.
- CI upload staging directories are `.gitea-artifacts/pr-fast-feedback`, `.gitea-artifacts/main-confidence`, `.gitea-artifacts/heavy-governance`, and `.gitea-artifacts/browser`.
- Scheduled Heavy Governance is gated by `TENANTATLAS_ENABLE_HEAVY_GOVERNANCE_SCHEDULE=1` and scheduled Browser is gated by `TENANTATLAS_ENABLE_BROWSER_SCHEDULE=1`.
## Implementation Order
1. Extend repo truth first.
- Add trigger-aware CI policy fields to `TestLaneManifest`.
- Extend `TestLaneBudget` and `TestLaneReport` with failure-class and CI-summary metadata.
- Add `scripts/platform-test-artifacts` for deterministic export and upload staging.
2. Land the blocking pull request path.
- Add `.gitea/workflows/test-pr-fast-feedback.yml`.
- Use only `scripts/platform-test-lane fast-feedback`.
- Stage and upload the Fast Feedback artifacts.
3. Land the broader mainline path.
- Add `.gitea/workflows/test-main-confidence.yml`.
- Trigger it on `push` to `dev`.
- Publish the Confidence lane JUnit XML, summary, report, and budget outputs from the same run.
4. Land the expensive lanes as separate workflows.
- Add `.gitea/workflows/test-heavy-governance.yml` for `workflow_dispatch`, with the schedule enabled only after the first successful manual validation.
- Add `.gitea/workflows/test-browser.yml` for `workflow_dispatch`, with the schedule enabled only after the first successful manual validation.
- Keep their failure semantics non-blocking at first unless stability evidence justifies more.
5. Add guards and documentation.
- Add CI-governance guard tests, including wrong-lane and unresolved-entry-point failure classification.
- Update `README.md` with trigger policy, local reproduction, artifact locations, and blocking semantics.
## Local Validation Commands
Run the existing wrappers before pushing workflow changes:
```bash
./scripts/platform-test-lane fast-feedback
./scripts/platform-test-lane confidence
./scripts/platform-test-lane heavy-governance
./scripts/platform-test-lane browser
./scripts/platform-test-report fast-feedback
./scripts/platform-test-report confidence
```
Run the focused guard suites after implementation changes:
```bash
cd apps/platform && ./vendor/bin/sail artisan test --compact tests/Feature/Guards/CiFastFeedbackWorkflowContractTest.php tests/Feature/Guards/CiConfidenceWorkflowContractTest.php tests/Feature/Guards/CiHeavyBrowserWorkflowContractTest.php tests/Feature/Guards/CiLaneFailureClassificationContractTest.php tests/Feature/Guards/FastFeedbackLaneContractTest.php tests/Feature/Guards/ConfidenceLaneContractTest.php tests/Feature/Guards/HeavyGovernanceLaneContractTest.php tests/Feature/Guards/BrowserLaneIsolationTest.php tests/Feature/Guards/FixtureLaneImpactBudgetTest.php tests/Feature/Guards/TestLaneManifestTest.php tests/Feature/Guards/TestLaneArtifactsContractTest.php tests/Feature/Guards/TestLaneCommandContractTest.php
```
## First Live Gitea Validation
1. Push the workflow files and helper changes on a feature branch.
2. Open or update a pull request and confirm only the Fast Feedback workflow triggers.
3. Verify the uploaded artifact bundle contains lane-specific summary, report, budget, and JUnit files.
4. Push or merge to `dev` and confirm the Confidence workflow triggers and publishes the broader artifact set.
5. Manually dispatch Heavy Governance and Browser workflows and confirm they remain separate.
6. Review the manual run summaries and guard results to confirm failure classification stays legible and the CI-governance changes did not widen fast-path fixture cost or lane membership unexpectedly.
7. Enable schedules only after the manual runs succeed and their artifact bundles are legible.
8. Capture one scheduled Heavy Governance run and one scheduled Browser run, then record trigger, executed lane, bundle names, budget outcome, and the primary failure class or explicit clean-success result in this quickstart, and record the Fast Feedback variance tolerance or any material recalibration note in `specs/210-ci-matrix-budget-enforcement/spec.md` or the active PR.
## Review Checklist
- Pull request workflow uses the repo wrapper and no inline test paths.
- `dev` push workflow publishes Confidence artifacts from one lane run.
- Heavy Governance and Browser have separate workflow files and separate upload bundles.
- Artifact names are deterministic per run and per lane.
- Failure summaries distinguish test, budget, artifact, wrapper, manifest, and infrastructure problems.
- README explains what blocks a pull request and what only warns.
- CI-governance changes do not widen default fixtures or promote guard coverage into Heavy Governance or Browser unexpectedly.