108 lines
7.1 KiB
Markdown
108 lines
7.1 KiB
Markdown
# Quickstart: CI Test Matrix & Runtime Budget Enforcement
|
|
|
|
## Preconditions
|
|
|
|
- Repository Actions are enabled in Gitea for this repository.
|
|
- At least one trusted `act_runner` is registered with a single stable runner label.
|
|
- The runner can reach the Gitea instance and the action source used by the workflows.
|
|
- `dev` remains the integration branch for broader shared validation.
|
|
- The existing local lane wrappers still pass before CI wiring begins.
|
|
|
|
## Required Validation Evidence
|
|
|
|
- One representative `pull_request` Fast Feedback run.
|
|
- One representative `push` to `dev` Confidence run.
|
|
- One manual Heavy Governance workflow run.
|
|
- One scheduled Heavy Governance workflow run after schedules are enabled.
|
|
- One manual Browser workflow run.
|
|
- One scheduled Browser workflow run after schedules are enabled.
|
|
- Each evidence record must capture the trigger, executed lane, artifact bundle names, budget outcome class, and primary failure class or explicit clean-success result. The Fast Feedback evidence must reference the chosen CI variance tolerance, and any material runtime recalibration must be recorded in `specs/210-ci-matrix-budget-enforcement/spec.md` or the active PR and may be linked from the affected evidence records.
|
|
|
|
## Frozen Trigger Matrix
|
|
|
|
| Trigger | Workflow | Lane | Expected artifact bundle |
|
|
|---|---|---|---|
|
|
| `pull_request` (`opened`, `reopened`, `synchronize`) | `test-pr-fast-feedback.yml` | `fast-feedback` | summary, report, budget, JUnit |
|
|
| `push` to `dev` | `test-main-confidence.yml` | `confidence` | summary, report, budget, JUnit |
|
|
| `workflow_dispatch` heavy | `test-heavy-governance.yml` | `heavy-governance` | summary, report, budget, JUnit |
|
|
| scheduled heavy | `test-heavy-governance.yml` | `heavy-governance` | summary, report, budget, JUnit |
|
|
| `workflow_dispatch` browser | `test-browser.yml` | `browser` | summary, report, budget, JUnit |
|
|
| scheduled browser | `test-browser.yml` | `browser` | summary, report, budget, JUnit |
|
|
|
|
## No-New-Fixture-Cost Rule
|
|
|
|
- Keep CI-governance additions limited to manifest or budget metadata, lightweight guard tests, workflow files, and the artifact staging helper.
|
|
- Do not add shared product fixtures, broaden default guard setup, or move CI-governance checks into Heavy Governance or Browser by default.
|
|
- Treat the Fast Feedback CI variance allowance as `15s`; a pull request lane run between `200s` and `215s` is warning-only, while a run above `215s` becomes a blocking budget breach.
|
|
|
|
## Artifact Locations
|
|
|
|
- Lane-local machine-readable outputs remain under `apps/platform/storage/logs/test-lanes`.
|
|
- CI upload staging directories are `.gitea-artifacts/pr-fast-feedback`, `.gitea-artifacts/main-confidence`, `.gitea-artifacts/heavy-governance`, and `.gitea-artifacts/browser`.
|
|
- Scheduled Heavy Governance is gated by `TENANTATLAS_ENABLE_HEAVY_GOVERNANCE_SCHEDULE=1` and scheduled Browser is gated by `TENANTATLAS_ENABLE_BROWSER_SCHEDULE=1`.
|
|
|
|
## Implementation Order
|
|
|
|
1. Extend repo truth first.
|
|
- Add trigger-aware CI policy fields to `TestLaneManifest`.
|
|
- Extend `TestLaneBudget` and `TestLaneReport` with failure-class and CI-summary metadata.
|
|
- Add `scripts/platform-test-artifacts` for deterministic export and upload staging.
|
|
|
|
2. Land the blocking pull request path.
|
|
- Add `.gitea/workflows/test-pr-fast-feedback.yml`.
|
|
- Use only `scripts/platform-test-lane fast-feedback`.
|
|
- Stage and upload the Fast Feedback artifacts.
|
|
|
|
3. Land the broader mainline path.
|
|
- Add `.gitea/workflows/test-main-confidence.yml`.
|
|
- Trigger it on `push` to `dev`.
|
|
- Publish the Confidence lane JUnit XML, summary, report, and budget outputs from the same run.
|
|
|
|
4. Land the expensive lanes as separate workflows.
|
|
- Add `.gitea/workflows/test-heavy-governance.yml` for `workflow_dispatch`, with the schedule enabled only after the first successful manual validation.
|
|
- Add `.gitea/workflows/test-browser.yml` for `workflow_dispatch`, with the schedule enabled only after the first successful manual validation.
|
|
- Keep their failure semantics non-blocking at first unless stability evidence justifies more.
|
|
|
|
5. Add guards and documentation.
|
|
- Add CI-governance guard tests, including wrong-lane and unresolved-entry-point failure classification.
|
|
- Update `README.md` with trigger policy, local reproduction, artifact locations, and blocking semantics.
|
|
|
|
## Local Validation Commands
|
|
|
|
Run the existing wrappers before pushing workflow changes:
|
|
|
|
```bash
|
|
./scripts/platform-test-lane fast-feedback
|
|
./scripts/platform-test-lane confidence
|
|
./scripts/platform-test-lane heavy-governance
|
|
./scripts/platform-test-lane browser
|
|
./scripts/platform-test-report fast-feedback
|
|
./scripts/platform-test-report confidence
|
|
```
|
|
|
|
Run the focused guard suites after implementation changes:
|
|
|
|
```bash
|
|
cd apps/platform && ./vendor/bin/sail artisan test --compact tests/Feature/Guards/CiFastFeedbackWorkflowContractTest.php tests/Feature/Guards/CiConfidenceWorkflowContractTest.php tests/Feature/Guards/CiHeavyBrowserWorkflowContractTest.php tests/Feature/Guards/CiLaneFailureClassificationContractTest.php tests/Feature/Guards/FastFeedbackLaneContractTest.php tests/Feature/Guards/ConfidenceLaneContractTest.php tests/Feature/Guards/HeavyGovernanceLaneContractTest.php tests/Feature/Guards/BrowserLaneIsolationTest.php tests/Feature/Guards/FixtureLaneImpactBudgetTest.php tests/Feature/Guards/TestLaneManifestTest.php tests/Feature/Guards/TestLaneArtifactsContractTest.php tests/Feature/Guards/TestLaneCommandContractTest.php
|
|
```
|
|
|
|
## First Live Gitea Validation
|
|
|
|
1. Push the workflow files and helper changes on a feature branch.
|
|
2. Open or update a pull request and confirm only the Fast Feedback workflow triggers.
|
|
3. Verify the uploaded artifact bundle contains lane-specific summary, report, budget, and JUnit files.
|
|
4. Push or merge to `dev` and confirm the Confidence workflow triggers and publishes the broader artifact set.
|
|
5. Manually dispatch Heavy Governance and Browser workflows and confirm they remain separate.
|
|
6. Review the manual run summaries and guard results to confirm failure classification stays legible and the CI-governance changes did not widen fast-path fixture cost or lane membership unexpectedly.
|
|
7. Enable schedules only after the manual runs succeed and their artifact bundles are legible.
|
|
8. Capture one scheduled Heavy Governance run and one scheduled Browser run, then record trigger, executed lane, bundle names, budget outcome, and the primary failure class or explicit clean-success result in this quickstart, and record the Fast Feedback variance tolerance or any material recalibration note in `specs/210-ci-matrix-budget-enforcement/spec.md` or the active PR.
|
|
|
|
## Review Checklist
|
|
|
|
- Pull request workflow uses the repo wrapper and no inline test paths.
|
|
- `dev` push workflow publishes Confidence artifacts from one lane run.
|
|
- Heavy Governance and Browser have separate workflow files and separate upload bundles.
|
|
- Artifact names are deterministic per run and per lane.
|
|
- Failure summaries distinguish test, budget, artifact, wrapper, manifest, and infrastructure problems.
|
|
- README explains what blocks a pull request and what only warns.
|
|
- CI-governance changes do not widen default fixtures or promote guard coverage into Heavy Governance or Browser unexpectedly. |