202 lines
17 KiB
Markdown
202 lines
17 KiB
Markdown
# Tasks: CI Test Matrix & Runtime Budget Enforcement
|
|
|
|
**Input**: Design documents from `/Users/ahmeddarrazi/Documents/projects/TenantAtlas/specs/210-ci-matrix-budget-enforcement/`
|
|
**Prerequisites**: `plan.md` (required), `spec.md` (required), `research.md`, `data-model.md`, `contracts/`, `quickstart.md`
|
|
|
|
**Tests**: Required. This feature changes repository CI and test-governance behavior, so each user story includes Pest guard coverage and focused validation through Sail and the repo-root lane wrappers.
|
|
|
|
**Organization**: Tasks are grouped by user story so each story can be implemented and validated independently where possible.
|
|
|
|
## Phase 1: Setup (Shared Context)
|
|
|
|
**Purpose**: Freeze shared CI assumptions, lane classification, and rollout evidence expectations before workflow implementation.
|
|
|
|
- [X] T001 [P] Audit `scripts/platform-test-lane`, `scripts/platform-test-report`, `apps/platform/tests/Support/TestLaneManifest.php`, `apps/platform/tests/Support/TestLaneBudget.php`, and `apps/platform/tests/Support/TestLaneReport.php` as the only valid CI execution and policy seams before writing `.gitea/workflows/*.yml`
|
|
- [X] T002 [P] Update `specs/210-ci-matrix-budget-enforcement/spec.md`, `specs/210-ci-matrix-budget-enforcement/plan.md`, and `specs/210-ci-matrix-budget-enforcement/quickstart.md` with the frozen trigger matrix, required validation evidence set, and no-new-fixture-cost rule before workflow implementation
|
|
|
|
---
|
|
|
|
## Phase 2: Foundational (Blocking Prerequisites)
|
|
|
|
**Purpose**: Extend the shared lane manifest, budget, report, and wrapper seams that every workflow path depends on.
|
|
|
|
**Critical**: No user story work should begin until this phase is complete.
|
|
|
|
- [X] T003 Extend `apps/platform/tests/Support/TestLaneManifest.php` with workflow profiles, lane bindings, artifact publication contracts, and trigger-to-lane metadata aligned to `specs/210-ci-matrix-budget-enforcement/contracts/ci-lane-matrix.schema.json`
|
|
- [X] T004 [P] Extend `apps/platform/tests/Support/TestLaneBudget.php` with trigger-aware enforcement profiles, variance allowance, and blocking-status evaluation for `hard-fail`, `soft-warn`, and `trend-only`
|
|
- [X] T005 [P] Extend `apps/platform/tests/Support/TestLaneReport.php` with CI summary metadata, primary failure classification, and artifact publication status aligned to `specs/210-ci-matrix-budget-enforcement/contracts/ci-lane-governance.logical.openapi.yaml`
|
|
- [X] T006 [P] Update `apps/platform/tests/Feature/Guards/TestLaneManifestTest.php`, `apps/platform/tests/Feature/Guards/TestLaneCommandContractTest.php`, `apps/platform/tests/Feature/Guards/TestLaneArtifactsContractTest.php`, and add `apps/platform/tests/Feature/Guards/CiLaneFailureClassificationContractTest.php` to lock the shared CI matrix metadata, artifact publication contract, wrong-lane drift signals, unresolved checked-in entry-point handling, and single-primary-failure classification
|
|
- [X] T007 Update `scripts/platform-test-lane`, `scripts/platform-test-report`, and `scripts/platform-test-artifacts` so local and CI executions share the same lane and report export contract
|
|
|
|
**Checkpoint**: The shared CI-governance seams are ready for story-specific workflow implementation.
|
|
|
|
---
|
|
|
|
## Phase 3: User Story 1 - Enforce The Fast Pull Request Path (Priority: P1) 🎯 MVP
|
|
|
|
**Goal**: Make pull request validation run only the Fast Feedback lane and fail clearly on blocking problems.
|
|
|
|
**Independent Test**: Open or update a pull request and confirm only the Fast Feedback workflow runs, publishes the required artifacts, and blocks on test, wrapper, artifact, or hard-fail budget problems.
|
|
|
|
### Tests for User Story 1
|
|
|
|
- [X] T008 [P] [US1] Add `apps/platform/tests/Feature/Guards/CiFastFeedbackWorkflowContractTest.php` and expand `apps/platform/tests/Feature/Guards/FastFeedbackLaneContractTest.php` to assert pull request trigger mapping, fast-only lane execution, and blocking Fast Feedback failure semantics
|
|
|
|
### Implementation for User Story 1
|
|
|
|
- [X] T009 [US1] Implement `.gitea/workflows/test-pr-fast-feedback.yml` for `pull_request` events (`opened`, `reopened`, `synchronize`) using `scripts/platform-test-lane fast-feedback`
|
|
- [X] T010 [US1] Update `apps/platform/tests/Support/TestLaneManifest.php` and `apps/platform/tests/Support/TestLaneBudget.php` with the `pr-fast-feedback` workflow profile, Fast Feedback artifact bundle requirements, and hard-fail policy for test, wrapper, manifest, artifact, and mature budget failures
|
|
- [X] T011 [US1] Wire deterministic Fast Feedback artifact staging and upload in `.gitea/workflows/test-pr-fast-feedback.yml` and `scripts/platform-test-artifacts` so pull request runs publish summary, report, budget, and JUnit bundles
|
|
|
|
**Checkpoint**: Pull requests now have one explicit blocking fast path that is lane-correct and artifact-complete.
|
|
|
|
---
|
|
|
|
## Phase 4: User Story 2 - Publish Mainline Confidence Evidence (Priority: P1)
|
|
|
|
**Goal**: Make the `dev` branch run the broader Confidence lane and publish machine-readable evidence from that same run.
|
|
|
|
**Independent Test**: Push to `dev` and confirm the Confidence workflow runs, publishes summary/report/budget/JUnit artifacts, and keeps budget warnings distinct from blocking test or artifact failures.
|
|
|
|
### Tests for User Story 2
|
|
|
|
- [X] T012 [P] [US2] Add `apps/platform/tests/Feature/Guards/CiConfidenceWorkflowContractTest.php` and expand `apps/platform/tests/Feature/Guards/ConfidenceLaneContractTest.php` plus `apps/platform/tests/Feature/Guards/TestLaneArtifactsContractTest.php` to assert `dev` push mapping, Confidence artifact completeness, and soft budget warning semantics
|
|
|
|
### Implementation for User Story 2
|
|
|
|
- [X] T013 [US2] Implement `.gitea/workflows/test-main-confidence.yml` for `push` to `dev` using `scripts/platform-test-lane confidence`
|
|
- [X] T014 [US2] Update `apps/platform/tests/Support/TestLaneManifest.php` with the `main-confidence` workflow profile, `dev` branch filter, and Confidence artifact publication contract
|
|
- [X] T015 [US2] Update `apps/platform/tests/Support/TestLaneBudget.php` and `apps/platform/tests/Support/TestLaneReport.php` so mainline Confidence distinguishes blocking test/artifact failures from non-blocking budget warnings and publishes the lane's own JUnit XML without a separate `junit` rerun
|
|
- [X] T016 [US2] Wire Confidence artifact staging and upload in `.gitea/workflows/test-main-confidence.yml` and `scripts/platform-test-artifacts` so `dev` runs publish summary, report, budget, and JUnit bundles from a single Confidence lane execution
|
|
|
|
**Checkpoint**: The integration branch now has a broader, reviewable Confidence path with standardized machine-readable evidence.
|
|
|
|
---
|
|
|
|
## Phase 5: User Story 3 - Keep Heavy And Browser Validation Separate (Priority: P2)
|
|
|
|
**Goal**: Run Heavy Governance and Browser as separate scheduled or manual CI lanes with their own semantics and contributor guidance.
|
|
|
|
**Independent Test**: Trigger Heavy Governance and Browser manually, confirm they run through separate workflows and artifact bundles, and verify the documented trigger matrix explains when each lane is expected.
|
|
|
|
### Tests for User Story 3
|
|
|
|
- [X] T017 [P] [US3] Add `apps/platform/tests/Feature/Guards/CiHeavyBrowserWorkflowContractTest.php` and expand `apps/platform/tests/Feature/Guards/HeavyGovernanceLaneContractTest.php` plus `apps/platform/tests/Feature/Guards/BrowserLaneIsolationTest.php` to assert separate manual or scheduled workflows, non-pull-request isolation, and trigger-specific budget modes
|
|
|
|
### Implementation for User Story 3
|
|
|
|
- [X] T018 [P] [US3] Implement `.gitea/workflows/test-heavy-governance.yml` for `workflow_dispatch` using `scripts/platform-test-lane heavy-governance`, with scheduled execution enabled only after the first successful manual validation
|
|
- [X] T019 [P] [US3] Implement `.gitea/workflows/test-browser.yml` for `workflow_dispatch` using `scripts/platform-test-lane browser`, with scheduled execution enabled only after the first successful manual validation
|
|
- [X] T020 [US3] Update `apps/platform/tests/Support/TestLaneManifest.php` and `apps/platform/tests/Support/TestLaneBudget.php` with `manual` and `scheduled` workflow profiles, governance-contract threshold handling for Heavy Governance, and warning or trend-only modes for Heavy Governance and Browser
|
|
- [X] T021 [US3] Update `README.md`, `specs/210-ci-matrix-budget-enforcement/quickstart.md`, and `specs/210-ci-matrix-budget-enforcement/spec.md` with the final trigger policy, local reproduction commands, artifact locations, blocking versus non-blocking guidance for Fast Feedback, Confidence, Heavy Governance, and Browser, and the documented Fast Feedback CI variance tolerance
|
|
- [X] T022 [US3] Wire Heavy Governance and Browser artifact staging and upload in `.gitea/workflows/test-heavy-governance.yml`, `.gitea/workflows/test-browser.yml`, and `scripts/platform-test-artifacts` so both lanes publish separate bundles with deterministic names
|
|
|
|
**Checkpoint**: Heavy Governance and Browser are now visible, reproducible, and isolated from the default fast path.
|
|
|
|
---
|
|
|
|
## Phase 6: Polish & Cross-Cutting Concerns
|
|
|
|
**Purpose**: Validate the full matrix, record rollout evidence, and finish formatting and cleanup.
|
|
|
|
- [X] T023 Run focused Pest coverage for `apps/platform/tests/Feature/Guards/CiFastFeedbackWorkflowContractTest.php`, `apps/platform/tests/Feature/Guards/CiConfidenceWorkflowContractTest.php`, `apps/platform/tests/Feature/Guards/CiHeavyBrowserWorkflowContractTest.php`, `apps/platform/tests/Feature/Guards/CiLaneFailureClassificationContractTest.php`, `apps/platform/tests/Feature/Guards/FastFeedbackLaneContractTest.php`, `apps/platform/tests/Feature/Guards/ConfidenceLaneContractTest.php`, `apps/platform/tests/Feature/Guards/HeavyGovernanceLaneContractTest.php`, `apps/platform/tests/Feature/Guards/BrowserLaneIsolationTest.php`, `apps/platform/tests/Feature/Guards/FixtureLaneImpactBudgetTest.php`, `apps/platform/tests/Feature/Guards/TestLaneManifestTest.php`, `apps/platform/tests/Feature/Guards/TestLaneArtifactsContractTest.php`, and `apps/platform/tests/Feature/Guards/TestLaneCommandContractTest.php` with `cd apps/platform && ./vendor/bin/sail artisan test --compact tests/Feature/Guards/CiFastFeedbackWorkflowContractTest.php tests/Feature/Guards/CiConfidenceWorkflowContractTest.php tests/Feature/Guards/CiHeavyBrowserWorkflowContractTest.php tests/Feature/Guards/CiLaneFailureClassificationContractTest.php tests/Feature/Guards/FastFeedbackLaneContractTest.php tests/Feature/Guards/ConfidenceLaneContractTest.php tests/Feature/Guards/HeavyGovernanceLaneContractTest.php tests/Feature/Guards/BrowserLaneIsolationTest.php tests/Feature/Guards/FixtureLaneImpactBudgetTest.php tests/Feature/Guards/TestLaneManifestTest.php tests/Feature/Guards/TestLaneArtifactsContractTest.php tests/Feature/Guards/TestLaneCommandContractTest.php`
|
|
- [X] T024 Run local wrapper validation via `./scripts/platform-test-lane fast-feedback`, `./scripts/platform-test-lane confidence`, `./scripts/platform-test-lane heavy-governance`, `./scripts/platform-test-lane browser`, `./scripts/platform-test-report fast-feedback`, and `./scripts/platform-test-report confidence`, then confirm the CI-governance helper and guard changes do not widen default fixtures or promote fast-path coverage into Heavy Governance or Browser lane membership
|
|
- [ ] T025 [P] Execute the required live Gitea validation evidence set: one `pull_request` Fast Feedback run, one `push` to `dev` Confidence run, one manual Heavy Governance run, one scheduled Heavy Governance run, one manual Browser run, and one scheduled Browser run using `.gitea/workflows/test-pr-fast-feedback.yml`, `.gitea/workflows/test-main-confidence.yml`, `.gitea/workflows/test-heavy-governance.yml`, and `.gitea/workflows/test-browser.yml`, then record trigger, executed lane, artifact bundle, budget outcome, and the primary failure class or explicit clean-success result in `specs/210-ci-matrix-budget-enforcement/quickstart.md`, and record the chosen Fast Feedback variance tolerance and any material runtime recalibration note in `specs/210-ci-matrix-budget-enforcement/spec.md` or the active PR
|
|
- [X] T026 Run `cd apps/platform && ./vendor/bin/sail bin pint --dirty --format agent` for PHP changes in `apps/platform/tests/Support/TestLaneManifest.php`, `apps/platform/tests/Support/TestLaneBudget.php`, `apps/platform/tests/Support/TestLaneReport.php`, and the new guard tests under `apps/platform/tests/Feature/Guards/`, then manually normalize shell and YAML formatting in `scripts/platform-test-artifacts` and `.gitea/workflows/*.yml`
|
|
|
|
---
|
|
|
|
## Dependencies & Execution Order
|
|
|
|
### Phase Dependencies
|
|
|
|
- **Setup (Phase 1)**: No dependencies and can start immediately.
|
|
- **Foundational (Phase 2)**: Depends on Phase 1 and blocks all user story work.
|
|
- **User Story 1 (Phase 3)**: Depends on Phase 2 only.
|
|
- **User Story 2 (Phase 4)**: Depends on Phase 2 only.
|
|
- **User Story 3 (Phase 5)**: Depends on Phase 2 for workflow implementation; final contributor guidance and full-matrix validation should wait until User Stories 1 and 2 are in place.
|
|
- **Polish (Phase 6)**: Depends on all desired user stories being complete.
|
|
|
|
### User Story Dependencies
|
|
|
|
- **User Story 1 (P1)**: Can begin immediately after Foundational and is the MVP slice.
|
|
- **User Story 2 (P1)**: Can begin immediately after Foundational and remains independently valuable even if Heavy Governance and Browser are not yet wired.
|
|
- **User Story 3 (P2)**: Uses the same shared seams as User Stories 1 and 2 and should finish after them so the final documented trigger matrix is complete.
|
|
|
|
### Within Each User Story
|
|
|
|
- Story-specific guard tests must be written and fail before implementation.
|
|
- Workflow metadata in `TestLaneManifest.php` and policy logic in `TestLaneBudget.php` should be in place before finalizing the matching workflow YAML.
|
|
- Artifact upload wiring should happen after the workflow file and lane policy both exist.
|
|
- Story validation should complete before moving on to the next priority slice.
|
|
|
|
### Parallel Opportunities
|
|
|
|
- T001 and T002 can run in parallel during Setup.
|
|
- T004, T005, and T006 can run in parallel once T003 defines the shared manifest shape.
|
|
- User Stories 1 and 2 can proceed in parallel after Foundational if different contributors own the workflow YAML and guard files.
|
|
- In User Story 3, T018 and T019 can run in parallel because Heavy Governance and Browser use separate workflow files.
|
|
- T025 can run in parallel with final formatting only after all workflows and guards are stable.
|
|
|
|
---
|
|
|
|
## Parallel Example: User Story 1
|
|
|
|
```bash
|
|
# After T008 defines the PR contract, these can proceed in parallel:
|
|
Task: "Implement .gitea/workflows/test-pr-fast-feedback.yml for pull_request events using scripts/platform-test-lane fast-feedback"
|
|
Task: "Update apps/platform/tests/Support/TestLaneManifest.php and apps/platform/tests/Support/TestLaneBudget.php with the pr-fast-feedback workflow profile and hard-fail policy"
|
|
```
|
|
|
|
---
|
|
|
|
## Parallel Example: User Story 2
|
|
|
|
```bash
|
|
# After T012 locks the Confidence workflow contract, these can proceed in parallel:
|
|
Task: "Implement .gitea/workflows/test-main-confidence.yml for push to dev using scripts/platform-test-lane confidence"
|
|
Task: "Update apps/platform/tests/Support/TestLaneManifest.php with the main-confidence workflow profile and Confidence artifact publication contract"
|
|
```
|
|
|
|
---
|
|
|
|
## Parallel Example: User Story 3
|
|
|
|
```bash
|
|
# After T017 defines the heavy/browser workflow contract, these can proceed in parallel:
|
|
Task: "Implement .gitea/workflows/test-heavy-governance.yml for workflow_dispatch and scheduled execution"
|
|
Task: "Implement .gitea/workflows/test-browser.yml for workflow_dispatch and scheduled execution"
|
|
```
|
|
|
|
---
|
|
|
|
## Implementation Strategy
|
|
|
|
### MVP First (User Story 1 Only)
|
|
|
|
1. Complete Phase 1: Setup.
|
|
2. Complete Phase 2: Foundational.
|
|
3. Complete Phase 3: User Story 1.
|
|
4. Stop and validate the pull request fast path independently.
|
|
|
|
### Incremental Delivery
|
|
|
|
1. Deliver the blocking pull request path first.
|
|
2. Add the broader `dev` Confidence path next.
|
|
3. Add Heavy Governance and Browser as separate scheduled or manual lanes.
|
|
4. Finish with live validation evidence and final contributor guidance.
|
|
|
|
### Parallel Team Strategy
|
|
|
|
1. One contributor extends the shared support seams in `apps/platform/tests/Support/` while another prepares workflow YAML files.
|
|
2. After Foundational completes, separate owners can implement the PR and `dev` workflows in parallel.
|
|
3. Heavy Governance and Browser can then be split across two contributors because they live in separate workflow files.
|
|
|
|
---
|
|
|
|
## Notes
|
|
|
|
- `[P]` tasks operate on different files and can run in parallel once their dependencies are satisfied.
|
|
- `[US1]`, `[US2]`, and `[US3]` map each task to the corresponding user story in `spec.md`.
|
|
- This feature changes runtime validation behavior, so focused Pest guards and the narrowest relevant lane reruns are part of the definition of done.
|
|
- Live Gitea validation is required because local wrapper tests alone cannot prove trigger behavior or uploaded artifact bundles. |