TenantAtlas/specs/210-ci-matrix-budget-enforcement/tasks.md
ahmido bf38ec1780
Some checks failed
Main Confidence / confidence (push) Failing after 3m36s
Spec 210: implement CI test matrix budget enforcement (#243)
## Summary
- add explicit Gitea workflow files for PR Fast Feedback, `dev` Confidence, Heavy Governance, and Browser lanes
- extend the repo-truth lane support seams with workflow profiles, trigger-aware budget enforcement, artifact publication contracts, CI summaries, and failure classification
- add deterministic artifact staging, new CI governance guard coverage, and Spec 210 planning/contracts/docs updates

## Validation
- `cd apps/platform && ./vendor/bin/sail bin pint --dirty --format agent`
- `cd apps/platform && ./vendor/bin/sail artisan test --compact tests/Feature/Guards/CiFastFeedbackWorkflowContractTest.php tests/Feature/Guards/CiConfidenceWorkflowContractTest.php tests/Feature/Guards/CiHeavyBrowserWorkflowContractTest.php tests/Feature/Guards/CiLaneFailureClassificationContractTest.php tests/Feature/Guards/FastFeedbackLaneContractTest.php tests/Feature/Guards/ConfidenceLaneContractTest.php tests/Feature/Guards/HeavyGovernanceLaneContractTest.php tests/Feature/Guards/BrowserLaneIsolationTest.php tests/Feature/Guards/FixtureLaneImpactBudgetTest.php tests/Feature/Guards/TestLaneManifestTest.php tests/Feature/Guards/TestLaneArtifactsContractTest.php tests/Feature/Guards/TestLaneCommandContractTest.php`
- `./scripts/platform-test-lane fast-feedback`
- `./scripts/platform-test-lane confidence`
- `./scripts/platform-test-lane heavy-governance`
- `./scripts/platform-test-lane browser`
- `./scripts/platform-test-report fast-feedback`
- `./scripts/platform-test-report confidence`

## Notes
- scheduled Heavy Governance and Browser workflows stay gated behind `TENANTATLAS_ENABLE_HEAVY_GOVERNANCE_SCHEDULE=1` and `TENANTATLAS_ENABLE_BROWSER_SCHEDULE=1`
- the remaining rollout evidence task is capturing the live Gitea run set this PR enables: PR Fast Feedback, `dev` Confidence, manual and scheduled Heavy Governance, and manual and scheduled Browser runs

Co-authored-by: Ahmed Darrazi <ahmed.darrazi@live.de>
Reviewed-on: #243
2026-04-17 18:04:35 +00:00

17 KiB

Tasks: CI Test Matrix & Runtime Budget Enforcement

Input: Design documents from /Users/ahmeddarrazi/Documents/projects/TenantAtlas/specs/210-ci-matrix-budget-enforcement/ Prerequisites: plan.md (required), spec.md (required), research.md, data-model.md, contracts/, quickstart.md

Tests: Required. This feature changes repository CI and test-governance behavior, so each user story includes Pest guard coverage and focused validation through Sail and the repo-root lane wrappers.

Organization: Tasks are grouped by user story so each story can be implemented and validated independently where possible.

Phase 1: Setup (Shared Context)

Purpose: Freeze shared CI assumptions, lane classification, and rollout evidence expectations before workflow implementation.

  • T001 [P] Audit scripts/platform-test-lane, scripts/platform-test-report, apps/platform/tests/Support/TestLaneManifest.php, apps/platform/tests/Support/TestLaneBudget.php, and apps/platform/tests/Support/TestLaneReport.php as the only valid CI execution and policy seams before writing .gitea/workflows/*.yml
  • T002 [P] Update specs/210-ci-matrix-budget-enforcement/spec.md, specs/210-ci-matrix-budget-enforcement/plan.md, and specs/210-ci-matrix-budget-enforcement/quickstart.md with the frozen trigger matrix, required validation evidence set, and no-new-fixture-cost rule before workflow implementation

Phase 2: Foundational (Blocking Prerequisites)

Purpose: Extend the shared lane manifest, budget, report, and wrapper seams that every workflow path depends on.

Critical: No user story work should begin until this phase is complete.

  • T003 Extend apps/platform/tests/Support/TestLaneManifest.php with workflow profiles, lane bindings, artifact publication contracts, and trigger-to-lane metadata aligned to specs/210-ci-matrix-budget-enforcement/contracts/ci-lane-matrix.schema.json
  • T004 [P] Extend apps/platform/tests/Support/TestLaneBudget.php with trigger-aware enforcement profiles, variance allowance, and blocking-status evaluation for hard-fail, soft-warn, and trend-only
  • T005 [P] Extend apps/platform/tests/Support/TestLaneReport.php with CI summary metadata, primary failure classification, and artifact publication status aligned to specs/210-ci-matrix-budget-enforcement/contracts/ci-lane-governance.logical.openapi.yaml
  • T006 [P] Update apps/platform/tests/Feature/Guards/TestLaneManifestTest.php, apps/platform/tests/Feature/Guards/TestLaneCommandContractTest.php, apps/platform/tests/Feature/Guards/TestLaneArtifactsContractTest.php, and add apps/platform/tests/Feature/Guards/CiLaneFailureClassificationContractTest.php to lock the shared CI matrix metadata, artifact publication contract, wrong-lane drift signals, unresolved checked-in entry-point handling, and single-primary-failure classification
  • T007 Update scripts/platform-test-lane, scripts/platform-test-report, and scripts/platform-test-artifacts so local and CI executions share the same lane and report export contract

Checkpoint: The shared CI-governance seams are ready for story-specific workflow implementation.


Phase 3: User Story 1 - Enforce The Fast Pull Request Path (Priority: P1) 🎯 MVP

Goal: Make pull request validation run only the Fast Feedback lane and fail clearly on blocking problems.

Independent Test: Open or update a pull request and confirm only the Fast Feedback workflow runs, publishes the required artifacts, and blocks on test, wrapper, artifact, or hard-fail budget problems.

Tests for User Story 1

  • T008 [P] [US1] Add apps/platform/tests/Feature/Guards/CiFastFeedbackWorkflowContractTest.php and expand apps/platform/tests/Feature/Guards/FastFeedbackLaneContractTest.php to assert pull request trigger mapping, fast-only lane execution, and blocking Fast Feedback failure semantics

Implementation for User Story 1

  • T009 [US1] Implement .gitea/workflows/test-pr-fast-feedback.yml for pull_request events (opened, reopened, synchronize) using scripts/platform-test-lane fast-feedback
  • T010 [US1] Update apps/platform/tests/Support/TestLaneManifest.php and apps/platform/tests/Support/TestLaneBudget.php with the pr-fast-feedback workflow profile, Fast Feedback artifact bundle requirements, and hard-fail policy for test, wrapper, manifest, artifact, and mature budget failures
  • T011 [US1] Wire deterministic Fast Feedback artifact staging and upload in .gitea/workflows/test-pr-fast-feedback.yml and scripts/platform-test-artifacts so pull request runs publish summary, report, budget, and JUnit bundles

Checkpoint: Pull requests now have one explicit blocking fast path that is lane-correct and artifact-complete.


Phase 4: User Story 2 - Publish Mainline Confidence Evidence (Priority: P1)

Goal: Make the dev branch run the broader Confidence lane and publish machine-readable evidence from that same run.

Independent Test: Push to dev and confirm the Confidence workflow runs, publishes summary/report/budget/JUnit artifacts, and keeps budget warnings distinct from blocking test or artifact failures.

Tests for User Story 2

  • T012 [P] [US2] Add apps/platform/tests/Feature/Guards/CiConfidenceWorkflowContractTest.php and expand apps/platform/tests/Feature/Guards/ConfidenceLaneContractTest.php plus apps/platform/tests/Feature/Guards/TestLaneArtifactsContractTest.php to assert dev push mapping, Confidence artifact completeness, and soft budget warning semantics

Implementation for User Story 2

  • T013 [US2] Implement .gitea/workflows/test-main-confidence.yml for push to dev using scripts/platform-test-lane confidence
  • T014 [US2] Update apps/platform/tests/Support/TestLaneManifest.php with the main-confidence workflow profile, dev branch filter, and Confidence artifact publication contract
  • T015 [US2] Update apps/platform/tests/Support/TestLaneBudget.php and apps/platform/tests/Support/TestLaneReport.php so mainline Confidence distinguishes blocking test/artifact failures from non-blocking budget warnings and publishes the lane's own JUnit XML without a separate junit rerun
  • T016 [US2] Wire Confidence artifact staging and upload in .gitea/workflows/test-main-confidence.yml and scripts/platform-test-artifacts so dev runs publish summary, report, budget, and JUnit bundles from a single Confidence lane execution

Checkpoint: The integration branch now has a broader, reviewable Confidence path with standardized machine-readable evidence.


Phase 5: User Story 3 - Keep Heavy And Browser Validation Separate (Priority: P2)

Goal: Run Heavy Governance and Browser as separate scheduled or manual CI lanes with their own semantics and contributor guidance.

Independent Test: Trigger Heavy Governance and Browser manually, confirm they run through separate workflows and artifact bundles, and verify the documented trigger matrix explains when each lane is expected.

Tests for User Story 3

  • T017 [P] [US3] Add apps/platform/tests/Feature/Guards/CiHeavyBrowserWorkflowContractTest.php and expand apps/platform/tests/Feature/Guards/HeavyGovernanceLaneContractTest.php plus apps/platform/tests/Feature/Guards/BrowserLaneIsolationTest.php to assert separate manual or scheduled workflows, non-pull-request isolation, and trigger-specific budget modes

Implementation for User Story 3

  • T018 [P] [US3] Implement .gitea/workflows/test-heavy-governance.yml for workflow_dispatch using scripts/platform-test-lane heavy-governance, with scheduled execution enabled only after the first successful manual validation
  • T019 [P] [US3] Implement .gitea/workflows/test-browser.yml for workflow_dispatch using scripts/platform-test-lane browser, with scheduled execution enabled only after the first successful manual validation
  • T020 [US3] Update apps/platform/tests/Support/TestLaneManifest.php and apps/platform/tests/Support/TestLaneBudget.php with manual and scheduled workflow profiles, governance-contract threshold handling for Heavy Governance, and warning or trend-only modes for Heavy Governance and Browser
  • T021 [US3] Update README.md, specs/210-ci-matrix-budget-enforcement/quickstart.md, and specs/210-ci-matrix-budget-enforcement/spec.md with the final trigger policy, local reproduction commands, artifact locations, blocking versus non-blocking guidance for Fast Feedback, Confidence, Heavy Governance, and Browser, and the documented Fast Feedback CI variance tolerance
  • T022 [US3] Wire Heavy Governance and Browser artifact staging and upload in .gitea/workflows/test-heavy-governance.yml, .gitea/workflows/test-browser.yml, and scripts/platform-test-artifacts so both lanes publish separate bundles with deterministic names

Checkpoint: Heavy Governance and Browser are now visible, reproducible, and isolated from the default fast path.


Phase 6: Polish & Cross-Cutting Concerns

Purpose: Validate the full matrix, record rollout evidence, and finish formatting and cleanup.

  • T023 Run focused Pest coverage for apps/platform/tests/Feature/Guards/CiFastFeedbackWorkflowContractTest.php, apps/platform/tests/Feature/Guards/CiConfidenceWorkflowContractTest.php, apps/platform/tests/Feature/Guards/CiHeavyBrowserWorkflowContractTest.php, apps/platform/tests/Feature/Guards/CiLaneFailureClassificationContractTest.php, apps/platform/tests/Feature/Guards/FastFeedbackLaneContractTest.php, apps/platform/tests/Feature/Guards/ConfidenceLaneContractTest.php, apps/platform/tests/Feature/Guards/HeavyGovernanceLaneContractTest.php, apps/platform/tests/Feature/Guards/BrowserLaneIsolationTest.php, apps/platform/tests/Feature/Guards/FixtureLaneImpactBudgetTest.php, apps/platform/tests/Feature/Guards/TestLaneManifestTest.php, apps/platform/tests/Feature/Guards/TestLaneArtifactsContractTest.php, and apps/platform/tests/Feature/Guards/TestLaneCommandContractTest.php with cd apps/platform && ./vendor/bin/sail artisan test --compact tests/Feature/Guards/CiFastFeedbackWorkflowContractTest.php tests/Feature/Guards/CiConfidenceWorkflowContractTest.php tests/Feature/Guards/CiHeavyBrowserWorkflowContractTest.php tests/Feature/Guards/CiLaneFailureClassificationContractTest.php tests/Feature/Guards/FastFeedbackLaneContractTest.php tests/Feature/Guards/ConfidenceLaneContractTest.php tests/Feature/Guards/HeavyGovernanceLaneContractTest.php tests/Feature/Guards/BrowserLaneIsolationTest.php tests/Feature/Guards/FixtureLaneImpactBudgetTest.php tests/Feature/Guards/TestLaneManifestTest.php tests/Feature/Guards/TestLaneArtifactsContractTest.php tests/Feature/Guards/TestLaneCommandContractTest.php
  • T024 Run local wrapper validation via ./scripts/platform-test-lane fast-feedback, ./scripts/platform-test-lane confidence, ./scripts/platform-test-lane heavy-governance, ./scripts/platform-test-lane browser, ./scripts/platform-test-report fast-feedback, and ./scripts/platform-test-report confidence, then confirm the CI-governance helper and guard changes do not widen default fixtures or promote fast-path coverage into Heavy Governance or Browser lane membership
  • T025 [P] Execute the required live Gitea validation evidence set: one pull_request Fast Feedback run, one push to dev Confidence run, one manual Heavy Governance run, one scheduled Heavy Governance run, one manual Browser run, and one scheduled Browser run using .gitea/workflows/test-pr-fast-feedback.yml, .gitea/workflows/test-main-confidence.yml, .gitea/workflows/test-heavy-governance.yml, and .gitea/workflows/test-browser.yml, then record trigger, executed lane, artifact bundle, budget outcome, and the primary failure class or explicit clean-success result in specs/210-ci-matrix-budget-enforcement/quickstart.md, and record the chosen Fast Feedback variance tolerance and any material runtime recalibration note in specs/210-ci-matrix-budget-enforcement/spec.md or the active PR
  • T026 Run cd apps/platform && ./vendor/bin/sail bin pint --dirty --format agent for PHP changes in apps/platform/tests/Support/TestLaneManifest.php, apps/platform/tests/Support/TestLaneBudget.php, apps/platform/tests/Support/TestLaneReport.php, and the new guard tests under apps/platform/tests/Feature/Guards/, then manually normalize shell and YAML formatting in scripts/platform-test-artifacts and .gitea/workflows/*.yml

Dependencies & Execution Order

Phase Dependencies

  • Setup (Phase 1): No dependencies and can start immediately.
  • Foundational (Phase 2): Depends on Phase 1 and blocks all user story work.
  • User Story 1 (Phase 3): Depends on Phase 2 only.
  • User Story 2 (Phase 4): Depends on Phase 2 only.
  • User Story 3 (Phase 5): Depends on Phase 2 for workflow implementation; final contributor guidance and full-matrix validation should wait until User Stories 1 and 2 are in place.
  • Polish (Phase 6): Depends on all desired user stories being complete.

User Story Dependencies

  • User Story 1 (P1): Can begin immediately after Foundational and is the MVP slice.
  • User Story 2 (P1): Can begin immediately after Foundational and remains independently valuable even if Heavy Governance and Browser are not yet wired.
  • User Story 3 (P2): Uses the same shared seams as User Stories 1 and 2 and should finish after them so the final documented trigger matrix is complete.

Within Each User Story

  • Story-specific guard tests must be written and fail before implementation.
  • Workflow metadata in TestLaneManifest.php and policy logic in TestLaneBudget.php should be in place before finalizing the matching workflow YAML.
  • Artifact upload wiring should happen after the workflow file and lane policy both exist.
  • Story validation should complete before moving on to the next priority slice.

Parallel Opportunities

  • T001 and T002 can run in parallel during Setup.
  • T004, T005, and T006 can run in parallel once T003 defines the shared manifest shape.
  • User Stories 1 and 2 can proceed in parallel after Foundational if different contributors own the workflow YAML and guard files.
  • In User Story 3, T018 and T019 can run in parallel because Heavy Governance and Browser use separate workflow files.
  • T025 can run in parallel with final formatting only after all workflows and guards are stable.

Parallel Example: User Story 1

# After T008 defines the PR contract, these can proceed in parallel:
Task: "Implement .gitea/workflows/test-pr-fast-feedback.yml for pull_request events using scripts/platform-test-lane fast-feedback"
Task: "Update apps/platform/tests/Support/TestLaneManifest.php and apps/platform/tests/Support/TestLaneBudget.php with the pr-fast-feedback workflow profile and hard-fail policy"

Parallel Example: User Story 2

# After T012 locks the Confidence workflow contract, these can proceed in parallel:
Task: "Implement .gitea/workflows/test-main-confidence.yml for push to dev using scripts/platform-test-lane confidence"
Task: "Update apps/platform/tests/Support/TestLaneManifest.php with the main-confidence workflow profile and Confidence artifact publication contract"

Parallel Example: User Story 3

# After T017 defines the heavy/browser workflow contract, these can proceed in parallel:
Task: "Implement .gitea/workflows/test-heavy-governance.yml for workflow_dispatch and scheduled execution"
Task: "Implement .gitea/workflows/test-browser.yml for workflow_dispatch and scheduled execution"

Implementation Strategy

MVP First (User Story 1 Only)

  1. Complete Phase 1: Setup.
  2. Complete Phase 2: Foundational.
  3. Complete Phase 3: User Story 1.
  4. Stop and validate the pull request fast path independently.

Incremental Delivery

  1. Deliver the blocking pull request path first.
  2. Add the broader dev Confidence path next.
  3. Add Heavy Governance and Browser as separate scheduled or manual lanes.
  4. Finish with live validation evidence and final contributor guidance.

Parallel Team Strategy

  1. One contributor extends the shared support seams in apps/platform/tests/Support/ while another prepares workflow YAML files.
  2. After Foundational completes, separate owners can implement the PR and dev workflows in parallel.
  3. Heavy Governance and Browser can then be split across two contributors because they live in separate workflow files.

Notes

  • [P] tasks operate on different files and can run in parallel once their dependencies are satisfied.
  • [US1], [US2], and [US3] map each task to the corresponding user story in spec.md.
  • This feature changes runtime validation behavior, so focused Pest guards and the narrowest relevant lane reruns are part of the definition of done.
  • Live Gitea validation is required because local wrapper tests alone cannot prove trigger behavior or uploaded artifact bundles.