173 lines
8.6 KiB
Markdown
173 lines
8.6 KiB
Markdown
# Data Model: CI Test Matrix & Runtime Budget Enforcement
|
|
|
|
## Model Scope
|
|
|
|
This feature adds no product database tables. The model below describes checked-in repository governance objects that live in workflow YAML, repo-root scripts, and the existing `TestLaneManifest` / `TestLaneBudget` / `TestLaneReport` seams.
|
|
|
|
## 1. WorkflowProfile
|
|
|
|
Represents one checked-in Gitea workflow file that owns a single trigger class.
|
|
|
|
| Field | Type | Description |
|
|
|---|---|---|
|
|
| `workflowId` | string | Stable identifier such as `pr-fast-feedback` or `main-confidence`. |
|
|
| `filePath` | string | Checked-in workflow path under `.gitea/workflows/`. |
|
|
| `triggerClass` | enum | One of `pull-request`, `mainline-push`, `scheduled`, `manual`. |
|
|
| `gitEvents` | list<string> | Concrete Gitea events that activate the workflow. |
|
|
| `branchFilters` | list<string> | Branch list or schedule scope, such as `dev`. |
|
|
| `runnerLabel` | string | Single runner label used by the workflow. |
|
|
| `blockingDefault` | boolean | Whether this workflow blocks the associated shared path by default. |
|
|
| `scheduleCron` | string or null | Cron expression for scheduled workflows. |
|
|
| `laneBindings` | list<string> | Ordered lane ids executed by the workflow. |
|
|
|
|
**Validation rules**
|
|
|
|
- One `WorkflowProfile` owns exactly one trigger class.
|
|
- Pull request workflows cannot bind `heavy-governance` or `browser`.
|
|
- The mainline workflow targets `dev`.
|
|
- Scheduled and manual workflows must remain explicit about which heavy lane they own.
|
|
|
|
## 2. LaneBinding
|
|
|
|
Represents the workflow-to-lane mapping for an existing `TestLaneManifest` lane.
|
|
|
|
| Field | Type | Description |
|
|
|---|---|---|
|
|
| `laneId` | enum | Existing manifest lane id: `fast-feedback`, `confidence`, `heavy-governance`, `browser`, `profiling`, or `junit`. |
|
|
| `executionWrapper` | string | Repo-root wrapper path, expected to be `scripts/platform-test-lane`. |
|
|
| `reportWrapper` | string | Repo-root report wrapper path, expected to be `scripts/platform-test-report`. |
|
|
| `commandRef` | string | Existing manifest command reference such as `test` or `test:confidence`. |
|
|
| `governanceClass` | enum | `fast`, `confidence`, `heavy`, or `support`. |
|
|
| `parallelMode` | enum | `required`, `optional`, or `forbidden`, inherited from the manifest. |
|
|
| `requiredArtifacts` | list<string> | Artifact types that must exist before upload. |
|
|
| `optionalArtifacts` | list<string> | Artifact types that may exist for this lane only. |
|
|
| `artifactExportProfile` | string | Stable export profile used by the staging helper. |
|
|
|
|
**Validation rules**
|
|
|
|
- `laneId` must exist in `TestLaneManifest::lanes()`.
|
|
- Workflows may not inline selectors or test paths that bypass the lane binding.
|
|
- Confidence and Fast Feedback must both require JUnit, summary, budget, and report artifacts.
|
|
- `profiling` may remain unbound from default CI workflows while still existing in the model.
|
|
|
|
## 3. BudgetEnforcementProfile
|
|
|
|
Represents how one lane budget behaves for one trigger class.
|
|
|
|
| Field | Type | Description |
|
|
|---|---|---|
|
|
| `policyId` | string | Stable identifier such as `pr-fast-feedback-budget`. |
|
|
| `laneId` | string | Lane governed by the policy. |
|
|
| `triggerClass` | string | Trigger class where the policy applies. |
|
|
| `thresholdSource` | enum | `lane-budget` or `governance-contract`. |
|
|
| `baseThresholdSeconds` | number | Threshold before CI tolerance is applied. |
|
|
| `varianceAllowanceSeconds` | number | Tolerance added for CI runner variance. |
|
|
| `effectiveThresholdSeconds` | number | Threshold used to classify the run. |
|
|
| `enforcementMode` | enum | `hard-fail`, `soft-warn`, or `trend-only`. |
|
|
| `lifecycleState` | string | Existing maturity state such as `documented`, `draft`, or `recalibrated`. |
|
|
| `reviewCadence` | string | Human review cadence for re-evaluating the policy. |
|
|
|
|
**Validation rules**
|
|
|
|
- `hard-fail` requires a documented tolerance strategy.
|
|
- `governance-contract` is reserved for `heavy-governance`.
|
|
- `soft-warn` or `trend-only` are the default for immature or scheduled lanes.
|
|
|
|
**State transitions**
|
|
|
|
- `within-budget` → `warning` → `over-budget`
|
|
- The mapped workflow outcome is trigger-aware:
|
|
- `hard-fail`: `over-budget` fails the run.
|
|
- `soft-warn`: `warning` or `over-budget` marks the run as non-blocking warning.
|
|
- `trend-only`: any budget drift is informational only.
|
|
|
|
## 4. ArtifactPublicationContract
|
|
|
|
Represents the per-lane artifact staging and upload rules.
|
|
|
|
| Field | Type | Description |
|
|
|---|---|---|
|
|
| `contractId` | string | Stable artifact contract id per lane. |
|
|
| `laneId` | string | Lane whose artifacts are being exported. |
|
|
| `sourceDirectory` | string | Local artifact root, expected to be `storage/logs/test-lanes`. |
|
|
| `sourcePatterns` | list<string> | Local file patterns such as `fast-feedback-latest.summary.md`. |
|
|
| `requiredFiles` | list<string> | Files that must be exported for a successful lane publication. |
|
|
| `optionalFiles` | list<string> | Files exported only when the lane supports them, such as `profile.txt`. |
|
|
| `stagingDirectory` | string | Per-run upload directory created by the staging helper. |
|
|
| `stagedNamePattern` | string | Deterministic upload naming pattern per lane and run. |
|
|
| `uploadGroupName` | string | Artifact bundle name shown in CI. |
|
|
| `retentionClass` | enum | `pr-short`, `mainline-medium`, or `scheduled-medium`. |
|
|
|
|
**Validation rules**
|
|
|
|
- `summary.md`, `budget.json`, `report.json`, and `junit.xml` are required for all governed lanes.
|
|
- `profile.txt` is optional and only expected for `profiling` unless future policy widens it.
|
|
- Upload bundles must remain lane-specific and reproducibly named.
|
|
|
|
## 5. FailureClassification
|
|
|
|
Represents the one primary reason a governed CI run did not succeed cleanly.
|
|
|
|
| Field | Type | Description |
|
|
|---|---|---|
|
|
| `failureClassId` | enum | `test-failure`, `wrapper-failure`, `budget-breach`, `artifact-publication-failure`, or `infrastructure-failure`. |
|
|
| `sourceStep` | string | Workflow step or repo helper that originated the classification. |
|
|
| `blockingOn` | list<string> | Trigger classes where this failure class is blocking. |
|
|
| `summaryLabel` | string | Human-readable label exposed in the CI summary. |
|
|
| `remediationHint` | string | Primary next action for contributors or maintainers. |
|
|
|
|
**Validation rules**
|
|
|
|
- Every non-success or warning run records exactly one primary failure class.
|
|
- Budget warnings cannot obscure wrapper or artifact failures.
|
|
- Infrastructure failures remain reserved for checkout, runner, or environment setup issues.
|
|
|
|
**State transitions**
|
|
|
|
- `unclassified` → `classified`
|
|
- `classified` then maps to workflow outcome: `blocked`, `warning`, or `informational`.
|
|
|
|
## 6. CiRunOutcome
|
|
|
|
Represents the summarized outcome of one workflow-lane execution pair.
|
|
|
|
| Field | Type | Description |
|
|
|---|---|---|
|
|
| `runId` | string | Workflow run identifier from CI. |
|
|
| `workflowId` | string | Workflow profile that executed. |
|
|
| `laneId` | string | Lane that was executed. |
|
|
| `testStatus` | enum | `passed` or `failed`. |
|
|
| `budgetStatus` | enum | `within-budget`, `warning`, or `over-budget`. |
|
|
| `artifactStatus` | enum | `complete` or `incomplete`. |
|
|
| `primaryFailureClassId` | string or null | Primary failure class when not cleanly successful. |
|
|
| `blockingStatus` | enum | `blocking`, `non-blocking-warning`, or `informational`. |
|
|
| `publishedArtifacts` | list<string> | Exported artifact names for the run. |
|
|
|
|
**State transitions**
|
|
|
|
- `pending` → `executing` → `succeeded`
|
|
- `pending` → `executing` → `warning`
|
|
- `pending` → `executing` → `failed`
|
|
|
|
## 7. ValidationEvidencePack
|
|
|
|
Represents the acceptance evidence collected for one workflow path during rollout.
|
|
|
|
| Field | Type | Description |
|
|
|---|---|---|
|
|
| `evidenceId` | string | Stable validation id per workflow path. |
|
|
| `workflowId` | string | Workflow being validated. |
|
|
| `sampleTrigger` | string | Example event used during validation. |
|
|
| `expectedLaneIds` | list<string> | Lanes expected to run for that workflow. |
|
|
| `expectedArtifacts` | list<string> | Artifacts that must be present. |
|
|
| `expectedBudgetMode` | string | Expected budget enforcement mode during validation. |
|
|
| `expectedFailureClasses` | list<string> | Failure classes that must remain distinguishable. |
|
|
| `verificationMethod` | enum | `local-guard`, `manual-ci-run`, or `scheduled-ci-run`. |
|
|
| `recordedAt` | datetime string | When the validation evidence was captured. |
|
|
|
|
## Relationships
|
|
|
|
- `WorkflowProfile` has many `LaneBinding` records.
|
|
- `LaneBinding` has one `BudgetEnforcementProfile` and one `ArtifactPublicationContract`.
|
|
- `CiRunOutcome` references one `WorkflowProfile`, one `LaneBinding`, and zero or one `FailureClassification`.
|
|
- `ValidationEvidencePack` references one `WorkflowProfile` and the expected `LaneBinding` set. |