ahmido/TenantAtlas
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects 1 Releases Wiki Activity
platform-dev
TenantAtlas/specs/295-full-suite-ci-baseline/data-model.md
ahmido f03555eae1 Spec 295: full suite CI lane baseline (#350) 
## Summary
- add the Spec 295 artifacts for full-suite failure classification and CI lane baseline work
- fix `scripts/platform-test-artifacts` so Sail passes artifact staging inputs into the embedded PHP script via argv
- add a guard test covering the artifact staging input contract

## Scope guards
- no browser screenshot baselines included
- no generated test artifacts included
- no runtime application code changes included

## Notes
- classification evidence and follow-up ownership are documented in `specs/295-full-suite-ci-baseline/failure-classification.md`
- this PR is intentionally limited to the CI/lane/artifact contract slice for Spec 295

Co-authored-by: Ahmed Darrazi <ahmed.darrazi@live.de>
Reviewed-on: #350
2026-05-11 11:14:56 +00:00

4.5 KiB
Raw Permalink Blame History

Data Model: Full Suite Failure Classification & CI Lane Baseline

Spec 295 introduces no application entity, table, model, enum, migration, or persisted runtime artifact. Its only modeled data is spec-local workflow truth for implementation: a bounded failure-classification artifact and one final CI readiness decision.

Spec-Local Artifact: failure-classification.md

failure-classification.md records the classification state for implementation. It is not product truth and must not be read by the application.

Failure Group

Field Meaning
Group One failing test file, assertion cluster, wrapper failure, report failure, artifact failure, budget breach, or environment failure sharing the same cause and owner
Observed Command Exact command that produced the signal
Seam One pinned CI/suite seam
Category One pinned failure-classification category
Observed Failure Concise description of the failure evidence
Candidate Owner Existing script, support class, guard test, product area, or follow-up package likely responsible
Fix In 295? yes, no, or only-if-ci-contract-proven
Follow-up Follow-up spec/lane decision, or none when resolved
Status pending, classified, fixed, resolved, follow-up-required, or environment-blocked

Pinned Categories

Category Meaning
ci-signal-restored Full suite or lane split is green and usable as a CI signal
ci-wrapper-or-manifest-regression Wrapper, composer script, workflow binding, or lane manifest no longer invokes the intended lane
artifact-publication-regression Required report/JUnit/budget/profile/trend artifacts are not generated or staged as contracted
budget-or-trend-baseline-drift Tests pass or mostly pass, but runtime budget/trend baseline output is stale or no longer interpretable
product-runtime-or-test-regression A real app/test behavior failure outside the CI wrapper/report/artifact contract
browser-lane-regression Existing browser lane or smoke failure that needs browser-specific follow-up unless it is a CI artifact issue
flaky-or-environment Nondeterministic, local container, browser runtime, database, queue, or runner issue
follow-up-spec-required Confirmed out-of-scope failure needing a separate spec/lane owner
resolved-or-not-needed Initially suspected group that no longer needs work after rerun or adjacent classification

Pinned CI / Suite Seams

Seam Meaning
raw-full-suite Direct sail artisan test --compact complete suite signal
fast-feedback-lane Existing fast-feedback wrapper and manifest selection
confidence-lane Existing confidence wrapper and manifest selection
heavy-governance-lane Existing heavy-governance wrapper and manifest selection
browser-lane Existing browser wrapper and smoke selection
profiling-or-junit-support Support lanes used for profiling or durable machine-readable output
lane-reporting scripts/platform-test-report and TestLaneReport output
artifact-publication scripts/platform-test-artifacts and lane artifact contracts
budget-trend-baseline TestLaneBudget, lane thresholds, and trend-history classification
legacy-cutover-regression-guard Failures that appear to challenge the retired route/panel baseline from Specs 287 to 293
provider-verification-regression-guard Failures that appear to challenge Spec 294 provider/verification semantics

Final Readiness Decision

Decision Meaning
restored-ci-signal Full suite and required lanes are green or have only non-blocking documented budget notes
classified-follow-up-required CI is not fully green, but every remaining red group is classified with follow-up ownership
blocked-by-environment Classification cannot complete because local/runner environment failures prevent trustworthy signal collection

Invariants

  • Every failing group must use exactly one pinned category and one pinned seam.
  • ci-signal-restored may be used only when the relevant command output is green.
  • Product/runtime failures cannot be fixed in 295 unless the classification proves a direct CI/lane contract defect.
  • Retired /admin/t/... or TenantPanelProvider behavior must never be restored as a remedy.
  • The same category and seam names must appear in spec.md, plan.md, tasks.md, quickstart.md, checklists/requirements.md, and failure-classification.md.