TenantAtlas/spec-candidates/382-baseline-compare-result-semantics.md
ahmido dbff2a0a90 feat(report): implement management report pdf runtime (#450)
Added jobs, controllers, and PDF generation logic for management report runtime as defined in Spec 379. Includes artifact migrations, payload builders, and testing coverage.

Co-authored-by: Ahmed Darrazi <ahmed.darrazi@live.de>
Reviewed-on: #450
2026-06-15 11:36:29 +00:00

9.8 KiB

Spec Candidate 382 - Baseline Compare Result Semantics & Gap Classification v1

Candidate Status

Candidate for implementation after Specs 380 and 381.

This candidate cleans up overloaded compare/gap result states so evidence, reviews, operations, and UI can distinguish real blockers from limitations.

Depends On

  • Spec 380 - Provider Resource Identity & Binding Foundation v1
  • Spec 381 - Baseline Matching Pipeline & Canonicalization v1

Spec Candidate Check

  • Problem: Baseline compare states and gap classifications are overloaded, so operators and customer-readiness gates cannot reliably distinguish unresolved governance risk from limitations or expected provider defaults.
  • Today's failure: ambiguous_match, policy_record_missing, foundation_not_policy_backed, and missing_policy can each carry multiple meanings, causing noisy follow-ups or misleading blockers.
  • User-visible improvement: Compare details and OperationRun follow-ups can show action-required, limitation, resolved, unsupported, missing evidence, and missing resource with safer publication consequences.
  • Smallest enterprise-capable version: Introduce provider-neutral reason codes, blocker categories, legacy mapping, OperationRun context payload updates, and internal table display updates; do not add UI resolution or evidence/review integration yet.
  • Explicit non-goals: No binding model, no matching pipeline, no resolution UI, no full historical data rewrite, no customer Review Pack wording changes beyond internal state preparation.
  • Permanent complexity imported: New reason-code family, blocker categories, legacy mapper, OperationRun context changes, table display logic, and compatibility tests.
  • Why now: Specs 380-381 can resolve identity, but downstream consumers still need result semantics that separate real blockers from accepted/provider limitations.
  • Why not local: Local label changes would not change OperationRun next steps, customer-blocking logic, legacy compatibility, or evidence/review consumers.
  • Approval class: Core Enterprise.
  • Red flags triggered: New semantic axes and many reason labels. The defense is that each reason changes operator action, publication blocking, limitation disclosure, or compatibility mapping.
  • Score: Nutzen: 2 | Dringlichkeit: 2 | Scope: 1 | Komplexitaet: 1 | Produktnaehe: 2 | Wiederverwendung: 2 | Gesamt: 10/12
  • Decision: approve after Spec 381, with a strict legacy mapper and no broad UI/review changes in this slice.

Proportionality Review

  1. Current operator problem: Operators cannot trust whether a compare issue requires action, limitation acceptance, evidence refresh, or no action.
  2. Why existing structure is insufficient: Existing reason codes are overloaded and policy-specific, so they cannot safely drive provider-neutral publication/readiness behavior.
  3. Narrowest correct implementation: Add result semantics and compatibility mapping only; defer resolution UI and review readiness to Specs 383-384.
  4. Ownership cost: Baseline compare, OperationRun, and UI owners must maintain a reason-code contract and legacy mapper.
  5. Rejected alternative: Presentation-only labels were rejected because they would not change blocker behavior or OperationRun next actions.
  6. Current-release truth or future prep: Current-release trust/readiness work because existing compare outputs already feed Evidence and Review surfaces.

Problem

Current compare states and gap classifications are overloaded.

Examples:

  • ambiguous_match can mean real duplicate tenant-owned resources or false ambiguity caused by built-ins/defaults.
  • policy_record_missing can mean missing local evidence, not necessarily missing provider resource.
  • foundation_not_policy_backed can mean inventory-only foundation coverage, unsupported coverage, or false policy-centric modeling.
  • missing_policy is policy-specific and not provider/resource-class neutral.

This makes OperationRun follow-ups, Evidence Snapshot completeness, Review Output Readiness, and customer-safe Review Packs hard to trust.

Goal

Create clearer provider-neutral compare result semantics.

The system must distinguish:

  • real drift,
  • no drift,
  • unresolved ambiguity,
  • resolved built-in/default,
  • resolved manual binding,
  • missing provider resource,
  • missing local evidence,
  • unsupported resource class,
  • inventory-only foundation limitation,
  • excluded non-governed subject,
  • accepted limitation,
  • compare failure,
  • coverage unproven.

Scope

In Scope

  • Introduce or refine provider-neutral compare result reason codes.
  • Map existing legacy states to new semantics.
  • Update gap normalization.
  • Update OperationRun context payloads for new reason semantics.
  • Update baseline compare detail tables to display blocker vs limitation vs resolved.
  • Update tests around ambiguous/gap classification.
  • Preserve backward compatibility for existing compare records.
  • Add translator/mapper for legacy reason codes.

Out of Scope

  • New resolution UI.
  • Evidence/review readiness full integration.
  • Provider binding data model.
  • Matching pipeline implementation.
  • Full historical data rewrite.
  • Customer Review Pack wording changes beyond internal state preparation.

Proposed Result Reasons

resolved_exact_identity
resolved_active_binding
resolved_canonical_builtin
resolved_canonical_virtual_target
resolved_unique_fallback
unresolved_ambiguous_match
missing_provider_resource
missing_local_evidence
unsupported_resource_class
foundation_inventory_only
excluded_non_governed
accepted_limitation
drift_detected
no_drift
compare_failed
coverage_unproven

Mapping from Existing States

Existing State / Reason New Semantic
ambiguous_match unresolved_ambiguous_match unless resolved by binding/canonicalization
policy_record_missing split into missing_local_evidence or missing_provider_resource
foundation_not_policy_backed split into foundation_inventory_only, unsupported_resource_class, or accepted_limitation
missing_policy missing_provider_resource where a trusted prior/current identity exists
unexpected_policy future-compatible unexpected_resource or preserved legacy display
unsupported_subjects unsupported_resource_class
coverage_unproven keep as run-level trust limitation
strategy_failed compare_failed

Blocker Categories

Each compare issue must classify into one of:

customer_blocker
operator_action_required
internal_limitation
accepted_limitation
resolved
informational

Customer Blocking Rules

Customer Blocking

  • unresolved ambiguity for governed in-scope subject,
  • missing provider resource after trusted identity/binding,
  • compare failure preventing required posture claim,
  • missing local evidence for required governed subject without accepted limitation,
  • coverage unproven for required customer-facing control.

Not Customer Blocking by Default

  • resolved built-in/default,
  • resolved virtual assignment target,
  • excluded non-governed subject,
  • accepted limitation,
  • unsupported foundation coverage when disclosed as limitation,
  • inventory-only foundation where no drift claim is made.

OperationRun Impact

OperationRun next steps should become specific:

Resolve ambiguous baseline subjects before publication.
Re-run inventory sync; required local evidence is missing.
Accept limitation or exclude unsupported foundation objects.
Provider default objects were canonicalized automatically.

Noisy follow-ups should not appear for resolved built-ins/defaults.

UI Impact

Baseline compare detail should clearly show:

Resolved
Action required
Limitation
Unsupported
Missing evidence
Missing resource
Drift
No drift

Do not show all limitations as blockers.

Compatibility

Existing compare records must remain readable.

Add a compatibility mapper for legacy payloads.

Do not rewrite old OperationRun context in v1 unless explicitly needed for a focused backfill.

Acceptance Criteria

  • ambiguous_match no longer represents resolved built-ins/defaults.
  • policy_record_missing is split into evidence-missing vs provider-resource-missing semantics.
  • foundation_not_policy_backed no longer appears as a false policy-missing blocker for foundation resources.
  • OperationRun follow-ups distinguish action-required from limitation.
  • Compare detail tables display blocker/limitation/resolved status clearly.
  • Legacy compare records remain readable.
  • Tests prove customer-blocking logic is not triggered by accepted limitations or resolved built-ins.

Required Tests

  • Ambiguous tenant-owned duplicate produces action-required blocker.
  • Resolved canonical built-in produces resolved/informational state.
  • Missing local policy/evidence produces missing local evidence, not missing provider resource.
  • Missing provider resource after trusted binding produces true blocker.
  • Unsupported foundation resource produces limitation.
  • Accepted limitation does not produce publication blocker.
  • Legacy ambiguous_match payload can still render.
  • Legacy foundation_not_policy_backed payload maps to safe legacy diagnostic.

Validation Commands

cd apps/platform && ./vendor/bin/sail artisan test --compact tests/Feature/Baselines/BaselineCompareAmbiguousMatchGapTest.php tests/Feature/Baselines/BaselineCompareGapClassificationTest.php
cd apps/platform && ./vendor/bin/sail artisan test --compact tests/Feature/Evidence/BaselineDriftPostureSourceTest.php

Risks

  • Renaming states too aggressively and breaking UI/tests.
  • Treating limitations as green.
  • Treating all missing evidence as missing provider resources.
  • Breaking historical compare rendering.

Recommendation

Implement this third.

This candidate makes the compare output trustworthy before exposing operator resolution UI and review-pack readiness changes.