TenantAtlas/spec-candidates/384-evidence-review-readiness-integration.md
ahmido dbff2a0a90 feat(report): implement management report pdf runtime (#450)
Added jobs, controllers, and PDF generation logic for management report runtime as defined in Spec 379. Includes artifact migrations, payload builders, and testing coverage.

Co-authored-by: Ahmed Darrazi <ahmed.darrazi@live.de>
Reviewed-on: #450
2026-06-15 11:36:29 +00:00

12 KiB

Spec Candidate 384 - Evidence & Review Readiness Integration v1

Candidate Status

Candidate for implementation after Specs 380, 381, 382, and 383.

This candidate integrates the new baseline subject identity/resolution semantics into Evidence Snapshot completeness, Baseline Drift Posture, Review Output Readiness, publication blockers, and Review Pack export.

Depends On

  • Spec 380 - Provider Resource Identity & Binding Foundation v1
  • Spec 381 - Baseline Matching Pipeline & Canonicalization v1
  • Spec 382 - Baseline Compare Result Semantics & Gap Classification v1
  • Spec 383 - Baseline Subject Resolution UI & Operator Decisions v1

Spec Candidate Check

  • Problem: Evidence and Review readiness consume compare results that may still contain overloaded or false blocker states.
  • Today's failure: Customer-ready output can be blocked by expected provider defaults, internal limitations can be presented as hard blockers, and unsupported foundation coverage can look like missing input.
  • User-visible improvement: Evidence, reviews, and Review Packs can distinguish customer-ready, published-with-limitations, internal-only, and blocked states based on true unresolved governance risk.
  • Smallest enterprise-capable version: Update Baseline Drift Posture, Evidence Snapshot completeness mapping, Environment Review readiness, Review Pack readiness/guidance, and customer-safe limitation wording only.
  • Explicit non-goals: No new binding data model, no matching pipeline implementation, no resolution UI, no workflow engine, no new report profile architecture, no broad management report redesign, and no historical review migration.
  • Permanent complexity imported: Readiness reason mapping, limitation categories in evidence details, review/readiness gate updates, Review Pack wording changes, internal technical rendering, and cross-surface tests.
  • Why now: After Specs 380-383, the system has trustworthy subject resolution and operator decisions; customer-facing readiness must consume those semantics or continue showing false blockers.
  • Why not local: A local Review Pack label patch would leave Evidence Snapshot, Baseline Drift Posture, OperationRun guidance, and Environment Review readiness inconsistent.
  • Approval class: Core Enterprise.
  • Red flags triggered: Many surfaces touched and new readiness classifications. The defense is that all surfaces already consume baseline compare truth and must share publication safety semantics.
  • Score: Nutzen: 2 | Dringlichkeit: 1 | Scope: 1 | Komplexitaet: 1 | Produktnaehe: 2 | Wiederverwendung: 2 | Gesamt: 9/12
  • Decision: approve after Specs 380-383 if scope stays limited to readiness integration and customer-safe wording.

UI Surface Impact

  • Reachable surface changed/added: yes, existing Evidence, Environment Review, Review Pack, Stored Report/Review Pack readiness labels, and OperationRun guidance may change.
  • Page archetype: readiness/status integration over existing surfaces, not a new primary workflow page.
  • Design depth: update labels, limitation summaries, blocker messaging, and internal technical appendix rules; no new layout system.
  • Customer/operator safety: customer-safe output must avoid raw provider IDs, provider_resource_bindings, canonical_subject_key, internal policy version IDs, and raw OperationRun payloads.
  • Dangerous/high-impact actions: none added; publication/readiness gate behavior changes require tests because they affect customer-ready output.

Proportionality Review

  1. Current operator problem: Operators cannot safely publish customer-ready review output when false blockers and limitations are mixed together.
  2. Why existing structure is insufficient: Evidence and Review readiness currently consume overloaded compare states and cannot distinguish resolved defaults from unresolved governed blockers.
  3. Narrowest correct implementation: Consume the new semantics in existing readiness and export paths only; no new report architecture or workflow.
  4. Ownership cost: Evidence, Environment Review, Review Pack, OperationRun guidance, and test owners must maintain shared readiness mappings.
  5. Rejected alternative: Review Pack-only wording was rejected because Evidence and Environment Review would still disagree about customer readiness.
  6. Current-release truth or future prep: Current-release customer trust work because compare output already influences Evidence, Reviews, and Review Pack export.

Problem

Evidence and Review readiness currently consume baseline compare results that can contain overloaded or false blocker states.

This can cause:

  • customer-ready output blocked by expected provider defaults,
  • internal limitations presented as hard blockers,
  • unsupported foundation coverage presented as missing input,
  • unresolved ambiguity not clearly separated from accepted limitations,
  • review packs that are too conservative or not precise enough.

TenantPilot needs evidence/review integration that reflects the improved compare semantics.

Goal

Make customer readiness and publication blockers depend on real unresolved governance risk, not on expected provider defaults or accepted limitations.

Scope

In Scope

  • Update Baseline Drift Posture source consumption of new compare semantics.
  • Update Evidence Snapshot completeness mapping.
  • Update Environment Review readiness.
  • Update Review Pack output readiness.
  • Update Review Pack output resolution guidance.
  • Update customer-safe limitation wording.
  • Update internal technical detail rendering.
  • Add tests for blocker vs limitation behavior.
  • Ensure Review Pack export does not leak raw provider IDs.

Out of Scope

  • New binding data model.
  • Matching pipeline implementation.
  • Operator resolution UI.
  • Generic workflow engine.
  • New report profile architecture.
  • Broad management report redesign.
  • Full historical review migration.

Evidence Completeness Mapping

Evidence should distinguish:

complete
complete_with_limitations
partial_action_required
partial_limitation_only
missing
stale

If existing evaluator only supports:

complete
partial
missing
stale

then the additional classification can live in evidence details/readiness reason payloads while preserving public enum compatibility.

Baseline Drift Posture Rules

Baseline Drift Posture must distinguish:

No drift found
Drift found
Compare blocked by unresolved ambiguity
Compare partial due to unsupported limitations
Compare incomplete due to missing evidence
Compare failed

Zero findings must not be treated as missing input when compare completed successfully.

Publication Rules

Customer-ready

Allowed when:

  • all required governed subjects are resolved,
  • no unresolved ambiguity hides real drift,
  • no required evidence is missing,
  • accepted limitations are allowed by profile/report policy,
  • customer-safe limitation summary is available where needed.

Published with limitations

Allowed when:

  • unsupported/foundation inventory-only limitations are accepted,
  • excluded non-governed subjects are audited,
  • no unresolved governed blocker remains,
  • limitations are disclosed in customer-safe language.

Internal-only

Required when:

  • unresolved ambiguity remains,
  • required evidence is missing,
  • compare cannot support customer-safe posture claim,
  • missing provider resource cannot be confidently interpreted,
  • limitation wording would be misleading externally.

Publication blocked

Required when:

  • unresolved governed blockers exist,
  • required review sections cannot be composed,
  • evidence basis is missing,
  • compare failed for required dimension,
  • output would imply false assurance.

Review Pack Wording

Customer-safe wording must avoid internal implementation jargon.

Do not expose:

provider_resource_bindings
canonical_subject_key
raw provider IDs
internal policy version IDs
operation_run context payloads

Customer-safe examples:

Some baseline resources could not be verified because multiple matching resources require operator confirmation.
Some foundation configuration classes are currently monitored as inventory-only and are disclosed as limitations.
No drift was identified for the verified baseline scope.

Internal technical appendix may include more detail if the report profile allows it.

OperationRun and Review Guidance

Review guidance should produce next steps such as:

Resolve ambiguous baseline subjects before publication.
Refresh evidence after baseline subject resolutions are applied.
Accept or exclude unsupported foundation resources before creating a customer-ready pack.
Re-run baseline compare after inventory sync completes.

It should not say:

Resolve review blockers

without explaining the real blocker class.

Integration Points

Expected areas:

  • BaselineDriftPostureSource
  • EvidenceCompletenessEvaluator
  • Evidence Snapshot sources
  • EnvironmentReviewReadinessGate
  • review composer services
  • ReviewPackOutputReadiness
  • ReviewPackOutputResolutionGuidance
  • Review Pack export/rendering services
  • Stored Report / Review Pack UI readiness labels
  • OperationRun follow-up support

Acceptance Criteria

  • Resolved built-ins/defaults do not create review publication blockers.
  • Accepted limitations produce "published with limitations" where policy allows.
  • Unsupported foundation inventory-only coverage is not mislabeled as missing input.
  • Real unresolved ambiguity blocks customer-ready output.
  • Missing provider resource after trusted identity blocks or surfaces as drift according to semantics.
  • Missing local evidence is described as evidence limitation, not resource drift.
  • Review Pack output remains customer-safe.
  • Internal technical details remain available for operators.
  • YPTW2-like compare results can become internal-only, customer-ready, or published-with-limitations based on true unresolved state.

Required Tests

  • Evidence test: resolved built-in does not make baseline drift posture partial action-required.
  • Evidence test: unresolved ambiguity makes posture partial action-required.
  • Evidence test: accepted limitation makes posture complete-with-limitations or equivalent.
  • Review test: publication blocker appears only for unresolved governed blocker.
  • Review test: accepted limitation does not block customer-ready output when report policy allows limitations.
  • Review test: unsupported foundation appears as limitation, not missing input.
  • Review Pack test: customer-safe output does not expose raw provider IDs.
  • Review Pack test: internal profile can include technical diagnostics.
  • Regression test: zero drift findings after completed compare is valid complete state.
  • Regression test: historical/legacy compare payload still renders safely.

Validation Commands

cd apps/platform && ./vendor/bin/sail artisan test --compact tests/Feature/Evidence/BaselineDriftPostureSourceTest.php
cd apps/platform && ./vendor/bin/sail artisan test --compact tests/Unit/EnvironmentReview/EnvironmentReviewComposerTest.php
cd apps/platform && ./vendor/bin/sail artisan test --compact tests/Feature/ReviewPack/Spec349ReviewPackResolutionGuidanceTest.php

Add focused tests for new readiness reason mapping.

Risks

  • Accidentally allowing customer-ready output with unresolved risk.
  • Blocking customer output for expected provider defaults.
  • Leaking raw provider identities in customer pack.
  • Creating inconsistent semantics between Evidence, Review, and Review Pack.
  • Producing too many limitation labels and lowering trust.

Recommendation

Implement this fifth.

This candidate completes the full product loop: identity -> matching -> result semantics -> operator resolution -> evidence/review/customer-ready output.