TenantAtlas/specs/120-secret-redaction-integrity/spec.md

Feature Specification: Secret Redaction Hardening & Snapshot Data Integrity

Feature Branch: 120-secret-redaction-integrity
Created: 2026-03-06
Updated: 2026-03-07
Status: Draft
Owner: Platform / Security
Input: User description: "Spec 120 — Secret Redaction Hardening & Snapshot Data Integrity (Enterprise)"

Spec Scope Fields (mandatory)

• Scope: workspace + tenant
• Primary Routes:
  • Tenant-context drift and finding detail views
  • Tenant-context baseline compare landing and related run detail links
  • Workspace and tenant monitoring views that show sanitized run or audit context
  • Existing verification and review/export surfaces that render policy evidence
• Data Ownership:
  • Tenant-owned: policy snapshots, drift evidence, restore evidence, compare inputs, operation context tied to a tenant
  • Workspace-owned: audit records, review/export artifacts, workspace-level monitoring context
• RBAC:
  • No new roles or capabilities are introduced
  • Existing membership and capability rules continue to govern snapshot capture, drift/compare visibility, restore flows, monitoring views, and exports
  • Non-members remain 404; members without capability remain 403

For canonical-view specs: not applicable.

Problem Statement

Current secret redaction rules treat broad words such as “password”, “token”, and “certificate” as automatic matches whenever they appear inside a key name. That behavior incorrectly masks normal policy configuration data before it is persisted. Once those values are replaced, downstream experiences such as drift detection, compare views, restore fidelity, evidence exports, and audit review cannot recover the original meaning.

The product needs a hardened redaction model that protects actual secrets while preserving legitimate configuration data and maintaining trustworthy governance evidence.

Goals

• Zero false positives for configuration fields: legitimate policy settings remain visible and comparable.
• Reliable secret protection: true credentials, keys, passphrases, and tokens stay hidden across storage, logs, exports, and UI output.
• Meaningful change detection: secret changes remain detectable without exposing secret values.
• Single source of truth: one classification policy governs all snapshot, audit, and output redaction behavior.

Non-Goals

• No vault or external secret-management project.
• No historical-remediation workflow for pre-go-live data.
• No expansion of RBAC scope or new operator personas.
• No requirement to extend GraphContractRegistry with secret_fields metadata in the first release.

Definitions

• Secret field: a value that must never be exposed verbatim to operators or exported artifacts.
• Configuration field: a policy setting whose actual value must remain available for compare, drift, restore, and audit comprehension.
• Protected snapshot: a persisted policy snapshot that preserves non-secret values and stores only protected representations for secret values.
• Output context: any user-facing or operator-facing rendering surface, export, notification, or log message derived from protected data.

Assumptions

• Existing capture, drift, compare, restore, audit, and export workflows remain the same functional entry points.
• Pre-go-live development data is disposable and no historical policy snapshot inventory will be carried into production.
• This feature may change how existing viewers render evidence, but it does not introduce new primary user workflows.

Clarifications

Session 2026-03-06

• Q: Where should protected snapshot fingerprinting/versioning data live? → A: Add dedicated policy_versions.secret_fingerprints and policy_versions.redaction_version columns.
• Q: How should protected secret fields be represented inside persisted snapshots? → A: Persist them as a constant placeholder like [REDACTED], with secret_fingerprints holding the change signal.
• Q: How should workspace-scoped fingerprint HMACs be derived? → A: Derive them from the app key plus a stable workspace-specific salt or identifier.
• Q: Should schema-driven GraphContractRegistry secret metadata be part of the initial implementation? → A: No. Keep the first release focused on a central exact/path-based classifier and defer schema-driven contract metadata to a follow-up phase.

Session 2026-03-07

• Q: Do we need a legacy-data remediation workflow before go-live? → A: No. Pre-go-live data is disposable, production starts with fresh compliant captures, and no historical snapshot inventory is being migrated in.

User Scenarios & Testing (mandatory)

User Story 1 - Preserve trustworthy drift and compare evidence (Priority: P1)

As a tenant operator, I can compare snapshots and investigate drift without harmless configuration values being masked as secrets, so findings remain trustworthy and actionable.

Why this priority: Preventing silent data corruption is the core value of this feature and unblocks every downstream governance workflow.

Independent Test: Capture or compare policies containing known safe configuration fields and verify that those values remain visible in snapshot-backed evidence while actual secret values remain protected.

Acceptance Scenarios:

1. Given a policy snapshot contains configuration fields whose names include words like “password”, “token”, or “certificate”, When the snapshot is persisted and later used for drift or compare, Then those configuration values remain available for meaningful comparison.
2. Given two snapshots differ only in a protected secret value, When drift or compare is run, Then the system reports that a protected value changed without revealing the value itself.

---

User Story 2 - Protect secrets consistently across operational surfaces (Priority: P2)

As a workspace or tenant operator, I can use monitoring, audit, verification, and export surfaces without secret values leaking into visible output.

Why this priority: Defense-in-depth must remain intact even after storage integrity is fixed.

Independent Test: View monitoring details, verification output, review/export artifacts, and audit context for records that include both true secrets and similarly named configuration fields, then verify that only the true secrets are hidden.

Acceptance Scenarios:

1. Given an audit or monitoring record includes both protected fields and harmless configuration values, When the record is rendered, Then only the protected values are hidden and the diagnostic meaning of the record remains intact.
2. Given a verification or export artifact includes policy evidence with protected values, When an operator views or downloads the artifact, Then no secret value is exposed while non-secret configuration evidence remains understandable.

Edge Cases

• Unknown field names that contain sensitive-looking words but behave like configuration fields must default to preserving operator-meaningful data unless the field is explicitly classified as protected.
• Nested structures and repeated field names must be classified consistently regardless of depth or path segment position.
• The same secret value used in different workspaces must remain non-correlatable across workspaces while still allowing change detection within a workspace.
• Free-text messages that mention words like passwordMinimumLength or tokenType must remain readable unless they contain an actual secret value pattern.

Requirements (mandatory)

Constitution alignment (required)

• This feature changes how snapshot and audit data are protected before downstream use. It does not add new Microsoft Graph endpoints, and it does not add a new operator workflow.
• Existing capture, compare, restore, and export behaviors remain observable through the existing operations surfaces.

Constitution alignment (OPS-UX)

• Existing capture, compare, restore, and export operations MUST continue to follow the three-surface feedback contract when user-initiated.
• OperationRun lifecycle ownership remains unchanged; this feature does not allow direct status or outcome writes outside the service-owned lifecycle.
• Regression tests MUST cover any new run summary or redaction-related operations behavior introduced by this feature.

Constitution alignment (RBAC-UX)

• Authorization planes involved: tenant/admin /admin with tenant-context views plus workspace/admin monitoring and audit surfaces.
• Non-members or users outside the permitted workspace or tenant scope continue to receive 404 responses.
• Members lacking the existing capability to view findings, exports, monitoring, or audits continue to receive 403 responses.
• Any operator action that starts snapshot capture, restore, or export remains enforced server-side and may not rely on UI visibility alone.
• No raw capability strings or role-string checks are introduced by this feature.

Constitution alignment (BADGE-001)

• If existing UI surfaces display a “protected change” indicator, those labels MUST use centralized badge semantics rather than page-specific mappings.
• The initial release does not require new badge-only indicators; explanatory copy may remain text-first.

Constitution alignment (Filament Action Surfaces)

• This feature changes rendering behavior on existing Filament surfaces but does not add new destructive actions or new mutation entry points in scope.
• The action surface contract remains satisfied because all existing actions keep their current behavior; only the evidence shown behind those actions changes.

Constitution alignment (UX-001 — Layout & Information Architecture)

• Existing Filament screens may display clearer protected-value messaging, but this feature does not introduce new create/edit forms.
• Any new empty or blocked state related to protected evidence MUST keep a single clear call to action and maintain existing information architecture patterns.

Functional Requirements

• FR-120-001 Exact secret classification: The system MUST classify protected fields using an exact, path-aware classification policy rather than broad substring matching.
• FR-120-002 Zero false positives for known configuration data: The system MUST preserve legitimate configuration values even when their field names include sensitive-looking words.
• FR-120-003 Single classification authority: Snapshot protection, audit sanitization, and output sanitization MUST all rely on the same authoritative classification rules.
• FR-120-003a Initial classifier scope: The first release MUST implement the classification authority as a central exact/path-based classifier and MUST NOT require new GraphContractRegistry secret-field metadata to be complete.
• FR-120-004 Single ownership of snapshot protection: The system MUST apply snapshot protection at one clearly defined point in the persistence flow and MUST NOT apply duplicate masking passes to the same snapshot write.
• FR-120-005 Dedicated persistence fields: The system MUST persist protected-change data in dedicated policy_versions.secret_fingerprints and policy_versions.redaction_version columns rather than embedding it in generic metadata.
• FR-120-006 Persisted secret placeholder contract: The system MUST preserve the original field shape in persisted snapshots by replacing protected secret values with a constant placeholder token such as [REDACTED]; the placeholder itself MUST NOT be used as the change-detection signal.
• FR-120-007 Protected change detectability: The system MUST retain a non-reversible protected-value change signal so compare, drift, and review workflows can report that a secret changed without exposing the secret.
• FR-120-007a Workspace-scoped fingerprint derivation: The system MUST derive protected-value fingerprints from the application key plus a stable workspace-specific salt or identifier so the same secret cannot be correlated across workspaces.
• FR-120-008 Snapshot integrity for downstream workflows: Protected snapshots MUST remain usable as the source of truth for drift detection, baseline comparison, restore fidelity, and evidence generation.
• FR-120-009 Audit integrity: Audit context protection MUST hide true secret values while preserving non-secret operational details needed for investigation and support.
• FR-120-010 Output-context hardening: Monitoring views, verification output, review/export artifacts, notifications, and human-readable failure messages MUST avoid leaking secret values while remaining readable and diagnostically useful.
• FR-120-011 Transparency of protection behavior: Operators MUST be able to tell when protected values were hidden without revealing the hidden values themselves.
• FR-120-012 Regression guardrails: The product MUST include automated guardrails that fail when new broad substring-based redaction logic is introduced into protected storage or audit paths.

Non-Functional Requirements

• NFR-120-001 Determinism: The same input snapshot and classification rules MUST always produce the same protected result and the same protected-change signal.
• NFR-120-002 Workspace isolation: Protected-change signals MUST not enable value correlation across workspace boundaries and MUST rely on the workspace-scoped derivation strategy defined for fingerprints.
• NFR-120-003 Explainability: Operators MUST be able to distinguish visible configuration data from intentionally hidden protected values.

Deferred Follow-Up

• Schema-driven secret classification via GraphContractRegistry metadata is explicitly deferred to a later phase after the central classifier and regression corpus are stable.

UI Action Matrix (mandatory when Filament is changed)

Surface	Location	Header Actions	Inspect Affordance (List/Table)	Row Actions (max 2 visible)	Bulk Actions (grouped)	Empty-State CTA(s)	View Header Actions	Create/Edit Save+Cancel	Audit log?	Notes / Exemptions
Resource Page	app/Filament/Resources/FindingResource/Pages/ViewFinding.php	No change	Existing finding list → view	No change	No change	No change	Existing workflow actions unchanged	N/A	Yes	Read-only drift/compare evidence may show protected-change messaging; no new actions are introduced.
Widget / Support Viewer	app/Filament/Widgets/Tenant/TenantVerificationReport.php and app/Filament/Support/VerificationReportViewer.php	No change	Existing verification widget drill-in	N/A	N/A	Existing verification empty state remains single-CTA	N/A	N/A	No new mutation	Output sanitization changes only; action surface contract unchanged.
Page	app/Filament/Pages/Operations/TenantlessOperationRunViewer.php	No change	Existing operations list → run detail	No change	No change	No change	No change	N/A	Yes	Human-readable failure and context rendering becomes more precise; no new actions or destructive flows are added.

Key Entities (include if feature involves data)

• Protected Snapshot Record: the persisted source-of-truth record used by compare, drift, restore, and evidence workflows, containing visible configuration data plus dedicated secret_fingerprints and redaction_version fields for protected secret tracking.
• Protected Placeholder: the constant persisted token used to preserve snapshot field shape for secret values while the real change signal is stored separately in secret_fingerprints.
• Protected Change Signal: a non-reversible marker that allows the system to determine whether a secret changed without revealing the secret value.
• Workspace-Scoped Fingerprint Salt: the stable workspace-specific input combined with the app key to derive non-correlatable secret fingerprints.
• Secret Classification Policy: the authoritative rule set that determines whether a field is a true secret or a visible configuration field.

Success Criteria (mandatory)

Measurable Outcomes

• SC-120-001 Zero safe-field regressions: 100% of the approved regression corpus of known configuration fields remains visible after protection is applied.
• SC-120-002 Protected-field coverage: 100% of the approved regression corpus of known secret fields is hidden in snapshot, audit, and output-context tests.
• SC-120-003 Meaningful protected-change detection: When only a protected value changes, compare and drift workflows report a protected change without exposing the value in 100% of covered regression scenarios.
• SC-120-004 Diagnostic readability: Verification, monitoring, and failure-message regression tests preserve operator-readable messages for known safe phrases such as policy configuration names in 100% of covered cases.