ahmido
d3158f5103
## Summary - align verification-start tests with the canonical credential-enabled provider fixture - seed required tenant-permission evidence for provider operation start tests so inventory/compliance assertions exercise the real queued and `scopeBusy` contracts - refresh stale provider-connection and verification-report test baselines to current shared output - add the complete Spec 294 artifacts for the bounded provider/verification stabilization follow-up ## Scope - bounded to `apps/platform/tests`, shared Pest test helpers, and `specs/294-provider-verification-runtime-semantics` - no runtime application code changes under `apps/platform/app` - no schema, route-cutover, framework, or asset changes ## Validation - `cd apps/platform && ./vendor/bin/sail artisan test --compact tests/Feature/Verification/VerificationAuthorizationTest.php tests/Feature/Verification/VerificationStartAfterCompletionTest.php tests/Feature/Verification/VerificationStartDedupeTest.php` - `cd apps/platform && ./vendor/bin/sail artisan test --compact tests/Feature/ProviderConnections/ProviderDispatchGateStartSurfaceTest.php tests/Feature/ProviderConnections/ProviderOperationConcurrencyTest.php` - `cd apps/platform && ./vendor/bin/sail artisan test --compact tests/Feature/ProviderConnections/ProviderConnectionNeutralitySpec238Test.php tests/Feature/Verification/ProviderConnectionHealthCheckWritesReportTest.php` - `cd apps/platform && ./vendor/bin/sail artisan test --compact tests/Feature/ProviderConnections tests/Feature/Verification` - `cd apps/platform && ./vendor/bin/sail bin pint --dirty --format agent` ## Notes - browser smoke was not run because the final diff contains no runtime app or UI changes; only tests, shared test helpers, and spec artifacts changed - provider registration remains unchanged in `apps/platform/bootstrap/providers.php` - no new globally searchable resource or destructive action behavior was introduced Co-authored-by: Ahmed Darrazi <ahmed.darrazi@live.de> Reviewed-on: #349
52 lines
5.0 KiB
Markdown
52 lines
5.0 KiB
Markdown
# Failure Classification: Provider Verification Runtime Semantics Stabilization
|
|
|
|
## Baseline
|
|
|
|
Observed from:
|
|
|
|
```bash
|
|
export PATH="/bin:/usr/bin:/usr/local/bin:$PATH" && cd apps/platform && ./vendor/bin/sail artisan test --compact tests/Feature/ProviderConnections tests/Feature/Verification
|
|
```
|
|
|
|
Current confirmed baseline at prep time: **11 failed, 98 passed, 755 assertions**.
|
|
|
|
Post-implementation verification for the bounded lane: **0 failed, 109 passed, 782 assertions**.
|
|
|
|
Grouping unit: one row below equals one failing test file or one failing scenario cluster that shares one seam and one category. The current 11 failing assertions collapse into 7 tracked groups.
|
|
|
|
## Pinned Categories
|
|
|
|
- `surface-or-report-baseline-drift`
|
|
- `fixture-contract-drift`
|
|
- `provider-verification-runtime-regression`
|
|
- `dedupe-concurrency-contract-drift`
|
|
- `out-of-scope-existing-debt`
|
|
- `resolved-or-not-needed`
|
|
|
|
## Pinned Seams
|
|
|
|
- `provider-neutrality-surface`
|
|
- `shared-startable-fixtures`
|
|
- `verification-start-contract`
|
|
- `provider-dispatch-concurrency`
|
|
- `verification-report-summary`
|
|
|
|
## Current Failing Groups
|
|
|
|
| Group | Seam | Category | Observed Failure | Candidate Owner | Fix In 294? | Follow-up | Status |
|
|
|---|---|---|---|---|---|---|---|
|
|
| `ProviderConnectionNeutralitySpec238Test` | `provider-neutrality-surface` | `surface-or-report-baseline-drift` | Current provider connection disclosure no longer matches one or more stale Spec 238 wording expectations, while the neutral target-scope summary remains the intended owner seam. | `ProviderConnectionSurfaceSummary`, target-scope descriptor/normalizer, touched resource detail/list expectations | yes | Resolved by refreshing the assertion to the current neutral `Provider context` baseline. | resolved |
|
|
| `ProviderDispatchGateStartSurfaceTest` | `provider-dispatch-concurrency` | `dedupe-concurrency-contract-drift` | Starting a different provider operation on the same active connection no longer returns the expected shared `scopeBusy` follow-through and queue/run count contract. | `ProviderOperationStartGate` | yes | Resolved after seeding the required provider-capability evidence the gate already expects. | resolved |
|
|
| `ProviderOperationConcurrencyTest` | `provider-dispatch-concurrency` | `dedupe-concurrency-contract-drift` | Multiple same-operation and cross-operation scenarios disagree with the current shared dedupe and `scopeBusy` contract, including run identity expectations. | `ProviderOperationStartGate`, one-hop `OperationRunService` identity path only if proven | yes | Resolved after aligning the startable provider-operation fixture prerequisites; no runtime owner change was needed. | resolved |
|
|
| `VerificationAuthorizationTest` | `shared-startable-fixtures` | `fixture-contract-drift` | An authorized operator receives `blocked` instead of `started`, which suggests the test's connection setup may no longer satisfy the canonical startable verification contract. | `tests/Pest.php`, touched verification fixture call sites, then `StartVerification` only if still wrong | yes | Resolved by using the canonical credential-enabled verification fixture path. | resolved |
|
|
| `VerificationStartAfterCompletionTest` | `verification-start-contract` | `provider-verification-runtime-regression` | A post-completion verification restart does not produce the expected fresh run and falls back to the wrong run identity or blocked behavior. | `StartVerification`, one-hop `OperationRunService` identity path only if proven | only-if-runtime-owner-proven | Resolved after fixture alignment; reruns proved the runtime owner contract was already correct. | resolved |
|
|
| `VerificationStartDedupeTest` | `verification-start-contract` | `provider-verification-runtime-regression` | Repeated verification starts return conflicting run keys instead of one canonical deduped run contract. | `StartVerification`, one-hop `OperationRunService` identity path only if proven | yes | Resolved after fixture alignment; reruns proved the runtime owner contract was already correct. | resolved |
|
|
| `ProviderConnectionHealthCheckWritesReportTest` | `verification-report-summary` | `surface-or-report-baseline-drift` | `summary.counts.total` now reflects a larger emitted check inventory than the stale hard-coded ceiling in the test. | `MicrosoftProviderHealthCheck`, directly implicated report writer seam, test expectation | yes | Resolved by removing the stale literal ceiling and pinning the count to the emitted check inventory. | resolved |
|
|
|
|
## Classification Rules
|
|
|
|
- Every in-scope failure group must keep exactly one seam and one category.
|
|
- If a rerun shows the failure is gone after adjacent fixture alignment, reclassify it to `resolved-or-not-needed` instead of widening runtime code.
|
|
- If a rerun proves the problem lives outside the bounded ProviderConnections/Verification scope, reclassify it to `out-of-scope-existing-debt` and stop widening `294`.
|
|
- Do not invent a new category or seam inside implementation without first updating `spec.md`, `plan.md`, `research.md`, `data-model.md`, `quickstart.md`, `tasks.md`, and `checklists/requirements.md`.
|