43 lines
2.0 KiB
Markdown
43 lines
2.0 KiB
Markdown
# Quickstart — Spec 115 (Baseline Operability & Alert Integration)
|
|
|
|
## Prereqs
|
|
- Run the app via Sail.
|
|
|
|
## Local setup
|
|
- Start containers: `vendor/bin/sail up -d`
|
|
|
|
## How to exercise the feature (manual)
|
|
|
|
### 1) Ensure baseline compare can run
|
|
- In Filament tenant-context, start a baseline compare (existing UI surface).
|
|
- Confirm an `OperationRun` of type `baseline_compare` appears in Monitoring → Operations.
|
|
|
|
### 2) Verify auto-close safety gate
|
|
- Create/open at least one baseline compare finding (`source = baseline.compare`) by running a compare with drift.
|
|
- Remediate drift (or modify baseline/current so it no longer appears).
|
|
- Run baseline compare again.
|
|
- Expected:
|
|
- If the run outcome is `succeeded` AND `summary_counts.processed == summary_counts.total` AND `summary_counts.failed == 0`, stale findings are resolved with `resolved_reason = no_longer_drifting`.
|
|
- If the run fails/partial/incomplete, no findings are auto-resolved.
|
|
|
|
### 3) Verify baseline drift alert events
|
|
- Ensure workspace settings are configured:
|
|
- `baseline.severity_mapping` has the three required keys.
|
|
- `baseline.alert_min_severity` is set (defaults to `high`).
|
|
- Run baseline compare to create new/reopened baseline findings.
|
|
- Trigger alerts evaluation:
|
|
- `vendor/bin/sail artisan tenantpilot:dispatch-alerts --once`
|
|
- Expected:
|
|
- `baseline_high_drift` events are produced only for findings that are new/reopened within the evaluation window.
|
|
- Repeat compares do not re-alert the same open finding.
|
|
|
|
### 4) Verify baseline compare failed alerts
|
|
- Force a baseline compare to fail (e.g., by making required preconditions fail or simulating a job failure).
|
|
- Run alerts evaluation again.
|
|
- Expected: `baseline_compare_failed` event is produced, subject to the existing per-rule cooldown and quiet-hours suppression.
|
|
|
|
## Tests (Pest)
|
|
- Run focused suite for this spec once implemented:
|
|
- `vendor/bin/sail artisan test --compact --filter=BaselineOperability`
|
|
- Or run specific test files under `tests/Feature/Alerts/` and `tests/Feature/Baselines/`.
|