tenantpilot/specs/004-policy-explorer-v2/plan.md

Implementation Plan: Policy Explorer V2

Branch: 004-policy-explorer-v2 | Date: 2025-12-09 | Spec: spec.md
Input: Feature specification from /specs/004-policy-explorer-v2/spec.md

Summary

Upgrade the existing Policy Explorer (/search) from basic search/table view to advanced data table with:

• Server-side pagination (10/25/50/100 rows per page)
• Multi-column sorting with ASC/DESC toggle
• Column management (show/hide, resize, reorder) persisted in localStorage
• PolicyType filtering with multi-select checkboxes
• Bulk export (CSV) for selected rows (client-side) and all filtered results (server-side, max 5000)
• Enhanced detail view with copy-to-clipboard, raw JSON, and "Open in Intune" link
• URL state for shareable filtered/sorted views
• Sticky header and compact/comfortable density modes

Technical approach: TanStack Table v8 for client-side table state management, Server Actions for data fetching/export, shadcn/ui primitives for UI consistency, nuqs for URL state, and localStorage for user preferences.

Technical Context

Language/Version: TypeScript 5.x strict mode
Primary Dependencies:

• Next.js 16+ App Router
• TanStack Table v8 (@tanstack/react-table)
• Drizzle ORM for database queries
• Shadcn UI components
• NextAuth.js v4 for tenant isolation
• URL state: nuqs or native useSearchParams
• CSV export: papaparse or native string builder

Storage: PostgreSQL (existing policy_settings table)
Testing: Jest/Vitest for utils, Playwright for E2E table interactions
Target Platform: Docker containers, modern web browsers (Chrome, Firefox, Safari, Edge)
Project Type: Next.js App Router web application
Performance Goals:

• Page load: <500ms (50 rows with filters)
• Sorting/filtering: <200ms
• CSV export (1000 rows): <2s client-side
• CSV export (5000 rows): <5s server-side

Constraints:

• Server-first architecture (all data via Server Actions)
• No client-side fetching (useEffect + fetch prohibited)
• TypeScript strict mode (no any types)
• Shadcn UI for all components
• Azure AD tenant isolation enforced

Scale/Scope: Multi-tenant SaaS, 1000+ policy settings per tenant, 100+ concurrent users

Constitution Check

GATE: Must pass before Phase 0 research. Re-check after Phase 1 design.

• Uses Next.js App Router with Server Actions (pagination/filtering/export via Server Actions)
• TypeScript strict mode enabled (existing codebase already strict)
• Drizzle ORM for all database operations (policy settings queries use Drizzle)
• Shadcn UI for all new components (Table, Button, Sheet, etc.)
• Azure AD multi-tenant authentication (existing auth, tenant isolation via session)
• Docker deployment with standalone build (existing Dockerfile)

Result: ✅ No constitution violations

Project Structure

Documentation (this feature)

specs/004-policy-explorer-v2/
├── plan.md              # This file
├── research.md          # Phase 0 output (TanStack Table patterns, CSV strategies)
├── data-model.md        # Phase 1 output (DataTableState, FilterState types)
├── quickstart.md        # Phase 1 output (how to add new columns, filters)
├── contracts/           # Phase 1 output (Server Action signatures)
│   ├── getPolicySettings.yaml
│   ├── exportPolicySettingsCSV.yaml
│   └── types.ts
└── tasks.md             # Phase 2 output (NOT created by this command)

Source Code (repository root)

Structure Decision: Next.js App Router structure (existing pattern in codebase)

app/
└── (app)/
    └── search/
        ├── page.tsx                         # Server Component (data fetching)
        ├── PolicyExplorerClient.tsx         # UPDATED: Client Component wrapper
        └── PolicyExplorerTable.tsx          # NEW: TanStack Table component

components/
└── policy-explorer/
    ├── PolicyTable.tsx                      # NEW: Main data table
    ├── PolicyTableColumns.tsx               # NEW: Column definitions
    ├── PolicyTableToolbar.tsx               # NEW: Filters, density toggle, export
    ├── PolicyTablePagination.tsx            # NEW: Pagination controls
    ├── PolicyDetailSheet.tsx                # UPDATED: Add copy buttons, raw JSON
    ├── ColumnVisibilityMenu.tsx             # NEW: Show/hide columns
    └── ExportButton.tsx                     # NEW: CSV export trigger

lib/
├── actions/
│   └── policySettings.ts                    # UPDATED: Add pagination, sorting, export
├── hooks/
│   ├── usePolicyTable.ts                    # NEW: TanStack Table hook
│   ├── useTablePreferences.ts               # NEW: localStorage persistence
│   └── useURLState.ts                       # NEW: URL state sync
└── utils/
    ├── csv-export.ts                        # NEW: Client-side CSV builder
    └── policy-table-helpers.ts              # NEW: Column formatters, sorters

tests/
├── unit/
│   ├── csv-export.test.ts                   # CSV generation logic
│   └── policy-table-helpers.test.ts         # Column utilities
└── e2e/
    └── policy-explorer.spec.ts              # Pagination, sorting, filtering, export

Complexity Tracking

No violations - All constitution checks pass. TanStack Table is a state management library (not a data fetching library), so it complements Server Actions rather than replacing them.

---

Phase 0: Research & Analysis

Unknowns to Resolve

1. TanStack Table Integration Pattern

   • Question: How to integrate TanStack Table with Next.js Server Actions for server-side pagination/sorting?
   • Research: Review TanStack Table docs for "manual pagination" mode, check existing patterns in similar Next.js projects
   • Output: research.md section on TanStack Table + Server Actions integration

2. CSV Export Strategy

   • Question: When to use client-side vs server-side CSV generation? What's the performance breakpoint?
   • Research: Test papaparse performance with 100/1000/5000 rows, measure memory usage, compare to server-side stream
   • Output: research.md section with performance benchmarks and decision matrix

3. URL State Management

   • Question: Use nuqs library or native useSearchParams + useRouter?
   • Research: Compare type safety, SSR compatibility, bundle size
   • Output: research.md section on URL state library choice

4. LocalStorage Schema

   • Question: How to version localStorage schema for forward compatibility when adding new features?
   • Research: Review best practices for localStorage versioning, schema migration patterns
   • Output: research.md section on localStorage structure

5. Sticky Header Implementation

   • Question: Use CSS position: sticky or JavaScript scroll listeners? Performance trade-offs?
   • Research: Test both approaches with large tables (1000+ rows), measure scroll jank
   • Output: research.md section on sticky header strategy

Dependencies & Best Practices

1. TanStack Table Best Practices

   • Task: Research recommended patterns for server-side pagination, column definitions, type safety
   • Output: research.md section with code examples

2. Shadcn Table + TanStack Integration

   • Task: Find examples of shadcn/ui Table primitives used with TanStack Table
   • Output: research.md section with integration patterns

3. CSV Escaping & Edge Cases

   • Task: Research proper CSV escaping (commas, quotes, newlines), Excel compatibility
   • Output: research.md section on CSV generation rules

---

Phase 1: Design & Contracts

Data Model Design

File: specs/004-policy-explorer-v2/data-model.md

Entities

DataTableState (Client-side ephemeral state)

interface DataTableState {
  pagination: {
    pageIndex: number;        // 0-based
    pageSize: 10 | 25 | 50 | 100;
  };
  sorting: Array<{
    id: string;               // column ID
    desc: boolean;            // true = DESC, false = ASC
  }>;
  columnVisibility: {
    [columnId: string]: boolean;
  };
  columnSizing: {
    [columnId: string]: number; // px
  };
  rowSelection: {
    [rowId: string]: boolean;
  };
  density: 'compact' | 'comfortable';
}

FilterState (Synced with URL + stored in localStorage)

interface FilterState {
  policyTypes: string[];      // ['deviceConfiguration', 'compliancePolicy']
  searchQuery: string;         // existing search functionality
  // Future: dateRange, tenant filter (Phase 2)
}

TablePreferences (Persisted in localStorage)

interface TablePreferences {
  version: 1;                  // schema version for migrations
  columnVisibility: { [columnId: string]: boolean };
  columnSizing: { [columnId: string]: number };
  columnOrder: string[];       // ordered column IDs
  density: 'compact' | 'comfortable';
  defaultPageSize: 10 | 25 | 50 | 100;
}

Database Schema Changes

None required - Existing policy_settings table has all necessary fields:

• policyName, policyType, settingName, settingValue, graphPolicyId, lastSyncedAt
• Indexes already exist for tenantId, settingName
• Recommendation: Add composite index for sorting performance: (tenantId, policyType, settingName)

API Contracts

File: specs/004-policy-explorer-v2/contracts/

Contract 1: getPolicySettings

File: contracts/getPolicySettings.yaml

action: getPolicySettings
description: Fetch policy settings with pagination, sorting, and filtering
method: Server Action

input:
  type: object
  properties:
    page:
      type: number
      minimum: 0
      description: 0-based page index
    pageSize:
      type: number
      enum: [10, 25, 50, 100]
      default: 50
    sortBy:
      type: string
      enum: [settingName, policyName, policyType, lastSyncedAt]
      optional: true
    sortDir:
      type: string
      enum: [asc, desc]
      default: asc
    policyTypes:
      type: array
      items:
        type: string
      optional: true
      description: Filter by policy types
    searchQuery:
      type: string
      optional: true
      description: Text search in settingName/policyName

output:
  type: object
  properties:
    data:
      type: array
      items:
        type: PolicySetting
    meta:
      type: object
      properties:
        totalCount: number
        pageCount: number
        currentPage: number
        pageSize: number
        hasNextPage: boolean
        hasPreviousPage: boolean

errors:
  - UNAUTHORIZED: User not authenticated
  - FORBIDDEN: User not in tenant
  - INVALID_PARAMS: Invalid pagination/sort params

Contract 2: exportPolicySettingsCSV

File: contracts/exportPolicySettingsCSV.yaml

action: exportPolicySettingsCSV
description: Export policy settings as CSV (server-side, max 5000 rows)
method: Server Action

input:
  type: object
  properties:
    policyTypes:
      type: array
      items:
        type: string
      optional: true
    searchQuery:
      type: string
      optional: true
    sortBy:
      type: string
      optional: true
    sortDir:
      type: string
      enum: [asc, desc]
      default: asc
    maxRows:
      type: number
      maximum: 5000
      default: 5000

output:
  type: object
  properties:
    csv:
      type: string
      description: CSV content as string
    filename:
      type: string
      description: Suggested filename (e.g., "policy-settings-2025-12-09.csv")
    rowCount:
      type: number
      description: Number of rows in CSV

errors:
  - UNAUTHORIZED: User not authenticated
  - FORBIDDEN: User not in tenant
  - TOO_MANY_ROWS: Result exceeds maxRows limit

Quickstart Guide

File: specs/004-policy-explorer-v2/quickstart.md

# Policy Explorer V2 - Quickstart

## Adding a New Column

1. Define column in `PolicyTableColumns.tsx`:
   ```typescript
   {
     accessorKey: 'myNewField',
     header: 'My Field',
     cell: ({ row }) => <span>{row.original.myNewField}</span>,
   }

2. Add to localStorage schema version if changing defaults
3. Update CSV export to include new column

Adding a New Filter

1. Update FilterState type in lib/types/policy-table.ts
2. Add UI control in PolicyTableToolbar.tsx
3. Update getPolicySettings Server Action to handle new filter param
4. Add to URL state in useURLState.ts

Testing Checklist

• Pagination: Navigate through pages, verify correct data
• Sorting: Click column headers, verify ASC/DESC toggle
• Filtering: Select policy types, verify filtered results
• Column visibility: Hide/show columns, reload page
• CSV export: Export selected rows, verify content
• Accessibility: Keyboard navigation, screen reader labels

---

## Phase 1.5: Agent Context Update

**Action**: Run `.specify/scripts/bash/update-agent-context.sh copilot`

This script will:
1. Detect AI agent in use (Copilot)
2. Update `.github/copilot-instructions.md`
3. Add new technologies from this plan:
   - TanStack Table v8 for data tables
   - CSV export patterns (client vs server-side)
   - URL state management with nuqs
   - LocalStorage persistence patterns

**Manual additions to preserve**:
- Existing Intune API patterns
- Project-specific conventions

---

## Re-Evaluation: Constitution Check

*Run after Phase 1 design completion*

- [X] TanStack Table uses Server Actions for data (no client-side fetch)
- [X] All types strictly defined (DataTableState, FilterState, contracts)
- [X] Drizzle ORM for queries (no raw SQL)
- [X] Shadcn UI Table primitives (no custom table components)
- [X] Azure AD session enforced in Server Actions
- [X] Docker build unaffected (no new runtime dependencies)

**Result**: ✅ Constitution compliance maintained

---

## Implementation Notes

### Critical Paths

1. **TanStack Table Integration** (Blocking)
   - Must establish pattern for Server Action + TanStack Table manual pagination
   - All other features depend on this foundation

2. **Server Action Updates** (Blocking)
   - `getPolicySettings` must support pagination/sorting before UI can be built

3. **CSV Export** (Parallel after data fetching works)
   - Client-side export can be built independently
   - Server-side export requires Server Action

### Parallel Work Opportunities

- **Phase 1**: Data model design + API contracts can happen in parallel
- **Implementation**: Client-side CSV export + column management can be built while server-side export is in progress
- **Testing**: Unit tests for utils can be written early, E2E tests require full integration

### Risk Mitigation

**Risk**: TanStack Table performance with large datasets (1000+ rows)
- **Mitigation**: Use virtualization (`@tanstack/react-virtual`) if needed
- **Fallback**: Reduce default page size to 25 rows

**Risk**: CSV export memory issues with 5000 rows
- **Mitigation**: Use streaming approach in Server Action
- **Fallback**: Reduce max export to 2500 rows

**Risk**: localStorage quota exceeded (5-10MB limit)
- **Mitigation**: Only store column preferences, not data
- **Fallback**: Clear old preferences, show user warning

---

## Success Metrics

### Performance Benchmarks

- [ ] Page load (50 rows): <500ms
- [ ] Sorting operation: <200ms
- [ ] Filtering operation: <300ms
- [ ] Column resize: <16ms (60fps)
- [ ] CSV export (1000 rows, client): <2s
- [ ] CSV export (5000 rows, server): <5s

### Functional Validation

- [ ] Pagination: All pages load correctly, no duplicate rows
- [ ] Sorting: Correct order for all column types (string, date, number)
- [ ] Filtering: AND logic for multiple filters
- [ ] Column management: Preferences persist across sessions
- [ ] CSV export: Proper escaping, Excel-compatible
- [ ] URL state: Shareable links work correctly
- [ ] Accessibility: Keyboard navigation, ARIA labels

---

## Next Steps

1. **Run Phase 0 Research**:
   ```bash
   # Research TanStack Table patterns
   # Benchmark CSV strategies
   # Choose URL state library

2. Generate research.md:

   • Document findings for each unknown
   • Include code examples and performance data

3. Generate data-model.md:

   • Define TypeScript interfaces
   • Document database schema (no changes)

4. Generate Contracts:

   • Create YAML specs for Server Actions
   • Define input/output types

5. Update Agent Context:

   .specify/scripts/bash/update-agent-context.sh copilot
   

6. Generate tasks.md:

   # After Phase 1 complete, run:
   /speckit.tasks
   

---

Plan Version: 1.0
Last Updated: 2025-12-09
Status: Ready for Phase 0 Research