Files
pn-new-crm/src/lib/db/schema/insights.ts

111 lines
4.4 KiB
TypeScript
Raw Normal View History

feat(insights): Phase B schema + service skeletons PR1 of Phase B per docs/superpowers/specs/2026-04-28-phase-b-insights-alerts-design.md. Lays the foundation that PRs 2-10 will fill in with behaviour. Schema (migration 0014): - alerts table with rule-engine fields (rule_id, severity, link, entity_type/id, fingerprint, fired/dismissed/acknowledged/resolved timestamps, jsonb metadata). Partial-unique fingerprint index keeps one open row per (port, rule, entity); separate indexes power severity-filtered and time-ordered queries. - analytics_snapshots (port_id, metric_id) -> jsonb cache + computedAt for the 15-min recurring refresh. - expenses: duplicate_of self-FK, dedup_scanned_at, ocr_status/raw/ confidence; partial index on (port, vendor, amount, date) where duplicate_of IS NULL drives the dedup heuristic. - audit_logs.search_text: GENERATED ALWAYS tsvector over action+entity_type+entity_id+user_id, GIN-indexed (drizzle can't model GENERATED ALWAYS in TS yet, so the migration appends manual ALTER + the GIN index). Service skeletons in src/lib/services/: - alerts.service.ts: fingerprintFor, reconcileAlertsForPort (upsert + auto-resolve), dismiss, acknowledge, listAlertsForPort. - alert-rules.ts: RULE_REGISTRY of 10 rule evaluators (currently no-op); PR2 fills in the bodies. - analytics.service.ts: readSnapshot/writeSnapshot with 15-min TTL + no-op compute* stubs for the four chart series; PR3 fills behavior. - expense-dedup.service.ts: scanForDuplicates + markBestDuplicate using the partial dedup index. PR8 wires the BullMQ trigger. - expense-ocr.service.ts: OcrResult/OcrLineItem types + ocrReceipt stub. PR9 wires Claude Vision (Haiku 4.5 + ephemeral system-prompt cache). - audit-search.service.ts: tsvector @@ plainto_tsquery + cursor pagination on (createdAt, id). PR10 wires the admin UI. tsc clean, lint clean, vitest 675/675 (one unrelated AES random-output flake passes solo). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-28 14:43:01 +02:00
/**
* Phase B - operational insight surfaces.
feat(insights): Phase B schema + service skeletons PR1 of Phase B per docs/superpowers/specs/2026-04-28-phase-b-insights-alerts-design.md. Lays the foundation that PRs 2-10 will fill in with behaviour. Schema (migration 0014): - alerts table with rule-engine fields (rule_id, severity, link, entity_type/id, fingerprint, fired/dismissed/acknowledged/resolved timestamps, jsonb metadata). Partial-unique fingerprint index keeps one open row per (port, rule, entity); separate indexes power severity-filtered and time-ordered queries. - analytics_snapshots (port_id, metric_id) -> jsonb cache + computedAt for the 15-min recurring refresh. - expenses: duplicate_of self-FK, dedup_scanned_at, ocr_status/raw/ confidence; partial index on (port, vendor, amount, date) where duplicate_of IS NULL drives the dedup heuristic. - audit_logs.search_text: GENERATED ALWAYS tsvector over action+entity_type+entity_id+user_id, GIN-indexed (drizzle can't model GENERATED ALWAYS in TS yet, so the migration appends manual ALTER + the GIN index). Service skeletons in src/lib/services/: - alerts.service.ts: fingerprintFor, reconcileAlertsForPort (upsert + auto-resolve), dismiss, acknowledge, listAlertsForPort. - alert-rules.ts: RULE_REGISTRY of 10 rule evaluators (currently no-op); PR2 fills in the bodies. - analytics.service.ts: readSnapshot/writeSnapshot with 15-min TTL + no-op compute* stubs for the four chart series; PR3 fills behavior. - expense-dedup.service.ts: scanForDuplicates + markBestDuplicate using the partial dedup index. PR8 wires the BullMQ trigger. - expense-ocr.service.ts: OcrResult/OcrLineItem types + ocrReceipt stub. PR9 wires Claude Vision (Haiku 4.5 + ephemeral system-prompt cache). - audit-search.service.ts: tsvector @@ plainto_tsquery + cursor pagination on (createdAt, id). PR10 wires the admin UI. tsc clean, lint clean, vitest 675/675 (one unrelated AES random-output flake passes solo). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-28 14:43:01 +02:00
*
* - `alerts`: rule-engine-fired actionable cards. The fingerprint column
* dedupes re-evaluations of the same condition; the partial unique
* index keeps a single open row per `(port, fingerprint)` while
* resolved/dismissed history accumulates.
* - `analytics_snapshots`: cached aggregate JSON keyed by metric+range,
* refreshed by a recurring job so dashboard hits don't recompute.
*/
import { pgTable, text, timestamp, jsonb, index, uniqueIndex } from 'drizzle-orm/pg-core';
import { sql } from 'drizzle-orm';
import { ports } from './ports';
import { user } from './users';
export const alerts = pgTable(
'alerts',
{
id: text('id')
.primaryKey()
.$defaultFn(() => crypto.randomUUID()),
portId: text('port_id')
.notNull()
.references(() => ports.id, { onDelete: 'cascade' }),
/** Stable rule identifier: 'reservation.no_agreement', 'interest.stale', ... */
ruleId: text('rule_id').notNull(),
/** 'info' | 'warning' | 'critical' */
severity: text('severity').notNull(),
title: text('title').notNull(),
body: text('body'),
/** Relative path the card deep-links to. */
link: text('link').notNull(),
/** Optional FK target: 'interest', 'reservation', 'document', 'expense', ... */
entityType: text('entity_type'),
entityId: text('entity_id'),
/** Hash of (rule_id + entity_type + entity_id) - dedupes re-evaluations. */
feat(insights): Phase B schema + service skeletons PR1 of Phase B per docs/superpowers/specs/2026-04-28-phase-b-insights-alerts-design.md. Lays the foundation that PRs 2-10 will fill in with behaviour. Schema (migration 0014): - alerts table with rule-engine fields (rule_id, severity, link, entity_type/id, fingerprint, fired/dismissed/acknowledged/resolved timestamps, jsonb metadata). Partial-unique fingerprint index keeps one open row per (port, rule, entity); separate indexes power severity-filtered and time-ordered queries. - analytics_snapshots (port_id, metric_id) -> jsonb cache + computedAt for the 15-min recurring refresh. - expenses: duplicate_of self-FK, dedup_scanned_at, ocr_status/raw/ confidence; partial index on (port, vendor, amount, date) where duplicate_of IS NULL drives the dedup heuristic. - audit_logs.search_text: GENERATED ALWAYS tsvector over action+entity_type+entity_id+user_id, GIN-indexed (drizzle can't model GENERATED ALWAYS in TS yet, so the migration appends manual ALTER + the GIN index). Service skeletons in src/lib/services/: - alerts.service.ts: fingerprintFor, reconcileAlertsForPort (upsert + auto-resolve), dismiss, acknowledge, listAlertsForPort. - alert-rules.ts: RULE_REGISTRY of 10 rule evaluators (currently no-op); PR2 fills in the bodies. - analytics.service.ts: readSnapshot/writeSnapshot with 15-min TTL + no-op compute* stubs for the four chart series; PR3 fills behavior. - expense-dedup.service.ts: scanForDuplicates + markBestDuplicate using the partial dedup index. PR8 wires the BullMQ trigger. - expense-ocr.service.ts: OcrResult/OcrLineItem types + ocrReceipt stub. PR9 wires Claude Vision (Haiku 4.5 + ephemeral system-prompt cache). - audit-search.service.ts: tsvector @@ plainto_tsquery + cursor pagination on (createdAt, id). PR10 wires the admin UI. tsc clean, lint clean, vitest 675/675 (one unrelated AES random-output flake passes solo). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-28 14:43:01 +02:00
fingerprint: text('fingerprint').notNull(),
firedAt: timestamp('fired_at', { withTimezone: true }).notNull().defaultNow(),
dismissedAt: timestamp('dismissed_at', { withTimezone: true }),
dismissedBy: text('dismissed_by').references(() => user.id),
/** "Someone is on it" - alert stays visible but stops nagging. */
feat(insights): Phase B schema + service skeletons PR1 of Phase B per docs/superpowers/specs/2026-04-28-phase-b-insights-alerts-design.md. Lays the foundation that PRs 2-10 will fill in with behaviour. Schema (migration 0014): - alerts table with rule-engine fields (rule_id, severity, link, entity_type/id, fingerprint, fired/dismissed/acknowledged/resolved timestamps, jsonb metadata). Partial-unique fingerprint index keeps one open row per (port, rule, entity); separate indexes power severity-filtered and time-ordered queries. - analytics_snapshots (port_id, metric_id) -> jsonb cache + computedAt for the 15-min recurring refresh. - expenses: duplicate_of self-FK, dedup_scanned_at, ocr_status/raw/ confidence; partial index on (port, vendor, amount, date) where duplicate_of IS NULL drives the dedup heuristic. - audit_logs.search_text: GENERATED ALWAYS tsvector over action+entity_type+entity_id+user_id, GIN-indexed (drizzle can't model GENERATED ALWAYS in TS yet, so the migration appends manual ALTER + the GIN index). Service skeletons in src/lib/services/: - alerts.service.ts: fingerprintFor, reconcileAlertsForPort (upsert + auto-resolve), dismiss, acknowledge, listAlertsForPort. - alert-rules.ts: RULE_REGISTRY of 10 rule evaluators (currently no-op); PR2 fills in the bodies. - analytics.service.ts: readSnapshot/writeSnapshot with 15-min TTL + no-op compute* stubs for the four chart series; PR3 fills behavior. - expense-dedup.service.ts: scanForDuplicates + markBestDuplicate using the partial dedup index. PR8 wires the BullMQ trigger. - expense-ocr.service.ts: OcrResult/OcrLineItem types + ocrReceipt stub. PR9 wires Claude Vision (Haiku 4.5 + ephemeral system-prompt cache). - audit-search.service.ts: tsvector @@ plainto_tsquery + cursor pagination on (createdAt, id). PR10 wires the admin UI. tsc clean, lint clean, vitest 675/675 (one unrelated AES random-output flake passes solo). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-28 14:43:01 +02:00
acknowledgedAt: timestamp('acknowledged_at', { withTimezone: true }),
acknowledgedBy: text('acknowledged_by').references(() => user.id),
/** Set by the engine when the underlying condition no longer fires. */
resolvedAt: timestamp('resolved_at', { withTimezone: true }),
/** Per-rule extras: days_stale, amount_at_risk, etc. */
metadata: jsonb('metadata').default({}),
},
(table) => [
// Only one open alert per (port, fingerprint) - re-evaluation upserts.
feat(insights): Phase B schema + service skeletons PR1 of Phase B per docs/superpowers/specs/2026-04-28-phase-b-insights-alerts-design.md. Lays the foundation that PRs 2-10 will fill in with behaviour. Schema (migration 0014): - alerts table with rule-engine fields (rule_id, severity, link, entity_type/id, fingerprint, fired/dismissed/acknowledged/resolved timestamps, jsonb metadata). Partial-unique fingerprint index keeps one open row per (port, rule, entity); separate indexes power severity-filtered and time-ordered queries. - analytics_snapshots (port_id, metric_id) -> jsonb cache + computedAt for the 15-min recurring refresh. - expenses: duplicate_of self-FK, dedup_scanned_at, ocr_status/raw/ confidence; partial index on (port, vendor, amount, date) where duplicate_of IS NULL drives the dedup heuristic. - audit_logs.search_text: GENERATED ALWAYS tsvector over action+entity_type+entity_id+user_id, GIN-indexed (drizzle can't model GENERATED ALWAYS in TS yet, so the migration appends manual ALTER + the GIN index). Service skeletons in src/lib/services/: - alerts.service.ts: fingerprintFor, reconcileAlertsForPort (upsert + auto-resolve), dismiss, acknowledge, listAlertsForPort. - alert-rules.ts: RULE_REGISTRY of 10 rule evaluators (currently no-op); PR2 fills in the bodies. - analytics.service.ts: readSnapshot/writeSnapshot with 15-min TTL + no-op compute* stubs for the four chart series; PR3 fills behavior. - expense-dedup.service.ts: scanForDuplicates + markBestDuplicate using the partial dedup index. PR8 wires the BullMQ trigger. - expense-ocr.service.ts: OcrResult/OcrLineItem types + ocrReceipt stub. PR9 wires Claude Vision (Haiku 4.5 + ephemeral system-prompt cache). - audit-search.service.ts: tsvector @@ plainto_tsquery + cursor pagination on (createdAt, id). PR10 wires the admin UI. tsc clean, lint clean, vitest 675/675 (one unrelated AES random-output flake passes solo). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-28 14:43:01 +02:00
uniqueIndex('idx_alerts_fingerprint_open')
.on(table.portId, table.fingerprint)
.where(sql`resolved_at IS NULL`),
index('idx_alerts_port_fired').on(table.portId, table.firedAt),
index('idx_alerts_port_severity_open')
.on(table.portId, table.severity)
.where(sql`resolved_at IS NULL AND dismissed_at IS NULL`),
],
);
export type Alert = typeof alerts.$inferSelect;
export type NewAlert = typeof alerts.$inferInsert;
export const analyticsSnapshots = pgTable(
'analytics_snapshots',
{
portId: text('port_id')
.notNull()
.references(() => ports.id, { onDelete: 'cascade' }),
/** Composite key: e.g. 'pipeline_funnel.30d', 'occupancy_timeline.90d'. */
metricId: text('metric_id').notNull(),
computedAt: timestamp('computed_at', { withTimezone: true }).notNull().defaultNow(),
/** Pre-shaped chart data. */
data: jsonb('data').notNull(),
},
(table) => [uniqueIndex('idx_analytics_pk').on(table.portId, table.metricId)],
);
export type AnalyticsSnapshot = typeof analyticsSnapshots.$inferSelect;
export type NewAnalyticsSnapshot = typeof analyticsSnapshots.$inferInsert;
/** Severity literal type for callers that want a typed enum. */
export type AlertSeverity = 'info' | 'warning' | 'critical';
chore(cleanup): Phase 1 — gap closure across audit, alerts, soft-delete, perms Multi-area cleanup pass closing partial-implementation gaps surfaced by the post-i18n audit. No behavior changes for happy-path users; closes real correctness/security holes. PR1a Public yacht-interest endpoint i18n. /api/public/interests now accepts phoneE164/phoneCountry, nationalityIso, address.{countryIso, subdivisionIso}, and company.{incorporationCountryIso, incorporationSubdivisionIso}. Server-side parsePhone() fallback for legacy raw phone strings. PR1b Alert rule registry trim. Two rule slots ('document.expiring_soon', 'audit.suspicious_login') were registered but evaluators returned []. Both required schema/instrumentation that hadn't landed. Removed from the registry; comments record the dependencies needed to revive them. Effective rule count: 8 active. PR1c vi.mock hoist + flake fix. Hoisted vi.mock calls to top-level in 5 integration test files; webhook-delivery uses vi.hoisted for the queue-add ref. Vitest no longer warns about non-top-level mocks. Deflaked the 'short value' assertion in security-encryption.test.ts by switching plaintext from 'ab' to 'XY' (non-hex chars). 5/5 runs green. PR1d Soft-delete reference audit. listClientOptions and listYachtsForOwner now filter by isNull(archivedAt). Berths use status (no archivedAt). PR1e Permission-matrix audit script + report. scripts/audit-permissions.ts walks every src/app/api/v1/**/route.ts and reports handlers without a withPermission() wrapper. Initial run found 33 violations. - Allow-listed 17 with explicit reasons (self-data, admin, alerts, search, currency, ai, custom-fields — some marked TODO). - Wrapped 7 routes with concrete permissions: clients/options (clients:view), berths/options (berths:view), dashboard/* (reports:view_dashboard), analytics (reports:view_analytics). Audit report at docs/runbooks/permission-audit.md. Script exits non-zero on any unallow-listed violation so it can become a CI gate. Vitest: 741 -> 741 (no new tests; existing suite covers the changes). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-28 18:48:22 +02:00
/**
* Rule IDs in the v1 catalog - keep in sync with `alert-rules.ts`.
chore(cleanup): Phase 1 — gap closure across audit, alerts, soft-delete, perms Multi-area cleanup pass closing partial-implementation gaps surfaced by the post-i18n audit. No behavior changes for happy-path users; closes real correctness/security holes. PR1a Public yacht-interest endpoint i18n. /api/public/interests now accepts phoneE164/phoneCountry, nationalityIso, address.{countryIso, subdivisionIso}, and company.{incorporationCountryIso, incorporationSubdivisionIso}. Server-side parsePhone() fallback for legacy raw phone strings. PR1b Alert rule registry trim. Two rule slots ('document.expiring_soon', 'audit.suspicious_login') were registered but evaluators returned []. Both required schema/instrumentation that hadn't landed. Removed from the registry; comments record the dependencies needed to revive them. Effective rule count: 8 active. PR1c vi.mock hoist + flake fix. Hoisted vi.mock calls to top-level in 5 integration test files; webhook-delivery uses vi.hoisted for the queue-add ref. Vitest no longer warns about non-top-level mocks. Deflaked the 'short value' assertion in security-encryption.test.ts by switching plaintext from 'ab' to 'XY' (non-hex chars). 5/5 runs green. PR1d Soft-delete reference audit. listClientOptions and listYachtsForOwner now filter by isNull(archivedAt). Berths use status (no archivedAt). PR1e Permission-matrix audit script + report. scripts/audit-permissions.ts walks every src/app/api/v1/**/route.ts and reports handlers without a withPermission() wrapper. Initial run found 33 violations. - Allow-listed 17 with explicit reasons (self-data, admin, alerts, search, currency, ai, custom-fields — some marked TODO). - Wrapped 7 routes with concrete permissions: clients/options (clients:view), berths/options (berths:view), dashboard/* (reports:view_dashboard), analytics (reports:view_analytics). Audit report at docs/runbooks/permission-audit.md. Script exits non-zero on any unallow-listed violation so it can become a CI gate. Vitest: 741 -> 741 (no new tests; existing suite covers the changes). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-28 18:48:22 +02:00
*
* Two rules from the original spec (`document.expiring_soon`,
* `audit.suspicious_login`) are deferred until their data sources land:
* - `document.expiring_soon` needs a `documents.expires_at` column populated
* from Documenso responses (currently expiry isn't tracked).
* - `audit.suspicious_login` needs better-auth instrumentation that writes
* `login.failed` audit rows (the auth layer currently doesn't).
* Re-add the literal here + the evaluator in `alert-rules.ts` once their
* dependencies ship.
*/
feat(insights): Phase B schema + service skeletons PR1 of Phase B per docs/superpowers/specs/2026-04-28-phase-b-insights-alerts-design.md. Lays the foundation that PRs 2-10 will fill in with behaviour. Schema (migration 0014): - alerts table with rule-engine fields (rule_id, severity, link, entity_type/id, fingerprint, fired/dismissed/acknowledged/resolved timestamps, jsonb metadata). Partial-unique fingerprint index keeps one open row per (port, rule, entity); separate indexes power severity-filtered and time-ordered queries. - analytics_snapshots (port_id, metric_id) -> jsonb cache + computedAt for the 15-min recurring refresh. - expenses: duplicate_of self-FK, dedup_scanned_at, ocr_status/raw/ confidence; partial index on (port, vendor, amount, date) where duplicate_of IS NULL drives the dedup heuristic. - audit_logs.search_text: GENERATED ALWAYS tsvector over action+entity_type+entity_id+user_id, GIN-indexed (drizzle can't model GENERATED ALWAYS in TS yet, so the migration appends manual ALTER + the GIN index). Service skeletons in src/lib/services/: - alerts.service.ts: fingerprintFor, reconcileAlertsForPort (upsert + auto-resolve), dismiss, acknowledge, listAlertsForPort. - alert-rules.ts: RULE_REGISTRY of 10 rule evaluators (currently no-op); PR2 fills in the bodies. - analytics.service.ts: readSnapshot/writeSnapshot with 15-min TTL + no-op compute* stubs for the four chart series; PR3 fills behavior. - expense-dedup.service.ts: scanForDuplicates + markBestDuplicate using the partial dedup index. PR8 wires the BullMQ trigger. - expense-ocr.service.ts: OcrResult/OcrLineItem types + ocrReceipt stub. PR9 wires Claude Vision (Haiku 4.5 + ephemeral system-prompt cache). - audit-search.service.ts: tsvector @@ plainto_tsquery + cursor pagination on (createdAt, id). PR10 wires the admin UI. tsc clean, lint clean, vitest 675/675 (one unrelated AES random-output flake passes solo). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-28 14:43:01 +02:00
export const ALERT_RULES = [
'reservation.no_agreement',
'interest.stale',
'document.signer_overdue',
'berth.under_offer_stalled',
'expense.duplicate',
'expense.unscanned',
'interest.high_value_silent',
'eoi.unsigned_long',
] as const;
export type AlertRuleId = (typeof ALERT_RULES)[number];