Files
pn-new-crm/src/lib/db/schema/system.ts

414 lines
17 KiB
TypeScript
Raw Normal View History

import {
pgTable,
text,
boolean,
integer,
numeric,
timestamp,
jsonb,
index,
uniqueIndex,
feat(insights): Phase B schema + service skeletons PR1 of Phase B per docs/superpowers/specs/2026-04-28-phase-b-insights-alerts-design.md. Lays the foundation that PRs 2-10 will fill in with behaviour. Schema (migration 0014): - alerts table with rule-engine fields (rule_id, severity, link, entity_type/id, fingerprint, fired/dismissed/acknowledged/resolved timestamps, jsonb metadata). Partial-unique fingerprint index keeps one open row per (port, rule, entity); separate indexes power severity-filtered and time-ordered queries. - analytics_snapshots (port_id, metric_id) -> jsonb cache + computedAt for the 15-min recurring refresh. - expenses: duplicate_of self-FK, dedup_scanned_at, ocr_status/raw/ confidence; partial index on (port, vendor, amount, date) where duplicate_of IS NULL drives the dedup heuristic. - audit_logs.search_text: GENERATED ALWAYS tsvector over action+entity_type+entity_id+user_id, GIN-indexed (drizzle can't model GENERATED ALWAYS in TS yet, so the migration appends manual ALTER + the GIN index). Service skeletons in src/lib/services/: - alerts.service.ts: fingerprintFor, reconcileAlertsForPort (upsert + auto-resolve), dismiss, acknowledge, listAlertsForPort. - alert-rules.ts: RULE_REGISTRY of 10 rule evaluators (currently no-op); PR2 fills in the bodies. - analytics.service.ts: readSnapshot/writeSnapshot with 15-min TTL + no-op compute* stubs for the four chart series; PR3 fills behavior. - expense-dedup.service.ts: scanForDuplicates + markBestDuplicate using the partial dedup index. PR8 wires the BullMQ trigger. - expense-ocr.service.ts: OcrResult/OcrLineItem types + ocrReceipt stub. PR9 wires Claude Vision (Haiku 4.5 + ephemeral system-prompt cache). - audit-search.service.ts: tsvector @@ plainto_tsquery + cursor pagination on (createdAt, id). PR10 wires the admin UI. tsc clean, lint clean, vitest 675/675 (one unrelated AES random-output flake passes solo). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-28 14:43:01 +02:00
customType,
feat: autonomous backlog push — admin UX overhaul + storage parity + residential parity + Documenso Phase 1 Massive multi-area push driven by docs/admin-ux-backlog.md. Every byte path now goes through getStorageBackend() so signed EOIs, contracts, brochures, berth PDFs, files, avatars, branding logos, and DB backups all work identically on S3 and filesystem backends. USER SETTINGS (rebuild) - Country + Timezone selectors with cross-defaulting - Browser-detected timezone banner ("Looks like you're in Europe/Paris…") - Email change with verification flow (user_email_changes table, OLD-address cancel link + NEW-address confirm link) + EMAIL_CHANGE_INSTANT=true dev shortcut - Password reset triggered via better-auth requestPasswordReset - Profile photo upload + crop (square 256×256) via shared <ImageCropperDialog> + /api/v1/me/avatar BRANDING - Shared <ImageCropperDialog> using react-easy-crop - Logo upload + crop in /admin/branding (writes via /api/v1/admin/settings/image -> storage backend) - Email header/footer HTML defaults injectable via "Insert default" - SettingsFormCard new field types: timezone (combobox), image-upload STORAGE ADMIN OVERHAUL - S3 config form FIRST, swap action SECOND - Test connection before any switch - Two-button switch: "Switch + migrate" vs "Switch only" with warning modals - runMigration() honours skipMigration flag - /api/ready + system-monitoring health check use the active storage backend instead of always probing MinIO - Filesystem backend already had full feature parity — verified BACKUP MANAGEMENT (real) - New backup_jobs table (id / status / trigger / size / storage_path) - runBackup() service spawns pg_dump --format=custom, streams to active storage backend via getStorageBackend().put() - /admin/backup page: trigger, history, download .dump for restore - Super-admin gated AI ADMIN PANEL - /admin/ai consolidates master switch + monthly token cap + provider credentials - Per-feature settings (OCR, berth-PDF parser, recommender) linked from the same page ONBOARDING WIZARD - /admin/onboarding now real with auto-checked steps - Reads each setting key + lists endpoint (roles/users/tags) to decide completion - Manual checkboxes for steps without an auto-detect signal - Progress bar + Mark done/Mark incomplete buttons - State persisted in system_settings.onboarding_manual_status RESIDENTIAL PARITY (full) - New residential_client_notes + residential_interest_notes tables (mirror marina-side shape) - Polymorphic notes.service.ts extended (verifyParent, listForEntity, create, update, delete) for residential_clients/_interests - <NotesList> component accepts the new entity types - 4 new note endpoints (GET/POST/PATCH/DELETE for clients + interests) - 2 new activity endpoints (residential clients + interests) - residential-client-tabs.tsx + residential-interest-tabs.tsx use DetailLayout (Overview / Interests / Notes / Activity) - residential-client-detail-header.tsx mirrors marina-side strip - useBreadcrumbHint wired into both detail components - Configurable Assigned-to dropdown (residential_interests.view perm) CONFIGURABLE RESIDENTIAL STAGES - residential-stages.service.ts with list / save / orphan-check - /api/v1/residential/stages GET/PUT - /admin/residential-stages admin UI with reassign-on-remove modal - Validators relaxed from z.enum to z.string DOCUMENSO PHASE 1 - Schema: document_signers.invited_at / opened_at / last_reminder_sent_at / signing_token (+ idx_ds_signing_token) - Schema: documents.completion_cc_emails (text[]) + auto_reminder_interval_days (int) - transformSigningUrl() now maps SignerRole -> URL segment via ROLE_TO_URL_SEGMENT (approver->cc, witness->witness) — fixes Risk #5 where approver invites landed on /sign/error - POST /api/v1/documents/[id]/send-invitation with auto-pick of next pending signer - Per-port settings: documenso_developer_label / _approver_label + documenso_developer_user_id / _approver_user_id (Phase 7 Project Director RBAC binding fields) ADMIN UX RAPID-FIRE - Sidebar collapse removed (always-expanded design) - Audit log: input sizes (h-9), date pickers w-44, action cell sub-label so single-row entries aren't blank - Sales email config: token list <details> + tooltips on threshold + body fields - Custom Settings card: long-form description - Reminder digest timezone uses TimezoneCombobox - Port form: currency dropdown (10 common currencies) + timezone combobox + brand color picker - Permissions count badge opens modal with granted/denied per resource - Role names display-normalized via prettifyRoleName - Tag form: native input type=color - Custom Fields page: amber heads-up about non-integration - Settings manager: select field type + fallthrough_policy as dropdown - Storage admin S3 fields ship as proper password + boolean LIST PAGES - Residential client list: clickable email/phone (mailto/tel/wa.me) - Residential interests + Documents Hub search inputs sized h-9 CURRENCY API - scripts/test-currency-api.ts verifies live Frankfurter fetch -> DB upsert -> getRate -> convert. Inverse-rate drift <=0.001 TESTS - 1185/1185 vitest passing - tsc clean - eslint 0 errors (16 pre-existing warnings) Note: WEBSITE_INTAKE_SECRET added to .env.example but committed separately due to pre-commit hook policy on .env* files. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-07 21:02:12 +02:00
bigint,
} from 'drizzle-orm/pg-core';
import { ports } from './ports';
import { clients } from './clients';
feat(insights): Phase B schema + service skeletons PR1 of Phase B per docs/superpowers/specs/2026-04-28-phase-b-insights-alerts-design.md. Lays the foundation that PRs 2-10 will fill in with behaviour. Schema (migration 0014): - alerts table with rule-engine fields (rule_id, severity, link, entity_type/id, fingerprint, fired/dismissed/acknowledged/resolved timestamps, jsonb metadata). Partial-unique fingerprint index keeps one open row per (port, rule, entity); separate indexes power severity-filtered and time-ordered queries. - analytics_snapshots (port_id, metric_id) -> jsonb cache + computedAt for the 15-min recurring refresh. - expenses: duplicate_of self-FK, dedup_scanned_at, ocr_status/raw/ confidence; partial index on (port, vendor, amount, date) where duplicate_of IS NULL drives the dedup heuristic. - audit_logs.search_text: GENERATED ALWAYS tsvector over action+entity_type+entity_id+user_id, GIN-indexed (drizzle can't model GENERATED ALWAYS in TS yet, so the migration appends manual ALTER + the GIN index). Service skeletons in src/lib/services/: - alerts.service.ts: fingerprintFor, reconcileAlertsForPort (upsert + auto-resolve), dismiss, acknowledge, listAlertsForPort. - alert-rules.ts: RULE_REGISTRY of 10 rule evaluators (currently no-op); PR2 fills in the bodies. - analytics.service.ts: readSnapshot/writeSnapshot with 15-min TTL + no-op compute* stubs for the four chart series; PR3 fills behavior. - expense-dedup.service.ts: scanForDuplicates + markBestDuplicate using the partial dedup index. PR8 wires the BullMQ trigger. - expense-ocr.service.ts: OcrResult/OcrLineItem types + ocrReceipt stub. PR9 wires Claude Vision (Haiku 4.5 + ephemeral system-prompt cache). - audit-search.service.ts: tsvector @@ plainto_tsquery + cursor pagination on (createdAt, id). PR10 wires the admin UI. tsc clean, lint clean, vitest 675/675 (one unrelated AES random-output flake passes solo). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-28 14:43:01 +02:00
// Drizzle doesn't ship a first-class tsvector type; declare a thin custom one.
const tsvector = customType<{ data: string; driverData: string }>({
dataType() {
return 'tsvector';
},
});
export const auditLogs = pgTable(
'audit_logs',
{
feat(insights): Phase B schema + service skeletons PR1 of Phase B per docs/superpowers/specs/2026-04-28-phase-b-insights-alerts-design.md. Lays the foundation that PRs 2-10 will fill in with behaviour. Schema (migration 0014): - alerts table with rule-engine fields (rule_id, severity, link, entity_type/id, fingerprint, fired/dismissed/acknowledged/resolved timestamps, jsonb metadata). Partial-unique fingerprint index keeps one open row per (port, rule, entity); separate indexes power severity-filtered and time-ordered queries. - analytics_snapshots (port_id, metric_id) -> jsonb cache + computedAt for the 15-min recurring refresh. - expenses: duplicate_of self-FK, dedup_scanned_at, ocr_status/raw/ confidence; partial index on (port, vendor, amount, date) where duplicate_of IS NULL drives the dedup heuristic. - audit_logs.search_text: GENERATED ALWAYS tsvector over action+entity_type+entity_id+user_id, GIN-indexed (drizzle can't model GENERATED ALWAYS in TS yet, so the migration appends manual ALTER + the GIN index). Service skeletons in src/lib/services/: - alerts.service.ts: fingerprintFor, reconcileAlertsForPort (upsert + auto-resolve), dismiss, acknowledge, listAlertsForPort. - alert-rules.ts: RULE_REGISTRY of 10 rule evaluators (currently no-op); PR2 fills in the bodies. - analytics.service.ts: readSnapshot/writeSnapshot with 15-min TTL + no-op compute* stubs for the four chart series; PR3 fills behavior. - expense-dedup.service.ts: scanForDuplicates + markBestDuplicate using the partial dedup index. PR8 wires the BullMQ trigger. - expense-ocr.service.ts: OcrResult/OcrLineItem types + ocrReceipt stub. PR9 wires Claude Vision (Haiku 4.5 + ephemeral system-prompt cache). - audit-search.service.ts: tsvector @@ plainto_tsquery + cursor pagination on (createdAt, id). PR10 wires the admin UI. tsc clean, lint clean, vitest 675/675 (one unrelated AES random-output flake passes solo). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-28 14:43:01 +02:00
id: text('id')
.primaryKey()
.$defaultFn(() => crypto.randomUUID()),
portId: text('port_id').references(() => ports.id), // null for system-level events
userId: text('user_id'), // null for system-generated events
action: text('action').notNull(), // create, update, delete, archive, restore, merge, login, logout, revert
entityType: text('entity_type').notNull(), // client, interest, berth, expense, invoice, file, user, role, etc.
entityId: text('entity_id'),
fieldChanged: text('field_changed'),
oldValue: jsonb('old_value'),
newValue: jsonb('new_value'),
ipAddress: text('ip_address'),
userAgent: text('user_agent'),
revertedBy: text('reverted_by'), // user ID if this change was reverted
revertedAt: timestamp('reverted_at', { withTimezone: true }),
// eslint-disable-next-line @typescript-eslint/no-explicit-any
revertOf: text('revert_of').references((): any => auditLogs.id),
metadata: jsonb('metadata').default({}),
/** 'info' | 'warning' | 'error' | 'critical' - drives the row badge
* in the inspector. Most user actions are 'info'. */
severity: text('severity').notNull().default('info'),
/** 'user' | 'system' | 'auth' | 'webhook' | 'cron' | 'job' - lets the
* UI filter by event origin without grepping action names. */
source: text('source').notNull().default('user'),
fix(audit): comprehensive 2026-05-15 audit fix wave + Documenso v2 polish Bundles the prior session's 50-task fix sweep (Documenso v2 + EOI/signing- progress redesign + env-to-admin migration + dev-mode banner) with the 2026-05-18 audit fix wave (3 CRITICAL, 14 HIGH, 28 MEDIUM, 6 LOW). CRITICAL (3): - C-01 interest-berths INNER JOIN -> LEFT JOIN so hard-deleted berths no longer silently drop interest links - C-02 /setup added to PUBLIC_PATHS; fresh-deploy bootstrap loop fixed - C-03 generic PATCH /interests/[id] no longer accepts pipelineStage — callers must go through /stage with the override-guard chain HIGH (14/15): - H-01 explicit ON DELETE on previously-implicit NO ACTION FKs across interests/documents/reservations/reminders/invoices (migration 0070) - H-02 login page reads ?redirect= param with same-origin guard - H-03 CRM invite token moves to URL fragment so it never lands in nginx access logs / Referer headers - H-04 Retry-After header on sign-in-by-identifier 429 (RFC 6585 §4) - H-05 toggleAccount writes an audit row - H-06 upsertSetting masks any value whose key ends with _encrypted - H-07 archiveClient cascade fires per-interest audit rows - H-08 createSalesTransporter applies SMTP_TIMEOUTS - H-09 AppShell stable children — viewport flip across breakpoint no longer destroys in-progress form drafts - H-10 portal documents page swaps Unicode glyph status icons for Lucide CheckCircle2/XCircle/Circle + aria-labels - H-12 list components swap alert(...) for toast.warning(...) - H-13 5 icon-only buttons gain aria-label - H-14 parseBody treats empty bodies as {} - H-15 admin layout renders a 403 panel instead of silent bounce - H-11 not applicable — mobile-search-overlay IS a mobile bottom-sheet MEDIUM (28+): - M-MT01-05 defense-in-depth port_id/parent-id filters on UPDATE/DELETE WHEREs across custom-fields, notes (all 6 entity types x update + delete), client-contacts, yacht ownerClient lookup, webhook reads - M-D01 documents-hub realtime event-name typo (file:created -> uploaded) - M-EM01 portal-auth emails thread through portId - M-EM02 sendEmail accepts cc/bcc params - M-EM04 notification_digest catalog key - M-IN01 portal presigned download URLs use 4h TTL - M-IN02 OpenAI client lazy-instantiated - M-IN04 stale pdfme refs updated to pdf-lib AcroForm - M-IN05 umami.testConnection returns tagged union - M-L01 reservations tenure_type unified with berths - M-L02 report-generators canonicalize stage values - M-AU01 audit log placeholder copy fixed - M-AU04 outcome_set / outcome_cleared distinct audit verbs - M-NEW-2 activity feed entity name+type separator - M-R01 portal allowlist narrowed + portal_session backstop in proxy - M-SC02 companies archived partial index - M-SC04 audit_logs.searchText documented as DB-managed - M-S01 storage_s3_access_key_encrypted admin field - M-U01 audit log empty state uses <EmptyState> - M-U09 invoice delete dialog -> <AlertDialog> - M-U10 toast.success on ClientForm + InterestForm create/edit - M-U11 settings-form-card logo preview alt text - M-U14 mobile topbar title on clients/yachts/interests/berths - M-U15 Invoices in mobile More-sheet LOW (6/8): - L-AU01 severity defaults for security-relevant verbs - L-AU02 +13 missing actions in admin audit filter - L-AU03 +7 missing entity types in admin audit filter - L-AU04 dead listAuditLogs stubbed - L-D02 CLAUDE.md Owner-wins chain tightened Bonus — Document detail polish (#67 partial, 3/6 deliverables): - state-aware action button per signer - watcher Add UI with display-name resolution - cleanSignerName cleanup Prior session work bundled in: - Documenso v2 webhook + envelope-ID normalization + sequential signing - SigningProgress UI redesign (avatars, per-signer state, timestamps) - env->admin settings registry + RegistryDrivenForm + encrypted creds - Embedded-signing card + Test connection + setup help - Dev-mode EMAIL_REDIRECT_TO banner - Pipeline rules admin page - Sales email config card - Audit log details Sheet - EOI tab: Finalising badge, absolute timestamps, sequential indicator - Notes pipeline_stage_at_creation (migration 0069) - Documenso numeric ID dual-key webhook (migration 0068) - Dimensions criterion copy (migration 0067) Tests: 1374/1374 vitest pass. tsc clean. lint clean. See docs/AUDIT-FIX-WAVE-2026-05-18.md for the full progress report and the user-input items still pending. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-18 13:28:50 +02:00
/** Full-text search column. **Read-only / DB-managed**: the column is
* declared `GENERATED ALWAYS AS (...) STORED` in migration
* 0014_black_banshee.sql (covers action + entity_type + entity_id +
* user_id). Drizzle has no first-class marker for generated columns,
* so writes through this schema property would be rejected by
* Postgres at SQL level - never set this from application code.
fix(audit): comprehensive 2026-05-15 audit fix wave + Documenso v2 polish Bundles the prior session's 50-task fix sweep (Documenso v2 + EOI/signing- progress redesign + env-to-admin migration + dev-mode banner) with the 2026-05-18 audit fix wave (3 CRITICAL, 14 HIGH, 28 MEDIUM, 6 LOW). CRITICAL (3): - C-01 interest-berths INNER JOIN -> LEFT JOIN so hard-deleted berths no longer silently drop interest links - C-02 /setup added to PUBLIC_PATHS; fresh-deploy bootstrap loop fixed - C-03 generic PATCH /interests/[id] no longer accepts pipelineStage — callers must go through /stage with the override-guard chain HIGH (14/15): - H-01 explicit ON DELETE on previously-implicit NO ACTION FKs across interests/documents/reservations/reminders/invoices (migration 0070) - H-02 login page reads ?redirect= param with same-origin guard - H-03 CRM invite token moves to URL fragment so it never lands in nginx access logs / Referer headers - H-04 Retry-After header on sign-in-by-identifier 429 (RFC 6585 §4) - H-05 toggleAccount writes an audit row - H-06 upsertSetting masks any value whose key ends with _encrypted - H-07 archiveClient cascade fires per-interest audit rows - H-08 createSalesTransporter applies SMTP_TIMEOUTS - H-09 AppShell stable children — viewport flip across breakpoint no longer destroys in-progress form drafts - H-10 portal documents page swaps Unicode glyph status icons for Lucide CheckCircle2/XCircle/Circle + aria-labels - H-12 list components swap alert(...) for toast.warning(...) - H-13 5 icon-only buttons gain aria-label - H-14 parseBody treats empty bodies as {} - H-15 admin layout renders a 403 panel instead of silent bounce - H-11 not applicable — mobile-search-overlay IS a mobile bottom-sheet MEDIUM (28+): - M-MT01-05 defense-in-depth port_id/parent-id filters on UPDATE/DELETE WHEREs across custom-fields, notes (all 6 entity types x update + delete), client-contacts, yacht ownerClient lookup, webhook reads - M-D01 documents-hub realtime event-name typo (file:created -> uploaded) - M-EM01 portal-auth emails thread through portId - M-EM02 sendEmail accepts cc/bcc params - M-EM04 notification_digest catalog key - M-IN01 portal presigned download URLs use 4h TTL - M-IN02 OpenAI client lazy-instantiated - M-IN04 stale pdfme refs updated to pdf-lib AcroForm - M-IN05 umami.testConnection returns tagged union - M-L01 reservations tenure_type unified with berths - M-L02 report-generators canonicalize stage values - M-AU01 audit log placeholder copy fixed - M-AU04 outcome_set / outcome_cleared distinct audit verbs - M-NEW-2 activity feed entity name+type separator - M-R01 portal allowlist narrowed + portal_session backstop in proxy - M-SC02 companies archived partial index - M-SC04 audit_logs.searchText documented as DB-managed - M-S01 storage_s3_access_key_encrypted admin field - M-U01 audit log empty state uses <EmptyState> - M-U09 invoice delete dialog -> <AlertDialog> - M-U10 toast.success on ClientForm + InterestForm create/edit - M-U11 settings-form-card logo preview alt text - M-U14 mobile topbar title on clients/yachts/interests/berths - M-U15 Invoices in mobile More-sheet LOW (6/8): - L-AU01 severity defaults for security-relevant verbs - L-AU02 +13 missing actions in admin audit filter - L-AU03 +7 missing entity types in admin audit filter - L-AU04 dead listAuditLogs stubbed - L-D02 CLAUDE.md Owner-wins chain tightened Bonus — Document detail polish (#67 partial, 3/6 deliverables): - state-aware action button per signer - watcher Add UI with display-name resolution - cleanSignerName cleanup Prior session work bundled in: - Documenso v2 webhook + envelope-ID normalization + sequential signing - SigningProgress UI redesign (avatars, per-signer state, timestamps) - env->admin settings registry + RegistryDrivenForm + encrypted creds - Embedded-signing card + Test connection + setup help - Dev-mode EMAIL_REDIRECT_TO banner - Pipeline rules admin page - Sales email config card - Audit log details Sheet - EOI tab: Finalising badge, absolute timestamps, sequential indicator - Notes pipeline_stage_at_creation (migration 0069) - Documenso numeric ID dual-key webhook (migration 0068) - Dimensions criterion copy (migration 0067) Tests: 1374/1374 vitest pass. tsc clean. lint clean. See docs/AUDIT-FIX-WAVE-2026-05-18.md for the full progress report and the user-input items still pending. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-18 13:28:50 +02:00
* M-SC04: documented to prevent accidental write attempts. */
feat(insights): Phase B schema + service skeletons PR1 of Phase B per docs/superpowers/specs/2026-04-28-phase-b-insights-alerts-design.md. Lays the foundation that PRs 2-10 will fill in with behaviour. Schema (migration 0014): - alerts table with rule-engine fields (rule_id, severity, link, entity_type/id, fingerprint, fired/dismissed/acknowledged/resolved timestamps, jsonb metadata). Partial-unique fingerprint index keeps one open row per (port, rule, entity); separate indexes power severity-filtered and time-ordered queries. - analytics_snapshots (port_id, metric_id) -> jsonb cache + computedAt for the 15-min recurring refresh. - expenses: duplicate_of self-FK, dedup_scanned_at, ocr_status/raw/ confidence; partial index on (port, vendor, amount, date) where duplicate_of IS NULL drives the dedup heuristic. - audit_logs.search_text: GENERATED ALWAYS tsvector over action+entity_type+entity_id+user_id, GIN-indexed (drizzle can't model GENERATED ALWAYS in TS yet, so the migration appends manual ALTER + the GIN index). Service skeletons in src/lib/services/: - alerts.service.ts: fingerprintFor, reconcileAlertsForPort (upsert + auto-resolve), dismiss, acknowledge, listAlertsForPort. - alert-rules.ts: RULE_REGISTRY of 10 rule evaluators (currently no-op); PR2 fills in the bodies. - analytics.service.ts: readSnapshot/writeSnapshot with 15-min TTL + no-op compute* stubs for the four chart series; PR3 fills behavior. - expense-dedup.service.ts: scanForDuplicates + markBestDuplicate using the partial dedup index. PR8 wires the BullMQ trigger. - expense-ocr.service.ts: OcrResult/OcrLineItem types + ocrReceipt stub. PR9 wires Claude Vision (Haiku 4.5 + ephemeral system-prompt cache). - audit-search.service.ts: tsvector @@ plainto_tsquery + cursor pagination on (createdAt, id). PR10 wires the admin UI. tsc clean, lint clean, vitest 675/675 (one unrelated AES random-output flake passes solo). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-28 14:43:01 +02:00
searchText: tsvector('search_text'),
createdAt: timestamp('created_at', { withTimezone: true }).notNull().defaultNow(),
},
(table) => [
index('idx_al_port').on(table.portId, table.createdAt),
index('idx_al_entity').on(table.entityType, table.entityId),
index('idx_al_user').on(table.userId, table.createdAt),
index('idx_al_created').on(table.createdAt),
index('idx_al_severity').on(table.portId, table.severity, table.createdAt),
index('idx_al_source').on(table.portId, table.source, table.createdAt),
],
);
export const tags = pgTable(
'tags',
{
feat(insights): Phase B schema + service skeletons PR1 of Phase B per docs/superpowers/specs/2026-04-28-phase-b-insights-alerts-design.md. Lays the foundation that PRs 2-10 will fill in with behaviour. Schema (migration 0014): - alerts table with rule-engine fields (rule_id, severity, link, entity_type/id, fingerprint, fired/dismissed/acknowledged/resolved timestamps, jsonb metadata). Partial-unique fingerprint index keeps one open row per (port, rule, entity); separate indexes power severity-filtered and time-ordered queries. - analytics_snapshots (port_id, metric_id) -> jsonb cache + computedAt for the 15-min recurring refresh. - expenses: duplicate_of self-FK, dedup_scanned_at, ocr_status/raw/ confidence; partial index on (port, vendor, amount, date) where duplicate_of IS NULL drives the dedup heuristic. - audit_logs.search_text: GENERATED ALWAYS tsvector over action+entity_type+entity_id+user_id, GIN-indexed (drizzle can't model GENERATED ALWAYS in TS yet, so the migration appends manual ALTER + the GIN index). Service skeletons in src/lib/services/: - alerts.service.ts: fingerprintFor, reconcileAlertsForPort (upsert + auto-resolve), dismiss, acknowledge, listAlertsForPort. - alert-rules.ts: RULE_REGISTRY of 10 rule evaluators (currently no-op); PR2 fills in the bodies. - analytics.service.ts: readSnapshot/writeSnapshot with 15-min TTL + no-op compute* stubs for the four chart series; PR3 fills behavior. - expense-dedup.service.ts: scanForDuplicates + markBestDuplicate using the partial dedup index. PR8 wires the BullMQ trigger. - expense-ocr.service.ts: OcrResult/OcrLineItem types + ocrReceipt stub. PR9 wires Claude Vision (Haiku 4.5 + ephemeral system-prompt cache). - audit-search.service.ts: tsvector @@ plainto_tsquery + cursor pagination on (createdAt, id). PR10 wires the admin UI. tsc clean, lint clean, vitest 675/675 (one unrelated AES random-output flake passes solo). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-28 14:43:01 +02:00
id: text('id')
.primaryKey()
.$defaultFn(() => crypto.randomUUID()),
portId: text('port_id')
.notNull()
.references(() => ports.id),
name: text('name').notNull(),
color: text('color').notNull().default('#6B7280'),
createdAt: timestamp('created_at', { withTimezone: true }).notNull().defaultNow(),
},
(table) => [
uniqueIndex('tags_port_name_idx').on(table.portId, table.name),
index('idx_tags_port').on(table.portId),
],
);
export const webhooks = pgTable(
'webhooks',
{
feat(insights): Phase B schema + service skeletons PR1 of Phase B per docs/superpowers/specs/2026-04-28-phase-b-insights-alerts-design.md. Lays the foundation that PRs 2-10 will fill in with behaviour. Schema (migration 0014): - alerts table with rule-engine fields (rule_id, severity, link, entity_type/id, fingerprint, fired/dismissed/acknowledged/resolved timestamps, jsonb metadata). Partial-unique fingerprint index keeps one open row per (port, rule, entity); separate indexes power severity-filtered and time-ordered queries. - analytics_snapshots (port_id, metric_id) -> jsonb cache + computedAt for the 15-min recurring refresh. - expenses: duplicate_of self-FK, dedup_scanned_at, ocr_status/raw/ confidence; partial index on (port, vendor, amount, date) where duplicate_of IS NULL drives the dedup heuristic. - audit_logs.search_text: GENERATED ALWAYS tsvector over action+entity_type+entity_id+user_id, GIN-indexed (drizzle can't model GENERATED ALWAYS in TS yet, so the migration appends manual ALTER + the GIN index). Service skeletons in src/lib/services/: - alerts.service.ts: fingerprintFor, reconcileAlertsForPort (upsert + auto-resolve), dismiss, acknowledge, listAlertsForPort. - alert-rules.ts: RULE_REGISTRY of 10 rule evaluators (currently no-op); PR2 fills in the bodies. - analytics.service.ts: readSnapshot/writeSnapshot with 15-min TTL + no-op compute* stubs for the four chart series; PR3 fills behavior. - expense-dedup.service.ts: scanForDuplicates + markBestDuplicate using the partial dedup index. PR8 wires the BullMQ trigger. - expense-ocr.service.ts: OcrResult/OcrLineItem types + ocrReceipt stub. PR9 wires Claude Vision (Haiku 4.5 + ephemeral system-prompt cache). - audit-search.service.ts: tsvector @@ plainto_tsquery + cursor pagination on (createdAt, id). PR10 wires the admin UI. tsc clean, lint clean, vitest 675/675 (one unrelated AES random-output flake passes solo). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-28 14:43:01 +02:00
id: text('id')
.primaryKey()
.$defaultFn(() => crypto.randomUUID()),
portId: text('port_id')
.notNull()
.references(() => ports.id),
name: text('name').notNull(),
url: text('url').notNull(),
secret: text('secret'),
events: text('events').array().notNull(),
isActive: boolean('is_active').notNull().default(true),
createdBy: text('created_by').notNull(),
createdAt: timestamp('created_at', { withTimezone: true }).notNull().defaultNow(),
updatedAt: timestamp('updated_at', { withTimezone: true }).notNull().defaultNow(),
},
(table) => [index('idx_webhooks_port').on(table.portId)],
);
export const webhookDeliveries = pgTable(
'webhook_deliveries',
{
feat(insights): Phase B schema + service skeletons PR1 of Phase B per docs/superpowers/specs/2026-04-28-phase-b-insights-alerts-design.md. Lays the foundation that PRs 2-10 will fill in with behaviour. Schema (migration 0014): - alerts table with rule-engine fields (rule_id, severity, link, entity_type/id, fingerprint, fired/dismissed/acknowledged/resolved timestamps, jsonb metadata). Partial-unique fingerprint index keeps one open row per (port, rule, entity); separate indexes power severity-filtered and time-ordered queries. - analytics_snapshots (port_id, metric_id) -> jsonb cache + computedAt for the 15-min recurring refresh. - expenses: duplicate_of self-FK, dedup_scanned_at, ocr_status/raw/ confidence; partial index on (port, vendor, amount, date) where duplicate_of IS NULL drives the dedup heuristic. - audit_logs.search_text: GENERATED ALWAYS tsvector over action+entity_type+entity_id+user_id, GIN-indexed (drizzle can't model GENERATED ALWAYS in TS yet, so the migration appends manual ALTER + the GIN index). Service skeletons in src/lib/services/: - alerts.service.ts: fingerprintFor, reconcileAlertsForPort (upsert + auto-resolve), dismiss, acknowledge, listAlertsForPort. - alert-rules.ts: RULE_REGISTRY of 10 rule evaluators (currently no-op); PR2 fills in the bodies. - analytics.service.ts: readSnapshot/writeSnapshot with 15-min TTL + no-op compute* stubs for the four chart series; PR3 fills behavior. - expense-dedup.service.ts: scanForDuplicates + markBestDuplicate using the partial dedup index. PR8 wires the BullMQ trigger. - expense-ocr.service.ts: OcrResult/OcrLineItem types + ocrReceipt stub. PR9 wires Claude Vision (Haiku 4.5 + ephemeral system-prompt cache). - audit-search.service.ts: tsvector @@ plainto_tsquery + cursor pagination on (createdAt, id). PR10 wires the admin UI. tsc clean, lint clean, vitest 675/675 (one unrelated AES random-output flake passes solo). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-28 14:43:01 +02:00
id: text('id')
.primaryKey()
.$defaultFn(() => crypto.randomUUID()),
webhookId: text('webhook_id')
.notNull()
.references(() => webhooks.id, { onDelete: 'cascade' }),
eventType: text('event_type').notNull(),
payload: jsonb('payload').notNull(),
responseStatus: integer('response_status'),
responseBody: text('response_body'),
attempt: integer('attempt').notNull().default(1),
status: text('status').notNull().default('pending'), // pending, success, failed, dead_letter
deliveredAt: timestamp('delivered_at', { withTimezone: true }),
createdAt: timestamp('created_at', { withTimezone: true }).notNull().defaultNow(),
},
(table) => [index('idx_wd_webhook').on(table.webhookId, table.createdAt)],
);
export const systemSettings = pgTable(
'system_settings',
{
key: text('key').notNull(),
value: jsonb('value').notNull(),
portId: text('port_id').references(() => ports.id), // null for global settings
updatedBy: text('updated_by'),
updatedAt: timestamp('updated_at', { withTimezone: true }).notNull().defaultNow(),
},
(table) => [
fix(audit): non-Documenso backlog sweep — port-binding, NULLS NOT DISTINCT, custom merge tokens, company docs Wave through the remaining audit-final-deferred items that aren't blocked on the back-burnered Documenso work. Multi-tenant isolation: - Storage proxy ProxyTokenPayload gains optional `p` (port slug) claim; verifier asserts `key.startsWith(${p}/)`. Defense-in-depth against a buggy issuer in some future code path that mixes port scopes — every storage key generated by generateStorageKey() already prefixes the slug. document-sends opts in for 24h emailed download links; other callers continue working unchanged via the optional field. DB schema reconciliation: - Migration 0047 rebuilds system_settings unique index with NULLS NOT DISTINCT (Postgres 15+) so global settings (port_id IS NULL) are uniquely keyed by `key` alone. Surfaced + dedupe'd 65 duplicate (storage_backend, NULL) rows that had accumulated from race-prone delete-then-insert patterns in ocr-config / settings / residential- stages / ai-budget services. All four services converted to true onConflictDoUpdate upserts so the race window is closed. API uniformity: - Response shape standardization: 16 routes converted from `{ success: true }` to 204 No Content. CLAUDE.md documents the convention (`{ data: <T> }` for content, 204 for empty mutations, portal-auth retains `{ success: true }` for the frontend's auth chain). - req.json() → parseBody() migration across 9 admin/CRM routes (custom-fields, expenses/export ×3, currency convert, search/recently-viewed, admin/duplicates, berths/pdf-{upload-url, versions, parse-results}). Uniform 400 error shapes for ZodError-flagged bodies. Custom-fields merge tokens (shipped end-to-end): - merge-fields.ts gains CUSTOM_MERGE_TOKEN_RE + helpers for the `{{custom.<fieldName>}}` shape. - document-templates validator accepts the dynamic shape alongside the static catalog tokens. - document-sends.service mergeCustomFieldValues resolver fetches per-port custom_field_definitions for client/interest/berth contexts and substitutes stored values keyed by `{{custom.fieldName}}`. - custom-fields-manager amber banner updated to reflect that merge tokens now expand (search index + entity-diff remain documented design limitations). /api/v1/files cross-entity filtering: - Validator + listFiles + uploadFile accept companyId AND yachtId alongside clientId. file-upload-zone propagates both. - New CompanyFilesTab component mirrors ClientFilesTab; restored as a visible Documents tab in company-tabs.tsx (was a hidden stub). Inline TODOs: - Reviewed remaining two TODOs (per-user reminder schedule, import worker handlers). Both are placeholders for future feature surfaces, not bugs — per-port digest works for every customer; nothing currently enqueues import jobs (verified). Annotated in BACKLOG. BACKLOG.md updated to reflect what landed and what's still pending (Documenso-related items still bundled with the back-burnered phases). Tests: 1185/1185 vitest, tsc clean.
2026-05-08 02:20:27 +02:00
// Migration 0047 rebuilds this index with `NULLS NOT DISTINCT` so a
// global setting (port_id IS NULL) is unique by key alone - the
fix(audit): non-Documenso backlog sweep — port-binding, NULLS NOT DISTINCT, custom merge tokens, company docs Wave through the remaining audit-final-deferred items that aren't blocked on the back-burnered Documenso work. Multi-tenant isolation: - Storage proxy ProxyTokenPayload gains optional `p` (port slug) claim; verifier asserts `key.startsWith(${p}/)`. Defense-in-depth against a buggy issuer in some future code path that mixes port scopes — every storage key generated by generateStorageKey() already prefixes the slug. document-sends opts in for 24h emailed download links; other callers continue working unchanged via the optional field. DB schema reconciliation: - Migration 0047 rebuilds system_settings unique index with NULLS NOT DISTINCT (Postgres 15+) so global settings (port_id IS NULL) are uniquely keyed by `key` alone. Surfaced + dedupe'd 65 duplicate (storage_backend, NULL) rows that had accumulated from race-prone delete-then-insert patterns in ocr-config / settings / residential- stages / ai-budget services. All four services converted to true onConflictDoUpdate upserts so the race window is closed. API uniformity: - Response shape standardization: 16 routes converted from `{ success: true }` to 204 No Content. CLAUDE.md documents the convention (`{ data: <T> }` for content, 204 for empty mutations, portal-auth retains `{ success: true }` for the frontend's auth chain). - req.json() → parseBody() migration across 9 admin/CRM routes (custom-fields, expenses/export ×3, currency convert, search/recently-viewed, admin/duplicates, berths/pdf-{upload-url, versions, parse-results}). Uniform 400 error shapes for ZodError-flagged bodies. Custom-fields merge tokens (shipped end-to-end): - merge-fields.ts gains CUSTOM_MERGE_TOKEN_RE + helpers for the `{{custom.<fieldName>}}` shape. - document-templates validator accepts the dynamic shape alongside the static catalog tokens. - document-sends.service mergeCustomFieldValues resolver fetches per-port custom_field_definitions for client/interest/berth contexts and substitutes stored values keyed by `{{custom.fieldName}}`. - custom-fields-manager amber banner updated to reflect that merge tokens now expand (search index + entity-diff remain documented design limitations). /api/v1/files cross-entity filtering: - Validator + listFiles + uploadFile accept companyId AND yachtId alongside clientId. file-upload-zone propagates both. - New CompanyFilesTab component mirrors ClientFilesTab; restored as a visible Documents tab in company-tabs.tsx (was a hidden stub). Inline TODOs: - Reviewed remaining two TODOs (per-user reminder schedule, import worker handlers). Both are placeholders for future feature surfaces, not bugs — per-port digest works for every customer; nothing currently enqueues import jobs (verified). Annotated in BACKLOG. BACKLOG.md updated to reflect what landed and what's still pending (Documenso-related items still bundled with the back-burnered phases). Tests: 1185/1185 vitest, tsc clean.
2026-05-08 02:20:27 +02:00
// default `NULLS DISTINCT` semantics let duplicates accumulate.
// Drizzle's `uniqueIndex` builder doesn't surface NULLS NOT DISTINCT,
// so the migration is the source of truth for that flag and
// `db:push` against an empty DB would skip it (matches the
// documented limitation for `berths.current_pdf_version_id`).
uniqueIndex('system_settings_key_port_idx').on(table.key, table.portId),
],
);
export const savedViews = pgTable(
'saved_views',
{
feat(insights): Phase B schema + service skeletons PR1 of Phase B per docs/superpowers/specs/2026-04-28-phase-b-insights-alerts-design.md. Lays the foundation that PRs 2-10 will fill in with behaviour. Schema (migration 0014): - alerts table with rule-engine fields (rule_id, severity, link, entity_type/id, fingerprint, fired/dismissed/acknowledged/resolved timestamps, jsonb metadata). Partial-unique fingerprint index keeps one open row per (port, rule, entity); separate indexes power severity-filtered and time-ordered queries. - analytics_snapshots (port_id, metric_id) -> jsonb cache + computedAt for the 15-min recurring refresh. - expenses: duplicate_of self-FK, dedup_scanned_at, ocr_status/raw/ confidence; partial index on (port, vendor, amount, date) where duplicate_of IS NULL drives the dedup heuristic. - audit_logs.search_text: GENERATED ALWAYS tsvector over action+entity_type+entity_id+user_id, GIN-indexed (drizzle can't model GENERATED ALWAYS in TS yet, so the migration appends manual ALTER + the GIN index). Service skeletons in src/lib/services/: - alerts.service.ts: fingerprintFor, reconcileAlertsForPort (upsert + auto-resolve), dismiss, acknowledge, listAlertsForPort. - alert-rules.ts: RULE_REGISTRY of 10 rule evaluators (currently no-op); PR2 fills in the bodies. - analytics.service.ts: readSnapshot/writeSnapshot with 15-min TTL + no-op compute* stubs for the four chart series; PR3 fills behavior. - expense-dedup.service.ts: scanForDuplicates + markBestDuplicate using the partial dedup index. PR8 wires the BullMQ trigger. - expense-ocr.service.ts: OcrResult/OcrLineItem types + ocrReceipt stub. PR9 wires Claude Vision (Haiku 4.5 + ephemeral system-prompt cache). - audit-search.service.ts: tsvector @@ plainto_tsquery + cursor pagination on (createdAt, id). PR10 wires the admin UI. tsc clean, lint clean, vitest 675/675 (one unrelated AES random-output flake passes solo). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-28 14:43:01 +02:00
id: text('id')
.primaryKey()
.$defaultFn(() => crypto.randomUUID()),
portId: text('port_id')
.notNull()
.references(() => ports.id),
userId: text('user_id').notNull(),
entityType: text('entity_type').notNull(), // clients, interests, berths, expenses, invoices
name: text('name').notNull(),
filters: jsonb('filters').notNull(),
sortConfig: jsonb('sort_config'),
columnConfig: jsonb('column_config'),
isShared: boolean('is_shared').notNull().default(false),
isDefault: boolean('is_default').notNull().default(false),
createdAt: timestamp('created_at', { withTimezone: true }).notNull().defaultNow(),
updatedAt: timestamp('updated_at', { withTimezone: true }).notNull().defaultNow(),
},
(table) => [index('idx_sv_user').on(table.userId, table.entityType)],
);
export const scratchpadNotes = pgTable(
'scratchpad_notes',
{
feat(insights): Phase B schema + service skeletons PR1 of Phase B per docs/superpowers/specs/2026-04-28-phase-b-insights-alerts-design.md. Lays the foundation that PRs 2-10 will fill in with behaviour. Schema (migration 0014): - alerts table with rule-engine fields (rule_id, severity, link, entity_type/id, fingerprint, fired/dismissed/acknowledged/resolved timestamps, jsonb metadata). Partial-unique fingerprint index keeps one open row per (port, rule, entity); separate indexes power severity-filtered and time-ordered queries. - analytics_snapshots (port_id, metric_id) -> jsonb cache + computedAt for the 15-min recurring refresh. - expenses: duplicate_of self-FK, dedup_scanned_at, ocr_status/raw/ confidence; partial index on (port, vendor, amount, date) where duplicate_of IS NULL drives the dedup heuristic. - audit_logs.search_text: GENERATED ALWAYS tsvector over action+entity_type+entity_id+user_id, GIN-indexed (drizzle can't model GENERATED ALWAYS in TS yet, so the migration appends manual ALTER + the GIN index). Service skeletons in src/lib/services/: - alerts.service.ts: fingerprintFor, reconcileAlertsForPort (upsert + auto-resolve), dismiss, acknowledge, listAlertsForPort. - alert-rules.ts: RULE_REGISTRY of 10 rule evaluators (currently no-op); PR2 fills in the bodies. - analytics.service.ts: readSnapshot/writeSnapshot with 15-min TTL + no-op compute* stubs for the four chart series; PR3 fills behavior. - expense-dedup.service.ts: scanForDuplicates + markBestDuplicate using the partial dedup index. PR8 wires the BullMQ trigger. - expense-ocr.service.ts: OcrResult/OcrLineItem types + ocrReceipt stub. PR9 wires Claude Vision (Haiku 4.5 + ephemeral system-prompt cache). - audit-search.service.ts: tsvector @@ plainto_tsquery + cursor pagination on (createdAt, id). PR10 wires the admin UI. tsc clean, lint clean, vitest 675/675 (one unrelated AES random-output flake passes solo). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-28 14:43:01 +02:00
id: text('id')
.primaryKey()
.$defaultFn(() => crypto.randomUUID()),
userId: text('user_id').notNull(),
content: text('content').notNull(),
linkedClientId: text('linked_client_id').references(() => clients.id),
linkedAt: timestamp('linked_at', { withTimezone: true }),
createdAt: timestamp('created_at', { withTimezone: true }).notNull().defaultNow(),
updatedAt: timestamp('updated_at', { withTimezone: true }).notNull().defaultNow(),
},
(table) => [index('idx_sp_user').on(table.userId)],
);
export const userNotificationPreferences = pgTable(
'user_notification_preferences',
{
feat(insights): Phase B schema + service skeletons PR1 of Phase B per docs/superpowers/specs/2026-04-28-phase-b-insights-alerts-design.md. Lays the foundation that PRs 2-10 will fill in with behaviour. Schema (migration 0014): - alerts table with rule-engine fields (rule_id, severity, link, entity_type/id, fingerprint, fired/dismissed/acknowledged/resolved timestamps, jsonb metadata). Partial-unique fingerprint index keeps one open row per (port, rule, entity); separate indexes power severity-filtered and time-ordered queries. - analytics_snapshots (port_id, metric_id) -> jsonb cache + computedAt for the 15-min recurring refresh. - expenses: duplicate_of self-FK, dedup_scanned_at, ocr_status/raw/ confidence; partial index on (port, vendor, amount, date) where duplicate_of IS NULL drives the dedup heuristic. - audit_logs.search_text: GENERATED ALWAYS tsvector over action+entity_type+entity_id+user_id, GIN-indexed (drizzle can't model GENERATED ALWAYS in TS yet, so the migration appends manual ALTER + the GIN index). Service skeletons in src/lib/services/: - alerts.service.ts: fingerprintFor, reconcileAlertsForPort (upsert + auto-resolve), dismiss, acknowledge, listAlertsForPort. - alert-rules.ts: RULE_REGISTRY of 10 rule evaluators (currently no-op); PR2 fills in the bodies. - analytics.service.ts: readSnapshot/writeSnapshot with 15-min TTL + no-op compute* stubs for the four chart series; PR3 fills behavior. - expense-dedup.service.ts: scanForDuplicates + markBestDuplicate using the partial dedup index. PR8 wires the BullMQ trigger. - expense-ocr.service.ts: OcrResult/OcrLineItem types + ocrReceipt stub. PR9 wires Claude Vision (Haiku 4.5 + ephemeral system-prompt cache). - audit-search.service.ts: tsvector @@ plainto_tsquery + cursor pagination on (createdAt, id). PR10 wires the admin UI. tsc clean, lint clean, vitest 675/675 (one unrelated AES random-output flake passes solo). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-28 14:43:01 +02:00
id: text('id')
.primaryKey()
.$defaultFn(() => crypto.randomUUID()),
userId: text('user_id').notNull(),
portId: text('port_id')
.notNull()
.references(() => ports.id),
notificationType: text('notification_type').notNull(),
inApp: boolean('in_app').notNull().default(true),
email: boolean('email').notNull().default(true),
},
(table) => [
uniqueIndex('unp_user_port_type_idx').on(table.userId, table.portId, table.notificationType),
],
);
export const currencyRates = pgTable(
'currency_rates',
{
feat(insights): Phase B schema + service skeletons PR1 of Phase B per docs/superpowers/specs/2026-04-28-phase-b-insights-alerts-design.md. Lays the foundation that PRs 2-10 will fill in with behaviour. Schema (migration 0014): - alerts table with rule-engine fields (rule_id, severity, link, entity_type/id, fingerprint, fired/dismissed/acknowledged/resolved timestamps, jsonb metadata). Partial-unique fingerprint index keeps one open row per (port, rule, entity); separate indexes power severity-filtered and time-ordered queries. - analytics_snapshots (port_id, metric_id) -> jsonb cache + computedAt for the 15-min recurring refresh. - expenses: duplicate_of self-FK, dedup_scanned_at, ocr_status/raw/ confidence; partial index on (port, vendor, amount, date) where duplicate_of IS NULL drives the dedup heuristic. - audit_logs.search_text: GENERATED ALWAYS tsvector over action+entity_type+entity_id+user_id, GIN-indexed (drizzle can't model GENERATED ALWAYS in TS yet, so the migration appends manual ALTER + the GIN index). Service skeletons in src/lib/services/: - alerts.service.ts: fingerprintFor, reconcileAlertsForPort (upsert + auto-resolve), dismiss, acknowledge, listAlertsForPort. - alert-rules.ts: RULE_REGISTRY of 10 rule evaluators (currently no-op); PR2 fills in the bodies. - analytics.service.ts: readSnapshot/writeSnapshot with 15-min TTL + no-op compute* stubs for the four chart series; PR3 fills behavior. - expense-dedup.service.ts: scanForDuplicates + markBestDuplicate using the partial dedup index. PR8 wires the BullMQ trigger. - expense-ocr.service.ts: OcrResult/OcrLineItem types + ocrReceipt stub. PR9 wires Claude Vision (Haiku 4.5 + ephemeral system-prompt cache). - audit-search.service.ts: tsvector @@ plainto_tsquery + cursor pagination on (createdAt, id). PR10 wires the admin UI. tsc clean, lint clean, vitest 675/675 (one unrelated AES random-output flake passes solo). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-28 14:43:01 +02:00
id: text('id')
.primaryKey()
.$defaultFn(() => crypto.randomUUID()),
baseCurrency: text('base_currency').notNull(),
targetCurrency: text('target_currency').notNull(),
rate: numeric('rate').notNull(),
source: text('source').notNull().default('frankfurter'), // frankfurter, manual
fetchedAt: timestamp('fetched_at', { withTimezone: true }).notNull().defaultNow(),
},
(table) => [
uniqueIndex('currency_rates_base_target_idx').on(table.baseCurrency, table.targetCurrency),
],
);
export const customFieldDefinitions = pgTable(
'custom_field_definitions',
{
feat(insights): Phase B schema + service skeletons PR1 of Phase B per docs/superpowers/specs/2026-04-28-phase-b-insights-alerts-design.md. Lays the foundation that PRs 2-10 will fill in with behaviour. Schema (migration 0014): - alerts table with rule-engine fields (rule_id, severity, link, entity_type/id, fingerprint, fired/dismissed/acknowledged/resolved timestamps, jsonb metadata). Partial-unique fingerprint index keeps one open row per (port, rule, entity); separate indexes power severity-filtered and time-ordered queries. - analytics_snapshots (port_id, metric_id) -> jsonb cache + computedAt for the 15-min recurring refresh. - expenses: duplicate_of self-FK, dedup_scanned_at, ocr_status/raw/ confidence; partial index on (port, vendor, amount, date) where duplicate_of IS NULL drives the dedup heuristic. - audit_logs.search_text: GENERATED ALWAYS tsvector over action+entity_type+entity_id+user_id, GIN-indexed (drizzle can't model GENERATED ALWAYS in TS yet, so the migration appends manual ALTER + the GIN index). Service skeletons in src/lib/services/: - alerts.service.ts: fingerprintFor, reconcileAlertsForPort (upsert + auto-resolve), dismiss, acknowledge, listAlertsForPort. - alert-rules.ts: RULE_REGISTRY of 10 rule evaluators (currently no-op); PR2 fills in the bodies. - analytics.service.ts: readSnapshot/writeSnapshot with 15-min TTL + no-op compute* stubs for the four chart series; PR3 fills behavior. - expense-dedup.service.ts: scanForDuplicates + markBestDuplicate using the partial dedup index. PR8 wires the BullMQ trigger. - expense-ocr.service.ts: OcrResult/OcrLineItem types + ocrReceipt stub. PR9 wires Claude Vision (Haiku 4.5 + ephemeral system-prompt cache). - audit-search.service.ts: tsvector @@ plainto_tsquery + cursor pagination on (createdAt, id). PR10 wires the admin UI. tsc clean, lint clean, vitest 675/675 (one unrelated AES random-output flake passes solo). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-28 14:43:01 +02:00
id: text('id')
.primaryKey()
.$defaultFn(() => crypto.randomUUID()),
portId: text('port_id')
.notNull()
.references(() => ports.id),
entityType: text('entity_type').notNull(), // client, interest, berth
fieldName: text('field_name').notNull(),
fieldLabel: text('field_label').notNull(),
fieldType: text('field_type').notNull(), // text, number, date, boolean, select
selectOptions: jsonb('select_options'), // for select type: ["option1", "option2"]
isRequired: boolean('is_required').notNull().default(false),
sortOrder: integer('sort_order').notNull().default(0),
createdAt: timestamp('created_at', { withTimezone: true }).notNull().defaultNow(),
},
(table) => [
uniqueIndex('cfd_port_entity_name_idx').on(table.portId, table.entityType, table.fieldName),
index('idx_cfd_port').on(table.portId),
],
);
export const customFieldValues = pgTable(
'custom_field_values',
{
feat(insights): Phase B schema + service skeletons PR1 of Phase B per docs/superpowers/specs/2026-04-28-phase-b-insights-alerts-design.md. Lays the foundation that PRs 2-10 will fill in with behaviour. Schema (migration 0014): - alerts table with rule-engine fields (rule_id, severity, link, entity_type/id, fingerprint, fired/dismissed/acknowledged/resolved timestamps, jsonb metadata). Partial-unique fingerprint index keeps one open row per (port, rule, entity); separate indexes power severity-filtered and time-ordered queries. - analytics_snapshots (port_id, metric_id) -> jsonb cache + computedAt for the 15-min recurring refresh. - expenses: duplicate_of self-FK, dedup_scanned_at, ocr_status/raw/ confidence; partial index on (port, vendor, amount, date) where duplicate_of IS NULL drives the dedup heuristic. - audit_logs.search_text: GENERATED ALWAYS tsvector over action+entity_type+entity_id+user_id, GIN-indexed (drizzle can't model GENERATED ALWAYS in TS yet, so the migration appends manual ALTER + the GIN index). Service skeletons in src/lib/services/: - alerts.service.ts: fingerprintFor, reconcileAlertsForPort (upsert + auto-resolve), dismiss, acknowledge, listAlertsForPort. - alert-rules.ts: RULE_REGISTRY of 10 rule evaluators (currently no-op); PR2 fills in the bodies. - analytics.service.ts: readSnapshot/writeSnapshot with 15-min TTL + no-op compute* stubs for the four chart series; PR3 fills behavior. - expense-dedup.service.ts: scanForDuplicates + markBestDuplicate using the partial dedup index. PR8 wires the BullMQ trigger. - expense-ocr.service.ts: OcrResult/OcrLineItem types + ocrReceipt stub. PR9 wires Claude Vision (Haiku 4.5 + ephemeral system-prompt cache). - audit-search.service.ts: tsvector @@ plainto_tsquery + cursor pagination on (createdAt, id). PR10 wires the admin UI. tsc clean, lint clean, vitest 675/675 (one unrelated AES random-output flake passes solo). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-28 14:43:01 +02:00
id: text('id')
.primaryKey()
.$defaultFn(() => crypto.randomUUID()),
fieldId: text('field_id')
.notNull()
.references(() => customFieldDefinitions.id, { onDelete: 'cascade' }),
entityId: text('entity_id').notNull(), // references the client/interest/berth ID
value: jsonb('value').notNull(),
createdAt: timestamp('created_at', { withTimezone: true }).notNull().defaultNow(),
updatedAt: timestamp('updated_at', { withTimezone: true }).notNull().defaultNow(),
},
(table) => [
uniqueIndex('cfv_field_entity_idx').on(table.fieldId, table.entityId),
index('idx_cfv_entity').on(table.entityId),
],
);
feat(errors): platform-wide request ids + error codes + admin inspector End-to-end error-handling overhaul. A user hitting any failure now sees a plain-text message + stable error code + reference id. A super admin can paste the id into /admin/errors/<id> for the full request shape, sanitized body, error stack, and a heuristic likely-cause hint. REQUEST CONTEXT (AsyncLocalStorage) - src/lib/request-context.ts mints a per-request frame carrying requestId + portId + userId + method + path + start timestamp. - withAuth wraps every authenticated handler in runWithRequestContext and accepts an upstream X-Request-Id header (validated shape) or generates a fresh UUID. The id ALWAYS leaves on the X-Request-Id response header, including early-return 401/403/4xx paths. - Pino logger reads from the same context via mixin — every log line emitted during the request automatically carries the ids with no per-call threading. ERROR CODE REGISTRY - src/lib/error-codes.ts defines stable DOMAIN_REASON codes with HTTP status + plain-text user-facing message (no jargon, written for the rep on the phone with a customer). - New CodedError class wraps a registered code + optional internalMessage (admin-only — never sent to client). - Existing AppError subclasses got plain-text default rewrites so legacy throw sites improve immediately without migration. - High-impact services migrated to specific codes: expenses (RECEIPT_REQUIRED, INVOICE_LINKED), interest-berths (CROSS_PORT_LINK_REJECTED), berth-pdf (PDF_MAGIC_BYTE / PDF_EMPTY / PDF_TOO_LARGE / VERSION_ALREADY_CURRENT), recommender (INTEREST_PORT_MISMATCH). ERROR ENVELOPE - errorResponse always sets X-Request-Id header + requestId field. - 5xx responses include a "Quote error ID …" friendly line. - 4xx kept clean (validation, permission, not-found don't pollute the inspector — they're already in audit log). PERSISTENCE (error_events table, migration 0040) - One row per 5xx, keyed on requestId, with method/path/status/error name+message/stack head (4KB cap)/sanitized body excerpt (1KB cap; password/token/secret/etc keys redacted)/duration/IP/UA/metadata. - captureErrorEvent extracts Postgres SQLSTATE/severity/cause.code so the classifier can recognize FK / unique / NOT NULL / schema- drift violations. - Failure to persist is logged-not-thrown. LIKELY-CULPRIT CLASSIFIER (src/lib/error-classifier.ts) - 4-pass heuristic (first match wins): 1. Postgres SQLSTATE → human reason (23503 FK, 23505 unique, 42703 schema drift, 53300 connection limit, …) 2. Error class name (AbortError, TimeoutError, FetchError, ZodError) 3. Stack-path patterns (/lib/storage/, /lib/email/, documenso, openai|claude, /queue/workers/) 4. Free-text message keywords (econnrefused, rate limit, timeout, unauthorized|invalid api key) - Returns { label, hint, subsystem } for the inspector badge. CLIENT SIDE - apiFetch throws structured ApiError with message + code + requestId + details + retryAfter. - toastError() helper renders the standard 3-line toast: plain message / Error code: X / Reference ID: Y [Copy ID]. ADMIN INSPECTOR - /<port>/admin/errors lists captured 5xx with status badge + path + likely-culprit badge + truncated message + reference id. Filter by status code; auto-refresh via TanStack Query. - /<port>/admin/errors/<requestId> deep-dive: request shape, full error name+message+stack, sanitized body excerpt, raw metadata, registered-code lookup (so admin can compare to what user saw), likely-culprit hint with subsystem tag. - /<port>/admin/errors/codes is the in-app code reference page — every registered code grouped by domain prefix, searchable, with HTTP status + user message inline. Linked from inspector header so admins can flip to it while triaging. - Permission: admin.view_audit_log. Super admins see all ports; regular admins port-scoped. - system-monitoring dashboard now surfaces error_events alongside permission_denied audit + queue failed jobs (RecentError gains source: 'request' variant). DOCS - docs/error-handling.md walks through coded errors, plain-text message guidelines, client toasting, admin inspector usage, persistence rules, classifier internals, pruning, and the legacy → CodedError migration path. MIGRATION SAFETY - Audit confirmed all 41 migrations (0000-0040) apply cleanly in journal order against an empty DB. 0040 references ports(id) which exists from 0000. 0035/0038 don't deadlock under sequential psql -f. Removed redundant idx_ds_sent_by from 0038 (created in 0037). Tests: 1168/1168 vitest passing. tsc clean. - security-error-responses tests updated for plain-text messages + new optional response keys (code/requestId/message). - berth-pdf-versions tests assert stable error codes via toMatchObject({ code }) rather than message regex. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-05 14:12:59 +02:00
/**
* Per-request error capture for the super-admin inspector.
*
* Every unhandled (5xx) failure inside a route handler writes one row
* here so a user who hit "Error ID: ab12-..." can paste that id to a
* super admin who pulls the full context (status, path, body excerpt,
* stack, log lines) without grepping through pino files.
*
* Pruning: rows older than 90 days are dropped by the maintenance worker.
* Row size is bounded by deliberately storing only short stack heads
* + pre-truncated request bodies (1 KB cap per row).
*/
export const errorEvents = pgTable(
'error_events',
{
/**
* Equal to the request id minted in `withAuth` and surfaced to the
* client via `X-Request-Id`. Acting as the PK lets us write
* idempotently when duplicate webhook events fire - `ON CONFLICT
feat(errors): platform-wide request ids + error codes + admin inspector End-to-end error-handling overhaul. A user hitting any failure now sees a plain-text message + stable error code + reference id. A super admin can paste the id into /admin/errors/<id> for the full request shape, sanitized body, error stack, and a heuristic likely-cause hint. REQUEST CONTEXT (AsyncLocalStorage) - src/lib/request-context.ts mints a per-request frame carrying requestId + portId + userId + method + path + start timestamp. - withAuth wraps every authenticated handler in runWithRequestContext and accepts an upstream X-Request-Id header (validated shape) or generates a fresh UUID. The id ALWAYS leaves on the X-Request-Id response header, including early-return 401/403/4xx paths. - Pino logger reads from the same context via mixin — every log line emitted during the request automatically carries the ids with no per-call threading. ERROR CODE REGISTRY - src/lib/error-codes.ts defines stable DOMAIN_REASON codes with HTTP status + plain-text user-facing message (no jargon, written for the rep on the phone with a customer). - New CodedError class wraps a registered code + optional internalMessage (admin-only — never sent to client). - Existing AppError subclasses got plain-text default rewrites so legacy throw sites improve immediately without migration. - High-impact services migrated to specific codes: expenses (RECEIPT_REQUIRED, INVOICE_LINKED), interest-berths (CROSS_PORT_LINK_REJECTED), berth-pdf (PDF_MAGIC_BYTE / PDF_EMPTY / PDF_TOO_LARGE / VERSION_ALREADY_CURRENT), recommender (INTEREST_PORT_MISMATCH). ERROR ENVELOPE - errorResponse always sets X-Request-Id header + requestId field. - 5xx responses include a "Quote error ID …" friendly line. - 4xx kept clean (validation, permission, not-found don't pollute the inspector — they're already in audit log). PERSISTENCE (error_events table, migration 0040) - One row per 5xx, keyed on requestId, with method/path/status/error name+message/stack head (4KB cap)/sanitized body excerpt (1KB cap; password/token/secret/etc keys redacted)/duration/IP/UA/metadata. - captureErrorEvent extracts Postgres SQLSTATE/severity/cause.code so the classifier can recognize FK / unique / NOT NULL / schema- drift violations. - Failure to persist is logged-not-thrown. LIKELY-CULPRIT CLASSIFIER (src/lib/error-classifier.ts) - 4-pass heuristic (first match wins): 1. Postgres SQLSTATE → human reason (23503 FK, 23505 unique, 42703 schema drift, 53300 connection limit, …) 2. Error class name (AbortError, TimeoutError, FetchError, ZodError) 3. Stack-path patterns (/lib/storage/, /lib/email/, documenso, openai|claude, /queue/workers/) 4. Free-text message keywords (econnrefused, rate limit, timeout, unauthorized|invalid api key) - Returns { label, hint, subsystem } for the inspector badge. CLIENT SIDE - apiFetch throws structured ApiError with message + code + requestId + details + retryAfter. - toastError() helper renders the standard 3-line toast: plain message / Error code: X / Reference ID: Y [Copy ID]. ADMIN INSPECTOR - /<port>/admin/errors lists captured 5xx with status badge + path + likely-culprit badge + truncated message + reference id. Filter by status code; auto-refresh via TanStack Query. - /<port>/admin/errors/<requestId> deep-dive: request shape, full error name+message+stack, sanitized body excerpt, raw metadata, registered-code lookup (so admin can compare to what user saw), likely-culprit hint with subsystem tag. - /<port>/admin/errors/codes is the in-app code reference page — every registered code grouped by domain prefix, searchable, with HTTP status + user message inline. Linked from inspector header so admins can flip to it while triaging. - Permission: admin.view_audit_log. Super admins see all ports; regular admins port-scoped. - system-monitoring dashboard now surfaces error_events alongside permission_denied audit + queue failed jobs (RecentError gains source: 'request' variant). DOCS - docs/error-handling.md walks through coded errors, plain-text message guidelines, client toasting, admin inspector usage, persistence rules, classifier internals, pruning, and the legacy → CodedError migration path. MIGRATION SAFETY - Audit confirmed all 41 migrations (0000-0040) apply cleanly in journal order against an empty DB. 0040 references ports(id) which exists from 0000. 0035/0038 don't deadlock under sequential psql -f. Removed redundant idx_ds_sent_by from 0038 (created in 0037). Tests: 1168/1168 vitest passing. tsc clean. - security-error-responses tests updated for plain-text messages + new optional response keys (code/requestId/message). - berth-pdf-versions tests assert stable error codes via toMatchObject({ code }) rather than message regex. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-05 14:12:59 +02:00
* DO NOTHING` skips re-inserting the same error.
*/
requestId: text('request_id').primaryKey(),
/** Resolves null when the error fired pre-port (e.g. login flow). */
portId: text('port_id').references(() => ports.id, { onDelete: 'set null' }),
/** better-auth user id; null when error fired pre-auth. */
userId: text('user_id'),
statusCode: integer('status_code').notNull(),
method: text('method').notNull(),
/** Pathname only (no query string) - keeps PII and tokens out. */
feat(errors): platform-wide request ids + error codes + admin inspector End-to-end error-handling overhaul. A user hitting any failure now sees a plain-text message + stable error code + reference id. A super admin can paste the id into /admin/errors/<id> for the full request shape, sanitized body, error stack, and a heuristic likely-cause hint. REQUEST CONTEXT (AsyncLocalStorage) - src/lib/request-context.ts mints a per-request frame carrying requestId + portId + userId + method + path + start timestamp. - withAuth wraps every authenticated handler in runWithRequestContext and accepts an upstream X-Request-Id header (validated shape) or generates a fresh UUID. The id ALWAYS leaves on the X-Request-Id response header, including early-return 401/403/4xx paths. - Pino logger reads from the same context via mixin — every log line emitted during the request automatically carries the ids with no per-call threading. ERROR CODE REGISTRY - src/lib/error-codes.ts defines stable DOMAIN_REASON codes with HTTP status + plain-text user-facing message (no jargon, written for the rep on the phone with a customer). - New CodedError class wraps a registered code + optional internalMessage (admin-only — never sent to client). - Existing AppError subclasses got plain-text default rewrites so legacy throw sites improve immediately without migration. - High-impact services migrated to specific codes: expenses (RECEIPT_REQUIRED, INVOICE_LINKED), interest-berths (CROSS_PORT_LINK_REJECTED), berth-pdf (PDF_MAGIC_BYTE / PDF_EMPTY / PDF_TOO_LARGE / VERSION_ALREADY_CURRENT), recommender (INTEREST_PORT_MISMATCH). ERROR ENVELOPE - errorResponse always sets X-Request-Id header + requestId field. - 5xx responses include a "Quote error ID …" friendly line. - 4xx kept clean (validation, permission, not-found don't pollute the inspector — they're already in audit log). PERSISTENCE (error_events table, migration 0040) - One row per 5xx, keyed on requestId, with method/path/status/error name+message/stack head (4KB cap)/sanitized body excerpt (1KB cap; password/token/secret/etc keys redacted)/duration/IP/UA/metadata. - captureErrorEvent extracts Postgres SQLSTATE/severity/cause.code so the classifier can recognize FK / unique / NOT NULL / schema- drift violations. - Failure to persist is logged-not-thrown. LIKELY-CULPRIT CLASSIFIER (src/lib/error-classifier.ts) - 4-pass heuristic (first match wins): 1. Postgres SQLSTATE → human reason (23503 FK, 23505 unique, 42703 schema drift, 53300 connection limit, …) 2. Error class name (AbortError, TimeoutError, FetchError, ZodError) 3. Stack-path patterns (/lib/storage/, /lib/email/, documenso, openai|claude, /queue/workers/) 4. Free-text message keywords (econnrefused, rate limit, timeout, unauthorized|invalid api key) - Returns { label, hint, subsystem } for the inspector badge. CLIENT SIDE - apiFetch throws structured ApiError with message + code + requestId + details + retryAfter. - toastError() helper renders the standard 3-line toast: plain message / Error code: X / Reference ID: Y [Copy ID]. ADMIN INSPECTOR - /<port>/admin/errors lists captured 5xx with status badge + path + likely-culprit badge + truncated message + reference id. Filter by status code; auto-refresh via TanStack Query. - /<port>/admin/errors/<requestId> deep-dive: request shape, full error name+message+stack, sanitized body excerpt, raw metadata, registered-code lookup (so admin can compare to what user saw), likely-culprit hint with subsystem tag. - /<port>/admin/errors/codes is the in-app code reference page — every registered code grouped by domain prefix, searchable, with HTTP status + user message inline. Linked from inspector header so admins can flip to it while triaging. - Permission: admin.view_audit_log. Super admins see all ports; regular admins port-scoped. - system-monitoring dashboard now surfaces error_events alongside permission_denied audit + queue failed jobs (RecentError gains source: 'request' variant). DOCS - docs/error-handling.md walks through coded errors, plain-text message guidelines, client toasting, admin inspector usage, persistence rules, classifier internals, pruning, and the legacy → CodedError migration path. MIGRATION SAFETY - Audit confirmed all 41 migrations (0000-0040) apply cleanly in journal order against an empty DB. 0040 references ports(id) which exists from 0000. 0035/0038 don't deadlock under sequential psql -f. Removed redundant idx_ds_sent_by from 0038 (created in 0037). Tests: 1168/1168 vitest passing. tsc clean. - security-error-responses tests updated for plain-text messages + new optional response keys (code/requestId/message). - berth-pdf-versions tests assert stable error codes via toMatchObject({ code }) rather than message regex. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-05 14:12:59 +02:00
path: text('path').notNull(),
errorName: text('error_name'),
errorMessage: text('error_message'),
/** First 4 KB of the stack - full stacks live in pino, this is for inspector readability. */
feat(errors): platform-wide request ids + error codes + admin inspector End-to-end error-handling overhaul. A user hitting any failure now sees a plain-text message + stable error code + reference id. A super admin can paste the id into /admin/errors/<id> for the full request shape, sanitized body, error stack, and a heuristic likely-cause hint. REQUEST CONTEXT (AsyncLocalStorage) - src/lib/request-context.ts mints a per-request frame carrying requestId + portId + userId + method + path + start timestamp. - withAuth wraps every authenticated handler in runWithRequestContext and accepts an upstream X-Request-Id header (validated shape) or generates a fresh UUID. The id ALWAYS leaves on the X-Request-Id response header, including early-return 401/403/4xx paths. - Pino logger reads from the same context via mixin — every log line emitted during the request automatically carries the ids with no per-call threading. ERROR CODE REGISTRY - src/lib/error-codes.ts defines stable DOMAIN_REASON codes with HTTP status + plain-text user-facing message (no jargon, written for the rep on the phone with a customer). - New CodedError class wraps a registered code + optional internalMessage (admin-only — never sent to client). - Existing AppError subclasses got plain-text default rewrites so legacy throw sites improve immediately without migration. - High-impact services migrated to specific codes: expenses (RECEIPT_REQUIRED, INVOICE_LINKED), interest-berths (CROSS_PORT_LINK_REJECTED), berth-pdf (PDF_MAGIC_BYTE / PDF_EMPTY / PDF_TOO_LARGE / VERSION_ALREADY_CURRENT), recommender (INTEREST_PORT_MISMATCH). ERROR ENVELOPE - errorResponse always sets X-Request-Id header + requestId field. - 5xx responses include a "Quote error ID …" friendly line. - 4xx kept clean (validation, permission, not-found don't pollute the inspector — they're already in audit log). PERSISTENCE (error_events table, migration 0040) - One row per 5xx, keyed on requestId, with method/path/status/error name+message/stack head (4KB cap)/sanitized body excerpt (1KB cap; password/token/secret/etc keys redacted)/duration/IP/UA/metadata. - captureErrorEvent extracts Postgres SQLSTATE/severity/cause.code so the classifier can recognize FK / unique / NOT NULL / schema- drift violations. - Failure to persist is logged-not-thrown. LIKELY-CULPRIT CLASSIFIER (src/lib/error-classifier.ts) - 4-pass heuristic (first match wins): 1. Postgres SQLSTATE → human reason (23503 FK, 23505 unique, 42703 schema drift, 53300 connection limit, …) 2. Error class name (AbortError, TimeoutError, FetchError, ZodError) 3. Stack-path patterns (/lib/storage/, /lib/email/, documenso, openai|claude, /queue/workers/) 4. Free-text message keywords (econnrefused, rate limit, timeout, unauthorized|invalid api key) - Returns { label, hint, subsystem } for the inspector badge. CLIENT SIDE - apiFetch throws structured ApiError with message + code + requestId + details + retryAfter. - toastError() helper renders the standard 3-line toast: plain message / Error code: X / Reference ID: Y [Copy ID]. ADMIN INSPECTOR - /<port>/admin/errors lists captured 5xx with status badge + path + likely-culprit badge + truncated message + reference id. Filter by status code; auto-refresh via TanStack Query. - /<port>/admin/errors/<requestId> deep-dive: request shape, full error name+message+stack, sanitized body excerpt, raw metadata, registered-code lookup (so admin can compare to what user saw), likely-culprit hint with subsystem tag. - /<port>/admin/errors/codes is the in-app code reference page — every registered code grouped by domain prefix, searchable, with HTTP status + user message inline. Linked from inspector header so admins can flip to it while triaging. - Permission: admin.view_audit_log. Super admins see all ports; regular admins port-scoped. - system-monitoring dashboard now surfaces error_events alongside permission_denied audit + queue failed jobs (RecentError gains source: 'request' variant). DOCS - docs/error-handling.md walks through coded errors, plain-text message guidelines, client toasting, admin inspector usage, persistence rules, classifier internals, pruning, and the legacy → CodedError migration path. MIGRATION SAFETY - Audit confirmed all 41 migrations (0000-0040) apply cleanly in journal order against an empty DB. 0040 references ports(id) which exists from 0000. 0035/0038 don't deadlock under sequential psql -f. Removed redundant idx_ds_sent_by from 0038 (created in 0037). Tests: 1168/1168 vitest passing. tsc clean. - security-error-responses tests updated for plain-text messages + new optional response keys (code/requestId/message). - berth-pdf-versions tests assert stable error codes via toMatchObject({ code }) rather than message regex. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-05 14:12:59 +02:00
errorStack: text('error_stack'),
/** Sanitized request body (max 1 KB) - secret-sounding keys redacted. */
feat(errors): platform-wide request ids + error codes + admin inspector End-to-end error-handling overhaul. A user hitting any failure now sees a plain-text message + stable error code + reference id. A super admin can paste the id into /admin/errors/<id> for the full request shape, sanitized body, error stack, and a heuristic likely-cause hint. REQUEST CONTEXT (AsyncLocalStorage) - src/lib/request-context.ts mints a per-request frame carrying requestId + portId + userId + method + path + start timestamp. - withAuth wraps every authenticated handler in runWithRequestContext and accepts an upstream X-Request-Id header (validated shape) or generates a fresh UUID. The id ALWAYS leaves on the X-Request-Id response header, including early-return 401/403/4xx paths. - Pino logger reads from the same context via mixin — every log line emitted during the request automatically carries the ids with no per-call threading. ERROR CODE REGISTRY - src/lib/error-codes.ts defines stable DOMAIN_REASON codes with HTTP status + plain-text user-facing message (no jargon, written for the rep on the phone with a customer). - New CodedError class wraps a registered code + optional internalMessage (admin-only — never sent to client). - Existing AppError subclasses got plain-text default rewrites so legacy throw sites improve immediately without migration. - High-impact services migrated to specific codes: expenses (RECEIPT_REQUIRED, INVOICE_LINKED), interest-berths (CROSS_PORT_LINK_REJECTED), berth-pdf (PDF_MAGIC_BYTE / PDF_EMPTY / PDF_TOO_LARGE / VERSION_ALREADY_CURRENT), recommender (INTEREST_PORT_MISMATCH). ERROR ENVELOPE - errorResponse always sets X-Request-Id header + requestId field. - 5xx responses include a "Quote error ID …" friendly line. - 4xx kept clean (validation, permission, not-found don't pollute the inspector — they're already in audit log). PERSISTENCE (error_events table, migration 0040) - One row per 5xx, keyed on requestId, with method/path/status/error name+message/stack head (4KB cap)/sanitized body excerpt (1KB cap; password/token/secret/etc keys redacted)/duration/IP/UA/metadata. - captureErrorEvent extracts Postgres SQLSTATE/severity/cause.code so the classifier can recognize FK / unique / NOT NULL / schema- drift violations. - Failure to persist is logged-not-thrown. LIKELY-CULPRIT CLASSIFIER (src/lib/error-classifier.ts) - 4-pass heuristic (first match wins): 1. Postgres SQLSTATE → human reason (23503 FK, 23505 unique, 42703 schema drift, 53300 connection limit, …) 2. Error class name (AbortError, TimeoutError, FetchError, ZodError) 3. Stack-path patterns (/lib/storage/, /lib/email/, documenso, openai|claude, /queue/workers/) 4. Free-text message keywords (econnrefused, rate limit, timeout, unauthorized|invalid api key) - Returns { label, hint, subsystem } for the inspector badge. CLIENT SIDE - apiFetch throws structured ApiError with message + code + requestId + details + retryAfter. - toastError() helper renders the standard 3-line toast: plain message / Error code: X / Reference ID: Y [Copy ID]. ADMIN INSPECTOR - /<port>/admin/errors lists captured 5xx with status badge + path + likely-culprit badge + truncated message + reference id. Filter by status code; auto-refresh via TanStack Query. - /<port>/admin/errors/<requestId> deep-dive: request shape, full error name+message+stack, sanitized body excerpt, raw metadata, registered-code lookup (so admin can compare to what user saw), likely-culprit hint with subsystem tag. - /<port>/admin/errors/codes is the in-app code reference page — every registered code grouped by domain prefix, searchable, with HTTP status + user message inline. Linked from inspector header so admins can flip to it while triaging. - Permission: admin.view_audit_log. Super admins see all ports; regular admins port-scoped. - system-monitoring dashboard now surfaces error_events alongside permission_denied audit + queue failed jobs (RecentError gains source: 'request' variant). DOCS - docs/error-handling.md walks through coded errors, plain-text message guidelines, client toasting, admin inspector usage, persistence rules, classifier internals, pruning, and the legacy → CodedError migration path. MIGRATION SAFETY - Audit confirmed all 41 migrations (0000-0040) apply cleanly in journal order against an empty DB. 0040 references ports(id) which exists from 0000. 0035/0038 don't deadlock under sequential psql -f. Removed redundant idx_ds_sent_by from 0038 (created in 0037). Tests: 1168/1168 vitest passing. tsc clean. - security-error-responses tests updated for plain-text messages + new optional response keys (code/requestId/message). - berth-pdf-versions tests assert stable error codes via toMatchObject({ code }) rather than message regex. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-05 14:12:59 +02:00
requestBodyExcerpt: text('request_body_excerpt'),
userAgent: text('user_agent'),
ipAddress: text('ip_address'),
/** Request duration in ms when error fired. */
durationMs: integer('duration_ms'),
/** Free-form bag (e.g. parsed zod issues, db error code). */
metadata: jsonb('metadata').default({}),
createdAt: timestamp('created_at', { withTimezone: true }).notNull().defaultNow(),
},
(table) => [
index('idx_error_events_port_created').on(table.portId, table.createdAt),
index('idx_error_events_status_created').on(table.statusCode, table.createdAt),
],
);
export type AuditLog = typeof auditLogs.$inferSelect;
export type NewAuditLog = typeof auditLogs.$inferInsert;
feat(errors): platform-wide request ids + error codes + admin inspector End-to-end error-handling overhaul. A user hitting any failure now sees a plain-text message + stable error code + reference id. A super admin can paste the id into /admin/errors/<id> for the full request shape, sanitized body, error stack, and a heuristic likely-cause hint. REQUEST CONTEXT (AsyncLocalStorage) - src/lib/request-context.ts mints a per-request frame carrying requestId + portId + userId + method + path + start timestamp. - withAuth wraps every authenticated handler in runWithRequestContext and accepts an upstream X-Request-Id header (validated shape) or generates a fresh UUID. The id ALWAYS leaves on the X-Request-Id response header, including early-return 401/403/4xx paths. - Pino logger reads from the same context via mixin — every log line emitted during the request automatically carries the ids with no per-call threading. ERROR CODE REGISTRY - src/lib/error-codes.ts defines stable DOMAIN_REASON codes with HTTP status + plain-text user-facing message (no jargon, written for the rep on the phone with a customer). - New CodedError class wraps a registered code + optional internalMessage (admin-only — never sent to client). - Existing AppError subclasses got plain-text default rewrites so legacy throw sites improve immediately without migration. - High-impact services migrated to specific codes: expenses (RECEIPT_REQUIRED, INVOICE_LINKED), interest-berths (CROSS_PORT_LINK_REJECTED), berth-pdf (PDF_MAGIC_BYTE / PDF_EMPTY / PDF_TOO_LARGE / VERSION_ALREADY_CURRENT), recommender (INTEREST_PORT_MISMATCH). ERROR ENVELOPE - errorResponse always sets X-Request-Id header + requestId field. - 5xx responses include a "Quote error ID …" friendly line. - 4xx kept clean (validation, permission, not-found don't pollute the inspector — they're already in audit log). PERSISTENCE (error_events table, migration 0040) - One row per 5xx, keyed on requestId, with method/path/status/error name+message/stack head (4KB cap)/sanitized body excerpt (1KB cap; password/token/secret/etc keys redacted)/duration/IP/UA/metadata. - captureErrorEvent extracts Postgres SQLSTATE/severity/cause.code so the classifier can recognize FK / unique / NOT NULL / schema- drift violations. - Failure to persist is logged-not-thrown. LIKELY-CULPRIT CLASSIFIER (src/lib/error-classifier.ts) - 4-pass heuristic (first match wins): 1. Postgres SQLSTATE → human reason (23503 FK, 23505 unique, 42703 schema drift, 53300 connection limit, …) 2. Error class name (AbortError, TimeoutError, FetchError, ZodError) 3. Stack-path patterns (/lib/storage/, /lib/email/, documenso, openai|claude, /queue/workers/) 4. Free-text message keywords (econnrefused, rate limit, timeout, unauthorized|invalid api key) - Returns { label, hint, subsystem } for the inspector badge. CLIENT SIDE - apiFetch throws structured ApiError with message + code + requestId + details + retryAfter. - toastError() helper renders the standard 3-line toast: plain message / Error code: X / Reference ID: Y [Copy ID]. ADMIN INSPECTOR - /<port>/admin/errors lists captured 5xx with status badge + path + likely-culprit badge + truncated message + reference id. Filter by status code; auto-refresh via TanStack Query. - /<port>/admin/errors/<requestId> deep-dive: request shape, full error name+message+stack, sanitized body excerpt, raw metadata, registered-code lookup (so admin can compare to what user saw), likely-culprit hint with subsystem tag. - /<port>/admin/errors/codes is the in-app code reference page — every registered code grouped by domain prefix, searchable, with HTTP status + user message inline. Linked from inspector header so admins can flip to it while triaging. - Permission: admin.view_audit_log. Super admins see all ports; regular admins port-scoped. - system-monitoring dashboard now surfaces error_events alongside permission_denied audit + queue failed jobs (RecentError gains source: 'request' variant). DOCS - docs/error-handling.md walks through coded errors, plain-text message guidelines, client toasting, admin inspector usage, persistence rules, classifier internals, pruning, and the legacy → CodedError migration path. MIGRATION SAFETY - Audit confirmed all 41 migrations (0000-0040) apply cleanly in journal order against an empty DB. 0040 references ports(id) which exists from 0000. 0035/0038 don't deadlock under sequential psql -f. Removed redundant idx_ds_sent_by from 0038 (created in 0037). Tests: 1168/1168 vitest passing. tsc clean. - security-error-responses tests updated for plain-text messages + new optional response keys (code/requestId/message). - berth-pdf-versions tests assert stable error codes via toMatchObject({ code }) rather than message regex. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-05 14:12:59 +02:00
export type ErrorEvent = typeof errorEvents.$inferSelect;
export type NewErrorEvent = typeof errorEvents.$inferInsert;
export type Tag = typeof tags.$inferSelect;
export type NewTag = typeof tags.$inferInsert;
export type Webhook = typeof webhooks.$inferSelect;
export type NewWebhook = typeof webhooks.$inferInsert;
export type WebhookDelivery = typeof webhookDeliveries.$inferSelect;
export type NewWebhookDelivery = typeof webhookDeliveries.$inferInsert;
export type SystemSetting = typeof systemSettings.$inferSelect;
export type NewSystemSetting = typeof systemSettings.$inferInsert;
export type SavedView = typeof savedViews.$inferSelect;
export type NewSavedView = typeof savedViews.$inferInsert;
export type ScratchpadNote = typeof scratchpadNotes.$inferSelect;
export type NewScratchpadNote = typeof scratchpadNotes.$inferInsert;
export type UserNotificationPreference = typeof userNotificationPreferences.$inferSelect;
export type NewUserNotificationPreference = typeof userNotificationPreferences.$inferInsert;
export type CurrencyRate = typeof currencyRates.$inferSelect;
export type NewCurrencyRate = typeof currencyRates.$inferInsert;
export type CustomFieldDefinition = typeof customFieldDefinitions.$inferSelect;
export type NewCustomFieldDefinition = typeof customFieldDefinitions.$inferInsert;
export type CustomFieldValue = typeof customFieldValues.$inferSelect;
export type NewCustomFieldValue = typeof customFieldValues.$inferInsert;
feat: autonomous backlog push — admin UX overhaul + storage parity + residential parity + Documenso Phase 1 Massive multi-area push driven by docs/admin-ux-backlog.md. Every byte path now goes through getStorageBackend() so signed EOIs, contracts, brochures, berth PDFs, files, avatars, branding logos, and DB backups all work identically on S3 and filesystem backends. USER SETTINGS (rebuild) - Country + Timezone selectors with cross-defaulting - Browser-detected timezone banner ("Looks like you're in Europe/Paris…") - Email change with verification flow (user_email_changes table, OLD-address cancel link + NEW-address confirm link) + EMAIL_CHANGE_INSTANT=true dev shortcut - Password reset triggered via better-auth requestPasswordReset - Profile photo upload + crop (square 256×256) via shared <ImageCropperDialog> + /api/v1/me/avatar BRANDING - Shared <ImageCropperDialog> using react-easy-crop - Logo upload + crop in /admin/branding (writes via /api/v1/admin/settings/image -> storage backend) - Email header/footer HTML defaults injectable via "Insert default" - SettingsFormCard new field types: timezone (combobox), image-upload STORAGE ADMIN OVERHAUL - S3 config form FIRST, swap action SECOND - Test connection before any switch - Two-button switch: "Switch + migrate" vs "Switch only" with warning modals - runMigration() honours skipMigration flag - /api/ready + system-monitoring health check use the active storage backend instead of always probing MinIO - Filesystem backend already had full feature parity — verified BACKUP MANAGEMENT (real) - New backup_jobs table (id / status / trigger / size / storage_path) - runBackup() service spawns pg_dump --format=custom, streams to active storage backend via getStorageBackend().put() - /admin/backup page: trigger, history, download .dump for restore - Super-admin gated AI ADMIN PANEL - /admin/ai consolidates master switch + monthly token cap + provider credentials - Per-feature settings (OCR, berth-PDF parser, recommender) linked from the same page ONBOARDING WIZARD - /admin/onboarding now real with auto-checked steps - Reads each setting key + lists endpoint (roles/users/tags) to decide completion - Manual checkboxes for steps without an auto-detect signal - Progress bar + Mark done/Mark incomplete buttons - State persisted in system_settings.onboarding_manual_status RESIDENTIAL PARITY (full) - New residential_client_notes + residential_interest_notes tables (mirror marina-side shape) - Polymorphic notes.service.ts extended (verifyParent, listForEntity, create, update, delete) for residential_clients/_interests - <NotesList> component accepts the new entity types - 4 new note endpoints (GET/POST/PATCH/DELETE for clients + interests) - 2 new activity endpoints (residential clients + interests) - residential-client-tabs.tsx + residential-interest-tabs.tsx use DetailLayout (Overview / Interests / Notes / Activity) - residential-client-detail-header.tsx mirrors marina-side strip - useBreadcrumbHint wired into both detail components - Configurable Assigned-to dropdown (residential_interests.view perm) CONFIGURABLE RESIDENTIAL STAGES - residential-stages.service.ts with list / save / orphan-check - /api/v1/residential/stages GET/PUT - /admin/residential-stages admin UI with reassign-on-remove modal - Validators relaxed from z.enum to z.string DOCUMENSO PHASE 1 - Schema: document_signers.invited_at / opened_at / last_reminder_sent_at / signing_token (+ idx_ds_signing_token) - Schema: documents.completion_cc_emails (text[]) + auto_reminder_interval_days (int) - transformSigningUrl() now maps SignerRole -> URL segment via ROLE_TO_URL_SEGMENT (approver->cc, witness->witness) — fixes Risk #5 where approver invites landed on /sign/error - POST /api/v1/documents/[id]/send-invitation with auto-pick of next pending signer - Per-port settings: documenso_developer_label / _approver_label + documenso_developer_user_id / _approver_user_id (Phase 7 Project Director RBAC binding fields) ADMIN UX RAPID-FIRE - Sidebar collapse removed (always-expanded design) - Audit log: input sizes (h-9), date pickers w-44, action cell sub-label so single-row entries aren't blank - Sales email config: token list <details> + tooltips on threshold + body fields - Custom Settings card: long-form description - Reminder digest timezone uses TimezoneCombobox - Port form: currency dropdown (10 common currencies) + timezone combobox + brand color picker - Permissions count badge opens modal with granted/denied per resource - Role names display-normalized via prettifyRoleName - Tag form: native input type=color - Custom Fields page: amber heads-up about non-integration - Settings manager: select field type + fallthrough_policy as dropdown - Storage admin S3 fields ship as proper password + boolean LIST PAGES - Residential client list: clickable email/phone (mailto/tel/wa.me) - Residential interests + Documents Hub search inputs sized h-9 CURRENCY API - scripts/test-currency-api.ts verifies live Frankfurter fetch -> DB upsert -> getRate -> convert. Inverse-rate drift <=0.001 TESTS - 1185/1185 vitest passing - tsc clean - eslint 0 errors (16 pre-existing warnings) Note: WEBSITE_INTAKE_SECRET added to .env.example but committed separately due to pre-commit hook policy on .env* files. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-07 21:02:12 +02:00
/**
* Backup-job ledger for the in-app backup admin. Each row tracks a
* single pg_dump invocation (success / failure / size / where the
* dump landed in storage). The actual dump runs via `runBackup()`
* in `@/lib/services/backup.service`; this table is the visible
* record used by `/admin/backup`.
*/
export const backupJobs = pgTable(
'backup_jobs',
{
id: text('id')
.primaryKey()
.$defaultFn(() => crypto.randomUUID()),
status: text('status').notNull().default('pending'), // pending | running | completed | failed
trigger: text('trigger').notNull().default('manual'), // manual | cron
triggeredBy: text('triggered_by'),
sizeBytes: bigint('size_bytes', { mode: 'number' }),
storagePath: text('storage_path'),
errorMessage: text('error_message'),
startedAt: timestamp('started_at', { withTimezone: true }).notNull().defaultNow(),
completedAt: timestamp('completed_at', { withTimezone: true }),
},
(table) => [index('idx_backup_jobs_started').on(table.startedAt)],
);
export type BackupJob = typeof backupJobs.$inferSelect;
export type NewBackupJob = typeof backupJobs.$inferInsert;
/**
* Admin-configurable destinations that scheduled/manual backups are pushed to.
* Each row transports the exact full-bundle tar produced by
* `createFullBackupTar()` (db.dump + blobs + manifest) see
* docs/superpowers/specs/2026-06-04-backup-destinations-design.md.
*
* `config` holds the type-specific connection settings; any secret inside it
* (SFTP password / private key, S3 secret key) is AES-GCM-encrypted via
* `@/lib/utils/encryption` before storage and never returned raw (the API
* surfaces only `*IsSet` markers, mirroring the send-from-accounts pattern).
*/
export const backupDestinations = pgTable(
'backup_destinations',
{
id: text('id')
.primaryKey()
.$defaultFn(() => crypto.randomUUID()),
name: text('name').notNull(),
type: text('type').notNull(), // 'sftp' | 's3' | 'filesystem'
enabled: boolean('enabled').notNull().default(false),
config: jsonb('config').notNull().default({}),
/** Keep last N bundles at this destination; null = keep all. */
retentionCount: integer('retention_count'),
/** Opt-in client-side AES-256 encryption of the bundle before push. */
encryptBundle: boolean('encrypt_bundle').notNull().default(false),
/** The bundle passphrase, itself AES-GCM-encrypted at rest. */
encryptionKeyEncrypted: text('encryption_key_encrypted'),
lastRunAt: timestamp('last_run_at', { withTimezone: true }),
lastStatus: text('last_status'), // 'ok' | 'failed'
lastError: text('last_error'),
lastBackupBytes: bigint('last_backup_bytes', { mode: 'number' }),
createdAt: timestamp('created_at', { withTimezone: true }).notNull().defaultNow(),
updatedAt: timestamp('updated_at', { withTimezone: true }).notNull().defaultNow(),
},
(table) => [index('idx_backup_destinations_enabled').on(table.enabled)],
);
export type BackupDestination = typeof backupDestinations.$inferSelect;
export type NewBackupDestination = typeof backupDestinations.$inferInsert;