Files
pn-new-crm/src/lib/services/expenses.ts

373 lines
11 KiB
TypeScript
Raw Normal View History

feat(expenses+interests): trip/event grouping (lightweight) Per the trips/events design discussion: instead of building a full events domain (table + CRUD UI + calendar) for the 6–12 yacht shows a year, ship the cheap version that covers the actual asks. Expenses — `tripLabel` free-text: - New `expenses.trip_label` text column (migration 0039) + index for filter / autocomplete lookup. - Validator: createExpenseShape + listExpensesSchema + exportExpensePdfSchema.filter all accept tripLabel. - Service: createExpense + updateExpense persist; listExpenses filters; new `listTripLabels(portId, search?)` returns distinct values ordered by most-recent expenseDate so the autocomplete surfaces recently-used labels first. - New `GET /api/v1/expenses/trip-labels` endpoint (gated by expenses.view) backs the autocomplete. - Form dialog: native `<datalist>` powered by the autocomplete query so reps don't end up with "Palm Beach 2026" / "palm-beach 2026" fragmented across two PDF sections. - Expense list: new "Trip" column (badge) + free-text filter. - Detail page: trip label rendered alongside Category / Payer. - PDF export: GroupBy gains 'trip'; filter.tripLabel narrows the export. Untagged rows fall under "(no trip)". - Trim/normalize on write so " Palm Beach 2026 " === "Palm Beach 2026". Interests — event tagging via existing tag system: - Reps can tag interests with an event tag (e.g. "Palm Beach 2026") via the existing InlineTagEditor on the detail page; tags are port-scoped and reusable. - Interest list now has a TagPicker filter rendered next to the FilterBar so reps can sort prospects by event attended ("show me every lead from Palm Beach"). Hidden 'relation'-typed FilterDefinition for tagIds wires URL round-trip + saved-views capture without rendering inside the FilterBar. - FilterBar deserializer now handles `relation` types as comma-joined arrays on URL load. Why a free-text trip label and not a trips table: - 6–12 events/year doesn't justify a domain. The CRUD UI cost would be most of the engineering, and reps already have the events on their personal calendars. - If usage proves demand for per-event ROI dashboards or richer attribution, promote to a real `trips` table later. Migration path: trip_label → tripId is a backfill+swap. Test status: 1168/1168 vitest. tsc clean. Migration 0039 applied in dev (also caught + fixed an unrelated audit-v3 follow-up: 0037 had `idx_br_interest` colliding with the existing `berth_recommendations.idx_br_interest`; renamed to `idx_brr_interest` / `idx_brr_contract_file`). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-05 13:46:54 +02:00
import { eq, and, gte, lte, sql, isNull, ilike, isNotNull, desc, ne } from 'drizzle-orm';
import type { PgColumn } from 'drizzle-orm/pg-core';
import { db } from '@/lib/db';
import { expenses, invoices, invoiceExpenses } from '@/lib/db/schema/financial';
import { buildListQuery } from '@/lib/db/query-builder';
fix(audit-wave-10): types-auditor fixes — Tx type, BerthDetailData, parseBody, toAuditJson Address the CRITICAL + high-leverage HIGH items from the types-auditor: **C1 — `tx: any` in client-restore.service** Export a canonical `Tx` type from `lib/db/utils.ts` (derived from Drizzle's `db.transaction` callback shape) and use it in `applyReversal` so the 12+ downstream tx writes get full inference. **C2 — berth-detail page stacked `useQuery<any>` escape hatches** Export `BerthDetailData` from berth-detail-header and consume it through useQuery + apiFetch. Removed three `any` escapes in the highest-traffic detail page. Also collapsed the duplicate `BerthData` in berth-tabs.tsx to import from berth-detail-header so the two types can't drift. **C3 — parseBody migration for portal/public routes** Replace raw `await req.json() + schema.parse(body)` with the project-standard `parseBody(req, schema)` helper across 7 routes: - portal/auth/{change-password, activate, reset-password} - auth/set-password - public/{interests, residential-inquiries} Skipped the three anti-enumeration routes (forgot-password, sign-in, sign-in-by-identifier) where the manual validation gives opaque errors on purpose. website-inquiries already wraps the parse in a custom 400 — left as-is. **HIGH #5 — `toAuditJson<T>` helper (21 → 0 inline casts)** Introduce `toAuditJson<T extends object>(row: T): Record<string, unknown>` in lib/audit.ts (mirrors gdpr-bundle-builder's `toJsonRow` that already exists for the same reason). Codemod 21 `<row> as unknown as Record<string, unknown>` sites across: - invoices.ts × 6 - expenses.ts × 6 - berths.service × 2 - documents.service × 2 - ocr-config.service × 2 - ai-budget.service × 2 - yachts.service, companies.service, company-memberships.service × 1 each document-templates' `payload as unknown as Record<...>` is a different shape (Documenso form-values widening, not an audit log) — kept the manual cast there. Tests stay 1315/1315. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-13 12:27:08 +02:00
import { createAuditLog, toAuditJson, type AuditMeta } from '@/lib/audit';
import { diffEntity } from '@/lib/entity-diff';
import { softDelete, restore } from '@/lib/db/utils';
feat(errors): platform-wide request ids + error codes + admin inspector End-to-end error-handling overhaul. A user hitting any failure now sees a plain-text message + stable error code + reference id. A super admin can paste the id into /admin/errors/<id> for the full request shape, sanitized body, error stack, and a heuristic likely-cause hint. REQUEST CONTEXT (AsyncLocalStorage) - src/lib/request-context.ts mints a per-request frame carrying requestId + portId + userId + method + path + start timestamp. - withAuth wraps every authenticated handler in runWithRequestContext and accepts an upstream X-Request-Id header (validated shape) or generates a fresh UUID. The id ALWAYS leaves on the X-Request-Id response header, including early-return 401/403/4xx paths. - Pino logger reads from the same context via mixin — every log line emitted during the request automatically carries the ids with no per-call threading. ERROR CODE REGISTRY - src/lib/error-codes.ts defines stable DOMAIN_REASON codes with HTTP status + plain-text user-facing message (no jargon, written for the rep on the phone with a customer). - New CodedError class wraps a registered code + optional internalMessage (admin-only — never sent to client). - Existing AppError subclasses got plain-text default rewrites so legacy throw sites improve immediately without migration. - High-impact services migrated to specific codes: expenses (RECEIPT_REQUIRED, INVOICE_LINKED), interest-berths (CROSS_PORT_LINK_REJECTED), berth-pdf (PDF_MAGIC_BYTE / PDF_EMPTY / PDF_TOO_LARGE / VERSION_ALREADY_CURRENT), recommender (INTEREST_PORT_MISMATCH). ERROR ENVELOPE - errorResponse always sets X-Request-Id header + requestId field. - 5xx responses include a "Quote error ID …" friendly line. - 4xx kept clean (validation, permission, not-found don't pollute the inspector — they're already in audit log). PERSISTENCE (error_events table, migration 0040) - One row per 5xx, keyed on requestId, with method/path/status/error name+message/stack head (4KB cap)/sanitized body excerpt (1KB cap; password/token/secret/etc keys redacted)/duration/IP/UA/metadata. - captureErrorEvent extracts Postgres SQLSTATE/severity/cause.code so the classifier can recognize FK / unique / NOT NULL / schema- drift violations. - Failure to persist is logged-not-thrown. LIKELY-CULPRIT CLASSIFIER (src/lib/error-classifier.ts) - 4-pass heuristic (first match wins): 1. Postgres SQLSTATE → human reason (23503 FK, 23505 unique, 42703 schema drift, 53300 connection limit, …) 2. Error class name (AbortError, TimeoutError, FetchError, ZodError) 3. Stack-path patterns (/lib/storage/, /lib/email/, documenso, openai|claude, /queue/workers/) 4. Free-text message keywords (econnrefused, rate limit, timeout, unauthorized|invalid api key) - Returns { label, hint, subsystem } for the inspector badge. CLIENT SIDE - apiFetch throws structured ApiError with message + code + requestId + details + retryAfter. - toastError() helper renders the standard 3-line toast: plain message / Error code: X / Reference ID: Y [Copy ID]. ADMIN INSPECTOR - /<port>/admin/errors lists captured 5xx with status badge + path + likely-culprit badge + truncated message + reference id. Filter by status code; auto-refresh via TanStack Query. - /<port>/admin/errors/<requestId> deep-dive: request shape, full error name+message+stack, sanitized body excerpt, raw metadata, registered-code lookup (so admin can compare to what user saw), likely-culprit hint with subsystem tag. - /<port>/admin/errors/codes is the in-app code reference page — every registered code grouped by domain prefix, searchable, with HTTP status + user message inline. Linked from inspector header so admins can flip to it while triaging. - Permission: admin.view_audit_log. Super admins see all ports; regular admins port-scoped. - system-monitoring dashboard now surfaces error_events alongside permission_denied audit + queue failed jobs (RecentError gains source: 'request' variant). DOCS - docs/error-handling.md walks through coded errors, plain-text message guidelines, client toasting, admin inspector usage, persistence rules, classifier internals, pruning, and the legacy → CodedError migration path. MIGRATION SAFETY - Audit confirmed all 41 migrations (0000-0040) apply cleanly in journal order against an empty DB. 0040 references ports(id) which exists from 0000. 0035/0038 don't deadlock under sequential psql -f. Removed redundant idx_ds_sent_by from 0038 (created in 0037). Tests: 1168/1168 vitest passing. tsc clean. - security-error-responses tests updated for plain-text messages + new optional response keys (code/requestId/message). - berth-pdf-versions tests assert stable error codes via toMatchObject({ code }) rather than message regex. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-05 14:12:59 +02:00
import { CodedError, NotFoundError } from '@/lib/errors';
import { emitToRoom } from '@/lib/socket/server';
import { convert } from '@/lib/services/currency';
import { logger } from '@/lib/logger';
feat(phase-b): ship analytics dashboard, alerts, scanner PWA, dedup, audit view Phase B (Insights & Alerts) PR4-11 in one drop. Builds on the schema + service skeletons committed in PRs 1-3. PR4 Analytics dashboard — 4 chart types (funnel/timeline/breakdown/source), date-range picker (today/7d/30d/90d), CSV+PNG export per card. PR5 Alert rail UI + /alerts page — topbar bell w/ live count, dashboard right-rail, three-tab page (active/dismissed/resolved), socket-driven invalidation. Bell lazy-loads list on popover open to keep cold pages fast in non-dashboard routes. PR6 EOI queue tab on documents hub — filters to in-flight EOIs, count surfaces in tab label. PR7 Interests-by-berth tab on berth detail — replaces the stub. PR8 Expense duplicate detection — BullMQ job runs scan on create, yellow banner on detail w/ Merge / Not-a-duplicate, transactional merge consolidates receipts and archives the source. PR9 Receipt scanner PWA + multi-provider AI — port-scoped /scan route in its own (scanner) group with no dashboard chrome, dynamic per-port manifest, OpenAI + Claude provider abstraction, admin OCR settings page (port-level + super-admin global default w/ opt-in fallback), test-connection endpoint, manual-entry fallback when no key is configured. Verify form always shown before save — no ghost rows. PR10 Audit log read view — swap to tsvector full-text search on the existing GIN index, cursor pagination, filters for entity/action/user /date range, batched actor-email resolution. PR11 Real-API tests — opt-in receipt-ocr.spec (admin save+test, optional real-receipt parse via REALAPI_RECEIPT_FIXTURE) and alert-engine socket-fanout spec gated behind RUN_ALERT_ENGINE_REALAPI. Both skip cleanly without their gate envs so CI stays green. Test totals: vitest 690 -> 713, smoke 130 -> 138, realapi +2 opt-in. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-28 17:21:55 +02:00
import type {
CreateExpenseInput,
UpdateExpenseInput,
ListExpensesInput,
} from '@/lib/validators/expenses';
export type { ListExpensesInput };
feat(expenses+interests): trip/event grouping (lightweight) Per the trips/events design discussion: instead of building a full events domain (table + CRUD UI + calendar) for the 6–12 yacht shows a year, ship the cheap version that covers the actual asks. Expenses — `tripLabel` free-text: - New `expenses.trip_label` text column (migration 0039) + index for filter / autocomplete lookup. - Validator: createExpenseShape + listExpensesSchema + exportExpensePdfSchema.filter all accept tripLabel. - Service: createExpense + updateExpense persist; listExpenses filters; new `listTripLabels(portId, search?)` returns distinct values ordered by most-recent expenseDate so the autocomplete surfaces recently-used labels first. - New `GET /api/v1/expenses/trip-labels` endpoint (gated by expenses.view) backs the autocomplete. - Form dialog: native `<datalist>` powered by the autocomplete query so reps don't end up with "Palm Beach 2026" / "palm-beach 2026" fragmented across two PDF sections. - Expense list: new "Trip" column (badge) + free-text filter. - Detail page: trip label rendered alongside Category / Payer. - PDF export: GroupBy gains 'trip'; filter.tripLabel narrows the export. Untagged rows fall under "(no trip)". - Trim/normalize on write so " Palm Beach 2026 " === "Palm Beach 2026". Interests — event tagging via existing tag system: - Reps can tag interests with an event tag (e.g. "Palm Beach 2026") via the existing InlineTagEditor on the detail page; tags are port-scoped and reusable. - Interest list now has a TagPicker filter rendered next to the FilterBar so reps can sort prospects by event attended ("show me every lead from Palm Beach"). Hidden 'relation'-typed FilterDefinition for tagIds wires URL round-trip + saved-views capture without rendering inside the FilterBar. - FilterBar deserializer now handles `relation` types as comma-joined arrays on URL load. Why a free-text trip label and not a trips table: - 6–12 events/year doesn't justify a domain. The CRUD UI cost would be most of the engineering, and reps already have the events on their personal calendars. - If usage proves demand for per-event ROI dashboards or richer attribution, promote to a real `trips` table later. Migration path: trip_label → tripId is a backfill+swap. Test status: 1168/1168 vitest. tsc clean. Migration 0039 applied in dev (also caught + fixed an unrelated audit-v3 follow-up: 0037 had `idx_br_interest` colliding with the existing `berth_recommendations.idx_br_interest`; renamed to `idx_brr_interest` / `idx_brr_contract_file`). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-05 13:46:54 +02:00
/**
* Trim + collapse whitespace on free-text trip labels so "Palm Beach 2026"
* and " Palm Beach 2026 " end up identical for the group-by + filter.
* Empty / whitespace-only collapses to null.
*/
function normalizeTripLabel(input: string | null | undefined): string | null {
if (input == null) return null;
const trimmed = input.replace(/\s+/g, ' ').trim();
return trimmed.length > 0 ? trimmed : null;
}
/**
* Distinct trip labels used in this port, ordered by most-recent first
* so the autocomplete surfaces "what reps actually used lately" rather
* than alphabetically. Powers the `tripLabel` <Combobox> on the expense
* form. Read-only no mutation hooks.
*/
export async function listTripLabels(portId: string, search?: string): Promise<string[]> {
const conditions = [
eq(expenses.portId, portId),
isNotNull(expenses.tripLabel),
ne(expenses.tripLabel, ''),
isNull(expenses.archivedAt),
];
if (search && search.trim().length > 0) {
conditions.push(ilike(expenses.tripLabel, `%${search.trim()}%`));
}
const rows = await db
.selectDistinct({
tripLabel: expenses.tripLabel,
latest: sql<Date>`MAX(${expenses.expenseDate})`.as('latest'),
})
.from(expenses)
.where(and(...conditions))
.groupBy(expenses.tripLabel)
.orderBy(desc(sql`MAX(${expenses.expenseDate})`))
.limit(50);
return rows.map((r) => r.tripLabel as string).filter((s): s is string => Boolean(s));
}
export async function listExpenses(portId: string, query: ListExpensesInput) {
const filters = [];
if (query.category) {
filters.push(eq(expenses.category, query.category));
}
if (query.paymentStatus) {
filters.push(eq(expenses.paymentStatus, query.paymentStatus));
}
if (query.currency) {
filters.push(eq(expenses.currency, query.currency));
}
if (query.payer) {
filters.push(eq(expenses.payer, query.payer));
}
feat(expenses+interests): trip/event grouping (lightweight) Per the trips/events design discussion: instead of building a full events domain (table + CRUD UI + calendar) for the 6–12 yacht shows a year, ship the cheap version that covers the actual asks. Expenses — `tripLabel` free-text: - New `expenses.trip_label` text column (migration 0039) + index for filter / autocomplete lookup. - Validator: createExpenseShape + listExpensesSchema + exportExpensePdfSchema.filter all accept tripLabel. - Service: createExpense + updateExpense persist; listExpenses filters; new `listTripLabels(portId, search?)` returns distinct values ordered by most-recent expenseDate so the autocomplete surfaces recently-used labels first. - New `GET /api/v1/expenses/trip-labels` endpoint (gated by expenses.view) backs the autocomplete. - Form dialog: native `<datalist>` powered by the autocomplete query so reps don't end up with "Palm Beach 2026" / "palm-beach 2026" fragmented across two PDF sections. - Expense list: new "Trip" column (badge) + free-text filter. - Detail page: trip label rendered alongside Category / Payer. - PDF export: GroupBy gains 'trip'; filter.tripLabel narrows the export. Untagged rows fall under "(no trip)". - Trim/normalize on write so " Palm Beach 2026 " === "Palm Beach 2026". Interests — event tagging via existing tag system: - Reps can tag interests with an event tag (e.g. "Palm Beach 2026") via the existing InlineTagEditor on the detail page; tags are port-scoped and reusable. - Interest list now has a TagPicker filter rendered next to the FilterBar so reps can sort prospects by event attended ("show me every lead from Palm Beach"). Hidden 'relation'-typed FilterDefinition for tagIds wires URL round-trip + saved-views capture without rendering inside the FilterBar. - FilterBar deserializer now handles `relation` types as comma-joined arrays on URL load. Why a free-text trip label and not a trips table: - 6–12 events/year doesn't justify a domain. The CRUD UI cost would be most of the engineering, and reps already have the events on their personal calendars. - If usage proves demand for per-event ROI dashboards or richer attribution, promote to a real `trips` table later. Migration path: trip_label → tripId is a backfill+swap. Test status: 1168/1168 vitest. tsc clean. Migration 0039 applied in dev (also caught + fixed an unrelated audit-v3 follow-up: 0037 had `idx_br_interest` colliding with the existing `berth_recommendations.idx_br_interest`; renamed to `idx_brr_interest` / `idx_brr_contract_file`). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-05 13:46:54 +02:00
if (query.tripLabel) {
filters.push(eq(expenses.tripLabel, query.tripLabel));
}
if (query.dateFrom) {
filters.push(gte(expenses.expenseDate, new Date(query.dateFrom)));
}
if (query.dateTo) {
filters.push(lte(expenses.expenseDate, new Date(query.dateTo)));
}
return buildListQuery({
table: expenses,
portIdColumn: expenses.portId,
portId,
idColumn: expenses.id,
updatedAtColumn: expenses.updatedAt,
filters,
page: query.page,
pageSize: query.limit,
searchColumns: [expenses.establishmentName, expenses.description],
searchTerm: query.search,
includeArchived: query.includeArchived,
archivedAtColumn: expenses.archivedAt,
sort: query.sort
feat(phase-b): ship analytics dashboard, alerts, scanner PWA, dedup, audit view Phase B (Insights & Alerts) PR4-11 in one drop. Builds on the schema + service skeletons committed in PRs 1-3. PR4 Analytics dashboard — 4 chart types (funnel/timeline/breakdown/source), date-range picker (today/7d/30d/90d), CSV+PNG export per card. PR5 Alert rail UI + /alerts page — topbar bell w/ live count, dashboard right-rail, three-tab page (active/dismissed/resolved), socket-driven invalidation. Bell lazy-loads list on popover open to keep cold pages fast in non-dashboard routes. PR6 EOI queue tab on documents hub — filters to in-flight EOIs, count surfaces in tab label. PR7 Interests-by-berth tab on berth detail — replaces the stub. PR8 Expense duplicate detection — BullMQ job runs scan on create, yellow banner on detail w/ Merge / Not-a-duplicate, transactional merge consolidates receipts and archives the source. PR9 Receipt scanner PWA + multi-provider AI — port-scoped /scan route in its own (scanner) group with no dashboard chrome, dynamic per-port manifest, OpenAI + Claude provider abstraction, admin OCR settings page (port-level + super-admin global default w/ opt-in fallback), test-connection endpoint, manual-entry fallback when no key is configured. Verify form always shown before save — no ghost rows. PR10 Audit log read view — swap to tsvector full-text search on the existing GIN index, cursor pagination, filters for entity/action/user /date range, batched actor-email resolution. PR11 Real-API tests — opt-in receipt-ocr.spec (admin save+test, optional real-receipt parse via REALAPI_RECEIPT_FIXTURE) and alert-engine socket-fanout spec gated behind RUN_ALERT_ENGINE_REALAPI. Both skip cleanly without their gate envs so CI stays green. Test totals: vitest 690 -> 713, smoke 130 -> 138, realapi +2 opt-in. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-28 17:21:55 +02:00
? {
column: expenses[query.sort as keyof typeof expenses] as unknown as PgColumn,
direction: query.order,
}
: undefined,
});
}
export async function getExpenseById(id: string, portId: string) {
const expense = await db.query.expenses.findFirst({
where: and(eq(expenses.id, id), eq(expenses.portId, portId)),
});
if (!expense) throw new NotFoundError('Expense');
return expense;
}
export async function createExpense(portId: string, data: CreateExpenseInput, meta: AuditMeta) {
let amountUsd: string | null = null;
let exchangeRate: string | null = null;
if (data.currency !== 'USD') {
const conversion = await convert(data.amount, data.currency, 'USD');
if (conversion) {
amountUsd = String(conversion.result);
exchangeRate = String(conversion.rate);
} else {
// BR-040: if rate unavailable, save without conversion + log warning
feat(phase-b): ship analytics dashboard, alerts, scanner PWA, dedup, audit view Phase B (Insights & Alerts) PR4-11 in one drop. Builds on the schema + service skeletons committed in PRs 1-3. PR4 Analytics dashboard — 4 chart types (funnel/timeline/breakdown/source), date-range picker (today/7d/30d/90d), CSV+PNG export per card. PR5 Alert rail UI + /alerts page — topbar bell w/ live count, dashboard right-rail, three-tab page (active/dismissed/resolved), socket-driven invalidation. Bell lazy-loads list on popover open to keep cold pages fast in non-dashboard routes. PR6 EOI queue tab on documents hub — filters to in-flight EOIs, count surfaces in tab label. PR7 Interests-by-berth tab on berth detail — replaces the stub. PR8 Expense duplicate detection — BullMQ job runs scan on create, yellow banner on detail w/ Merge / Not-a-duplicate, transactional merge consolidates receipts and archives the source. PR9 Receipt scanner PWA + multi-provider AI — port-scoped /scan route in its own (scanner) group with no dashboard chrome, dynamic per-port manifest, OpenAI + Claude provider abstraction, admin OCR settings page (port-level + super-admin global default w/ opt-in fallback), test-connection endpoint, manual-entry fallback when no key is configured. Verify form always shown before save — no ghost rows. PR10 Audit log read view — swap to tsvector full-text search on the existing GIN index, cursor pagination, filters for entity/action/user /date range, batched actor-email resolution. PR11 Real-API tests — opt-in receipt-ocr.spec (admin save+test, optional real-receipt parse via REALAPI_RECEIPT_FIXTURE) and alert-engine socket-fanout spec gated behind RUN_ALERT_ENGINE_REALAPI. Both skip cleanly without their gate envs so CI stays green. Test totals: vitest 690 -> 713, smoke 130 -> 138, realapi +2 opt-in. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-28 17:21:55 +02:00
logger.warn(
{ currency: data.currency },
'Currency rate unavailable, saving expense without USD conversion',
);
}
} else {
amountUsd = String(data.amount);
exchangeRate = '1';
}
const [expense] = await db
.insert(expenses)
.values({
portId,
establishmentName: data.establishmentName,
amount: String(data.amount),
currency: data.currency,
amountUsd,
exchangeRate,
paymentMethod: data.paymentMethod,
category: data.category,
payer: data.payer,
expenseDate: data.expenseDate,
description: data.description,
feat(expenses+interests): trip/event grouping (lightweight) Per the trips/events design discussion: instead of building a full events domain (table + CRUD UI + calendar) for the 6–12 yacht shows a year, ship the cheap version that covers the actual asks. Expenses — `tripLabel` free-text: - New `expenses.trip_label` text column (migration 0039) + index for filter / autocomplete lookup. - Validator: createExpenseShape + listExpensesSchema + exportExpensePdfSchema.filter all accept tripLabel. - Service: createExpense + updateExpense persist; listExpenses filters; new `listTripLabels(portId, search?)` returns distinct values ordered by most-recent expenseDate so the autocomplete surfaces recently-used labels first. - New `GET /api/v1/expenses/trip-labels` endpoint (gated by expenses.view) backs the autocomplete. - Form dialog: native `<datalist>` powered by the autocomplete query so reps don't end up with "Palm Beach 2026" / "palm-beach 2026" fragmented across two PDF sections. - Expense list: new "Trip" column (badge) + free-text filter. - Detail page: trip label rendered alongside Category / Payer. - PDF export: GroupBy gains 'trip'; filter.tripLabel narrows the export. Untagged rows fall under "(no trip)". - Trim/normalize on write so " Palm Beach 2026 " === "Palm Beach 2026". Interests — event tagging via existing tag system: - Reps can tag interests with an event tag (e.g. "Palm Beach 2026") via the existing InlineTagEditor on the detail page; tags are port-scoped and reusable. - Interest list now has a TagPicker filter rendered next to the FilterBar so reps can sort prospects by event attended ("show me every lead from Palm Beach"). Hidden 'relation'-typed FilterDefinition for tagIds wires URL round-trip + saved-views capture without rendering inside the FilterBar. - FilterBar deserializer now handles `relation` types as comma-joined arrays on URL load. Why a free-text trip label and not a trips table: - 6–12 events/year doesn't justify a domain. The CRUD UI cost would be most of the engineering, and reps already have the events on their personal calendars. - If usage proves demand for per-event ROI dashboards or richer attribution, promote to a real `trips` table later. Migration path: trip_label → tripId is a backfill+swap. Test status: 1168/1168 vitest. tsc clean. Migration 0039 applied in dev (also caught + fixed an unrelated audit-v3 follow-up: 0037 had `idx_br_interest` colliding with the existing `berth_recommendations.idx_br_interest`; renamed to `idx_brr_interest` / `idx_brr_contract_file`). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-05 13:46:54 +02:00
tripLabel: normalizeTripLabel(data.tripLabel),
receiptFileIds: data.receiptFileIds ?? [],
feat(expenses): streaming expense-PDF export + receipt-less expense flag + audit-3 fixes Replaces the legacy text-only expense PDF (was just dumping rows into a single pdfme text field — no images, no pagination) with a proper streaming export modelled on the legacy Nuxt client-portal but re-architected for memory safety. The legacy implementation OOM'd on hundreds of receipts because it: - buffered every receipt image into memory simultaneously - accumulated PDF chunks into an array, concat'd at end - base64-encoded the whole PDF into a JSON response (3x peak memory) - had no image downscaling The new design: - `streamExpensePdf()` (src/lib/services/expense-pdf.service.ts): pdfkit pipes bytes directly to the HTTP response (no Buffer accumulation). Receipts are processed serially so peak heap is one image at a time. Sharp downscales any receipt > 500 KB or > 1500 px to JPEG q80 — typical 8 MB phone photo collapses to ~250 KB. For a 500-receipt export, peak RSS stays under ~100 MB; legacy needed >2 GB for the same input. - Pages: cover summary box (count, totals, currency equiv, optional processing fee), grouped expense table (groupBy=none|payer|category| date), one-page-per-receipt with header (establishment, amount, date, payer, category, file name) and full-bleed image. - Storage backend abstraction — receipts stream from `getStorageBackend().get(storageKey)`, works on MinIO/S3/filesystem. - Route: POST /api/v1/expenses/export/pdf streams binary application/pdf with cache-control:no-store. Validator caps expenseIds at 1000 to prevent runaway loops. Receipt-less expense flow (per user request): - Schema: 0033 migration adds `expenses.no_receipt_acknowledged` boolean (default false). - Validator: createExpenseSchema requires either receiptFileIds OR noReceiptAcknowledged=true; the .refine() error message tells the rep exactly what to do. updateExpenseSchema is partial and skips the rule (existing rows can be edited without re-acknowledging). - PDF: receiptless expenses get an inline red "(no receipt)" tag in the establishment cell + a red footer warning in the summary box showing the count and at-risk amount. - The legacy parent-company reimbursement queue may refuse to pay receiptless expenses, so the warning is load-bearing for ops. Audit-3 fixes piggy-backed: - 🔴 Tesseract OCR runtime now races a 30s timeout (CPU-bomb DoS protection — a crafted PDF rasterizing to high-res noise could pin the worker indefinitely). - 🟠 brochures.service.ts:listBrochures dropped a wasted query (the legacy single-brochure fast-path was discarding its result on the multi-brochure branch). - 🟠 berth-pdf.service.ts:listBerthPdfVersions now Promise.all's the presignDownload calls instead of awaiting each in a for-loop — 20-version berths went from 20× round-trip to 1×. - 🟡 public berths route no longer logs the full `row` object on enum drift (was dumping price + amenity columns into ops logs). - 🟡 dropped the dead `void sql` import from public berths route. Tests still 1163/1163. tsc clean. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-05 04:38:32 +02:00
noReceiptAcknowledged: data.noReceiptAcknowledged ?? false,
paymentStatus: data.paymentStatus,
paymentDate: data.paymentDate ?? null,
paymentReference: data.paymentReference ?? null,
paymentNotes: data.paymentNotes ?? null,
createdBy: meta.userId,
})
.returning();
if (!expense)
throw new CodedError('INSERT_RETURNING_EMPTY', {
internalMessage: 'Expense insert returned no row',
});
void createAuditLog({
userId: meta.userId,
portId,
action: 'create',
entityType: 'expense',
entityId: expense.id,
fix(audit-wave-10): types-auditor fixes — Tx type, BerthDetailData, parseBody, toAuditJson Address the CRITICAL + high-leverage HIGH items from the types-auditor: **C1 — `tx: any` in client-restore.service** Export a canonical `Tx` type from `lib/db/utils.ts` (derived from Drizzle's `db.transaction` callback shape) and use it in `applyReversal` so the 12+ downstream tx writes get full inference. **C2 — berth-detail page stacked `useQuery<any>` escape hatches** Export `BerthDetailData` from berth-detail-header and consume it through useQuery + apiFetch. Removed three `any` escapes in the highest-traffic detail page. Also collapsed the duplicate `BerthData` in berth-tabs.tsx to import from berth-detail-header so the two types can't drift. **C3 — parseBody migration for portal/public routes** Replace raw `await req.json() + schema.parse(body)` with the project-standard `parseBody(req, schema)` helper across 7 routes: - portal/auth/{change-password, activate, reset-password} - auth/set-password - public/{interests, residential-inquiries} Skipped the three anti-enumeration routes (forgot-password, sign-in, sign-in-by-identifier) where the manual validation gives opaque errors on purpose. website-inquiries already wraps the parse in a custom 400 — left as-is. **HIGH #5 — `toAuditJson<T>` helper (21 → 0 inline casts)** Introduce `toAuditJson<T extends object>(row: T): Record<string, unknown>` in lib/audit.ts (mirrors gdpr-bundle-builder's `toJsonRow` that already exists for the same reason). Codemod 21 `<row> as unknown as Record<string, unknown>` sites across: - invoices.ts × 6 - expenses.ts × 6 - berths.service × 2 - documents.service × 2 - ocr-config.service × 2 - ai-budget.service × 2 - yachts.service, companies.service, company-memberships.service × 1 each document-templates' `payload as unknown as Record<...>` is a different shape (Documenso form-values widening, not an audit log) — kept the manual cast there. Tests stay 1315/1315. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-13 12:27:08 +02:00
newValue: toAuditJson(expense),
ipAddress: meta.ipAddress,
userAgent: meta.userAgent,
});
emitToRoom(`port:${portId}`, 'expense:created', {
expenseId: expense.id,
amount: Number(expense.amount),
currency: expense.currency,
category: expense.category ?? '',
});
// Schedule a duplicate-detection sweep. Best-effort - we don't want a
feat(phase-b): ship analytics dashboard, alerts, scanner PWA, dedup, audit view Phase B (Insights & Alerts) PR4-11 in one drop. Builds on the schema + service skeletons committed in PRs 1-3. PR4 Analytics dashboard — 4 chart types (funnel/timeline/breakdown/source), date-range picker (today/7d/30d/90d), CSV+PNG export per card. PR5 Alert rail UI + /alerts page — topbar bell w/ live count, dashboard right-rail, three-tab page (active/dismissed/resolved), socket-driven invalidation. Bell lazy-loads list on popover open to keep cold pages fast in non-dashboard routes. PR6 EOI queue tab on documents hub — filters to in-flight EOIs, count surfaces in tab label. PR7 Interests-by-berth tab on berth detail — replaces the stub. PR8 Expense duplicate detection — BullMQ job runs scan on create, yellow banner on detail w/ Merge / Not-a-duplicate, transactional merge consolidates receipts and archives the source. PR9 Receipt scanner PWA + multi-provider AI — port-scoped /scan route in its own (scanner) group with no dashboard chrome, dynamic per-port manifest, OpenAI + Claude provider abstraction, admin OCR settings page (port-level + super-admin global default w/ opt-in fallback), test-connection endpoint, manual-entry fallback when no key is configured. Verify form always shown before save — no ghost rows. PR10 Audit log read view — swap to tsvector full-text search on the existing GIN index, cursor pagination, filters for entity/action/user /date range, batched actor-email resolution. PR11 Real-API tests — opt-in receipt-ocr.spec (admin save+test, optional real-receipt parse via REALAPI_RECEIPT_FIXTURE) and alert-engine socket-fanout spec gated behind RUN_ALERT_ENGINE_REALAPI. Both skip cleanly without their gate envs so CI stays green. Test totals: vitest 690 -> 713, smoke 130 -> 138, realapi +2 opt-in. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-28 17:21:55 +02:00
// queue-side hiccup to fail the user's create.
try {
const { getQueue } = await import('@/lib/queue');
await getQueue('maintenance').add('expense-dedup-scan', { expenseId: expense.id });
} catch (err) {
logger.warn({ err, expenseId: expense.id }, 'Failed to enqueue expense-dedup-scan');
}
return expense;
}
export async function updateExpense(
id: string,
portId: string,
data: UpdateExpenseInput,
meta: AuditMeta,
) {
const existing = await getExpenseById(id, portId);
fix(audit-final): pre-merge hardening + expense receipt UI Final audit pass on feat/berth-recommender (3 parallel Opus agents) caught 5 critical and ~12 high-severity findings. All addressed in-branch; medium/low items deferred to docs/audit-final-deferred.md. Critical: - Add filesystem-backend PUT handler at /api/storage/[token] so presigned uploads stop 405-ing in filesystem mode (every browser-driven berth-PDF + brochure upload was broken). Same token-verify + replay protection as GET, plus magic-byte gate when c=application/pdf. - Forward req.signal into streamExpensePdf so an aborted 1000-receipt export no longer keeps grinding for minutes. - Strengthen Content-Disposition filename sanitization: \s matches CR/LF which would let documentName forge headers; restrict to [\w. -]+ and add filename* RFC 5987 fallback. - Lock public berths feed behind an explicit slug allowlist instead of ?portSlug= enumeration. - Reject cross-port interest_berths upserts (defense-in-depth on top of the recommender SQL port filter). High: - Recommender: width-only feasibility now caps length via L/W ratio so a 200ft berth doesn't surface for a 30ft beam request; total_interest_count filters out junction rows whose interest is in another port. - Mooring normalization follow-up migration (0034) catches un-hyphenated padded forms (A01) the original 0024 WHERE missed. - Send-out rate limit moved AFTER validation and scoped per-(port, user) so typos don't burn a slot and a multi-port rep can't be DoS'd by another tenant. - Default-brochure path now blocks an archived row from sneaking through the partial unique index. - NocoDB import --update-snapshot honoured under --dry-run so reps can refresh the seed JSON without committing DB writes. - PDF export: orderBy desc(expenseDate); apply isNull(archivedAt) when expenseIds are passed (was bypassed); flag rate-unavailable rows with an amber footer instead of silently treating them as 1:1; skip the USD->EUR chain when source already matches target. - expense-form-dialog: revokeObjectURL captures the URL in the closure instead of revoking the still-displayed one; reset upload state on close. - scan/page: handleClearReceipt resets in-flight scan/upload mutations; Save disabled while upload pending. - updateExpense re-asserts receipt-or-acknowledgement at the merged row so PATCH can't slip past the create-time refine. Plus the in-progress receipt upload UI for the expense form dialog (receipt picker + "I have no receipt" checkbox + warning banner) and a noReceiptAcknowledged flag on ExpenseRow for edit-mode hydration. Includes the canonical plan doc (referenced in CLAUDE.md), the handoff prompt, and a deferred-findings index for follow-up issues. 1163/1163 vitest passing. Typecheck clean. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-05 05:11:26 +02:00
// The create-time validator enforces "receipt OR no-receipt-ack" via
// `.refine`, but `updateExpenseSchema` is `.partial()` so the rule is
// dropped. Re-assert it here against the merged (existing + patch)
// shape so a PATCH can't slip an unexplained receipt-less expense
// past the create-time guard.
const mergedReceiptIds =
data.receiptFileIds !== undefined ? data.receiptFileIds : existing.receiptFileIds;
const mergedAck =
data.noReceiptAcknowledged !== undefined
? data.noReceiptAcknowledged
: existing.noReceiptAcknowledged;
const hasReceipts = Array.isArray(mergedReceiptIds) && mergedReceiptIds.length > 0;
if (!hasReceipts && !mergedAck) {
feat(errors): platform-wide request ids + error codes + admin inspector End-to-end error-handling overhaul. A user hitting any failure now sees a plain-text message + stable error code + reference id. A super admin can paste the id into /admin/errors/<id> for the full request shape, sanitized body, error stack, and a heuristic likely-cause hint. REQUEST CONTEXT (AsyncLocalStorage) - src/lib/request-context.ts mints a per-request frame carrying requestId + portId + userId + method + path + start timestamp. - withAuth wraps every authenticated handler in runWithRequestContext and accepts an upstream X-Request-Id header (validated shape) or generates a fresh UUID. The id ALWAYS leaves on the X-Request-Id response header, including early-return 401/403/4xx paths. - Pino logger reads from the same context via mixin — every log line emitted during the request automatically carries the ids with no per-call threading. ERROR CODE REGISTRY - src/lib/error-codes.ts defines stable DOMAIN_REASON codes with HTTP status + plain-text user-facing message (no jargon, written for the rep on the phone with a customer). - New CodedError class wraps a registered code + optional internalMessage (admin-only — never sent to client). - Existing AppError subclasses got plain-text default rewrites so legacy throw sites improve immediately without migration. - High-impact services migrated to specific codes: expenses (RECEIPT_REQUIRED, INVOICE_LINKED), interest-berths (CROSS_PORT_LINK_REJECTED), berth-pdf (PDF_MAGIC_BYTE / PDF_EMPTY / PDF_TOO_LARGE / VERSION_ALREADY_CURRENT), recommender (INTEREST_PORT_MISMATCH). ERROR ENVELOPE - errorResponse always sets X-Request-Id header + requestId field. - 5xx responses include a "Quote error ID …" friendly line. - 4xx kept clean (validation, permission, not-found don't pollute the inspector — they're already in audit log). PERSISTENCE (error_events table, migration 0040) - One row per 5xx, keyed on requestId, with method/path/status/error name+message/stack head (4KB cap)/sanitized body excerpt (1KB cap; password/token/secret/etc keys redacted)/duration/IP/UA/metadata. - captureErrorEvent extracts Postgres SQLSTATE/severity/cause.code so the classifier can recognize FK / unique / NOT NULL / schema- drift violations. - Failure to persist is logged-not-thrown. LIKELY-CULPRIT CLASSIFIER (src/lib/error-classifier.ts) - 4-pass heuristic (first match wins): 1. Postgres SQLSTATE → human reason (23503 FK, 23505 unique, 42703 schema drift, 53300 connection limit, …) 2. Error class name (AbortError, TimeoutError, FetchError, ZodError) 3. Stack-path patterns (/lib/storage/, /lib/email/, documenso, openai|claude, /queue/workers/) 4. Free-text message keywords (econnrefused, rate limit, timeout, unauthorized|invalid api key) - Returns { label, hint, subsystem } for the inspector badge. CLIENT SIDE - apiFetch throws structured ApiError with message + code + requestId + details + retryAfter. - toastError() helper renders the standard 3-line toast: plain message / Error code: X / Reference ID: Y [Copy ID]. ADMIN INSPECTOR - /<port>/admin/errors lists captured 5xx with status badge + path + likely-culprit badge + truncated message + reference id. Filter by status code; auto-refresh via TanStack Query. - /<port>/admin/errors/<requestId> deep-dive: request shape, full error name+message+stack, sanitized body excerpt, raw metadata, registered-code lookup (so admin can compare to what user saw), likely-culprit hint with subsystem tag. - /<port>/admin/errors/codes is the in-app code reference page — every registered code grouped by domain prefix, searchable, with HTTP status + user message inline. Linked from inspector header so admins can flip to it while triaging. - Permission: admin.view_audit_log. Super admins see all ports; regular admins port-scoped. - system-monitoring dashboard now surfaces error_events alongside permission_denied audit + queue failed jobs (RecentError gains source: 'request' variant). DOCS - docs/error-handling.md walks through coded errors, plain-text message guidelines, client toasting, admin inspector usage, persistence rules, classifier internals, pruning, and the legacy → CodedError migration path. MIGRATION SAFETY - Audit confirmed all 41 migrations (0000-0040) apply cleanly in journal order against an empty DB. 0040 references ports(id) which exists from 0000. 0035/0038 don't deadlock under sequential psql -f. Removed redundant idx_ds_sent_by from 0038 (created in 0037). Tests: 1168/1168 vitest passing. tsc clean. - security-error-responses tests updated for plain-text messages + new optional response keys (code/requestId/message). - berth-pdf-versions tests assert stable error codes via toMatchObject({ code }) rather than message regex. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-05 14:12:59 +02:00
throw new CodedError('EXPENSES_RECEIPT_REQUIRED');
fix(audit-final): pre-merge hardening + expense receipt UI Final audit pass on feat/berth-recommender (3 parallel Opus agents) caught 5 critical and ~12 high-severity findings. All addressed in-branch; medium/low items deferred to docs/audit-final-deferred.md. Critical: - Add filesystem-backend PUT handler at /api/storage/[token] so presigned uploads stop 405-ing in filesystem mode (every browser-driven berth-PDF + brochure upload was broken). Same token-verify + replay protection as GET, plus magic-byte gate when c=application/pdf. - Forward req.signal into streamExpensePdf so an aborted 1000-receipt export no longer keeps grinding for minutes. - Strengthen Content-Disposition filename sanitization: \s matches CR/LF which would let documentName forge headers; restrict to [\w. -]+ and add filename* RFC 5987 fallback. - Lock public berths feed behind an explicit slug allowlist instead of ?portSlug= enumeration. - Reject cross-port interest_berths upserts (defense-in-depth on top of the recommender SQL port filter). High: - Recommender: width-only feasibility now caps length via L/W ratio so a 200ft berth doesn't surface for a 30ft beam request; total_interest_count filters out junction rows whose interest is in another port. - Mooring normalization follow-up migration (0034) catches un-hyphenated padded forms (A01) the original 0024 WHERE missed. - Send-out rate limit moved AFTER validation and scoped per-(port, user) so typos don't burn a slot and a multi-port rep can't be DoS'd by another tenant. - Default-brochure path now blocks an archived row from sneaking through the partial unique index. - NocoDB import --update-snapshot honoured under --dry-run so reps can refresh the seed JSON without committing DB writes. - PDF export: orderBy desc(expenseDate); apply isNull(archivedAt) when expenseIds are passed (was bypassed); flag rate-unavailable rows with an amber footer instead of silently treating them as 1:1; skip the USD->EUR chain when source already matches target. - expense-form-dialog: revokeObjectURL captures the URL in the closure instead of revoking the still-displayed one; reset upload state on close. - scan/page: handleClearReceipt resets in-flight scan/upload mutations; Save disabled while upload pending. - updateExpense re-asserts receipt-or-acknowledgement at the merged row so PATCH can't slip past the create-time refine. Plus the in-progress receipt upload UI for the expense form dialog (receipt picker + "I have no receipt" checkbox + warning banner) and a noReceiptAcknowledged flag on ExpenseRow for edit-mode hydration. Includes the canonical plan doc (referenced in CLAUDE.md), the handoff prompt, and a deferred-findings index for follow-up issues. 1163/1163 vitest passing. Typecheck clean. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-05 05:11:26 +02:00
}
const updateData: Record<string, unknown> = { ...data, updatedAt: new Date() };
feat(expenses+interests): trip/event grouping (lightweight) Per the trips/events design discussion: instead of building a full events domain (table + CRUD UI + calendar) for the 6–12 yacht shows a year, ship the cheap version that covers the actual asks. Expenses — `tripLabel` free-text: - New `expenses.trip_label` text column (migration 0039) + index for filter / autocomplete lookup. - Validator: createExpenseShape + listExpensesSchema + exportExpensePdfSchema.filter all accept tripLabel. - Service: createExpense + updateExpense persist; listExpenses filters; new `listTripLabels(portId, search?)` returns distinct values ordered by most-recent expenseDate so the autocomplete surfaces recently-used labels first. - New `GET /api/v1/expenses/trip-labels` endpoint (gated by expenses.view) backs the autocomplete. - Form dialog: native `<datalist>` powered by the autocomplete query so reps don't end up with "Palm Beach 2026" / "palm-beach 2026" fragmented across two PDF sections. - Expense list: new "Trip" column (badge) + free-text filter. - Detail page: trip label rendered alongside Category / Payer. - PDF export: GroupBy gains 'trip'; filter.tripLabel narrows the export. Untagged rows fall under "(no trip)". - Trim/normalize on write so " Palm Beach 2026 " === "Palm Beach 2026". Interests — event tagging via existing tag system: - Reps can tag interests with an event tag (e.g. "Palm Beach 2026") via the existing InlineTagEditor on the detail page; tags are port-scoped and reusable. - Interest list now has a TagPicker filter rendered next to the FilterBar so reps can sort prospects by event attended ("show me every lead from Palm Beach"). Hidden 'relation'-typed FilterDefinition for tagIds wires URL round-trip + saved-views capture without rendering inside the FilterBar. - FilterBar deserializer now handles `relation` types as comma-joined arrays on URL load. Why a free-text trip label and not a trips table: - 6–12 events/year doesn't justify a domain. The CRUD UI cost would be most of the engineering, and reps already have the events on their personal calendars. - If usage proves demand for per-event ROI dashboards or richer attribution, promote to a real `trips` table later. Migration path: trip_label → tripId is a backfill+swap. Test status: 1168/1168 vitest. tsc clean. Migration 0039 applied in dev (also caught + fixed an unrelated audit-v3 follow-up: 0037 had `idx_br_interest` colliding with the existing `berth_recommendations.idx_br_interest`; renamed to `idx_brr_interest` / `idx_brr_contract_file`). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-05 13:46:54 +02:00
if (data.tripLabel !== undefined) {
updateData.tripLabel = normalizeTripLabel(data.tripLabel);
}
// Re-convert to USD if amount or currency changed
const newAmount = data.amount ?? Number(existing.amount);
const newCurrency = data.currency ?? existing.currency;
if (data.amount !== undefined || data.currency !== undefined) {
if (newCurrency !== 'USD') {
const conversion = await convert(newAmount, newCurrency, 'USD');
if (conversion) {
updateData.amountUsd = String(conversion.result);
updateData.exchangeRate = String(conversion.rate);
} else {
feat(phase-b): ship analytics dashboard, alerts, scanner PWA, dedup, audit view Phase B (Insights & Alerts) PR4-11 in one drop. Builds on the schema + service skeletons committed in PRs 1-3. PR4 Analytics dashboard — 4 chart types (funnel/timeline/breakdown/source), date-range picker (today/7d/30d/90d), CSV+PNG export per card. PR5 Alert rail UI + /alerts page — topbar bell w/ live count, dashboard right-rail, three-tab page (active/dismissed/resolved), socket-driven invalidation. Bell lazy-loads list on popover open to keep cold pages fast in non-dashboard routes. PR6 EOI queue tab on documents hub — filters to in-flight EOIs, count surfaces in tab label. PR7 Interests-by-berth tab on berth detail — replaces the stub. PR8 Expense duplicate detection — BullMQ job runs scan on create, yellow banner on detail w/ Merge / Not-a-duplicate, transactional merge consolidates receipts and archives the source. PR9 Receipt scanner PWA + multi-provider AI — port-scoped /scan route in its own (scanner) group with no dashboard chrome, dynamic per-port manifest, OpenAI + Claude provider abstraction, admin OCR settings page (port-level + super-admin global default w/ opt-in fallback), test-connection endpoint, manual-entry fallback when no key is configured. Verify form always shown before save — no ghost rows. PR10 Audit log read view — swap to tsvector full-text search on the existing GIN index, cursor pagination, filters for entity/action/user /date range, batched actor-email resolution. PR11 Real-API tests — opt-in receipt-ocr.spec (admin save+test, optional real-receipt parse via REALAPI_RECEIPT_FIXTURE) and alert-engine socket-fanout spec gated behind RUN_ALERT_ENGINE_REALAPI. Both skip cleanly without their gate envs so CI stays green. Test totals: vitest 690 -> 713, smoke 130 -> 138, realapi +2 opt-in. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-28 17:21:55 +02:00
logger.warn(
{ currency: newCurrency },
'Currency rate unavailable during update, clearing USD conversion',
);
updateData.amountUsd = null;
updateData.exchangeRate = null;
}
} else {
updateData.amountUsd = String(newAmount);
updateData.exchangeRate = '1';
}
}
if (data.amount !== undefined) updateData.amount = String(data.amount);
fix(audit-wave-10): types-auditor fixes — Tx type, BerthDetailData, parseBody, toAuditJson Address the CRITICAL + high-leverage HIGH items from the types-auditor: **C1 — `tx: any` in client-restore.service** Export a canonical `Tx` type from `lib/db/utils.ts` (derived from Drizzle's `db.transaction` callback shape) and use it in `applyReversal` so the 12+ downstream tx writes get full inference. **C2 — berth-detail page stacked `useQuery<any>` escape hatches** Export `BerthDetailData` from berth-detail-header and consume it through useQuery + apiFetch. Removed three `any` escapes in the highest-traffic detail page. Also collapsed the duplicate `BerthData` in berth-tabs.tsx to import from berth-detail-header so the two types can't drift. **C3 — parseBody migration for portal/public routes** Replace raw `await req.json() + schema.parse(body)` with the project-standard `parseBody(req, schema)` helper across 7 routes: - portal/auth/{change-password, activate, reset-password} - auth/set-password - public/{interests, residential-inquiries} Skipped the three anti-enumeration routes (forgot-password, sign-in, sign-in-by-identifier) where the manual validation gives opaque errors on purpose. website-inquiries already wraps the parse in a custom 400 — left as-is. **HIGH #5 — `toAuditJson<T>` helper (21 → 0 inline casts)** Introduce `toAuditJson<T extends object>(row: T): Record<string, unknown>` in lib/audit.ts (mirrors gdpr-bundle-builder's `toJsonRow` that already exists for the same reason). Codemod 21 `<row> as unknown as Record<string, unknown>` sites across: - invoices.ts × 6 - expenses.ts × 6 - berths.service × 2 - documents.service × 2 - ocr-config.service × 2 - ai-budget.service × 2 - yachts.service, companies.service, company-memberships.service × 1 each document-templates' `payload as unknown as Record<...>` is a different shape (Documenso form-values widening, not an audit log) — kept the manual cast there. Tests stay 1315/1315. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-13 12:27:08 +02:00
const { diff } = diffEntity(toAuditJson(existing), updateData);
const [updated] = await db
.update(expenses)
.set(updateData as Record<string, unknown>)
.where(and(eq(expenses.id, id), eq(expenses.portId, portId)))
.returning();
if (!updated) throw new NotFoundError('Expense');
void createAuditLog({
userId: meta.userId,
portId,
action: 'update',
entityType: 'expense',
entityId: id,
fix(audit-wave-10): types-auditor fixes — Tx type, BerthDetailData, parseBody, toAuditJson Address the CRITICAL + high-leverage HIGH items from the types-auditor: **C1 — `tx: any` in client-restore.service** Export a canonical `Tx` type from `lib/db/utils.ts` (derived from Drizzle's `db.transaction` callback shape) and use it in `applyReversal` so the 12+ downstream tx writes get full inference. **C2 — berth-detail page stacked `useQuery<any>` escape hatches** Export `BerthDetailData` from berth-detail-header and consume it through useQuery + apiFetch. Removed three `any` escapes in the highest-traffic detail page. Also collapsed the duplicate `BerthData` in berth-tabs.tsx to import from berth-detail-header so the two types can't drift. **C3 — parseBody migration for portal/public routes** Replace raw `await req.json() + schema.parse(body)` with the project-standard `parseBody(req, schema)` helper across 7 routes: - portal/auth/{change-password, activate, reset-password} - auth/set-password - public/{interests, residential-inquiries} Skipped the three anti-enumeration routes (forgot-password, sign-in, sign-in-by-identifier) where the manual validation gives opaque errors on purpose. website-inquiries already wraps the parse in a custom 400 — left as-is. **HIGH #5 — `toAuditJson<T>` helper (21 → 0 inline casts)** Introduce `toAuditJson<T extends object>(row: T): Record<string, unknown>` in lib/audit.ts (mirrors gdpr-bundle-builder's `toJsonRow` that already exists for the same reason). Codemod 21 `<row> as unknown as Record<string, unknown>` sites across: - invoices.ts × 6 - expenses.ts × 6 - berths.service × 2 - documents.service × 2 - ocr-config.service × 2 - ai-budget.service × 2 - yachts.service, companies.service, company-memberships.service × 1 each document-templates' `payload as unknown as Record<...>` is a different shape (Documenso form-values widening, not an audit log) — kept the manual cast there. Tests stay 1315/1315. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-13 12:27:08 +02:00
oldValue: toAuditJson(existing),
newValue: toAuditJson(updated),
metadata: { diff },
ipAddress: meta.ipAddress,
userAgent: meta.userAgent,
});
emitToRoom(`port:${portId}`, 'expense:updated', {
expenseId: id,
changedFields: Object.keys(diff),
});
return updated;
}
export async function archiveExpense(id: string, portId: string, meta: AuditMeta) {
const existing = await getExpenseById(id, portId);
// BR-045: Check if linked to non-draft invoice
const linkedInvoice = await db
.select({ invoiceId: invoiceExpenses.invoiceId })
.from(invoiceExpenses)
.innerJoin(invoices, eq(invoices.id, invoiceExpenses.invoiceId))
feat(phase-b): ship analytics dashboard, alerts, scanner PWA, dedup, audit view Phase B (Insights & Alerts) PR4-11 in one drop. Builds on the schema + service skeletons committed in PRs 1-3. PR4 Analytics dashboard — 4 chart types (funnel/timeline/breakdown/source), date-range picker (today/7d/30d/90d), CSV+PNG export per card. PR5 Alert rail UI + /alerts page — topbar bell w/ live count, dashboard right-rail, three-tab page (active/dismissed/resolved), socket-driven invalidation. Bell lazy-loads list on popover open to keep cold pages fast in non-dashboard routes. PR6 EOI queue tab on documents hub — filters to in-flight EOIs, count surfaces in tab label. PR7 Interests-by-berth tab on berth detail — replaces the stub. PR8 Expense duplicate detection — BullMQ job runs scan on create, yellow banner on detail w/ Merge / Not-a-duplicate, transactional merge consolidates receipts and archives the source. PR9 Receipt scanner PWA + multi-provider AI — port-scoped /scan route in its own (scanner) group with no dashboard chrome, dynamic per-port manifest, OpenAI + Claude provider abstraction, admin OCR settings page (port-level + super-admin global default w/ opt-in fallback), test-connection endpoint, manual-entry fallback when no key is configured. Verify form always shown before save — no ghost rows. PR10 Audit log read view — swap to tsvector full-text search on the existing GIN index, cursor pagination, filters for entity/action/user /date range, batched actor-email resolution. PR11 Real-API tests — opt-in receipt-ocr.spec (admin save+test, optional real-receipt parse via REALAPI_RECEIPT_FIXTURE) and alert-engine socket-fanout spec gated behind RUN_ALERT_ENGINE_REALAPI. Both skip cleanly without their gate envs so CI stays green. Test totals: vitest 690 -> 713, smoke 130 -> 138, realapi +2 opt-in. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-28 17:21:55 +02:00
.where(and(eq(invoiceExpenses.expenseId, id), sql`${invoices.status} != 'draft'`))
.limit(1);
if (linkedInvoice.length > 0) {
feat(errors): platform-wide request ids + error codes + admin inspector End-to-end error-handling overhaul. A user hitting any failure now sees a plain-text message + stable error code + reference id. A super admin can paste the id into /admin/errors/<id> for the full request shape, sanitized body, error stack, and a heuristic likely-cause hint. REQUEST CONTEXT (AsyncLocalStorage) - src/lib/request-context.ts mints a per-request frame carrying requestId + portId + userId + method + path + start timestamp. - withAuth wraps every authenticated handler in runWithRequestContext and accepts an upstream X-Request-Id header (validated shape) or generates a fresh UUID. The id ALWAYS leaves on the X-Request-Id response header, including early-return 401/403/4xx paths. - Pino logger reads from the same context via mixin — every log line emitted during the request automatically carries the ids with no per-call threading. ERROR CODE REGISTRY - src/lib/error-codes.ts defines stable DOMAIN_REASON codes with HTTP status + plain-text user-facing message (no jargon, written for the rep on the phone with a customer). - New CodedError class wraps a registered code + optional internalMessage (admin-only — never sent to client). - Existing AppError subclasses got plain-text default rewrites so legacy throw sites improve immediately without migration. - High-impact services migrated to specific codes: expenses (RECEIPT_REQUIRED, INVOICE_LINKED), interest-berths (CROSS_PORT_LINK_REJECTED), berth-pdf (PDF_MAGIC_BYTE / PDF_EMPTY / PDF_TOO_LARGE / VERSION_ALREADY_CURRENT), recommender (INTEREST_PORT_MISMATCH). ERROR ENVELOPE - errorResponse always sets X-Request-Id header + requestId field. - 5xx responses include a "Quote error ID …" friendly line. - 4xx kept clean (validation, permission, not-found don't pollute the inspector — they're already in audit log). PERSISTENCE (error_events table, migration 0040) - One row per 5xx, keyed on requestId, with method/path/status/error name+message/stack head (4KB cap)/sanitized body excerpt (1KB cap; password/token/secret/etc keys redacted)/duration/IP/UA/metadata. - captureErrorEvent extracts Postgres SQLSTATE/severity/cause.code so the classifier can recognize FK / unique / NOT NULL / schema- drift violations. - Failure to persist is logged-not-thrown. LIKELY-CULPRIT CLASSIFIER (src/lib/error-classifier.ts) - 4-pass heuristic (first match wins): 1. Postgres SQLSTATE → human reason (23503 FK, 23505 unique, 42703 schema drift, 53300 connection limit, …) 2. Error class name (AbortError, TimeoutError, FetchError, ZodError) 3. Stack-path patterns (/lib/storage/, /lib/email/, documenso, openai|claude, /queue/workers/) 4. Free-text message keywords (econnrefused, rate limit, timeout, unauthorized|invalid api key) - Returns { label, hint, subsystem } for the inspector badge. CLIENT SIDE - apiFetch throws structured ApiError with message + code + requestId + details + retryAfter. - toastError() helper renders the standard 3-line toast: plain message / Error code: X / Reference ID: Y [Copy ID]. ADMIN INSPECTOR - /<port>/admin/errors lists captured 5xx with status badge + path + likely-culprit badge + truncated message + reference id. Filter by status code; auto-refresh via TanStack Query. - /<port>/admin/errors/<requestId> deep-dive: request shape, full error name+message+stack, sanitized body excerpt, raw metadata, registered-code lookup (so admin can compare to what user saw), likely-culprit hint with subsystem tag. - /<port>/admin/errors/codes is the in-app code reference page — every registered code grouped by domain prefix, searchable, with HTTP status + user message inline. Linked from inspector header so admins can flip to it while triaging. - Permission: admin.view_audit_log. Super admins see all ports; regular admins port-scoped. - system-monitoring dashboard now surfaces error_events alongside permission_denied audit + queue failed jobs (RecentError gains source: 'request' variant). DOCS - docs/error-handling.md walks through coded errors, plain-text message guidelines, client toasting, admin inspector usage, persistence rules, classifier internals, pruning, and the legacy → CodedError migration path. MIGRATION SAFETY - Audit confirmed all 41 migrations (0000-0040) apply cleanly in journal order against an empty DB. 0040 references ports(id) which exists from 0000. 0035/0038 don't deadlock under sequential psql -f. Removed redundant idx_ds_sent_by from 0038 (created in 0037). Tests: 1168/1168 vitest passing. tsc clean. - security-error-responses tests updated for plain-text messages + new optional response keys (code/requestId/message). - berth-pdf-versions tests assert stable error codes via toMatchObject({ code }) rather than message regex. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-05 14:12:59 +02:00
throw new CodedError('EXPENSES_INVOICE_LINKED');
}
await softDelete(expenses, expenses.id, id);
void createAuditLog({
userId: meta.userId,
portId,
action: 'archive',
entityType: 'expense',
entityId: id,
fix(audit-wave-10): types-auditor fixes — Tx type, BerthDetailData, parseBody, toAuditJson Address the CRITICAL + high-leverage HIGH items from the types-auditor: **C1 — `tx: any` in client-restore.service** Export a canonical `Tx` type from `lib/db/utils.ts` (derived from Drizzle's `db.transaction` callback shape) and use it in `applyReversal` so the 12+ downstream tx writes get full inference. **C2 — berth-detail page stacked `useQuery<any>` escape hatches** Export `BerthDetailData` from berth-detail-header and consume it through useQuery + apiFetch. Removed three `any` escapes in the highest-traffic detail page. Also collapsed the duplicate `BerthData` in berth-tabs.tsx to import from berth-detail-header so the two types can't drift. **C3 — parseBody migration for portal/public routes** Replace raw `await req.json() + schema.parse(body)` with the project-standard `parseBody(req, schema)` helper across 7 routes: - portal/auth/{change-password, activate, reset-password} - auth/set-password - public/{interests, residential-inquiries} Skipped the three anti-enumeration routes (forgot-password, sign-in, sign-in-by-identifier) where the manual validation gives opaque errors on purpose. website-inquiries already wraps the parse in a custom 400 — left as-is. **HIGH #5 — `toAuditJson<T>` helper (21 → 0 inline casts)** Introduce `toAuditJson<T extends object>(row: T): Record<string, unknown>` in lib/audit.ts (mirrors gdpr-bundle-builder's `toJsonRow` that already exists for the same reason). Codemod 21 `<row> as unknown as Record<string, unknown>` sites across: - invoices.ts × 6 - expenses.ts × 6 - berths.service × 2 - documents.service × 2 - ocr-config.service × 2 - ai-budget.service × 2 - yachts.service, companies.service, company-memberships.service × 1 each document-templates' `payload as unknown as Record<...>` is a different shape (Documenso form-values widening, not an audit log) — kept the manual cast there. Tests stay 1315/1315. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-13 12:27:08 +02:00
oldValue: toAuditJson(existing),
ipAddress: meta.ipAddress,
userAgent: meta.userAgent,
});
emitToRoom(`port:${portId}`, 'expense:archived', { expenseId: id });
}
export async function restoreExpense(id: string, portId: string, meta: AuditMeta) {
await getExpenseById(id, portId);
await restore(expenses, expenses.id, id);
const restored = await getExpenseById(id, portId);
void createAuditLog({
userId: meta.userId,
portId,
action: 'restore',
entityType: 'expense',
entityId: id,
fix(audit-wave-10): types-auditor fixes — Tx type, BerthDetailData, parseBody, toAuditJson Address the CRITICAL + high-leverage HIGH items from the types-auditor: **C1 — `tx: any` in client-restore.service** Export a canonical `Tx` type from `lib/db/utils.ts` (derived from Drizzle's `db.transaction` callback shape) and use it in `applyReversal` so the 12+ downstream tx writes get full inference. **C2 — berth-detail page stacked `useQuery<any>` escape hatches** Export `BerthDetailData` from berth-detail-header and consume it through useQuery + apiFetch. Removed three `any` escapes in the highest-traffic detail page. Also collapsed the duplicate `BerthData` in berth-tabs.tsx to import from berth-detail-header so the two types can't drift. **C3 — parseBody migration for portal/public routes** Replace raw `await req.json() + schema.parse(body)` with the project-standard `parseBody(req, schema)` helper across 7 routes: - portal/auth/{change-password, activate, reset-password} - auth/set-password - public/{interests, residential-inquiries} Skipped the three anti-enumeration routes (forgot-password, sign-in, sign-in-by-identifier) where the manual validation gives opaque errors on purpose. website-inquiries already wraps the parse in a custom 400 — left as-is. **HIGH #5 — `toAuditJson<T>` helper (21 → 0 inline casts)** Introduce `toAuditJson<T extends object>(row: T): Record<string, unknown>` in lib/audit.ts (mirrors gdpr-bundle-builder's `toJsonRow` that already exists for the same reason). Codemod 21 `<row> as unknown as Record<string, unknown>` sites across: - invoices.ts × 6 - expenses.ts × 6 - berths.service × 2 - documents.service × 2 - ocr-config.service × 2 - ai-budget.service × 2 - yachts.service, companies.service, company-memberships.service × 1 each document-templates' `payload as unknown as Record<...>` is a different shape (Documenso form-values widening, not an audit log) — kept the manual cast there. Tests stay 1315/1315. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-13 12:27:08 +02:00
newValue: toAuditJson(restored),
ipAddress: meta.ipAddress,
userAgent: meta.userAgent,
});
emitToRoom(`port:${portId}`, 'expense:updated', {
expenseId: id,
changedFields: ['archivedAt'],
});
return restored;
}
export async function addReceiptFiles(
id: string,
portId: string,
fileIds: string[],
meta: AuditMeta,
) {
await getExpenseById(id, portId);
const [updated] = await db
.update(expenses)
.set({
receiptFileIds: sql`array_cat(receipt_file_ids, ${fileIds}::text[])`,
updatedAt: new Date(),
} as Record<string, unknown>)
.where(and(eq(expenses.id, id), eq(expenses.portId, portId)))
.returning();
if (!updated) throw new NotFoundError('Expense');
void createAuditLog({
userId: meta.userId,
portId,
action: 'update',
entityType: 'expense',
entityId: id,
metadata: { addedFileIds: fileIds },
ipAddress: meta.ipAddress,
userAgent: meta.userAgent,
});
return updated;
}