Files
pn-new-crm/src/lib/logger.ts

75 lines
2.3 KiB
TypeScript
Raw Normal View History

import pino from 'pino';
feat(errors): platform-wide request ids + error codes + admin inspector End-to-end error-handling overhaul. A user hitting any failure now sees a plain-text message + stable error code + reference id. A super admin can paste the id into /admin/errors/<id> for the full request shape, sanitized body, error stack, and a heuristic likely-cause hint. REQUEST CONTEXT (AsyncLocalStorage) - src/lib/request-context.ts mints a per-request frame carrying requestId + portId + userId + method + path + start timestamp. - withAuth wraps every authenticated handler in runWithRequestContext and accepts an upstream X-Request-Id header (validated shape) or generates a fresh UUID. The id ALWAYS leaves on the X-Request-Id response header, including early-return 401/403/4xx paths. - Pino logger reads from the same context via mixin — every log line emitted during the request automatically carries the ids with no per-call threading. ERROR CODE REGISTRY - src/lib/error-codes.ts defines stable DOMAIN_REASON codes with HTTP status + plain-text user-facing message (no jargon, written for the rep on the phone with a customer). - New CodedError class wraps a registered code + optional internalMessage (admin-only — never sent to client). - Existing AppError subclasses got plain-text default rewrites so legacy throw sites improve immediately without migration. - High-impact services migrated to specific codes: expenses (RECEIPT_REQUIRED, INVOICE_LINKED), interest-berths (CROSS_PORT_LINK_REJECTED), berth-pdf (PDF_MAGIC_BYTE / PDF_EMPTY / PDF_TOO_LARGE / VERSION_ALREADY_CURRENT), recommender (INTEREST_PORT_MISMATCH). ERROR ENVELOPE - errorResponse always sets X-Request-Id header + requestId field. - 5xx responses include a "Quote error ID …" friendly line. - 4xx kept clean (validation, permission, not-found don't pollute the inspector — they're already in audit log). PERSISTENCE (error_events table, migration 0040) - One row per 5xx, keyed on requestId, with method/path/status/error name+message/stack head (4KB cap)/sanitized body excerpt (1KB cap; password/token/secret/etc keys redacted)/duration/IP/UA/metadata. - captureErrorEvent extracts Postgres SQLSTATE/severity/cause.code so the classifier can recognize FK / unique / NOT NULL / schema- drift violations. - Failure to persist is logged-not-thrown. LIKELY-CULPRIT CLASSIFIER (src/lib/error-classifier.ts) - 4-pass heuristic (first match wins): 1. Postgres SQLSTATE → human reason (23503 FK, 23505 unique, 42703 schema drift, 53300 connection limit, …) 2. Error class name (AbortError, TimeoutError, FetchError, ZodError) 3. Stack-path patterns (/lib/storage/, /lib/email/, documenso, openai|claude, /queue/workers/) 4. Free-text message keywords (econnrefused, rate limit, timeout, unauthorized|invalid api key) - Returns { label, hint, subsystem } for the inspector badge. CLIENT SIDE - apiFetch throws structured ApiError with message + code + requestId + details + retryAfter. - toastError() helper renders the standard 3-line toast: plain message / Error code: X / Reference ID: Y [Copy ID]. ADMIN INSPECTOR - /<port>/admin/errors lists captured 5xx with status badge + path + likely-culprit badge + truncated message + reference id. Filter by status code; auto-refresh via TanStack Query. - /<port>/admin/errors/<requestId> deep-dive: request shape, full error name+message+stack, sanitized body excerpt, raw metadata, registered-code lookup (so admin can compare to what user saw), likely-culprit hint with subsystem tag. - /<port>/admin/errors/codes is the in-app code reference page — every registered code grouped by domain prefix, searchable, with HTTP status + user message inline. Linked from inspector header so admins can flip to it while triaging. - Permission: admin.view_audit_log. Super admins see all ports; regular admins port-scoped. - system-monitoring dashboard now surfaces error_events alongside permission_denied audit + queue failed jobs (RecentError gains source: 'request' variant). DOCS - docs/error-handling.md walks through coded errors, plain-text message guidelines, client toasting, admin inspector usage, persistence rules, classifier internals, pruning, and the legacy → CodedError migration path. MIGRATION SAFETY - Audit confirmed all 41 migrations (0000-0040) apply cleanly in journal order against an empty DB. 0040 references ports(id) which exists from 0000. 0035/0038 don't deadlock under sequential psql -f. Removed redundant idx_ds_sent_by from 0038 (created in 0037). Tests: 1168/1168 vitest passing. tsc clean. - security-error-responses tests updated for plain-text messages + new optional response keys (code/requestId/message). - berth-pdf-versions tests assert stable error codes via toMatchObject({ code }) rather than message regex. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-05 14:12:59 +02:00
import { getRequestContext } from '@/lib/request-context';
export const logger = pino({
level: process.env.LOG_LEVEL ?? 'info',
feat(errors): platform-wide request ids + error codes + admin inspector End-to-end error-handling overhaul. A user hitting any failure now sees a plain-text message + stable error code + reference id. A super admin can paste the id into /admin/errors/<id> for the full request shape, sanitized body, error stack, and a heuristic likely-cause hint. REQUEST CONTEXT (AsyncLocalStorage) - src/lib/request-context.ts mints a per-request frame carrying requestId + portId + userId + method + path + start timestamp. - withAuth wraps every authenticated handler in runWithRequestContext and accepts an upstream X-Request-Id header (validated shape) or generates a fresh UUID. The id ALWAYS leaves on the X-Request-Id response header, including early-return 401/403/4xx paths. - Pino logger reads from the same context via mixin — every log line emitted during the request automatically carries the ids with no per-call threading. ERROR CODE REGISTRY - src/lib/error-codes.ts defines stable DOMAIN_REASON codes with HTTP status + plain-text user-facing message (no jargon, written for the rep on the phone with a customer). - New CodedError class wraps a registered code + optional internalMessage (admin-only — never sent to client). - Existing AppError subclasses got plain-text default rewrites so legacy throw sites improve immediately without migration. - High-impact services migrated to specific codes: expenses (RECEIPT_REQUIRED, INVOICE_LINKED), interest-berths (CROSS_PORT_LINK_REJECTED), berth-pdf (PDF_MAGIC_BYTE / PDF_EMPTY / PDF_TOO_LARGE / VERSION_ALREADY_CURRENT), recommender (INTEREST_PORT_MISMATCH). ERROR ENVELOPE - errorResponse always sets X-Request-Id header + requestId field. - 5xx responses include a "Quote error ID …" friendly line. - 4xx kept clean (validation, permission, not-found don't pollute the inspector — they're already in audit log). PERSISTENCE (error_events table, migration 0040) - One row per 5xx, keyed on requestId, with method/path/status/error name+message/stack head (4KB cap)/sanitized body excerpt (1KB cap; password/token/secret/etc keys redacted)/duration/IP/UA/metadata. - captureErrorEvent extracts Postgres SQLSTATE/severity/cause.code so the classifier can recognize FK / unique / NOT NULL / schema- drift violations. - Failure to persist is logged-not-thrown. LIKELY-CULPRIT CLASSIFIER (src/lib/error-classifier.ts) - 4-pass heuristic (first match wins): 1. Postgres SQLSTATE → human reason (23503 FK, 23505 unique, 42703 schema drift, 53300 connection limit, …) 2. Error class name (AbortError, TimeoutError, FetchError, ZodError) 3. Stack-path patterns (/lib/storage/, /lib/email/, documenso, openai|claude, /queue/workers/) 4. Free-text message keywords (econnrefused, rate limit, timeout, unauthorized|invalid api key) - Returns { label, hint, subsystem } for the inspector badge. CLIENT SIDE - apiFetch throws structured ApiError with message + code + requestId + details + retryAfter. - toastError() helper renders the standard 3-line toast: plain message / Error code: X / Reference ID: Y [Copy ID]. ADMIN INSPECTOR - /<port>/admin/errors lists captured 5xx with status badge + path + likely-culprit badge + truncated message + reference id. Filter by status code; auto-refresh via TanStack Query. - /<port>/admin/errors/<requestId> deep-dive: request shape, full error name+message+stack, sanitized body excerpt, raw metadata, registered-code lookup (so admin can compare to what user saw), likely-culprit hint with subsystem tag. - /<port>/admin/errors/codes is the in-app code reference page — every registered code grouped by domain prefix, searchable, with HTTP status + user message inline. Linked from inspector header so admins can flip to it while triaging. - Permission: admin.view_audit_log. Super admins see all ports; regular admins port-scoped. - system-monitoring dashboard now surfaces error_events alongside permission_denied audit + queue failed jobs (RecentError gains source: 'request' variant). DOCS - docs/error-handling.md walks through coded errors, plain-text message guidelines, client toasting, admin inspector usage, persistence rules, classifier internals, pruning, and the legacy → CodedError migration path. MIGRATION SAFETY - Audit confirmed all 41 migrations (0000-0040) apply cleanly in journal order against an empty DB. 0040 references ports(id) which exists from 0000. 0035/0038 don't deadlock under sequential psql -f. Removed redundant idx_ds_sent_by from 0038 (created in 0037). Tests: 1168/1168 vitest passing. tsc clean. - security-error-responses tests updated for plain-text messages + new optional response keys (code/requestId/message). - berth-pdf-versions tests assert stable error codes via toMatchObject({ code }) rather than message regex. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-05 14:12:59 +02:00
/**
* Mix the active request context (request id, port id, user id) into
* EVERY log line emitted within an API request this is what makes
* the super-admin error inspector usable: paste a request id into the
* search and every log line that fired during that request comes back
* keyed to it. Outside a request (queue jobs, scheduled tasks) the
* mixin returns an empty object so the log line is unchanged.
*/
mixin() {
const ctx = getRequestContext();
if (!ctx) return {};
return {
requestId: ctx.requestId,
portId: ctx.portId || undefined,
userId: ctx.userId || undefined,
};
},
redact: {
paths: [
'password',
'token',
'secret',
'accessKey',
'secretKey',
'creditCard',
'*.password',
'*.token',
'*.secret',
'*.accessKey',
'*.secretKey',
fix(audit-v3): platform-wide deferred-list cleanup (rounds 1-4) Working through the audit-v2 deferred backlog. Each round was tested (typecheck + 1168/1168 vitest) before moving on. Round 1 — DB performance + AI cost visibility: - Add missing FK indexes Postgres doesn't auto-create on berth_reservations.{interest_id, contract_file_id}, documents.{file_id, signed_file_id}, document_events.signer_id, document_templates.source_file_id, form_submissions.{form_template_id, client_id}, document_sends.{brochure_id, brochure_version_id, sent_by_user_id}. Without these, RESTRICT-checks on parent delete + reverse-lookups walk the child tables fully. Migration 0037. - AI worker now writes one ai_usage_ledger row per OpenAI call so admins can audit spend per port/user/feature and future per-port budgets have history to read from. Failure to write is logged-not-thrown so the user-facing email draft is unaffected. Round 2 — Boot-time + transport hardening: - S3 backend verifies the bucket exists at startup (or auto-creates when MINIO_AUTO_CREATE_BUCKET=true). A typo'd bucket name now surfaces with a clear boot error instead of a vague Minio error inside the first user-facing request. - Documenso v1 placeFields: 3-attempt exponential-backoff retry on 5xx + network errors, fail-fast on 4xx. Stops one transient flake from leaving a document with a partial field set. - FilesystemBackend logs a structured warn-once at boot when the dev HMAC fallback is in effect, so two processes started with different BETTER_AUTH_SECRET values are observable (random 401s on file downloads otherwise). - Logger redact paths extended to cover *.headers.{authorization, cookie}, *.config.headers.authorization, encrypted-credential blobs (secretKeyEncrypted, smtpPassEncrypted, etc.), the Documenso X-Documenso-Secret header, and 2-level nested forms. Round 3 — UI feedback + permission gates: - Storage admin migrate dialog: success toast with row count + error toast on both dryRun and migrate mutations. - Invoice detail Send + Record-payment buttons wrapped in PermissionGate (invoices.send / invoices.record_payment); both mutations now toast on success/error. - Admin user list Edit button wrapped in PermissionGate(admin.manage_users). - Scan-receipt page surfaces an amber warning when OCR fails so reps know they can fill the form manually instead of staring at a stalled spinner; the editable form now also opens on scanMutation.isError / uploadedFile, not only on success. - Email threads list now renders skeleton rows during load + shared EmptyState for the empty case (was a single "Loading…" line). Round 4 — Service / route correctness: - documentSends.sent_by_user_id was a free-text NOT NULL column with no FK. Now nullable + FK to user(id) ON DELETE SET NULL so the audit row survives a user being hard-deleted. Migration 0038 with a defensive null-out for any orphan ids before attaching the constraint. - Saved-views route: documented why withAuth alone is correct (the service strictly filters by (portId, userId) — owner-only by design). - Public-interests audit log: replaced "userId: null as unknown as string" cast with userId: null; AuditLogParams already accepts null for system-generated events. - EOI in-app PDF fill: extracted setBerthRange() that, when the AcroForm field is missing AND the context has a non-empty range string, logs a structured warn so the deployment gap (live Documenso template needs the field) is observable instead of silently dropping the multi-berth range. Test status: 1168/1168 vitest. tsc clean. Two new migrations (0037/0038) need pnpm db:push (or migration apply) on the dev DB. Deferred-doc updated with the remaining open items (bigger refactors). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-05 12:49:53 +02:00
// Encrypted credential blobs surface in storage / smtp config logs
// unintentionally; redact them defensively even though they're
// already AES-encrypted at rest.
'*.secretKeyEncrypted',
'*.smtpPassEncrypted',
'*.imapPassEncrypted',
'*.proxyHmacSecretEncrypted',
// HTTP authorization headers (Bearer tokens, Basic creds) leak via
// err.config.headers on http-client error logs.
'*.headers.authorization',
'*.headers.Authorization',
'*.headers["x-documenso-secret"]',
'*.config.headers.Authorization',
'*.config.headers.authorization',
// Cookie headers can carry session tokens.
'*.headers.cookie',
'*.headers.Cookie',
// Two-level nesting for things like `req.headers.authorization` or
// `cfg.s3.secretKeyEncrypted`.
'*.*.password',
'*.*.token',
'*.*.secret',
'*.*.secretKeyEncrypted',
'*.*.headers.authorization',
'*.*.headers.Authorization',
],
censor: '[REDACTED]',
},
transport:
process.env.NODE_ENV !== 'production'
? { target: 'pino-pretty', options: { colorize: true } }
: undefined,
serializers: {
err: pino.stdSerializers.err,
req: pino.stdSerializers.req,
res: pino.stdSerializers.res,
},
});