letsbe/pn-new-crm
1
0
Fork 0
Code Issues Pull Requests 1 Actions Packages Projects Releases Wiki Activity
357 Commits 8 Branches 0 Tags
5c8c12ba1faf26c45491c16e42f730ed4bfa340e
Commit Graph

5 Commits

Author SHA1 Message Date
Matt Ciaccio
4723994bdc feat(errors): platform-wide request ids + error codes + admin inspector 
End-to-end error-handling overhaul. A user hitting any failure now sees
a plain-text message + stable error code + reference id. A super admin
can paste the id into /admin/errors/<id> for the full request shape,
sanitized body, error stack, and a heuristic likely-cause hint.

REQUEST CONTEXT (AsyncLocalStorage)
- src/lib/request-context.ts mints a per-request frame carrying
  requestId + portId + userId + method + path + start timestamp.
- withAuth wraps every authenticated handler in runWithRequestContext
  and accepts an upstream X-Request-Id header (validated shape) or
  generates a fresh UUID. The id ALWAYS leaves on the X-Request-Id
  response header, including early-return 401/403/4xx paths.
- Pino logger reads from the same context via mixin — every log
  line emitted during the request automatically carries the ids
  with no per-call threading.

ERROR CODE REGISTRY
- src/lib/error-codes.ts defines stable DOMAIN_REASON codes with
  HTTP status + plain-text user-facing message (no jargon, written
  for the rep on the phone with a customer).
- New CodedError class wraps a registered code + optional
  internalMessage (admin-only — never sent to client).
- Existing AppError subclasses got plain-text default rewrites so
  legacy throw sites improve immediately without migration.
- High-impact services migrated to specific codes:
  expenses (RECEIPT_REQUIRED, INVOICE_LINKED), interest-berths
  (CROSS_PORT_LINK_REJECTED), berth-pdf (PDF_MAGIC_BYTE / PDF_EMPTY /
  PDF_TOO_LARGE / VERSION_ALREADY_CURRENT), recommender
  (INTEREST_PORT_MISMATCH).

ERROR ENVELOPE
- errorResponse always sets X-Request-Id header + requestId field.
- 5xx responses include a "Quote error ID …" friendly line.
- 4xx kept clean (validation, permission, not-found don't pollute
  the inspector — they're already in audit log).

PERSISTENCE (error_events table, migration 0040)
- One row per 5xx, keyed on requestId, with method/path/status/error
  name+message/stack head (4KB cap)/sanitized body excerpt (1KB cap;
  password/token/secret/etc keys redacted)/duration/IP/UA/metadata.
- captureErrorEvent extracts Postgres SQLSTATE/severity/cause.code
  so the classifier can recognize FK / unique / NOT NULL / schema-
  drift violations.
- Failure to persist is logged-not-thrown.

LIKELY-CULPRIT CLASSIFIER (src/lib/error-classifier.ts)
- 4-pass heuristic (first match wins):
  1. Postgres SQLSTATE → human reason (23503 FK, 23505 unique,
     42703 schema drift, 53300 connection limit, …)
  2. Error class name (AbortError, TimeoutError, FetchError,
     ZodError)
  3. Stack-path patterns (/lib/storage/, /lib/email/, documenso,
     openai|claude, /queue/workers/)
  4. Free-text message keywords (econnrefused, rate limit, timeout,
     unauthorized|invalid api key)
- Returns { label, hint, subsystem } for the inspector badge.

CLIENT SIDE
- apiFetch throws structured ApiError with message + code + requestId
  + details + retryAfter.
- toastError() helper renders the standard 3-line toast:
  plain message / Error code: X / Reference ID: Y [Copy ID].

ADMIN INSPECTOR
- /<port>/admin/errors lists captured 5xx with status badge + path +
  likely-culprit badge + truncated message + reference id. Filter by
  status code; auto-refresh via TanStack Query.
- /<port>/admin/errors/<requestId> deep-dive: request shape, full
  error name+message+stack, sanitized body excerpt, raw metadata,
  registered-code lookup (so admin can compare to what user saw),
  likely-culprit hint with subsystem tag.
- /<port>/admin/errors/codes is the in-app code reference page —
  every registered code grouped by domain prefix, searchable, with
  HTTP status + user message inline. Linked from inspector header
  so admins can flip to it while triaging.
- Permission: admin.view_audit_log. Super admins see all ports;
  regular admins port-scoped.
- system-monitoring dashboard now surfaces error_events alongside
  permission_denied audit + queue failed jobs (RecentError gains
  source: 'request' variant).

DOCS
- docs/error-handling.md walks through coded errors, plain-text
  message guidelines, client toasting, admin inspector usage,
  persistence rules, classifier internals, pruning, and the
  legacy → CodedError migration path.

MIGRATION SAFETY
- Audit confirmed all 41 migrations (0000-0040) apply cleanly in
  journal order against an empty DB. 0040 references ports(id)
  which exists from 0000. 0035/0038 don't deadlock under sequential
  psql -f. Removed redundant idx_ds_sent_by from 0038 (created in
  0037).

Tests: 1168/1168 vitest passing. tsc clean.
- security-error-responses tests updated for plain-text messages
  + new optional response keys (code/requestId/message).
- berth-pdf-versions tests assert stable error codes via
  toMatchObject({ code }) rather than message regex.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-05 14:12:59 +02:00
Matt Ciaccio
180912ba9f fix(audit-final): pre-merge hardening + expense receipt UI 
Final audit pass on feat/berth-recommender (3 parallel Opus agents)
caught 5 critical and ~12 high-severity findings. All addressed in-branch;
medium/low items deferred to docs/audit-final-deferred.md.

Critical:
- Add filesystem-backend PUT handler at /api/storage/[token] so
  presigned uploads stop 405-ing in filesystem mode (every browser-driven
  berth-PDF + brochure upload was broken). Same token-verify + replay
  protection as GET, plus magic-byte gate when c=application/pdf.
- Forward req.signal into streamExpensePdf so an aborted 1000-receipt
  export no longer keeps grinding for minutes.
- Strengthen Content-Disposition filename sanitization: \s matches CR/LF
  which would let documentName forge headers; restrict to [\w. -]+ and
  add filename* RFC 5987 fallback.
- Lock public berths feed behind an explicit slug allowlist instead of
  ?portSlug= enumeration.
- Reject cross-port interest_berths upserts (defense-in-depth on top of
  the recommender SQL port filter).

High:
- Recommender: width-only feasibility now caps length via L/W ratio so a
  200ft berth doesn't surface for a 30ft beam request; total_interest_count
  filters out junction rows whose interest is in another port.
- Mooring normalization follow-up migration (0034) catches un-hyphenated
  padded forms (A01) the original 0024 WHERE missed.
- Send-out rate limit moved AFTER validation and scoped per-(port, user)
  so typos don't burn a slot and a multi-port rep can't be DoS'd by
  another tenant.
- Default-brochure path now blocks an archived row from sneaking through
  the partial unique index.
- NocoDB import --update-snapshot honoured under --dry-run so reps can
  refresh the seed JSON without committing DB writes.
- PDF export: orderBy desc(expenseDate); apply isNull(archivedAt) when
  expenseIds are passed (was bypassed); flag rate-unavailable rows with
  an amber footer instead of silently treating them as 1:1; skip the
  USD->EUR chain when source already matches target.
- expense-form-dialog: revokeObjectURL captures the URL in the closure
  instead of revoking the still-displayed one; reset upload state on
  close.
- scan/page: handleClearReceipt resets in-flight scan/upload mutations;
  Save disabled while upload pending.
- updateExpense re-asserts receipt-or-acknowledgement at the merged
  row so PATCH can't slip past the create-time refine.

Plus the in-progress receipt upload UI for the expense form dialog
(receipt picker + "I have no receipt" checkbox + warning banner) and
a noReceiptAcknowledged flag on ExpenseRow for edit-mode hydration.

Includes the canonical plan doc (referenced in CLAUDE.md), the handoff
prompt, and a deferred-findings index for follow-up issues.

1163/1163 vitest passing. Typecheck clean.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-05 05:11:26 +02:00
Matt Ciaccio
a3e002852b fix(audit-2): integration regressions + data-integrity from second-pass review 
Two reviewer agents did a second-pass deep audit of the 21-commit
refactor. Eight findings; four fixed here (one was deferred with a
schema comment, three were 🟡 nice-to-haves left for follow-up).

Integration regressions (🟠 high):
- Outbound webhook `interest.berth_linked` now fires from the new
  junction-add handler. Was emitting a socket-only event, leaving
  external integrations silent post-refactor.
- Two new webhook events `interest.berth_unlinked` and
  `interest.berth_link_updated` added to WEBHOOK_EVENTS +
  INTERNAL_TO_WEBHOOK_MAP. PATCH and DELETE handlers now dispatch them
  alongside the existing socket emits — lifecycle parity restored.
- BerthInterestPulse adds useRealtimeInvalidation for berth-link
  events. The query key was berth-scoped while the linked-berths
  dialog invalidates interest-scoped keys (no prefix match), so the
  pulse went stale. Bridges via the realtime hook now.

Recommender semantic fix (🟠 medium-high):
- aggregates CTE: active_interest_count now filters on
  `ib.is_specific_interest = true`, matching the public-map "Under
  Offer" derivation. EOI-bundle-only links no longer demote a berth
  to Tier C for other reps. Smoke test confirms previously-all-Tier-C
  results now correctly classify as Tier A.
- Same CTE: `total_interest_count` uses COUNT(ib.berth_id) instead of
  COUNT(*) so a berth with no junction rows reports 0 (not 1 from
  the LEFT JOIN's NULL-right-side row). Prevents heat over-counting.

Data integrity (🟠):
- AcroForm tier rejects negative numerics in coerceFieldValue (was
  letting through `length_ft="-50"` which would poison the
  recommender feasibility filter on apply).
- FilesystemBackend.resolveHmacSecret throws in production when
  storage_proxy_hmac_secret_encrypted is null. Dev still derives from
  BETTER_AUTH_SECRET for ergonomics; prod must explicitly configure.
- Documented the circular FK between berths.current_pdf_version_id
  and berth_pdf_versions.id. Drizzle's `.references()` can't express
  the cycle so the schema column is plain text + a comment; the FK
  is authoritatively maintained by migration 0030.

Tests still 1163/1163. tsc clean.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-05 04:20:38 +02:00
Matt Ciaccio
86372a857f fix(audit): post-review hardening across phases 0-7 
15 of 17 findings from the consolidated audit (3 reviewer agents on
the previously-shipped phase commits). Remaining two are nice-to-have
follow-ups deferred.

Critical (data integrity / security):
- Public berths API: closed-deal junction rows no longer flip a berth
  to "Under Offer" - filter on `interests.outcome IS NULL` so won/
  lost/cancelled don't pollute public-map status. Both list +
  single-mooring routes.
- Recommender heat: cancelled outcomes now count as fall-throughs
  (SQL was `LIKE 'lost%'` which silently dropped them, leaving
  cancelled-only berths stuck in tier A).
- Filesystem presignDownload returns an absolute URL (origin from
  APP_URL) so emailed download links resolve from external mail
  clients.
- Magic-byte verification on the presigned-PUT path: both per-berth
  PDFs and brochures stream the first 5 bytes via the storage backend
  and reject + delete on `%PDF-` mismatch (was only enforced when the
  server saw the buffer; presign-PUT was wide open).
- Replay-protection TTL aligned to the token's own expiry (was a
  fixed 30 min, but send-out tokens live 24 h). Floor 60 s, ceiling
  25 days.
- Brochures unique partial index on (port_id) WHERE is_default=true
  + 0032 migration. Closes the read-then-write race in the create/
  update transactions.

Important:
- Recommender SQL: defense-in-depth `i.port_id = $portId` filter on
  the aggregates CTE.
- berth-pdf service: per-berth pg_advisory_xact_lock around the
  version-number SELECT + insert. Storage key is now UUID-based so
  concurrent uploads can't collide on blob paths. Replaces
  `nextVersionNumber` with the tx-bound variant.
- berth-pdf apply: rejects with ConflictError when parse_results
  contain a mooring-mismatch warning unless the caller passes
  `confirmMooringMismatch: true` (force-reconfirm gate was UI-only).
- Send-out body: HTML-escape brochure filename in the download-link
  fallback (XSS guard).
- parseDecimalWithUnit rejects negative numbers.
- listClients DISTINCT ON for primary contact resolution: bounds
  contact-row count to ~2 per client.

Defensive:
- verifyProxyToken rejects NaN/Infinity expiries via Number.isFinite.
- Replaced sql ANY() with inArray() in interest-berths.

Tests: 1145 -> 1163 passing.

Deferred: bulk-send rate limit (no bulk endpoint today), markdown
italic regex breaking links with asterisks (cosmetic).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-05 04:07:03 +02:00
Matt Ciaccio
b1e787e55c feat(recommender): SQL ranking + tier ladder + heat scoring 
Plan §4.4 + §13: pure SQL recommender, no AI. Single CTE chain
(feasible -> aggregates) + JS-side tier classification, fall-through
cooldown filter, heat scoring, and fit ranking. Per-port settings via
system_settings layered over global + DEFAULT_RECOMMENDER_SETTINGS.

Tier ladder (default):
  A : no interest history
  B : lost-only history (still recommendable + boosted by heat)
  C : active interest in early stage (open..eoi_signed)
  D : active interest at deposit_10pct or beyond (hidden by default)

Heat (only for tier B):
  recency        weight 30  full @ <=30 days, decays to 0 @ 365 days
  furthest stage weight 40  full when prior reached deposit
  interest count weight 15  saturates at 5+
  EOI count      weight 15  saturates at 3+

Multi-port isolation enforced (§14.10 critical): the SQL filters by
port_id AND the entry-point function rejects cross-port interest
lookups with an explicit error. Fall-through policy supports
immediate_with_heat (default), cooldown, and never_auto_recommend.

15 unit tests covering tier classification, heat saturation, weight
tuning, zero-weight guard. Smoke-tested end-to-end via
scripts/dev-recommender-smoke.ts.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-05 02:58:34 +02:00