letsbe/pn-new-crm
1
0
Fork 0
Code Issues Pull Requests 1 Actions Packages Projects Releases Wiki Activity
317 Commits 7 Branches 0 Tags
c90876abad778d83448b13f35aa442d22bbad8ae
Commit Graph

317 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Matt Ciaccio
c90876abad feat(admin): inquiry inbox, send log, email-template overrides, reports dashboard, recommender keys, role-editor coverage; replace placeholder pages 
Closes the bulk of audit-pass-#1 admin gaps in one batch.

New admin pages:
- /admin/inquiries reads website_submissions with filter chips for
  berth/residence/contact + payload viewer per row.
- /admin/sends reads document_sends with sent/failed filter chips and
  expandable body markdown; failures surface errorReason and any
  fallback-to-link reason from the SMTP retry.
- /admin/email-templates lets per-port admins override the subject of
  each transactional template (8 templates catalogued in
  template-catalog.ts). Body editing is a follow-on; portal_activation
  + portal_reset are wired to honor the override via loadSubjectOverride.
- /admin/reports replaces the "Coming in Layer 3" placeholder with a
  KPI dashboard: 4 KPI tiles, pipeline funnel bars, berth occupancy
  donut-bars, conversion %, refresh every 60s.
- backup/import/onboarding admin pages replace placeholders with
  actionable guidance: backup posture + planned features, available CLI
  imports + planned UI, ordered onboarding checklist linking to admin
  pages.

Existing pages widened:
- settings-manager exposes the 9 berth-recommender tunables that were
  previously code-only (recommender_*, heat_weight_*, fallthrough_*,
  tier_ladder_hide_late_stage).
- role-form covers all 19 RolePermissions schema groups; previously
  missing yachts/companies/memberships/reservations + missing
  documents.edit + files.edit checkboxes. snake_case residential
  labels replaced with friendly text.

portal-auth.service.ts now also writes audit_log rows for portal
invite, resend, activate, password-reset request, and reset (closes one
more audit-pass-#2 gap while we were touching the file).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-06 14:58:17 +02:00
Matt Ciaccio
8cdee99310 feat(activity): per-entity Activity timeline (clients/yachts/companies/berths) 
Until now only the global /admin/audit page surfaced audit_logs. Each
entity detail page either lacked the Activity tab entirely or rendered
"Activity log coming soon" text.

- entity-activity.service.loadEntityActivity wraps searchAuditLogs
  with actor-email resolution; reused by all 5 endpoints.
- New endpoints: /api/v1/{clients,yachts,companies,berths,interests}/[id]/activity,
  each gated on the per-entity .view permission and tenant-checked
  against ctx.portId.
- EntityActivityFeed renders a timeline with action verb ("Updated",
  "Archived"), actor name, relative time, and field old→new diff.
- client-tabs, yacht-tabs, company-tabs, berth-tabs now mount the feed
  on their Activity tab. Interest already has the richer
  InterestTimeline component.
- yacht-tabs YachtInterestsTab also gets a friendlier empty state with
  guidance copy.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-06 14:57:51 +02:00
Matt Ciaccio
d19b74b935 feat(profile): /settings/profile page + change-password endpoint 
The user-menu's Profile link previously 404'd, and CRM users had no way
to change their password from inside the app.

- /api/v1/me/password POST wraps better-auth changePassword, surfaces a
  friendlier "Current password is incorrect" on the typical failure
  mode, and writes an audit_log row with metadata.revokedOtherSessions.
- /{port}/settings/profile renders display name + email + change-password
  card with current/new/confirm fields and a 'Sign out other devices'
  toggle.

End-to-end verified: wrong current pw → 400 with mapped message;
correct → 200 + audit row; revert → 200.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-06 14:57:35 +02:00
Matt Ciaccio
1b78eadd36 feat(audit): extend AuditAction enum + audit logging on alerts + expense dedup 
- AuditAction gains password_change, portal_invite/activate/reset
  variants, send, view. AuditLogParams.ipAddress/userAgent now optional
  so background jobs and internal helpers can log without faking values.
- alerts.service.dismissAlert/acknowledgeAlert now write
  action='update' rows with metadata.kind so the audit log differentiates
  the two state changes.
- expense-dedup.service.clearDuplicate/mergeDuplicate accept userId
  and write action='update'/'merge' rows respectively. Routes pass
  ctx.userId.

Audit gaps surfaced by audit-pass-#2: 6 services bypassed audit_logs
entirely. This commit closes 2 of them; portal-auth lands in a later
commit alongside the email-template-override work that already touches
the same file.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-06 14:57:24 +02:00
Matt Ciaccio
1fb3aa3aeb fix(regressions): client-bundle ioredis + Drizzle ANY() array bindings 
Two regressions from yesterday's audit-tier-0 work that broke the dev
server and every clients API call.

- baseListQuerySchema lived in route-helpers.ts, which was made
  server-only by the rate-limit import. Every validator imported it,
  pulling ioredis (and dns/net/tls/fs/node:async_hooks) into the client
  bundle — every form/detail page returned 500 in dev. Extracted the
  schema to src/lib/api/list-query.ts and updated all 14 validators.
- clients.service.listClients and email-compose used raw SQL
  ANY(\${jsArray}) which Drizzle binds as JSON — Postgres rejects with
  42809 "op ANY/ALL (array) requires array on right side". Switched to
  the inArray helper.

GET /api/v1/clients now returns 200 again. Affects every form/detail
page that imports a validator.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-06 14:56:59 +02:00
Matt Ciaccio
7bd969b41a fix(audit-integrations): SMTP/PG/Socket.IO timeouts, prompt injection, secret-at-rest 
A focused review of every external integration surfaced six issues the
original audit missed.  Fixed here.

HIGH
* Socket.IO had an unconditional 30-second idle disconnect on every
  socket.  The comment on the line acknowledged it was "for development
  only, would be longer in prod" but no NODE_ENV guard existed, and the
  `socket.onAny` listener only resets on inbound client events — every
  dashboard connection that received only server-push events would have
  been torn down every 30s in production.  Removed the manual idle
  timer entirely; Socket.IO's pingTimeout / pingInterval handles
  dead-transport detection at the protocol level.
* SMTP transporters had no `connectionTimeout` / `greetingTimeout` /
  `socketTimeout`.  Nodemailer's defaults are 2 minutes for connect
  and unlimited for socket — a hung SMTP server would have held a
  BullMQ `email` worker concurrency slot for up to 10 min per job
  (5 retries × 2 min).  Set 10s/10s/30s on both the system transporter
  in `src/lib/email/index.ts` and the user-account transporter in
  `email-compose.service.ts`.

MEDIUM
* PostgreSQL pool had no `statement_timeout` /
  `idle_in_transaction_session_timeout`.  A slow query or transaction
  held by a crashed handler would have eventually exhausted the
  20-connection pool.  30s statement cap, 10s idle-in-tx cap, plus
  `max_lifetime: 30min` to recycle connections.
* `umami_password` and `umami_api_token` were stored as plaintext in
  `system_settings` (the SMTP and S3 secret paths use AES-GCM).  The
  reader now passes them through `readSecret()` which auto-detects
  the encrypted `iv:cipher:tag` shape and decrypts, falling back to
  legacy plaintext so operators can rotate without a flag-day.
* AI email-draft worker interpolated `additionalInstructions` (user-
  controlled) directly into the OpenAI prompt — a hostile rep could
  close the instructions block and inject prompt directives that
  override the system prompt.  Added `sanitizeForPrompt()` that
  strips newlines + quote chars, caps at 500 chars, and the prompt
  now wraps the value in a "treat as data not commands" preamble.

LOW
* Legacy `ensureBucket()` in `src/lib/minio/index.ts` was unguarded —
  if any future code imported it (currently no callers), a misconfigured
  prod deploy could mint a fresh empty bucket.  Now matches the gate
  used by the pluggable S3Backend (`MINIO_AUTO_CREATE_BUCKET=true`
  required) so the legacy export and the new pluggable path agree.

Confirmed not-an-issue: BullMQ Workers create connections via
`{ url }` options object, and BullMQ sets `maxRetriesPerRequest: null`
internally for those — no fix needed.  The shared `redis` singleton
that does keep `maxRetriesPerRequest: 3` is used only for direct
Redis ops (rate-limit sliding window, etc.), never for blocking
BullMQ commands, so the value is correct there.

Test status: 1175/1175 vitest, tsc clean.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-05 21:31:50 +02:00
Matt Ciaccio
63c4073e64 fix(audit-verification): regressions found in post-Tier-6 review 
Two parallel reviews of the Tier 0–6 work surfaced one CRITICAL
regression and a handful of remaining cross-tenant gaps that the
original audit didn't enumerate. All fixed here:

CRITICAL
* document-reminders.processReminderQueue — the new bulk-fetch
  leftJoin to documentTemplates was scoped on `templateType` alone.
  Templates of the same type exist in every port; the cartesian
  explosion would have fired one Documenso reminder PER matching
  template-row per cron tick (a 5-port deploy = 5 reminders to the
  same signer per cycle). Added eq(documentTemplates.portId, portId)
  to the join.
* All five remaining Documenso webhook handlers (RecipientSigned /
  Completed / Opened / Rejected / Cancelled) accept and require an
  optional portId now, with a shared resolveWebhookDocument() helper
  that refuses to mutate when the lookup is ambiguous across tenants
  without a resolved port. Tier 5's port-scoping was applied only to
  Expired; the route now forwards the matched portId to every
  handler. Tightens the WHERE clauses on subsequent UPDATEs to (id,
  portId) for defense-in-depth.

HIGH
* verifyDocumensoSecret rejects when `expected` is empty —
  timingSafeEqual(0-bytes, 0-bytes) was returning true, so a dev env
  with a blank DOCUMENSO_WEBHOOK_SECRET would accept a request whose
  X-Documenso-Secret header was also missing/empty.
  listDocumensoWebhookSecrets skips the env entry when blank.
* /api/public/health — the website-intake-secret comparison was a
  string `===` (not constant-time). Switched to timingSafeEqual via
  Buffer.from().

MEDIUM
* server.ts SIGTERM ordering — Socket.io closes BEFORE the HTTP
  drain so long-poll websockets stop holding the server open past
  the compose stop_grace_period.
* /api/v1/me PATCH preferences merge — allow-list filter on the
  merged JSONB so legacy rows from the old .passthrough() era stop
  silently re-shipping their bloat to disk.

Migration fixes (deploy-blocking)
* 0041 referenced `port_role_overrides.permissions` (column is
  `permission_overrides`) — overrides are partial JSONB and don't
  need backfilling at all (deepMerge resolves edit from the base
  role). Removed the override UPDATEs entirely.
* 0042 switched all FK + CHECK adds to NOT VALID + VALIDATE so the
  brief table-lock phase is decoupled from the row-scan validation,
  giving a cleaner abort-and-restart story if a constraint catches
  dirty production data. Added a pre-cleanup UPDATE for
  invoices.billing_entity_id = '' rows (backfills from clientName,
  falls back to the row id) so the new non-empty CHECK passes on a
  dirty table.

Test status: 1175/1175 vitest, tsc clean.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-05 21:19:39 +02:00
Matt Ciaccio
83239104e0 fix(audit-tier-6): validation, perms, ops/infra, per-port webhook secret 
Final audit polish — closes the remaining LOW + MED items the previous
tiers didn't reach:

* Validation hardening: me.preferences uses .strict() + 8KB cap
  instead of unbounded .passthrough(); files.uploadFile gains
  magic-byte verification (jpeg/png/gif/webp/pdf/doc/xlsx); OCR scan
  endpoint enforces 10MB cap + magic-byte check on receipt images;
  port logoUrl + me.avatarUrl reject javascript:/data: schemes via
  a shared httpUrl refinement.
* Permission gates: document-sends/{brochure,berth-pdf} now require
  email.send (was withAuth-only); document-sends/{preview,list} on
  email.view; ai/email-draft on email.send; documents/[id]/send
  uses send_for_signing (was create); expenses/export/parent-company
  flips from hard isSuperAdmin to expenses.export for parity;
  admin/users/options gated on reminders.assign_others (was withAuth).
* Envelope hygiene: auth/set-password switches the third {message}
  variant to errorResponse + {data: {email}}; ai/email-draft wraps
  jobId in {data: {jobId}}.
* UI polish: reports-list.handleDownload surfaces failures via
  toastError (was console-only).
* Ops/infra: pin pnpm@10.33.2 across all three Dockerfiles +
  packageManager field in package.json; Dockerfile.worker re-orders
  user creation BEFORE pnpm install so node_modules / .cache dirs
  are worker-owned (fixes tesseract.js + sharp EACCES at first PDF
  parse); add Redis-ping HEALTHCHECK to the worker container.
* Public health endpoint: returns full env+appUrl payload only when
  the caller presents X-Intake-Secret, otherwise a minimal {status}
  so generic uptime monitors still work but anonymous internet
  doesn't get deployment fingerprints.
* Per-port Documenso webhook secret: new system_settings key
  + listDocumensoWebhookSecrets() helper.  The webhook receiver
  iterates every configured per-port secret with timing-safe
  comparison + falls back to env, then forwards the resolved portId
  into handleDocumentExpired so two ports sharing a documensoId
  cannot cross-mutate.

Deferred (handled in dedicated follow-up PRs):
* Tier 5.1 — direct service tests for portal-auth / users /
  email-accounts / document-sends / sales-email-config.  MED, large
  test-writing scope.
* The {ok: true} → {data: null} envelope migration across
  alerts/expenses/admin-ocr-settings/storage routes.  Mechanical but
  needs coordinated client + test updates.
* CSP-nonce migration (drop unsafe-inline) — needs middleware-level
  nonce generation that the Next 15 router has to thread through.
* Idempotency-Key header on Documenso createDocument.  Requires
  schema column on documents to persist the key; deferred so it
  doesn't bundle a migration into this commit.
* The 16 better-auth user_id FKs — separate dedicated migration
  with care (some columns are NOT NULL today and cascade decisions
  matter).
* PermissionGate / Skeleton / EmptyState wraps across 5 admin lists
  (auditor-H §§36–37) and the residential-clients filter bar.

Test status: 1175/1175 vitest, tsc clean.

Refs: docs/audit-comprehensive-2026-05-05.md MED §§28,29,30 + LOW §§32–43
+ HIGH §9 (Documenso secrets follow-up).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-05 21:03:31 +02:00
Matt Ciaccio
4bab6de8be test(audit-tier-5): webhook + cross-port test coverage 
Closes the highest-priority gaps from audit HIGH §19 + MED §§20–21:

* New tests/integration/documenso-webhook-route.test.ts exercises the
  receiver route end-to-end: bad-secret rejection, valid-secret +
  DOCUMENT_SIGNED writes a documentEvents row, dedup via signatureHash
  refuses replays of the same body.
* tests/integration/documents-expired-webhook.test.ts gains a
  cross-port assertion: two ports holding the same documenso_id, port
  A receives the expired event, port B's document must NOT flip.  Made
  passing today by extending handleDocumentExpired to accept an
  optional `portId` and refuse to mutate when the lookup is ambiguous
  across multiple ports without one.
* tests/integration/custom-fields.test.ts gains a Cross-port Isolation
  describe: definitions in port A invisible from port B,
  setValues from port B with a port-A fieldId is rejected,
  getValues for a port-A entity from port B is empty.

Deferred: Tier 5.1 (new test suites for portal-auth / users /
email-accounts / document-sends / sales-email-config) is a multi-hour
test-writing task best handled in a dedicated PR.  Each service is
already covered indirectly via route + integration tests; the audit's
ask is direct service tests with cross-port negative paths, which
this commit doesn't address.

Test status: 1175/1175 vitest (was 1168), tsc clean.

Refs: docs/audit-comprehensive-2026-05-05.md HIGH §19 (auditor-J Issue 2)
+ MED §§20–21 (auditor-J Issues 3–4).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-05 20:53:34 +02:00
Matt Ciaccio
4eea4ceff9 fix(audit-tier-4): tenant-isolation defense-in-depth 
Closes the audit's HIGH §10 + MED §§17–22 isolation footguns. None of
these are user-impactful TODAY — every site is preceded by a port-
scoped read or pre-validated by ctx.portId — but each is a future-
refactor accident waiting to happen, so the SQL itself now pins the
tenant boundary:

* mergeClients gains a callerPortId option; the route caller passes
  ctx.portId.  removeInterestBerth now requires portId and verifies
  both the interest and the berth share it before deleting the
  junction row.  All three callers updated.
* Six service mutations now scope the WHERE to (id, portId):
  form-templates update + delete, invoices.detectOverdue per-row
  update, notifications.markRead, clients.deleteRelationship.
  company-memberships uses an inArray sub-select against port
  companies (no port_id column on the table itself), covering
  updateMembership / endMembership / setPrimary.
* Port-scoped file lookups in portal.getDocumentDownloadUrl,
  reports.getDownloadUrl (file presign), berth-reservations.activate
  (contractFileId attach guard), and residential.getResidentialInterestById
  (residentialClient join).

Test status: 1168/1168 vitest, tsc clean.

Refs: docs/audit-comprehensive-2026-05-05.md HIGH §10 + MED §§17–22
(auditor-B3 Issues 1–5,7).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-05 20:48:13 +02:00
Matt Ciaccio
7854cbabe4 perf(audit-tier-3): bulk-fetch the five hot N+1 loops 
Replaces per-row fan-out with grouped queries / inArray pre-fetches
across the five dashboard + cron hotspots flagged in the audit
(MED §13 / HIGH §11–14):

* reminders.processFollowUpReminders — was 3 round trips per
  enabled-and-due interest.  Now: filter in JS, single clients
  bulk-fetch, single reminders bulk-insert, single interests
  bulk-update, one summary socket emit.  1k due interests: 6 round
  trips total instead of 3000+.
* portal.getClientInvoices — was a full-table scan filtered in JS.
  Now an inArray push-down on lower(billingEmail) + defensive
  limit(100).  After 12mo this would have been the worst portal
  endpoint.
* interest-scoring.calculateBulkScores — was 6N round trips
  (1 redis + 1 findFirst + 4 counts per interest).  Now 4 grouped
  count queries on the port's interest set + a single redis pipeline
  to refresh the cache.  1k interests: ~7 round trips.
* document-reminders.processReminderQueue — was 5N round trips per
  cron tick (port + template + lastReminder + pendingSigners + send
  per doc).  Now hoists port + per-type template map + grouped
  lastReminder + bulk pendingSigners; per-row work collapses to a
  Map.get and the documenso send.  500 docs: ~7 round trips.
* inquiry-notifications.sendInquiryNotifications — was sequential
  createNotification + emailQueue.add per recipient inside a public
  POST.  Now Promise.all'd; a 20-user port stops blocking the public
  inquiry POST on ~80 round trips.

Test status: 1168/1168 vitest, tsc clean.

Refs: docs/audit-comprehensive-2026-05-05.md HIGH §§11–14 (auditor-I
Issues 1–4) + MED §13 (auditor-I Issue 5).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-05 20:41:23 +02:00
Matt Ciaccio
d3a6a9beef fix(audit-tier-2-routes): manual NextResponse.json error sweep + admin form banners 
Two final waves of error-surface hygiene closing the audit's MED §12 +
HIGH §15 + HIGH §17 findings:

* 50 route files swept (61 sites): manual NextResponse.json({error,
  status: 4xx|5xx}) early-returns replaced by typed throws +
  errorResponse(err) at the catch.
  - Super-admin gates (13 sites) use new requireSuperAdmin(ctx, action)
    helper from src/lib/api/helpers.ts so denials hit the audit log.
  - Path-param + body validation 400s become ValidationError throws.
  - 404s become NotFoundError or CodedError('NOT_FOUND') for AI
    feature-flag paths.
  - 11 manual 5xx returns now re-throw so error_events captures the
    request-id (the admin error inspector becomes usable from real
    incidents).
  - website-analytics 200-with-error anti-pattern flipped to 409 +
    UMAMI_NOT_CONFIGURED. 502 upstream paths use UMAMI_UPSTREAM_ERROR.
  - 11 sites intentionally preserved: storage/[token] anti-enumeration
    token-failure paths, webhook-secret 401, "Unknown port" 400 in
    public intake.

* 7 admin forms (roles, users, ports, webhooks, custom-fields,
  document-templates, tags) gain a formatErrorBanner() helper from
  src/lib/api/toast-error.ts that builds a multi-line "Error code / Reference ID"
  banner — the rep can copy the request id when reporting a failed
  save.  Banners get whitespace-pre-line so newlines render.

Test status: 1168/1168 vitest, tsc clean.

Refs: docs/audit-comprehensive-2026-05-05.md MED §12 (auditor-F Issue 1)
+ HIGH §15 (auditor-F Issue 2) + HIGH §17 (auditor-H Issue 2).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-05 20:36:59 +02:00
Matt Ciaccio
fc7595faf8 fix(audit-tier-2): error-surface hygiene — toastError + CodedError sweep 
Two mechanical sweeps closing the audit's HIGH §16 + MED §11 findings:

* 38 client components / 56 toast.error sites converted to
  toastError(err) so the new admin error inspector becomes usable from
  user-reported issues — every failed inline-edit, save, send, archive,
  upload, etc. now carries the request-id + error-code (Copy ID action).
* 26 service files / 62 bare-Error throws converted to CodedError or
  the existing AppError subclasses.  Adds new error codes:
  DOCUMENSO_UPSTREAM_ERROR (502), DOCUMENSO_AUTH_FAILURE (502),
  DOCUMENSO_TIMEOUT (504), OCR_UPSTREAM_ERROR (502),
  IMAP_UPSTREAM_ERROR (502), UMAMI_UPSTREAM_ERROR (502),
  UMAMI_NOT_CONFIGURED (409), and INSERT_RETURNING_EMPTY (500) for
  post-insert returning-empty guards.
* Five vitest assertions updated to match the new user-facing wording
  (client-merge "already been merged", expense/interest "couldn't find
  that …", documenso "signing service didn't respond").

Test status: 1168/1168 vitest, tsc clean.

Refs: docs/audit-comprehensive-2026-05-05.md HIGH §16 (auditor-H Issue 1)
+ MED §11 (auditor-G Issue 1).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-05 20:18:05 +02:00
Matt Ciaccio
6a609ecf94 fix(audit-tier-1): timeouts, lifecycle, per-port Documenso, FK constraints 
Closes the second wave of HIGH-priority audit findings:

* fetchWithTimeout helper (new src/lib/fetch-with-timeout.ts) wraps
  Documenso, OCR, currency, Umami, IMAP, etc. — a hung upstream can
  no longer pin a worker concurrency slot indefinitely.  OpenAI client
  passes timeout: 30_000.  ImapFlow gets socket / greeting / connection
  timeouts.
* SIGTERM / SIGINT handler in src/server.ts drains in-flight HTTP,
  closes Socket.io, and disconnects Redis before exit; compose
  stop_grace_period bumped to 30s.  Adds closeSocketServer() helper.
* env.ts gains zod-validated PORT and MULTI_NODE_DEPLOYMENT, and
  filesystem.ts now reads from env (a typo can no longer silently
  disable the multi-node guard).
* Per-port Documenso template + recipient IDs land in system_settings
  with env fallback (PortDocumensoConfig now exposes eoiTemplateId,
  clientRecipientId, developerRecipientId, approvalRecipientId).
  document-templates.ts uses the per-port config and threads portId
  into documensoGenerateFromTemplate().
* Migration 0042 wires the eleven HIGH-tier missing FK constraints
  (documents/files/interests/reminders/berth_waiting_list/
  form_submissions) plus polymorphic CHECK round 2
  (yacht_ownership_history.owner_type, document_sends.document_kind),
  invoices.billing_entity_id NOT EMPTY, and clients.merged_into self-FK.
  Drizzle schema columns updated to .references(...) where possible
  so the misleading "FK wired in relations.ts" comments are gone.

Test status: 1168/1168 vitest, tsc clean.

Refs: docs/audit-comprehensive-2026-05-05.md HIGH §§5,6,7,8,9,10 +
MED §§14,15,16,18.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-05 19:52:58 +02:00
Matt Ciaccio
cf430d70c3 fix(storage): route every file op through getStorageBackend() 
Removes 12 direct minioClient.{put,get,remove}Object call sites that
bypassed the pluggable storage abstraction.  Filesystem-mode deploys
(MULTI_NODE_DEPLOYMENT=false, storage_backend=filesystem) silently
broke at every site: GDPR export, invoice PDF, EOI generation, portal
download, file upload, folder create/rename/delete, signed PDF land,
maintenance cleanup, etc.  Each site now resolves the active backend
and uses its put/get/delete + the new presignDownloadUrl() helper.

Folder marker objects in /files/folders/* keep the same on-the-wire
shape but route through the backend.  A future refactor should move
folder bookkeeping to a DB-backed virtual-folder table (see audit
HIGH §3 follow-up note in the route file).

Sites left untouched: src/lib/services/system-monitoring.service.ts
and src/app/api/ready/route.ts use minioClient.bucketExists as an S3-
specific health probe — those are correctly mode-aware and stay.

Refs: docs/audit-comprehensive-2026-05-05.md HIGH §3 (auditor-D Issue 1)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-05 18:41:02 +02:00
Matt Ciaccio
312779c0c5 fix(security): tier-0 audit blockers (next CVE, role gate, perm traps, key validation, rate limits) 
Closes the five highest-risk findings from
docs/audit-comprehensive-2026-05-05.md so the platform is not exposed
while the rest of the audit backlog (1 CRIT + 18 HIGH + 32 MED + 23 LOW)
is worked through:

* CVE-2025-29927 — bump next 15.1.0 → 15.2.9; nginx strips
  X-Middleware-Subrequest at the edge as defense-in-depth.
* Cross-tenant role escalation — POST/PATCH/DELETE on /admin/roles now
  require super-admin (was: any holder of admin.manage_users).  Adds
  shared `requireSuperAdmin(ctx)` helper.
* Silent-403 traps — `documents.edit` and `files.edit` keys added to
  RolePermissions; seeded role values updated; migration 0041 backfills
  the new keys on every existing roles+port_role_overrides JSONB.  File
  routes remap the dead `create` action to `upload` / `manage_folders`.
* Berth-PDF / brochure register endpoints — reject body.storageKey
  unless it matches the namespace the matching presign endpoint issued
  (prevents repointing a tenant's PDF at foreign-port bytes).
* Portal auth rate limits — sign-in 5/15min/(ip,email),
  forgot-password 3/hr/IP, activate/reset/set-password 10/hr/IP.  Adds
  `enforcePublicRateLimit()` for non-`withAuth` routes.

Test status unchanged: 1168/1168 vitest, tsc clean.

Refs: docs/audit-comprehensive-2026-05-05.md (CRITICAL, HIGH §§1–4)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-05 18:33:13 +02:00
Matt Ciaccio
4723994bdc feat(errors): platform-wide request ids + error codes + admin inspector 
End-to-end error-handling overhaul. A user hitting any failure now sees
a plain-text message + stable error code + reference id. A super admin
can paste the id into /admin/errors/<id> for the full request shape,
sanitized body, error stack, and a heuristic likely-cause hint.

REQUEST CONTEXT (AsyncLocalStorage)
- src/lib/request-context.ts mints a per-request frame carrying
  requestId + portId + userId + method + path + start timestamp.
- withAuth wraps every authenticated handler in runWithRequestContext
  and accepts an upstream X-Request-Id header (validated shape) or
  generates a fresh UUID. The id ALWAYS leaves on the X-Request-Id
  response header, including early-return 401/403/4xx paths.
- Pino logger reads from the same context via mixin — every log
  line emitted during the request automatically carries the ids
  with no per-call threading.

ERROR CODE REGISTRY
- src/lib/error-codes.ts defines stable DOMAIN_REASON codes with
  HTTP status + plain-text user-facing message (no jargon, written
  for the rep on the phone with a customer).
- New CodedError class wraps a registered code + optional
  internalMessage (admin-only — never sent to client).
- Existing AppError subclasses got plain-text default rewrites so
  legacy throw sites improve immediately without migration.
- High-impact services migrated to specific codes:
  expenses (RECEIPT_REQUIRED, INVOICE_LINKED), interest-berths
  (CROSS_PORT_LINK_REJECTED), berth-pdf (PDF_MAGIC_BYTE / PDF_EMPTY /
  PDF_TOO_LARGE / VERSION_ALREADY_CURRENT), recommender
  (INTEREST_PORT_MISMATCH).

ERROR ENVELOPE
- errorResponse always sets X-Request-Id header + requestId field.
- 5xx responses include a "Quote error ID …" friendly line.
- 4xx kept clean (validation, permission, not-found don't pollute
  the inspector — they're already in audit log).

PERSISTENCE (error_events table, migration 0040)
- One row per 5xx, keyed on requestId, with method/path/status/error
  name+message/stack head (4KB cap)/sanitized body excerpt (1KB cap;
  password/token/secret/etc keys redacted)/duration/IP/UA/metadata.
- captureErrorEvent extracts Postgres SQLSTATE/severity/cause.code
  so the classifier can recognize FK / unique / NOT NULL / schema-
  drift violations.
- Failure to persist is logged-not-thrown.

LIKELY-CULPRIT CLASSIFIER (src/lib/error-classifier.ts)
- 4-pass heuristic (first match wins):
  1. Postgres SQLSTATE → human reason (23503 FK, 23505 unique,
     42703 schema drift, 53300 connection limit, …)
  2. Error class name (AbortError, TimeoutError, FetchError,
     ZodError)
  3. Stack-path patterns (/lib/storage/, /lib/email/, documenso,
     openai|claude, /queue/workers/)
  4. Free-text message keywords (econnrefused, rate limit, timeout,
     unauthorized|invalid api key)
- Returns { label, hint, subsystem } for the inspector badge.

CLIENT SIDE
- apiFetch throws structured ApiError with message + code + requestId
  + details + retryAfter.
- toastError() helper renders the standard 3-line toast:
  plain message / Error code: X / Reference ID: Y [Copy ID].

ADMIN INSPECTOR
- /<port>/admin/errors lists captured 5xx with status badge + path +
  likely-culprit badge + truncated message + reference id. Filter by
  status code; auto-refresh via TanStack Query.
- /<port>/admin/errors/<requestId> deep-dive: request shape, full
  error name+message+stack, sanitized body excerpt, raw metadata,
  registered-code lookup (so admin can compare to what user saw),
  likely-culprit hint with subsystem tag.
- /<port>/admin/errors/codes is the in-app code reference page —
  every registered code grouped by domain prefix, searchable, with
  HTTP status + user message inline. Linked from inspector header
  so admins can flip to it while triaging.
- Permission: admin.view_audit_log. Super admins see all ports;
  regular admins port-scoped.
- system-monitoring dashboard now surfaces error_events alongside
  permission_denied audit + queue failed jobs (RecentError gains
  source: 'request' variant).

DOCS
- docs/error-handling.md walks through coded errors, plain-text
  message guidelines, client toasting, admin inspector usage,
  persistence rules, classifier internals, pruning, and the
  legacy → CodedError migration path.

MIGRATION SAFETY
- Audit confirmed all 41 migrations (0000-0040) apply cleanly in
  journal order against an empty DB. 0040 references ports(id)
  which exists from 0000. 0035/0038 don't deadlock under sequential
  psql -f. Removed redundant idx_ds_sent_by from 0038 (created in
  0037).

Tests: 1168/1168 vitest passing. tsc clean.
- security-error-responses tests updated for plain-text messages
  + new optional response keys (code/requestId/message).
- berth-pdf-versions tests assert stable error codes via
  toMatchObject({ code }) rather than message regex.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-05 14:12:59 +02:00
Matt Ciaccio
c4a41d5f5b feat(expenses+interests): trip/event grouping (lightweight) 
Per the trips/events design discussion: instead of building a full
events domain (table + CRUD UI + calendar) for the 6–12 yacht shows
a year, ship the cheap version that covers the actual asks.

Expenses — `tripLabel` free-text:
- New `expenses.trip_label` text column (migration 0039) + index for
  filter / autocomplete lookup.
- Validator: createExpenseShape + listExpensesSchema +
  exportExpensePdfSchema.filter all accept tripLabel.
- Service: createExpense + updateExpense persist; listExpenses filters;
  new `listTripLabels(portId, search?)` returns distinct values
  ordered by most-recent expenseDate so the autocomplete surfaces
  recently-used labels first.
- New `GET /api/v1/expenses/trip-labels` endpoint (gated by
  expenses.view) backs the autocomplete.
- Form dialog: native `<datalist>` powered by the autocomplete query
  so reps don't end up with "Palm Beach 2026" / "palm-beach 2026"
  fragmented across two PDF sections.
- Expense list: new "Trip" column (badge) + free-text filter.
- Detail page: trip label rendered alongside Category / Payer.
- PDF export: GroupBy gains 'trip'; filter.tripLabel narrows the
  export. Untagged rows fall under "(no trip)".
- Trim/normalize on write so " Palm Beach 2026 " === "Palm Beach 2026".

Interests — event tagging via existing tag system:
- Reps can tag interests with an event tag (e.g. "Palm Beach 2026")
  via the existing InlineTagEditor on the detail page; tags are
  port-scoped and reusable.
- Interest list now has a TagPicker filter rendered next to the
  FilterBar so reps can sort prospects by event attended ("show me
  every lead from Palm Beach"). Hidden 'relation'-typed
  FilterDefinition for tagIds wires URL round-trip + saved-views
  capture without rendering inside the FilterBar.
- FilterBar deserializer now handles `relation` types as comma-joined
  arrays on URL load.

Why a free-text trip label and not a trips table:
- 6–12 events/year doesn't justify a domain. The CRUD UI cost would
  be most of the engineering, and reps already have the events on
  their personal calendars.
- If usage proves demand for per-event ROI dashboards or richer
  attribution, promote to a real `trips` table later. Migration
  path: trip_label → tripId is a backfill+swap.

Test status: 1168/1168 vitest. tsc clean. Migration 0039 applied
in dev (also caught + fixed an unrelated audit-v3 follow-up: 0037
had `idx_br_interest` colliding with the existing
`berth_recommendations.idx_br_interest`; renamed to
`idx_brr_interest` / `idx_brr_contract_file`).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-05 13:46:54 +02:00
Matt Ciaccio
687a1f1c2f fix(audit-v3): platform-wide deferred-list cleanup (rounds 1-4) 
Working through the audit-v2 deferred backlog. Each round was tested
(typecheck + 1168/1168 vitest) before moving on.

Round 1 — DB performance + AI cost visibility:
- Add missing FK indexes Postgres doesn't auto-create on
  berth_reservations.{interest_id, contract_file_id},
  documents.{file_id, signed_file_id}, document_events.signer_id,
  document_templates.source_file_id, form_submissions.{form_template_id,
  client_id}, document_sends.{brochure_id, brochure_version_id,
  sent_by_user_id}. Without these, RESTRICT-checks on parent delete +
  reverse-lookups walk the child tables fully. Migration 0037.
- AI worker now writes one ai_usage_ledger row per OpenAI call so admins
  can audit spend per port/user/feature and future per-port budgets have
  history to read from. Failure to write is logged-not-thrown so the
  user-facing email draft is unaffected.

Round 2 — Boot-time + transport hardening:
- S3 backend verifies the bucket exists at startup (or auto-creates
  when MINIO_AUTO_CREATE_BUCKET=true). A typo'd bucket name now
  surfaces with a clear boot error instead of a vague Minio error
  inside the first user-facing request.
- Documenso v1 placeFields: 3-attempt exponential-backoff retry on 5xx
  + network errors, fail-fast on 4xx. Stops one transient flake from
  leaving a document with a partial field set.
- FilesystemBackend logs a structured warn-once at boot when the dev
  HMAC fallback is in effect, so two processes started with different
  BETTER_AUTH_SECRET values are observable (random 401s on file
  downloads otherwise).
- Logger redact paths extended to cover *.headers.{authorization,
  cookie}, *.config.headers.authorization, encrypted-credential blobs
  (secretKeyEncrypted, smtpPassEncrypted, etc.), the Documenso
  X-Documenso-Secret header, and 2-level nested forms.

Round 3 — UI feedback + permission gates:
- Storage admin migrate dialog: success toast with row count + error
  toast on both dryRun and migrate mutations.
- Invoice detail Send + Record-payment buttons wrapped in
  PermissionGate (invoices.send / invoices.record_payment); both
  mutations now toast on success/error.
- Admin user list Edit button wrapped in PermissionGate(admin.manage_users).
- Scan-receipt page surfaces an amber warning when OCR fails so reps
  know they can fill the form manually instead of staring at a stalled
  spinner; the editable form now also opens on scanMutation.isError
  / uploadedFile, not only on success.
- Email threads list now renders skeleton rows during load + shared
  EmptyState for the empty case (was a single "Loading…" line).

Round 4 — Service / route correctness:
- documentSends.sent_by_user_id was a free-text NOT NULL column with no
  FK. Now nullable + FK to user(id) ON DELETE SET NULL so the audit row
  survives a user being hard-deleted. Migration 0038 with a defensive
  null-out for any orphan ids before attaching the constraint.
- Saved-views route: documented why withAuth alone is correct (the
  service strictly filters by (portId, userId) — owner-only by design).
- Public-interests audit log: replaced "userId: null as unknown as
  string" cast with userId: null; AuditLogParams already accepts null
  for system-generated events.
- EOI in-app PDF fill: extracted setBerthRange() that, when the
  AcroForm field is missing AND the context has a non-empty range
  string, logs a structured warn so the deployment gap (live Documenso
  template needs the field) is observable instead of silently dropping
  the multi-berth range.

Test status: 1168/1168 vitest. tsc clean. Two new migrations
(0037/0038) need pnpm db:push (or migration apply) on the dev DB.
Deferred-doc updated with the remaining open items (bigger refactors).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-05 12:49:53 +02:00
Matt Ciaccio
ade4c9e77d fix(audit-v2): platform-wide post-merge hardening across 5 domains 
Five-domain audit (security, routes, DB, integrations, UI/UX) ran after
the cf37d09 merge. Critical + high-impact items landed here; deferred
medium/low items indexed in docs/audit-final-deferred.md (now organised
into a "Audit-final v2" section).

Security:
- Storage proxy tokens now bind to op (`'get'` vs `'put'`). A long-lived
  download URL minted by `presignDownload` for an emailed brochure can no
  longer be replayed against the proxy PUT to overwrite the original
  storage object. `verifyProxyToken` requires `expectedOp` and rejects
  mismatches; legacy tokens missing `op` fail-closed. Regression tests
  added.
- Markdown email merge values are now markdown-escaped (`[`, `]`, `(`,
  `)`, `*`, `_`, `\`, backticks, braces) before substitution into the
  rep-authored body. A malicious value like `[click here](https://evil)`
  stored in `client.fullName` no longer survives `escapeHtml` to render
  as a real `<a href>` in the outbound email. Phishing-via-merge-field
  closed; regression tests added.
- Middleware now performs an Origin/Referer check on
  POST/PUT/PATCH/DELETE to `/api/v1/**`. Defense-in-depth on top of
  better-auth's SameSite=Lax cookie. Webhooks/public/auth/portal routes
  exempt as they don't carry the session cookie.

Routes:
- Template management routes were calling `withPermission('documents',
  'manage', ...)` — but `documents` doesn't have a `manage` action. The
  registry has `document_templates.manage`. Every non-superadmin was
  getting 403'd on the seven template endpoints. Fixed across the
  /admin/templates surface.
- Custom-fields permission resource is hardcoded to `clients` regardless
  of which entity (yacht/company/etc.) the values belong to. Documented
  as deferred (requires per-entity routes).

DB:
- documentSends: every parent FK (client_id, interest_id, berth_id,
  brochure_id, brochure_version_id) now uses ON DELETE SET NULL so the
  audit trail outlasts hard-deletes. The denormalized columns
  (recipient_email, document_kind, body_markdown, from_address) were
  added precisely for this. Migration 0035.
- Polymorphic discriminators on yachts.current_owner_type and
  invoices.billing_entity_type now have CHECK constraints — typos like
  `'clients'` vs `'client'` were silently inserting unreachable rows
  before. Migration 0036.

Integrations:
- Email attachment resolution (`src/lib/email/index.ts`) was importing
  MinIO directly instead of `getStorageBackend()`. Filesystem-backend
  deployments would have broken every email-with-attachment send. Now
  routes through the pluggable abstraction per CLAUDE.md.
- Documenso DOCUMENT_OPENED webhook filter relaxed: v2 may omit
  `readStatus` or send lowercase, so an event that was the SIGNAL of an
  open was being silently dropped. Now treats any recipient on a
  DOCUMENT_OPENED event as opened.

UI/UX:
- Expense detail used to render `receiptFileIds` as opaque UUID badges —
  reps couldn't view the receipt they uploaded. Now renders an image
  thumbnail (via `/api/v1/files/[id]/preview`) plus a Download link for
  PDFs. Closed the "where's my receipt?" loop in the expense flow.
- Expense detail Edit + Archive buttons now `<PermissionGate>` and the
  archive mutation surfaces success/error toasts instead of silent 403s.
- Brochures admin: setDefault/archive/create mutations now have onError
  toasts (only onSuccess existed before).
- Removed broken bulk-upload link in scan/page (route doesn't exist;
  used a raw `<a>` triggering a full reload to a 404).

Test status: 1168/1168 vitest passing. tsc clean.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-05 05:51:39 +02:00
Matt Ciaccio
d4b3a1338f fix(security): scope berth-pdf service entrypoints by portId 
Post-merge security review caught a cross-tenant authorization bypass
in the per-berth PDF endpoints (HIGH severity, confidence 10):

  GET    /api/v1/berths/[id]/pdf-versions
  POST   /api/v1/berths/[id]/pdf-versions
  POST   /api/v1/berths/[id]/pdf-upload-url
  POST   /api/v1/berths/[id]/pdf-versions/[versionId]/rollback
  POST   /api/v1/berths/[id]/pdf-versions/parse-results/apply

Each handler looked up the target berth by id only — `eq(berths.id, ...)`.
withAuth resolves ctx.portId from the user-controlled X-Port-Id header
(only verifying the user has SOME role on that port), and
withPermission('berths', 'view'|'edit', ...) is a coarse capability
check, not a row-level grant. A rep with berths:edit on Port A could
supply a Port B berth UUID and:
- list + receive 15-min presigned download URLs to every PDF version
- mint an upload URL targeting `berths/<port-B-id>/uploads/...`
- POST a new version (overwriting current_pdf_version_id on foreign berth)
- rollback to any prior version on a foreign berth
- apply rep-confirmed parse-result fields onto a foreign berth's columns

Sibling routes (waiting-list etc.) already pair the id filter with
`eq(berths.portId, ctx.portId)`, so this was an omission, not design.

Fix:
- Push `portId: string` into uploadBerthPdf, listBerthPdfVersions,
  rollbackToVersion, applyParseResults, reconcilePdfWithBerth.
- Each function now filters the berth lookup with
  `and(eq(berths.id, ...), eq(berths.portId, portId))` and throws
  NotFoundError on mismatch (no foreign-port disclosure).
- Inline the same `and(...)` filter in the pdf-upload-url handler.
- Every handler passes ctx.portId through.

Coverage:
- New `cross-port tenant guard` test exercises every entrypoint with a
  foreign-port id and asserts NotFoundError.
- 1164/1164 vitest passing. Typecheck clean.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-05 05:31:33 +02:00
Matt Ciaccio
cf37d09519 Merge feat/berth-recommender into main 
Multi-phase work bundle (24 commits, 159 files, ~127k LOC) covering plan
docs/berth-recommender-and-pdf-plan.md:

  Phase 0 — NocoDB berth import + mooring normalization (A-01 → A1)
  Phase 1 — /clients + /interests list-column redesign (contacts/yachts join)
  Phase 2 — M:M interest_berths junction with role flags
  Phase 3 — Public berths API + /api/public/health
  Phase 4 — Berth recommender (SQL ranking, tier ladder, heat scoring)
  Phase 5 — Multi-berth EOI bundle + range formatter
  Phase 6 — Pluggable storage backend + per-berth PDF parser
  Phase 7 — Sales send-outs + brochures + email-from settings
  Phase 8 — CLAUDE.md conventions update

Plus a memory-efficient streaming expense PDF export (replaces a legacy
implementation that OOM'd on hundreds of receipts), receipt-less expense
flag with PDF warning annotations, receipt upload UI in the expense
form dialog, and the scan-receipt page accepting device-uploaded photos
in parallel with the OCR scan.

Four audit passes (audit-1 → audit-final, mostly Opus 4.7 reviewers in
parallel) drove progressive hardening: ~50 findings landed; the last
audit's 5 critical / 12 high items are fixed in 180912b. Medium/low
items are deferred and indexed in docs/audit-final-deferred.md.

Tests: 1163/1163 vitest passing. tsc clean. 12 new migrations applied
in dev (0023..0034), three of which (0028/0029, 0034) involve careful
backfills.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-05 05:12:24 +02:00
Matt Ciaccio
180912ba9f fix(audit-final): pre-merge hardening + expense receipt UI 
Final audit pass on feat/berth-recommender (3 parallel Opus agents)
caught 5 critical and ~12 high-severity findings. All addressed in-branch;
medium/low items deferred to docs/audit-final-deferred.md.

Critical:
- Add filesystem-backend PUT handler at /api/storage/[token] so
  presigned uploads stop 405-ing in filesystem mode (every browser-driven
  berth-PDF + brochure upload was broken). Same token-verify + replay
  protection as GET, plus magic-byte gate when c=application/pdf.
- Forward req.signal into streamExpensePdf so an aborted 1000-receipt
  export no longer keeps grinding for minutes.
- Strengthen Content-Disposition filename sanitization: \s matches CR/LF
  which would let documentName forge headers; restrict to [\w. -]+ and
  add filename* RFC 5987 fallback.
- Lock public berths feed behind an explicit slug allowlist instead of
  ?portSlug= enumeration.
- Reject cross-port interest_berths upserts (defense-in-depth on top of
  the recommender SQL port filter).

High:
- Recommender: width-only feasibility now caps length via L/W ratio so a
  200ft berth doesn't surface for a 30ft beam request; total_interest_count
  filters out junction rows whose interest is in another port.
- Mooring normalization follow-up migration (0034) catches un-hyphenated
  padded forms (A01) the original 0024 WHERE missed.
- Send-out rate limit moved AFTER validation and scoped per-(port, user)
  so typos don't burn a slot and a multi-port rep can't be DoS'd by
  another tenant.
- Default-brochure path now blocks an archived row from sneaking through
  the partial unique index.
- NocoDB import --update-snapshot honoured under --dry-run so reps can
  refresh the seed JSON without committing DB writes.
- PDF export: orderBy desc(expenseDate); apply isNull(archivedAt) when
  expenseIds are passed (was bypassed); flag rate-unavailable rows with
  an amber footer instead of silently treating them as 1:1; skip the
  USD->EUR chain when source already matches target.
- expense-form-dialog: revokeObjectURL captures the URL in the closure
  instead of revoking the still-displayed one; reset upload state on
  close.
- scan/page: handleClearReceipt resets in-flight scan/upload mutations;
  Save disabled while upload pending.
- updateExpense re-asserts receipt-or-acknowledgement at the merged
  row so PATCH can't slip past the create-time refine.

Plus the in-progress receipt upload UI for the expense form dialog
(receipt picker + "I have no receipt" checkbox + warning banner) and
a noReceiptAcknowledged flag on ExpenseRow for edit-mode hydration.

Includes the canonical plan doc (referenced in CLAUDE.md), the handoff
prompt, and a deferred-findings index for follow-up issues.

1163/1163 vitest passing. Typecheck clean.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-05 05:11:26 +02:00
Matt Ciaccio
014bbe1923 feat(expenses): streaming expense-PDF export + receipt-less expense flag + audit-3 fixes 
Replaces the legacy text-only expense PDF (was just dumping rows into a
single pdfme text field — no images, no pagination) with a proper
streaming export modelled on the legacy Nuxt client-portal but
re-architected for memory safety. The legacy implementation OOM'd on
hundreds of receipts because it:
  - buffered every receipt image into memory simultaneously
  - accumulated PDF chunks into an array, concat'd at end
  - base64-encoded the whole PDF into a JSON response (3x peak memory)
  - had no image downscaling

The new design:
  - `streamExpensePdf()` (src/lib/services/expense-pdf.service.ts):
    pdfkit pipes bytes directly to the HTTP response (no Buffer
    accumulation). Receipts are processed serially so peak heap is one
    image at a time. Sharp downscales any receipt > 500 KB or > 1500 px
    to JPEG q80 — typical 8 MB phone photo collapses to ~250 KB. For a
    500-receipt export, peak RSS stays under ~100 MB; legacy needed >2
    GB for the same input.
  - Pages: cover summary box (count, totals, currency equiv, optional
    processing fee), grouped expense table (groupBy=none|payer|category|
    date), one-page-per-receipt with header (establishment, amount,
    date, payer, category, file name) and full-bleed image.
  - Storage backend abstraction — receipts stream from
    `getStorageBackend().get(storageKey)`, works on MinIO/S3/filesystem.
  - Route: POST /api/v1/expenses/export/pdf streams binary
    application/pdf with cache-control:no-store. Validator caps
    expenseIds at 1000 to prevent runaway loops.

Receipt-less expense flow (per user request):
  - Schema: 0033 migration adds `expenses.no_receipt_acknowledged`
    boolean (default false).
  - Validator: createExpenseSchema requires either receiptFileIds OR
    noReceiptAcknowledged=true; the .refine() error message tells the
    rep exactly what to do. updateExpenseSchema is partial and skips
    the rule (existing rows can be edited without re-acknowledging).
  - PDF: receiptless expenses get an inline red "(no receipt)" tag in
    the establishment cell + a red footer warning in the summary box
    showing the count and at-risk amount.
  - The legacy parent-company reimbursement queue may refuse to pay
    receiptless expenses, so the warning is load-bearing for ops.

Audit-3 fixes piggy-backed:
  - 🔴 Tesseract OCR runtime now races a 30s timeout (CPU-bomb DoS
    protection — a crafted PDF rasterizing to high-res noise could
    pin the worker indefinitely).
  - 🟠 brochures.service.ts:listBrochures dropped a wasted query (the
    legacy single-brochure fast-path was discarding its result on the
    multi-brochure branch).
  - 🟠 berth-pdf.service.ts:listBerthPdfVersions now Promise.all's the
    presignDownload calls instead of awaiting each in a for-loop —
    20-version berths went from 20× round-trip to 1×.
  - 🟡 public berths route no longer logs the full `row` object on
    enum drift (was dumping price + amenity columns into ops logs).
  - 🟡 dropped the dead `void sql` import from public berths route.

Tests still 1163/1163. tsc clean.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-05 04:38:32 +02:00
Matt Ciaccio
a3e002852b fix(audit-2): integration regressions + data-integrity from second-pass review 
Two reviewer agents did a second-pass deep audit of the 21-commit
refactor. Eight findings; four fixed here (one was deferred with a
schema comment, three were 🟡 nice-to-haves left for follow-up).

Integration regressions (🟠 high):
- Outbound webhook `interest.berth_linked` now fires from the new
  junction-add handler. Was emitting a socket-only event, leaving
  external integrations silent post-refactor.
- Two new webhook events `interest.berth_unlinked` and
  `interest.berth_link_updated` added to WEBHOOK_EVENTS +
  INTERNAL_TO_WEBHOOK_MAP. PATCH and DELETE handlers now dispatch them
  alongside the existing socket emits — lifecycle parity restored.
- BerthInterestPulse adds useRealtimeInvalidation for berth-link
  events. The query key was berth-scoped while the linked-berths
  dialog invalidates interest-scoped keys (no prefix match), so the
  pulse went stale. Bridges via the realtime hook now.

Recommender semantic fix (🟠 medium-high):
- aggregates CTE: active_interest_count now filters on
  `ib.is_specific_interest = true`, matching the public-map "Under
  Offer" derivation. EOI-bundle-only links no longer demote a berth
  to Tier C for other reps. Smoke test confirms previously-all-Tier-C
  results now correctly classify as Tier A.
- Same CTE: `total_interest_count` uses COUNT(ib.berth_id) instead of
  COUNT(*) so a berth with no junction rows reports 0 (not 1 from
  the LEFT JOIN's NULL-right-side row). Prevents heat over-counting.

Data integrity (🟠):
- AcroForm tier rejects negative numerics in coerceFieldValue (was
  letting through `length_ft="-50"` which would poison the
  recommender feasibility filter on apply).
- FilesystemBackend.resolveHmacSecret throws in production when
  storage_proxy_hmac_secret_encrypted is null. Dev still derives from
  BETTER_AUTH_SECRET for ergonomics; prod must explicitly configure.
- Documented the circular FK between berths.current_pdf_version_id
  and berth_pdf_versions.id. Drizzle's `.references()` can't express
  the cycle so the schema column is plain text + a comment; the FK
  is authoritatively maintained by migration 0030.

Tests still 1163/1163. tsc clean.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-05 04:20:38 +02:00
Matt Ciaccio
312ebf1a88 docs(eoi): document multi-berth Berth Range field + legacy parity 
The user asked us to confirm we copied the Documenso template's
auto-fill schema verbatim from the legacy system. Confirmed and
documented in the canonical mapping file:

- Every legacy formValues key (Name, Email, Address, Yacht Name,
  Length, Width, Draft, Berth Number, Lease_10, Purchase) is still
  emitted with identical names and types — single-berth EOIs are
  byte-for-byte compatible with template id 8.
- Phase 5 added one new field: `Berth Range` (compact range string
  for multi-berth EOIs from the is_in_eoi_bundle junction rows).
  Documenso silently drops unknown formValues, so the live template
  will simply not render the range until someone adds the field. The
  doc now flags this explicitly.
- Verified buildDocumensoPayload() populates all 11 fields from the
  resolved EoiContext; tests at tests/unit/services/documenso-payload
  cover every field.

The "rest is handled inside Documenso" (signature, date, terms) -
those fields live on the template itself and don't appear in our
formValues map.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-05 04:13:32 +02:00
Matt Ciaccio
0b8d08b57e docs(claude): add berth-recommender + storage + send-outs conventions 
Phase 8: capture the new conventions established by the 19-commit
berth-recommender refactor so future Claude sessions don't re-litigate
the design decisions.

Added to the Conventions section:
- Multi-berth interest model + interest_berths role flags
- Mooring number canonical format
- Public berths API + health env-match
- Berth recommender (pure SQL, no AI; tier ladder; heat tunables)
- EOI bundle range formatter
- Pluggable storage backend (filesystem single-node-only constraint)
- Per-berth PDFs (UUID storage keys + advisory lock + 3-tier parser)
- Brochures (default-uniqueness via partial unique index)
- Send-from accounts (encrypted creds, *PassIsSet boolean, XSS guard,
  size-threshold link fallback, 50/hour rate limit)
- NocoDB berth import script

Updated Architecture docs section to note:
- The Documenso template needs the new "Berth Range" field added.
- Pointer to the comprehensive plan doc.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-05 04:09:27 +02:00
Matt Ciaccio
86372a857f fix(audit): post-review hardening across phases 0-7 
15 of 17 findings from the consolidated audit (3 reviewer agents on
the previously-shipped phase commits). Remaining two are nice-to-have
follow-ups deferred.

Critical (data integrity / security):
- Public berths API: closed-deal junction rows no longer flip a berth
  to "Under Offer" - filter on `interests.outcome IS NULL` so won/
  lost/cancelled don't pollute public-map status. Both list +
  single-mooring routes.
- Recommender heat: cancelled outcomes now count as fall-throughs
  (SQL was `LIKE 'lost%'` which silently dropped them, leaving
  cancelled-only berths stuck in tier A).
- Filesystem presignDownload returns an absolute URL (origin from
  APP_URL) so emailed download links resolve from external mail
  clients.
- Magic-byte verification on the presigned-PUT path: both per-berth
  PDFs and brochures stream the first 5 bytes via the storage backend
  and reject + delete on `%PDF-` mismatch (was only enforced when the
  server saw the buffer; presign-PUT was wide open).
- Replay-protection TTL aligned to the token's own expiry (was a
  fixed 30 min, but send-out tokens live 24 h). Floor 60 s, ceiling
  25 days.
- Brochures unique partial index on (port_id) WHERE is_default=true
  + 0032 migration. Closes the read-then-write race in the create/
  update transactions.

Important:
- Recommender SQL: defense-in-depth `i.port_id = $portId` filter on
  the aggregates CTE.
- berth-pdf service: per-berth pg_advisory_xact_lock around the
  version-number SELECT + insert. Storage key is now UUID-based so
  concurrent uploads can't collide on blob paths. Replaces
  `nextVersionNumber` with the tx-bound variant.
- berth-pdf apply: rejects with ConflictError when parse_results
  contain a mooring-mismatch warning unless the caller passes
  `confirmMooringMismatch: true` (force-reconfirm gate was UI-only).
- Send-out body: HTML-escape brochure filename in the download-link
  fallback (XSS guard).
- parseDecimalWithUnit rejects negative numbers.
- listClients DISTINCT ON for primary contact resolution: bounds
  contact-row count to ~2 per client.

Defensive:
- verifyProxyToken rejects NaN/Infinity expiries via Number.isFinite.
- Replaced sql ANY() with inArray() in interest-berths.

Tests: 1145 -> 1163 passing.

Deferred: bulk-send rate limit (no bulk endpoint today), markdown
italic regex breaking links with asterisks (cosmetic).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-05 04:07:03 +02:00
Matt Ciaccio
b4776b4c3c feat(interests): linked berths list with role-flag toggles + EOI bypass 
Implements plan §5.5: a per-interest "Linked berths" panel mounted above the
recommender on the interest detail Overview tab. Each junction row exposes
the role-flag controls reps need to manage the M:M `interest_berths` link
without the legacy single-berth flow.

UI (`src/components/interests/linked-berths-list.tsx`)

* Rows ordered with primary first; mooring number links to /berths/[id], with
  area + a status pill (available/under_offer/sold) and a "Primary" chip.
* "Specifically pitching" Switch (writes `is_specific_interest`) with the
  consequence text from §1: "This berth will appear as under interest on the
  public map" / "This berth is hidden from the public map".
* "Mark in EOI bundle" Switch (writes `is_in_eoi_bundle`).
* "Set as primary" button when the row isn't primary - the existing
  `upsertInterestBerth` helper demotes the prior primary in the same tx.
* "Bypass EOI for this berth" with reason textarea, ONLY rendered when the
  parent interest's `eoiStatus === 'signed'`. Writes the bypass triple
  (`eoi_bypass_reason`, `eoi_bypassed_by` = caller, `eoi_bypassed_at` = now);
  also supports clearing.
* Remove-from-interest action gated by a confirmation dialog.

API (`src/app/api/v1/interests/[id]/berths/...`)

* `GET /` - list endpoint returning `listBerthsForInterest` plus the parent
  interest's `eoiStatus` in `meta.eoiStatus` so the UI can decide whether to
  show the bypass control.
* `PATCH /[berthId]` - partial update of the junction row's flags + bypass
  fields. Server-side guard: rejects bypass writes when `eoiStatus !==
  'signed'` (defence in depth - never trust the UI to gate this).
* `DELETE /[berthId]` - calls `removeInterestBerth`.
* The existing POST stays unchanged. All routes wrapped with
  `withAuth(withPermission('interests', view|edit, ...))`. portId from ctx;
  cross-port reads/writes return 404 for enumeration prevention (§14.10).

Service changes (`src/lib/services/interest-berths.service.ts`)

* `upsertInterestBerth` now accepts `eoiBypassReason` (tri-state: omit = no
  change, non-empty = record, null = clear) and `eoiBypassedBy`. The bypass
  triple moves as a unit, with `eoi_bypassed_at` stamped server-side.
* `listBerthsForInterest` now returns berth detail (area, status, dimensions)
  alongside the junction row, typed as `InterestBerthWithDetails`.

Socket: added `interest:berthLinkUpdated` event for live UI refreshes.

Tests: 18 new integration tests in `tests/integration/api/interest-berths.test.ts`
covering happy paths, primary-demotion in same tx, bypass write/clear, the
"requires signed EOI" guard, cross-port 404s, missing-link 404s, empty-body
400, and viewer 403 through the permission gate.
 2026-05-05 04:01:56 +02:00
Matt Ciaccio
a0091e4ca6 feat(emails): sales send-out flows + brochures + email-from settings 
Phase 7 of the berth-recommender refactor (plan §3.3, §4.8, §4.9, §5.7,
§5.8, §5.9, §11.1, §14.7, §14.9). Adds the rep-driven send-out path for
per-berth PDFs and port-wide brochures, the per-port sales SMTP/IMAP
config + body templates, and the supporting admin UI.

Migration: 0031_brochures_and_document_sends.sql

Schema additions:
  - brochures (port-wide, with isDefault marker + archive)
  - brochure_versions (versioned uploads, storageKey per §4.7a)
  - document_sends (audit log of every rep-initiated send; failures
    captured with failedAt + errorReason). berthPdfVersionId is a plain
    text column (no FK) — loose-coupled to Phase 6b's berth_pdf_versions
    so the two phases stay independent.

§14.7 critical mitigations:
  - Body XSS: rep-authored markdown goes through renderEmailBody()
    (HTML-escape first, then a tight allowlist of bold/italic/code/link
    rules). https:// + mailto: only — javascript:/data: URLs stripped.
    Tested against script/img/iframe/svg/onerror polyglots.
  - Recipient typo: strict email regex + two-step confirm modal that
    shows the exact recipient before send.
  - Unresolved merge fields: pre-send dry-run /preview endpoint blocks
    submission until findUnresolvedTokens() returns empty.
  - SMTP failure: every transport rejection writes a document_sends row
    with failedAt + errorReason; UI surfaces the message.
  - Hourly per-user rate limit: 50 sends/user/hour via existing
    checkRateLimit().
  - Size threshold fallback (§11.1): files above
    email_attach_threshold_mb (default 15) ship as a 24h signed-URL
    download link in the body instead of an attachment. Storage stream
    flows directly to nodemailer to avoid buffering 20MB+.

§14.10 critical mitigation:
  - SMTP/IMAP passwords encrypted at rest via the existing
    EMAIL_CREDENTIAL_KEY (AES-256-GCM). The /api/v1/admin/email/
    sales-config GET endpoint never returns the decrypted value — only
    a *PassIsSet boolean. PATCH treats empty string as "leave unchanged"
    and explicit null as "clear", so the masked-placeholder UI round-
    trips without forcing re-entry on every save.

system_settings keys (per-port unless noted):
  - sales_from_address, sales_smtp_{host,port,secure,user,pass_encrypted}
  - sales_imap_{host,port,user,pass_encrypted}
  - sales_auth_method (default app_password)
  - noreply_from_address
  - email_template_send_berth_pdf_body, email_template_send_brochure_body
  - brochure_max_upload_mb (default 50)
  - email_attach_threshold_mb (default 15)

UI surfaces (per §5.7, §5.8, §5.9):
  - <SendDocumentDialog> shared 2-step compose+confirm flow.
  - <SendBerthPdfDialog>, <SendDocumentsDialog>, <SendFromInterestButton>
    wrappers per detail page.
  - /[portSlug]/admin/brochures: list, upload (direct-to-storage
    presigned PUT for the 20MB+ files per §11.1), default toggle,
    archive.
  - /[portSlug]/admin/email extended with <SalesEmailConfigCard>:
    SMTP + IMAP creds, body templates, threshold/max settings.

Storage: every upload + download goes through getStorageBackend() —
no direct minio imports, per Phase 6a contract.

Tests: 1145 vitest passing (+ 50 new in
markdown-email-sanitization.test.ts, document-sends-validators.test.ts,
sales-email-config-validators.test.ts).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-05 03:38:47 +02:00
Matt Ciaccio
249ffe3e4a feat(berths): per-berth PDF storage (versioned) + reverse parser 
Phase 6b of the berth-recommender refactor (see
docs/berth-recommender-and-pdf-plan.md §3.2, §3.3, §4.7b, §11.1, §14.6).
Builds on the Phase 6a pluggable storage backend (commit 83693dd) — every
file write goes through `getStorageBackend()`; no direct minio imports.

Schema (migration 0030_berth_pdf_versions):
  - new table `berth_pdf_versions` with monotonic `version_number` per
    berth, `storage_key` (renamed convention from §4.7a), sha256, size,
    `download_url_expires_at` cache slot for §11.1 signed-URL throttling,
    and `parse_results` jsonb for the audit trail.
  - new column `berths.current_pdf_version_id` (deferred from Phase 0)
    with FK to `berth_pdf_versions(id)` ON DELETE SET NULL.
  - relations + types exported from `schema/berths.ts`.

3-tier reverse parser (`lib/services/berth-pdf-parser.ts`):
  1. AcroForm via pdf-lib — pulls named fields (`length_ft`,
     `mooring_number`, etc.) at confidence 1. Sample PDF has 0 such
     fields, so this is defensive coverage for future templates.
  2. OCR via Tesseract.js — positional/regex heuristics keyed off the
     §9.2 layout (Length/Width/Water Depth as `<imperial> / <metric>`,
     `WEEK HIGH / LOW`, `CONFIRMED THROUGH UNTIL <date>`, etc.). Returns
     per-field confidence + global mean; flags imperial-vs-metric drift
     >1% in `warnings`.
  3. AI fallback — gated via `getResolvedOcrConfig()` (existing
     openai/claude provider). Surfaced from the diff dialog only when
     `shouldOfferAiTier()` returns true (mean OCR confidence below
     0.55 threshold), so OPENAI_API_KEY isn't burned on every upload.

Service layer (`lib/services/berth-pdf.service.ts`):
  - `uploadBerthPdf()` — magic-byte check, size cap, version-number
    bump + current pointer in one transaction.
  - `reconcilePdfWithBerth()` — auto-applies fields where CRM is null;
    flags conflicts when CRM and PDF disagree; tolerates ±1% on numeric
    columns; warns on mooring-number-in-PDF mismatch (§14.6).
  - `applyParseResults()` — hard allowlist of writable columns;
    stamps `appliedFields` onto `parse_results` for audit.
  - `rollbackToVersion()` — pointer flip only, never re-parses (§14.6).
  - `listBerthPdfVersions()` — version list with 15-min signed URLs.
  - `getMaxUploadMb()` — port-override → global → default 15 lookup
    on `system_settings.berth_pdf_max_upload_mb`.

§14.6 critical mitigations:
  - Magic-byte check (`%PDF-`) on every upload; mismatch deletes the
    storage object and rejects the request.
  - Size cap from `system_settings.berth_pdf_max_upload_mb` (default
    15 MB); enforced in the upload-url presign AND server-side.
  - 0-byte uploads rejected.
  - Mooring-number mismatch surfaces as a `warnings[]` entry on the
    reconcile result so the rep sees it in the diff dialog.
  - Imperial vs metric ±1% tolerance in both the parser warnings and
    the reconcile equality check.
  - Path traversal already blocked at the storage layer (Phase 6a).

API + UI:
  - `POST /api/v1/berths/[id]/pdf-upload-url` — presigned URL (S3) or
    HMAC-signed proxy URL (filesystem) sized to the per-port cap.
  - `POST /api/v1/berths/[id]/pdf-versions` — verifies the upload via
    `backend.head()`, writes the row, bumps `current_pdf_version_id`.
  - `GET /api/v1/berths/[id]/pdf-versions` — version list + signed URLs.
  - `POST /api/v1/berths/[id]/pdf-versions/[versionId]/rollback`.
  - `POST /api/v1/berths/[id]/pdf-versions/parse-results/apply` —
    rep-confirmed diff payload.
  - New "Documents" tab on the berth detail page (`berth-tabs.tsx`)
    with current-PDF panel, version history, Replace PDF button, and
    `<PdfReconcileDialog>` for the auto-applied + conflicts UX.

System settings:
  - `berth_pdf_max_upload_mb` (default 15) — caps presigned-upload size
    + server-side validation. Resolved port-override → global → default.

Tests:
  - `tests/unit/services/berth-pdf-parser.test.ts` — magic bytes,
    feet-inches, human dates, full §9.2-shaped OCR text → 18 fields,
    drift warning, AI-tier gate.
  - `tests/unit/services/berth-pdf-acroform.test.ts` — synthetic
    pdf-lib AcroForm round-trip.
  - `tests/integration/berth-pdf-versions.test.ts` — upload, version-
    number bump, magic-byte rejection, reconcile auto-applied vs
    conflicts vs ±1% tolerance, mooring-number warning,
    applyParseResults allowlist enforcement, rollback semantics.

Acceptance: `pnpm exec tsc --noEmit` clean, `pnpm exec vitest run`
green at 1103/1103.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-05 03:34:24 +02:00
Matt Ciaccio
83693dd993 feat(storage): pluggable s3-or-filesystem backend + migration CLI + admin UI 
Phase 6a from docs/berth-recommender-and-pdf-plan.md §4.7a + §14.9a. Lays
the storage groundwork for Phase 6b/7 file-bearing schemas (per-berth PDFs,
brochures) without touching those domains yet.

New files:
- src/lib/storage/index.ts        StorageBackend interface + per-process
                                  factory keyed on system_settings.
- src/lib/storage/s3.ts           S3-compatible backend (MinIO/AWS/B2/R2/
                                  Wasabi/Tigris) wrapping the existing minio
                                  JS client. Includes a healthCheck() used
                                  by the admin "Test connection" button.
- src/lib/storage/filesystem.ts   Local filesystem backend with all §14.9a
                                  mitigations baked in.
- src/lib/storage/migrate.ts      Shared migration core — pg_advisory_lock,
                                  per-row resumable progress markers,
                                  sha256 round-trip verification, atomic
                                  storage_backend flip on success.
- scripts/migrate-storage.ts      Thin CLI shim around runMigration().
- src/app/api/storage/[token]/route.ts
                                  Filesystem proxy GET. Verifies HMAC,
                                  enforces single-use replay protection
                                  via Redis SET NX, streams via NextResponse
                                  ReadableStream with explicit Content-Type
                                  + Content-Disposition. Node runtime only.
- src/app/api/v1/admin/storage/route.ts
                                  GET status + POST connection test.
- src/app/api/v1/admin/storage/migrate/route.ts
                                  Super-admin-only POST that runs the
                                  exact same runMigration() as the CLI.
- src/app/(dashboard)/[portSlug]/admin/storage/page.tsx
                                  Super-admin admin UI (current backend,
                                  capacity stats, switch button with
                                  dry-run, test connection, backup hint).
- src/components/admin/storage-admin-panel.tsx
                                  Client component for the page above.

§14.9a critical mitigations implemented:
- Path-traversal: storage keys validated against ^[a-zA-Z0-9/_.-]+$;
  `..`, `.`, `//`, leading `/`, and overlength keys rejected.
- Realpath: storage root realpath'd at create time, every per-key
  resolution checked against the realpath'd prefix.
- Storage root created (or chmod'd) to 0o700.
- Multi-node refusal: FilesystemBackend.create() throws when
  MULTI_NODE_DEPLOYMENT=true.
- HMAC token: sha256-HMAC over the (key, expiry, nonce, filename,
  content-type) payload. Verified with timingSafeEqual; bad sig,
  expired, or invalid-key payloads all return 403.
- Single-use replay: token body cached in Redis SET NX EX 1800s.
- sha256 round-trip: copyAndVerify() re-fetches from the target after
  put() and aborts the migration on any mismatch.
- Free-disk pre-flight: when migrating to filesystem, sums byte counts
  via source.head() and aborts if free space < total * 1.2.
- pg_advisory_lock(0xc7000a01) prevents concurrent migrations.
- Resumable: per-row progress markers in _storage_migration_progress.

system_settings keys read by the factory (jsonb, no schema change):
storage_backend, storage_s3_endpoint, storage_s3_region,
storage_s3_bucket, storage_s3_access_key,
storage_s3_secret_key_encrypted, storage_s3_force_path_style,
storage_filesystem_root, storage_proxy_hmac_secret_encrypted.

Defaults: storage_backend=`s3`, storage_filesystem_root=`./storage`
(./storage added to .gitignore).

Tests added (34 tests, all green):
- tests/unit/storage/filesystem-backend.test.ts — key validation
  allow/reject matrix, realpath escape, 0o700 perms, multi-node
  refusal, HMAC token sign/verify/tamper/expire/invalid-key.
- tests/unit/storage/copy-and-verify.test.ts — sha256 mismatch on
  round-trip aborts the migration.
- tests/integration/storage/proxy-route.test.ts — happy path, wrong
  HMAC secret, expired token, replay rejection.

Phase 6a ships zero file-bearing tables — TABLES_WITH_STORAGE_KEYS is
intentionally empty. berth_pdf_versions and brochure_versions land in
Phase 6b and join the list there. Existing s3_key columns: only
gdpr_export_jobs.storage_key, already named correctly — no rename needed.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-05 03:15:59 +02:00
Matt Ciaccio
15d4849030 feat(recommender): API endpoint + interest-detail panel + add-to-interest dialog 2026-05-05 03:05:22 +02:00
Matt Ciaccio
e00e812199 feat(eoi): multi-berth EOI generation + berth-range formatter 
Plan §4.6 + §1: a render function that compresses every berth marked
is_in_eoi_bundle=true on an interest into a compact range string
("A1-A3, B5-B7"), wired into both EOI generation paths (the Documenso
template-generate call and the in-app pdf-lib AcroForm fill).

- src/lib/templates/berth-range.ts: pure formatBerthRange() with the
  full edge-case set from §4.6 - empty, single, run, gap, multiple
  prefixes, sort/dedup, multi-letter prefixes, non-canonical
  passthrough, long ranges. Sorts by (prefix, number); dedupes; passes
  non-canonical inputs through with a logger warning.
- src/lib/templates/merge-fields.ts: new {{eoi.berthRange}} token
  added to VALID_MERGE_TOKENS allow-list under a fresh `eoi` scope so
  unknown-token validation at template creation time still rejects
  typos.
- src/lib/services/eoi-context.ts: EoiContext gains eoiBerthRange.
  Resolved by joining interest_berths (is_in_eoi_bundle=true) →
  berths and feeding the mooring numbers through formatBerthRange.
- src/lib/services/documenso-payload.ts: formValues now includes
  "Berth Range" alongside the legacy "Berth Number". Multi-berth EOIs
  surface here; single-berth EOIs duplicate the primary.
- src/lib/pdf/fill-eoi-form.ts: in-app AcroForm fill mirrors the
  Documenso payload by populating "Berth Range". Falls back silently
  when older PDFs don't have the field (setText is no-op-on-missing).

15 unit tests on the formatter; existing EoiContext + Documenso
payload tests updated to assert the new field. 1022 -> 1037 passing.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-05 03:03:29 +02:00
Matt Ciaccio
b1e787e55c feat(recommender): SQL ranking + tier ladder + heat scoring 
Plan §4.4 + §13: pure SQL recommender, no AI. Single CTE chain
(feasible -> aggregates) + JS-side tier classification, fall-through
cooldown filter, heat scoring, and fit ranking. Per-port settings via
system_settings layered over global + DEFAULT_RECOMMENDER_SETTINGS.

Tier ladder (default):
  A : no interest history
  B : lost-only history (still recommendable + boosted by heat)
  C : active interest in early stage (open..eoi_signed)
  D : active interest at deposit_10pct or beyond (hidden by default)

Heat (only for tier B):
  recency        weight 30  full @ <=30 days, decays to 0 @ 365 days
  furthest stage weight 40  full when prior reached deposit
  interest count weight 15  saturates at 5+
  EOI count      weight 15  saturates at 3+

Multi-port isolation enforced (§14.10 critical): the SQL filters by
port_id AND the entry-point function rejects cross-port interest
lookups with an explicit error. Fall-through policy supports
immediate_with_heat (default), cooldown, and never_auto_recommend.

15 unit tests covering tier classification, heat saturation, weight
tuning, zero-weight guard. Smoke-tested end-to-end via
scripts/dev-recommender-smoke.ts.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-05 02:58:34 +02:00
Matt Ciaccio
fb1116f1d4 feat(berths): public berths API + health env-match endpoint 
Adds the read-only public-website data feed promised by plan §4.5 and
§7.3. The marketing site's `getBerths()` swap is now a one-line URL
change against the existing 5-min TTL behaviour.

- src/app/api/public/berths/route.ts: GET / unauth, returns the full
  port-nimara berth list as { list, pageInfo } in the verbatim NocoDB
  shape ("Mooring Number", "Side Pontoon", quoted-key fields). Cache:
  s-maxage=300 + stale-while-revalidate=60. portSlug query param lets
  future ports opt in.
- src/app/api/public/berths/[mooringNumber]/route.ts: GET single. Up-
  front regex validation (^[A-Z]+\\d+$) rejects malformed lookups with
  400 + cache-control:no-store before hitting the DB. 404 + no-store
  when not found.
- src/app/api/public/health/route.ts: returns { status, env, appUrl,
  timestamp } so the marketing site can refuse to start when its
  CRM_PUBLIC_URL points at a different deployment env (§14.8 critical
  env-mismatch protection).
- src/lib/services/public-berths.ts: pure mapper with derivePublicStatus
  ("sold" wins; otherwise specific-interest junction OR
  status='under_offer' -> "Under Offer"; else "Available").
- 11 unit tests covering numeric coercion, status derivation,
  archived-berth handling, missing-map-data omission, and the
  status-precedence rule that "sold" trumps the specific-interest
  signal.

Smoke-tested: /api/public/berths -> 117 rows, A1 correctly shows
"Under Offer" (has interest_berths.is_specific_interest=true link),
INVALID -> 400, Z99 -> 404. Total tests: 996 -> 1007.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-05 02:52:44 +02:00
Matt Ciaccio
5b70e9b04b feat(interests): desired-dimension form fields + size-desired column 
Surfaces the recommender inputs added in Phase 2a (interests
.desired_length_ft / desired_width_ft / desired_draft_ft) on the
two interfaces reps actually use:

- /interests list: new "Berth size desired" column rendered as a
  compact "60×18×6 ft" string. Cells with no dimensions show "-";
  partial dimensions render "?" for the missing axis (recommender
  treats null as "no constraint").
- New/Edit Interest form: three optional length/width/draft inputs
  with explanatory subhead. Empty submissions collapse to undefined
  so the API doesn't see "" -> numeric coercion errors.
- createInterestSchema gains the three optional desired-dim fields
  with a shared transform that coerces strings/numbers to a positive
  2-decimal numeric string for the postgres `numeric` column.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-05 02:49:01 +02:00
Matt Ciaccio
57cbc9a506 fix(tests): cascade interest_berths in global teardown 
The Phase 2b refactor (commit 6e3d910) added a junction table whose
berth_id has onDelete: 'restrict'. The vitest global teardown deletes
test-port berths but never explicitly clears interest_berths first,
so any test leaking junction rows (e.g. via the new createInterest
write path) leaves berths un-deletable and ports stranded.

Adds DELETE FROM interest_berths WHERE berth_id IN (test berths) to
the WITH-chain so cascading teardown completes cleanly.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-05 02:45:45 +02:00
Matt Ciaccio
6e3d910c76 refactor(interests): migrate callers to interest_berths junction + drop berth_id 
Phase 2b of the berth-recommender refactor (plan §3.4). Every caller of
the legacy `interests.berth_id` column now reads / writes through the
`interest_berths` junction via the helper service introduced in Phase 2a;
the column itself is dropped in a final migration.

Service-layer changes
- interests.service: filter `?berthId=X` becomes EXISTS-against-junction;
  list enrichment uses `getPrimaryBerthsForInterests`; create/update/
  linkBerth/unlinkBerth all dispatch through the junction helpers, with
  createInterest's row insert + junction write sharing a single transaction.
- clients / dashboard / report-generators / search: leftJoin chains pivot
  through `interest_berths` filtered by `is_primary=true`.
- eoi-context / document-templates / berth-rules-engine / portal /
  record-export / queue worker: read primary via `getPrimaryBerth(...)`.
- interest-scoring: berthLinked is now derived from any junction row count.
- dedup/migration-apply + public interest route: write a primary junction
  row alongside the interest insert when a berth is provided.

API contract preserved: list/detail responses still emit `berthId` and
`berthMooringNumber`, derived from the primary junction row, so frontend
consumers (interest-form, interest-detail-header) need no changes.

Schema + migration
- Drop `interestsRelations.berth` and `idx_interests_berth`.
- Replace `berthsRelations.interests` with `interestBerths`.
- Migration 0029_puzzling_romulus drops `interests.berth_id` + the index.
- Tests that previously inserted `interests.berthId` now seed a primary
  junction row alongside the interest.

Verified: vitest 995 passing (1 unrelated pre-existing flake in
maintenance-cleanup.test.ts), tsc clean.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-05 02:41:52 +02:00
Matt Ciaccio
ff92a08620 feat(db): m:m interest_berths junction + role flags 
Introduces the multi-berth interest model from plan §3.1: a junction
between interests and berths with three role flags so the same berth
can be linked as the primary deal target, an EOI-bundle inclusion,
or a "just exploring" link without conflating semantics.

- 0028 schema migration creates interest_berths with the unique
  partial index "≤1 primary per interest", a unique compound on
  (interest_id, berth_id), and indexes for the public-map "under
  offer" lookup (where is_specific_interest=true).
- Same migration adds desired_length_ft / desired_width_ft /
  desired_draft_ft to interests for the recommender.
- Same migration runs the Phase 2 data migration: every interest
  with a non-null berth_id gets one junction row marked
  is_primary=true, is_specific_interest=true, and is_in_eoi_bundle =
  (eoi_status='signed'). Pre-flight check halts on dangling FKs
  (§14.3 critical case).
- New service src/lib/services/interest-berths.service.ts owns reads
  + writes of the junction. getPrimaryBerth / getPrimaryBerthsForInterests
  feed list pages; upsertInterestBerth demotes the prior primary in
  the same transaction so the unique index is never violated.
- interests.berth_id stays in place this commit so existing callers
  keep working; Phase 2b migrates them onto the helper service and a
  later migration drops the column.

53 dev rows seeded into the junction; tests still green at 996.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-05 02:22:11 +02:00
Matt Ciaccio
05257723f6 fix(interests): list yacht join + EOI status column + col redesign 
Wire interests.yachtId -> yachts.name into the listInterests post-fetch
enrichment so the redesigned columns (Client · Yacht · Berth · Stage ·
EOI status · Source · Last activity) render the linked yacht.

- Add yachtId/yachtName to InterestRow.
- listInterests: fourth parallel join for yachts.name, Map merged
  alongside the existing client/berth/tag/notes joins.
- interest-columns: add Yacht column (with link to /yachts/[id] when
  the yacht has an id); replace Category with EOI status (badge
  driven by interests.eoi_status); drop default-view Tags.

The "Berth size desired" column called out in §5.2 is deferred to
Phase 2 since the underlying desired_*_ft columns don't exist yet.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-05 02:18:13 +02:00
Matt Ciaccio
3017ce4b3a fix(clients): list contacts join + nationality backfill + col redesign 
Wire primary email + primary phone into the /clients list service so
the redesigned columns (Name · Email · Phone · Country · Source ·
Latest stage · Created) actually have data. Picks the row marked
is_primary=true; falls back to most-recent created_at when the flag
is unset.

- 0026 schema migration: unique partial index
  idx_cc_one_primary_per_channel on (client_id, channel) WHERE
  is_primary=true. Prevents the §14.2 "multiple primaries" ambiguity.
- 0027 data migration: backfill clients.nationality_iso from the
  primary phone's value_country. 218 -> 36 missing on dev. Idempotent.
- listClients: add a fifth parallel query for client_contacts; build
  primaryEmailMap / primaryPhoneMap in-memory from the pre-sorted
  result.
- client-columns: drop Yachts/Companies/Tags from the default view
  per §5.1; add Email/Phone/Country/Latest-stage columns; rename
  "Nationality" -> "Country" since phone country is a proxy (§14.2).
- client-card: prefer email, fall back to phone, for the line under
  the name; replaces the old `contacts.find(isPrimary)` lookup.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-05 02:15:03 +02:00
Matt Ciaccio
a2588f2c4a chore(berths): refresh seed-data/berths.json from live NocoDB 
Regenerates the 117-row berth seed via:
  pnpm tsx scripts/import-berths-from-nocodb.ts --apply --update-snapshot

The JSON ordering matches the legacy seed-data.ts contract (idx 0..4
available, 5..9 under_offer, 10..11 sold, remainder by mooring number).
Mooring numbers are now in canonical form throughout.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-05 02:08:53 +02:00
Matt Ciaccio
18119644ae feat(berths): nocodb berth import script + helpers + unit tests 
Idempotent NocoDB Berths -> CRM `berths` import script with full
re-run safety. Re-running picks up NocoDB additions/edits without
clobbering CRM-side overrides (compares updated_at vs last_imported_at,
1-second tolerance for sub-second clock drift). --force overrides the
edit guard.

Mitigates the §14.1 critical/high cases:
- Mooring collisions: unique (port_id, mooring_number) on the table.
- Concurrent runs: pg_advisory_xact_lock on a stable BIGINT key.
- Numeric-with-units inputs: parseDecimalWithUnit() strips trailing
  ft/m/kw/v/usd/$ markers before parsing.
- Metric drift: NocoDB's metric formula columns are ignored; metric
  values recomputed from imperial via 0.3048 + round-to-2-decimals to
  match NocoDB's `precision: 2` columns and avoid spurious diffs.
- Map Data shape: zod-validated; failures are skipped rather than
  aborting the import.
- Status enum mapping: NocoDB display strings -> CRM snake_case.
- NocoDB row deleted: reported as "orphaned in CRM"; never auto-
  deleted (rep decides via admin UI in a future phase).

Pure helpers (parseDecimalWithUnit, mapStatus, parseMapData,
extractNumerics, mapRow, buildPlan) live in
src/lib/services/berth-import.ts so vitest can exercise the mapping
logic without triggering the script's top-level db connection.

40 new unit tests (956 -> 996 passing).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-05 02:07:58 +02:00
Matt Ciaccio
61e2fbb2db feat(berths): add per-berth pricing + last_imported_at columns 
Adds the 5 pricing columns surfaced by the per-berth PDFs (Phase 6b
will populate them via the OCR parser) and the last_imported_at marker
the NocoDB import script (Phase 0c) uses to detect human edits and
skip overwriting them.

- weekly_rate_high_usd / weekly_rate_low_usd
- daily_rate_high_usd  / daily_rate_low_usd
- pricing_valid_until (date) - drives the "stale pricing" chip on
  the berth detail page when older than today
- last_imported_at - compared against updated_at so re-running the
  import preserves CRM-side overrides

tenure_type comment widens to include 'fee_simple' and 'strata_lot'
to match the per-berth PDF tenure model; the column is plain text
so no DB-level enum change is required.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-05 02:00:46 +02:00
Matt Ciaccio
05be89ec6f feat(berths): normalize mooring numbers to canonical form 
Sweep CRM mooring numbers from the legacy hyphen+padded form ("A-01")
to the canonical bare form ("A1") used by NocoDB, the public website,
the per-berth PDFs, and the Documenso EOI templates. Drift was
introduced by the original load-berths-to-port-nimara.ts seed; this
gates the Phase 3 public-website cutover where /berths/A1 URLs would
404 against a CRM still storing "A-01".

- 0024 data migration: idempotent regexp_replace + post-update sanity
  check that surfaces any non-conforming rows for manual triage.
- Invert normalizeLegacyMooring in dedup/migration-apply: it now
  canonicalizes ("D-32" -> "D32") instead of legacy-izing.
- Update tiptap-to-pdfme example tokens, EOI fixture moorings, and
  smoke-test seed moorings.
- Refresh seed-data/berths.json to canonical form; drop the now-
  redundant legacyMooringNumber field.
- Delete scripts/load-berths-to-port-nimara.ts (superseded in 0c).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-05 01:59:26 +02:00
Matt Ciaccio
8699f81879 chore(style): codebase em-dash sweep + minor layout polish
Some checks failed
Build & Push Docker Images / lint (push) Failing after 1m18s
Details
Build & Push Docker Images / build-and-push (push) Has been skipped
Details 
Replaces every em-dash and en-dash with regular ASCII hyphens
across comments, JSX strings, and dev-facing logs. Mostly cosmetic
but stops the inconsistent mix that crept in over the last few
months (some files used em-dashes in comments, others didn't,
some used both).

Bundles two small dashboard-layout tweaks that touch a couple of
already-modified files:
- (dashboard)/layout.tsx main padding goes from p-6 to pt-3 px-6
  pb-6 so page content sits closer to the topbar.
- Sidebar now receives the ports list it needs for the footer
  port switcher.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-04 22:57:01 +02:00
Matt Ciaccio
d62822c284 fix(migration): NocoDB import safety + dedup helpers + lead-source backfill 
migration-apply: residential client + interest inserts now wrap in
db.transaction so a partial failure can't leave an orphan client
row without its interest (or vice versa).

migration-transform: buildPlannedDocument returns null when there
are no signers so the apply pass doesn't try to send a Documenso
envelope without recipients. mapDocumentStatus gets an explicit
"Awaiting Further Details" branch that no longer auto-promotes via
stale sign-time fields. parseFlexibleDate handles ISO and DD-MM-YYYY
inputs uniformly.

backfill-legacy-lead-source: chunk UPDATE WHERE clause now
isNull(source) on top of the inArray match, so a re-run can't
overwrite a more accurate source written between batches.

Adds 235 lines of vitest coverage on migration-transform.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-04 22:56:18 +02:00
Matt Ciaccio
089f4a67a4 feat(receipts): upload guide page + scanner head-tag fix 
Adds /invoices/upload-receipts as the dedicated explainer for the
mobile scanner PWA: install instructions for iOS/Android, direct
deep-link button, and a walkthrough of the scan -> verify -> save
flow. Sidebar entry replaces the old "Scan receipt" tab so the
desktop side picks up the install steps before sending users to
the mobile-only surface.

Scanner layout moves PWA manifest + apple-* meta tags from inline
JSX into Next.js's metadata/viewport exports so the App Router
doesn't try to render a second <head>, fixing a hydration error
that surfaced as two console warnings on the scan page.

Scanner shell gains a centered Port Nimara logo header so the
standalone PWA looks branded when launched from the home screen
without the dashboard chrome.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-04 22:55:42 +02:00
Matt Ciaccio
77ad10ced1 feat(dashboard): custom date range + KPI port-hydration gate 
DateRangePicker grows a "Custom range" mode (From/To inputs capped
at today, mutually-bounded so From <= To). dashboard-shell threads
the range through to /api/v1/analytics, which validates calendar
dates via ISO round-trip and enforces a 365-day cap as a backstop
against the occupancy timeline N+1.

KpiCards now gates its query on currentPortId so the early
unhydrated-store fetch can't cache a zeroed/error response and
display "-" until staleTime expires.

MyRemindersRail drops xl:h-full so the rail no longer stretches
past its grid row and overlaps ActivityFeed below.

useRealtimeInvalidation switches to partial-prefix queryKeys so a
realtime mutation invalidates every cached range bucket at once
instead of just the one currently visible.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-04 22:54:55 +02:00