letsbe/pn-new-crm
1
0
Fork 0
Code Issues Pull Requests 1 Actions Packages Projects Releases Wiki Activity
300 Commits 7 Branches 0 Tags
c4a41d5f5b8e3ec4d9f2c68ed18db55a4fa7a40c
Commit Graph

99 Commits

Author SHA1 Message Date
Matt Ciaccio
ade4c9e77d fix(audit-v2): platform-wide post-merge hardening across 5 domains 
Five-domain audit (security, routes, DB, integrations, UI/UX) ran after
the cf37d09 merge. Critical + high-impact items landed here; deferred
medium/low items indexed in docs/audit-final-deferred.md (now organised
into a "Audit-final v2" section).

Security:
- Storage proxy tokens now bind to op (`'get'` vs `'put'`). A long-lived
  download URL minted by `presignDownload` for an emailed brochure can no
  longer be replayed against the proxy PUT to overwrite the original
  storage object. `verifyProxyToken` requires `expectedOp` and rejects
  mismatches; legacy tokens missing `op` fail-closed. Regression tests
  added.
- Markdown email merge values are now markdown-escaped (`[`, `]`, `(`,
  `)`, `*`, `_`, `\`, backticks, braces) before substitution into the
  rep-authored body. A malicious value like `[click here](https://evil)`
  stored in `client.fullName` no longer survives `escapeHtml` to render
  as a real `<a href>` in the outbound email. Phishing-via-merge-field
  closed; regression tests added.
- Middleware now performs an Origin/Referer check on
  POST/PUT/PATCH/DELETE to `/api/v1/**`. Defense-in-depth on top of
  better-auth's SameSite=Lax cookie. Webhooks/public/auth/portal routes
  exempt as they don't carry the session cookie.

Routes:
- Template management routes were calling `withPermission('documents',
  'manage', ...)` — but `documents` doesn't have a `manage` action. The
  registry has `document_templates.manage`. Every non-superadmin was
  getting 403'd on the seven template endpoints. Fixed across the
  /admin/templates surface.
- Custom-fields permission resource is hardcoded to `clients` regardless
  of which entity (yacht/company/etc.) the values belong to. Documented
  as deferred (requires per-entity routes).

DB:
- documentSends: every parent FK (client_id, interest_id, berth_id,
  brochure_id, brochure_version_id) now uses ON DELETE SET NULL so the
  audit trail outlasts hard-deletes. The denormalized columns
  (recipient_email, document_kind, body_markdown, from_address) were
  added precisely for this. Migration 0035.
- Polymorphic discriminators on yachts.current_owner_type and
  invoices.billing_entity_type now have CHECK constraints — typos like
  `'clients'` vs `'client'` were silently inserting unreachable rows
  before. Migration 0036.

Integrations:
- Email attachment resolution (`src/lib/email/index.ts`) was importing
  MinIO directly instead of `getStorageBackend()`. Filesystem-backend
  deployments would have broken every email-with-attachment send. Now
  routes through the pluggable abstraction per CLAUDE.md.
- Documenso DOCUMENT_OPENED webhook filter relaxed: v2 may omit
  `readStatus` or send lowercase, so an event that was the SIGNAL of an
  open was being silently dropped. Now treats any recipient on a
  DOCUMENT_OPENED event as opened.

UI/UX:
- Expense detail used to render `receiptFileIds` as opaque UUID badges —
  reps couldn't view the receipt they uploaded. Now renders an image
  thumbnail (via `/api/v1/files/[id]/preview`) plus a Download link for
  PDFs. Closed the "where's my receipt?" loop in the expense flow.
- Expense detail Edit + Archive buttons now `<PermissionGate>` and the
  archive mutation surfaces success/error toasts instead of silent 403s.
- Brochures admin: setDefault/archive/create mutations now have onError
  toasts (only onSuccess existed before).
- Removed broken bulk-upload link in scan/page (route doesn't exist;
  used a raw `<a>` triggering a full reload to a 404).

Test status: 1168/1168 vitest passing. tsc clean.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-05 05:51:39 +02:00
Matt Ciaccio
d4b3a1338f fix(security): scope berth-pdf service entrypoints by portId 
Post-merge security review caught a cross-tenant authorization bypass
in the per-berth PDF endpoints (HIGH severity, confidence 10):

  GET    /api/v1/berths/[id]/pdf-versions
  POST   /api/v1/berths/[id]/pdf-versions
  POST   /api/v1/berths/[id]/pdf-upload-url
  POST   /api/v1/berths/[id]/pdf-versions/[versionId]/rollback
  POST   /api/v1/berths/[id]/pdf-versions/parse-results/apply

Each handler looked up the target berth by id only — `eq(berths.id, ...)`.
withAuth resolves ctx.portId from the user-controlled X-Port-Id header
(only verifying the user has SOME role on that port), and
withPermission('berths', 'view'|'edit', ...) is a coarse capability
check, not a row-level grant. A rep with berths:edit on Port A could
supply a Port B berth UUID and:
- list + receive 15-min presigned download URLs to every PDF version
- mint an upload URL targeting `berths/<port-B-id>/uploads/...`
- POST a new version (overwriting current_pdf_version_id on foreign berth)
- rollback to any prior version on a foreign berth
- apply rep-confirmed parse-result fields onto a foreign berth's columns

Sibling routes (waiting-list etc.) already pair the id filter with
`eq(berths.portId, ctx.portId)`, so this was an omission, not design.

Fix:
- Push `portId: string` into uploadBerthPdf, listBerthPdfVersions,
  rollbackToVersion, applyParseResults, reconcilePdfWithBerth.
- Each function now filters the berth lookup with
  `and(eq(berths.id, ...), eq(berths.portId, portId))` and throws
  NotFoundError on mismatch (no foreign-port disclosure).
- Inline the same `and(...)` filter in the pdf-upload-url handler.
- Every handler passes ctx.portId through.

Coverage:
- New `cross-port tenant guard` test exercises every entrypoint with a
  foreign-port id and asserts NotFoundError.
- 1164/1164 vitest passing. Typecheck clean.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-05 05:31:33 +02:00
Matt Ciaccio
86372a857f fix(audit): post-review hardening across phases 0-7 
15 of 17 findings from the consolidated audit (3 reviewer agents on
the previously-shipped phase commits). Remaining two are nice-to-have
follow-ups deferred.

Critical (data integrity / security):
- Public berths API: closed-deal junction rows no longer flip a berth
  to "Under Offer" - filter on `interests.outcome IS NULL` so won/
  lost/cancelled don't pollute public-map status. Both list +
  single-mooring routes.
- Recommender heat: cancelled outcomes now count as fall-throughs
  (SQL was `LIKE 'lost%'` which silently dropped them, leaving
  cancelled-only berths stuck in tier A).
- Filesystem presignDownload returns an absolute URL (origin from
  APP_URL) so emailed download links resolve from external mail
  clients.
- Magic-byte verification on the presigned-PUT path: both per-berth
  PDFs and brochures stream the first 5 bytes via the storage backend
  and reject + delete on `%PDF-` mismatch (was only enforced when the
  server saw the buffer; presign-PUT was wide open).
- Replay-protection TTL aligned to the token's own expiry (was a
  fixed 30 min, but send-out tokens live 24 h). Floor 60 s, ceiling
  25 days.
- Brochures unique partial index on (port_id) WHERE is_default=true
  + 0032 migration. Closes the read-then-write race in the create/
  update transactions.

Important:
- Recommender SQL: defense-in-depth `i.port_id = $portId` filter on
  the aggregates CTE.
- berth-pdf service: per-berth pg_advisory_xact_lock around the
  version-number SELECT + insert. Storage key is now UUID-based so
  concurrent uploads can't collide on blob paths. Replaces
  `nextVersionNumber` with the tx-bound variant.
- berth-pdf apply: rejects with ConflictError when parse_results
  contain a mooring-mismatch warning unless the caller passes
  `confirmMooringMismatch: true` (force-reconfirm gate was UI-only).
- Send-out body: HTML-escape brochure filename in the download-link
  fallback (XSS guard).
- parseDecimalWithUnit rejects negative numbers.
- listClients DISTINCT ON for primary contact resolution: bounds
  contact-row count to ~2 per client.

Defensive:
- verifyProxyToken rejects NaN/Infinity expiries via Number.isFinite.
- Replaced sql ANY() with inArray() in interest-berths.

Tests: 1145 -> 1163 passing.

Deferred: bulk-send rate limit (no bulk endpoint today), markdown
italic regex breaking links with asterisks (cosmetic).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-05 04:07:03 +02:00
Matt Ciaccio
b4776b4c3c feat(interests): linked berths list with role-flag toggles + EOI bypass 
Implements plan §5.5: a per-interest "Linked berths" panel mounted above the
recommender on the interest detail Overview tab. Each junction row exposes
the role-flag controls reps need to manage the M:M `interest_berths` link
without the legacy single-berth flow.

UI (`src/components/interests/linked-berths-list.tsx`)

* Rows ordered with primary first; mooring number links to /berths/[id], with
  area + a status pill (available/under_offer/sold) and a "Primary" chip.
* "Specifically pitching" Switch (writes `is_specific_interest`) with the
  consequence text from §1: "This berth will appear as under interest on the
  public map" / "This berth is hidden from the public map".
* "Mark in EOI bundle" Switch (writes `is_in_eoi_bundle`).
* "Set as primary" button when the row isn't primary - the existing
  `upsertInterestBerth` helper demotes the prior primary in the same tx.
* "Bypass EOI for this berth" with reason textarea, ONLY rendered when the
  parent interest's `eoiStatus === 'signed'`. Writes the bypass triple
  (`eoi_bypass_reason`, `eoi_bypassed_by` = caller, `eoi_bypassed_at` = now);
  also supports clearing.
* Remove-from-interest action gated by a confirmation dialog.

API (`src/app/api/v1/interests/[id]/berths/...`)

* `GET /` - list endpoint returning `listBerthsForInterest` plus the parent
  interest's `eoiStatus` in `meta.eoiStatus` so the UI can decide whether to
  show the bypass control.
* `PATCH /[berthId]` - partial update of the junction row's flags + bypass
  fields. Server-side guard: rejects bypass writes when `eoiStatus !==
  'signed'` (defence in depth - never trust the UI to gate this).
* `DELETE /[berthId]` - calls `removeInterestBerth`.
* The existing POST stays unchanged. All routes wrapped with
  `withAuth(withPermission('interests', view|edit, ...))`. portId from ctx;
  cross-port reads/writes return 404 for enumeration prevention (§14.10).

Service changes (`src/lib/services/interest-berths.service.ts`)

* `upsertInterestBerth` now accepts `eoiBypassReason` (tri-state: omit = no
  change, non-empty = record, null = clear) and `eoiBypassedBy`. The bypass
  triple moves as a unit, with `eoi_bypassed_at` stamped server-side.
* `listBerthsForInterest` now returns berth detail (area, status, dimensions)
  alongside the junction row, typed as `InterestBerthWithDetails`.

Socket: added `interest:berthLinkUpdated` event for live UI refreshes.

Tests: 18 new integration tests in `tests/integration/api/interest-berths.test.ts`
covering happy paths, primary-demotion in same tx, bypass write/clear, the
"requires signed EOI" guard, cross-port 404s, missing-link 404s, empty-body
400, and viewer 403 through the permission gate.
 2026-05-05 04:01:56 +02:00
Matt Ciaccio
a0091e4ca6 feat(emails): sales send-out flows + brochures + email-from settings 
Phase 7 of the berth-recommender refactor (plan §3.3, §4.8, §4.9, §5.7,
§5.8, §5.9, §11.1, §14.7, §14.9). Adds the rep-driven send-out path for
per-berth PDFs and port-wide brochures, the per-port sales SMTP/IMAP
config + body templates, and the supporting admin UI.

Migration: 0031_brochures_and_document_sends.sql

Schema additions:
  - brochures (port-wide, with isDefault marker + archive)
  - brochure_versions (versioned uploads, storageKey per §4.7a)
  - document_sends (audit log of every rep-initiated send; failures
    captured with failedAt + errorReason). berthPdfVersionId is a plain
    text column (no FK) — loose-coupled to Phase 6b's berth_pdf_versions
    so the two phases stay independent.

§14.7 critical mitigations:
  - Body XSS: rep-authored markdown goes through renderEmailBody()
    (HTML-escape first, then a tight allowlist of bold/italic/code/link
    rules). https:// + mailto: only — javascript:/data: URLs stripped.
    Tested against script/img/iframe/svg/onerror polyglots.
  - Recipient typo: strict email regex + two-step confirm modal that
    shows the exact recipient before send.
  - Unresolved merge fields: pre-send dry-run /preview endpoint blocks
    submission until findUnresolvedTokens() returns empty.
  - SMTP failure: every transport rejection writes a document_sends row
    with failedAt + errorReason; UI surfaces the message.
  - Hourly per-user rate limit: 50 sends/user/hour via existing
    checkRateLimit().
  - Size threshold fallback (§11.1): files above
    email_attach_threshold_mb (default 15) ship as a 24h signed-URL
    download link in the body instead of an attachment. Storage stream
    flows directly to nodemailer to avoid buffering 20MB+.

§14.10 critical mitigation:
  - SMTP/IMAP passwords encrypted at rest via the existing
    EMAIL_CREDENTIAL_KEY (AES-256-GCM). The /api/v1/admin/email/
    sales-config GET endpoint never returns the decrypted value — only
    a *PassIsSet boolean. PATCH treats empty string as "leave unchanged"
    and explicit null as "clear", so the masked-placeholder UI round-
    trips without forcing re-entry on every save.

system_settings keys (per-port unless noted):
  - sales_from_address, sales_smtp_{host,port,secure,user,pass_encrypted}
  - sales_imap_{host,port,user,pass_encrypted}
  - sales_auth_method (default app_password)
  - noreply_from_address
  - email_template_send_berth_pdf_body, email_template_send_brochure_body
  - brochure_max_upload_mb (default 50)
  - email_attach_threshold_mb (default 15)

UI surfaces (per §5.7, §5.8, §5.9):
  - <SendDocumentDialog> shared 2-step compose+confirm flow.
  - <SendBerthPdfDialog>, <SendDocumentsDialog>, <SendFromInterestButton>
    wrappers per detail page.
  - /[portSlug]/admin/brochures: list, upload (direct-to-storage
    presigned PUT for the 20MB+ files per §11.1), default toggle,
    archive.
  - /[portSlug]/admin/email extended with <SalesEmailConfigCard>:
    SMTP + IMAP creds, body templates, threshold/max settings.

Storage: every upload + download goes through getStorageBackend() —
no direct minio imports, per Phase 6a contract.

Tests: 1145 vitest passing (+ 50 new in
markdown-email-sanitization.test.ts, document-sends-validators.test.ts,
sales-email-config-validators.test.ts).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-05 03:38:47 +02:00
Matt Ciaccio
249ffe3e4a feat(berths): per-berth PDF storage (versioned) + reverse parser 
Phase 6b of the berth-recommender refactor (see
docs/berth-recommender-and-pdf-plan.md §3.2, §3.3, §4.7b, §11.1, §14.6).
Builds on the Phase 6a pluggable storage backend (commit 83693dd) — every
file write goes through `getStorageBackend()`; no direct minio imports.

Schema (migration 0030_berth_pdf_versions):
  - new table `berth_pdf_versions` with monotonic `version_number` per
    berth, `storage_key` (renamed convention from §4.7a), sha256, size,
    `download_url_expires_at` cache slot for §11.1 signed-URL throttling,
    and `parse_results` jsonb for the audit trail.
  - new column `berths.current_pdf_version_id` (deferred from Phase 0)
    with FK to `berth_pdf_versions(id)` ON DELETE SET NULL.
  - relations + types exported from `schema/berths.ts`.

3-tier reverse parser (`lib/services/berth-pdf-parser.ts`):
  1. AcroForm via pdf-lib — pulls named fields (`length_ft`,
     `mooring_number`, etc.) at confidence 1. Sample PDF has 0 such
     fields, so this is defensive coverage for future templates.
  2. OCR via Tesseract.js — positional/regex heuristics keyed off the
     §9.2 layout (Length/Width/Water Depth as `<imperial> / <metric>`,
     `WEEK HIGH / LOW`, `CONFIRMED THROUGH UNTIL <date>`, etc.). Returns
     per-field confidence + global mean; flags imperial-vs-metric drift
     >1% in `warnings`.
  3. AI fallback — gated via `getResolvedOcrConfig()` (existing
     openai/claude provider). Surfaced from the diff dialog only when
     `shouldOfferAiTier()` returns true (mean OCR confidence below
     0.55 threshold), so OPENAI_API_KEY isn't burned on every upload.

Service layer (`lib/services/berth-pdf.service.ts`):
  - `uploadBerthPdf()` — magic-byte check, size cap, version-number
    bump + current pointer in one transaction.
  - `reconcilePdfWithBerth()` — auto-applies fields where CRM is null;
    flags conflicts when CRM and PDF disagree; tolerates ±1% on numeric
    columns; warns on mooring-number-in-PDF mismatch (§14.6).
  - `applyParseResults()` — hard allowlist of writable columns;
    stamps `appliedFields` onto `parse_results` for audit.
  - `rollbackToVersion()` — pointer flip only, never re-parses (§14.6).
  - `listBerthPdfVersions()` — version list with 15-min signed URLs.
  - `getMaxUploadMb()` — port-override → global → default 15 lookup
    on `system_settings.berth_pdf_max_upload_mb`.

§14.6 critical mitigations:
  - Magic-byte check (`%PDF-`) on every upload; mismatch deletes the
    storage object and rejects the request.
  - Size cap from `system_settings.berth_pdf_max_upload_mb` (default
    15 MB); enforced in the upload-url presign AND server-side.
  - 0-byte uploads rejected.
  - Mooring-number mismatch surfaces as a `warnings[]` entry on the
    reconcile result so the rep sees it in the diff dialog.
  - Imperial vs metric ±1% tolerance in both the parser warnings and
    the reconcile equality check.
  - Path traversal already blocked at the storage layer (Phase 6a).

API + UI:
  - `POST /api/v1/berths/[id]/pdf-upload-url` — presigned URL (S3) or
    HMAC-signed proxy URL (filesystem) sized to the per-port cap.
  - `POST /api/v1/berths/[id]/pdf-versions` — verifies the upload via
    `backend.head()`, writes the row, bumps `current_pdf_version_id`.
  - `GET /api/v1/berths/[id]/pdf-versions` — version list + signed URLs.
  - `POST /api/v1/berths/[id]/pdf-versions/[versionId]/rollback`.
  - `POST /api/v1/berths/[id]/pdf-versions/parse-results/apply` —
    rep-confirmed diff payload.
  - New "Documents" tab on the berth detail page (`berth-tabs.tsx`)
    with current-PDF panel, version history, Replace PDF button, and
    `<PdfReconcileDialog>` for the auto-applied + conflicts UX.

System settings:
  - `berth_pdf_max_upload_mb` (default 15) — caps presigned-upload size
    + server-side validation. Resolved port-override → global → default.

Tests:
  - `tests/unit/services/berth-pdf-parser.test.ts` — magic bytes,
    feet-inches, human dates, full §9.2-shaped OCR text → 18 fields,
    drift warning, AI-tier gate.
  - `tests/unit/services/berth-pdf-acroform.test.ts` — synthetic
    pdf-lib AcroForm round-trip.
  - `tests/integration/berth-pdf-versions.test.ts` — upload, version-
    number bump, magic-byte rejection, reconcile auto-applied vs
    conflicts vs ±1% tolerance, mooring-number warning,
    applyParseResults allowlist enforcement, rollback semantics.

Acceptance: `pnpm exec tsc --noEmit` clean, `pnpm exec vitest run`
green at 1103/1103.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-05 03:34:24 +02:00
Matt Ciaccio
83693dd993 feat(storage): pluggable s3-or-filesystem backend + migration CLI + admin UI 
Phase 6a from docs/berth-recommender-and-pdf-plan.md §4.7a + §14.9a. Lays
the storage groundwork for Phase 6b/7 file-bearing schemas (per-berth PDFs,
brochures) without touching those domains yet.

New files:
- src/lib/storage/index.ts        StorageBackend interface + per-process
                                  factory keyed on system_settings.
- src/lib/storage/s3.ts           S3-compatible backend (MinIO/AWS/B2/R2/
                                  Wasabi/Tigris) wrapping the existing minio
                                  JS client. Includes a healthCheck() used
                                  by the admin "Test connection" button.
- src/lib/storage/filesystem.ts   Local filesystem backend with all §14.9a
                                  mitigations baked in.
- src/lib/storage/migrate.ts      Shared migration core — pg_advisory_lock,
                                  per-row resumable progress markers,
                                  sha256 round-trip verification, atomic
                                  storage_backend flip on success.
- scripts/migrate-storage.ts      Thin CLI shim around runMigration().
- src/app/api/storage/[token]/route.ts
                                  Filesystem proxy GET. Verifies HMAC,
                                  enforces single-use replay protection
                                  via Redis SET NX, streams via NextResponse
                                  ReadableStream with explicit Content-Type
                                  + Content-Disposition. Node runtime only.
- src/app/api/v1/admin/storage/route.ts
                                  GET status + POST connection test.
- src/app/api/v1/admin/storage/migrate/route.ts
                                  Super-admin-only POST that runs the
                                  exact same runMigration() as the CLI.
- src/app/(dashboard)/[portSlug]/admin/storage/page.tsx
                                  Super-admin admin UI (current backend,
                                  capacity stats, switch button with
                                  dry-run, test connection, backup hint).
- src/components/admin/storage-admin-panel.tsx
                                  Client component for the page above.

§14.9a critical mitigations implemented:
- Path-traversal: storage keys validated against ^[a-zA-Z0-9/_.-]+$;
  `..`, `.`, `//`, leading `/`, and overlength keys rejected.
- Realpath: storage root realpath'd at create time, every per-key
  resolution checked against the realpath'd prefix.
- Storage root created (or chmod'd) to 0o700.
- Multi-node refusal: FilesystemBackend.create() throws when
  MULTI_NODE_DEPLOYMENT=true.
- HMAC token: sha256-HMAC over the (key, expiry, nonce, filename,
  content-type) payload. Verified with timingSafeEqual; bad sig,
  expired, or invalid-key payloads all return 403.
- Single-use replay: token body cached in Redis SET NX EX 1800s.
- sha256 round-trip: copyAndVerify() re-fetches from the target after
  put() and aborts the migration on any mismatch.
- Free-disk pre-flight: when migrating to filesystem, sums byte counts
  via source.head() and aborts if free space < total * 1.2.
- pg_advisory_lock(0xc7000a01) prevents concurrent migrations.
- Resumable: per-row progress markers in _storage_migration_progress.

system_settings keys read by the factory (jsonb, no schema change):
storage_backend, storage_s3_endpoint, storage_s3_region,
storage_s3_bucket, storage_s3_access_key,
storage_s3_secret_key_encrypted, storage_s3_force_path_style,
storage_filesystem_root, storage_proxy_hmac_secret_encrypted.

Defaults: storage_backend=`s3`, storage_filesystem_root=`./storage`
(./storage added to .gitignore).

Tests added (34 tests, all green):
- tests/unit/storage/filesystem-backend.test.ts — key validation
  allow/reject matrix, realpath escape, 0o700 perms, multi-node
  refusal, HMAC token sign/verify/tamper/expire/invalid-key.
- tests/unit/storage/copy-and-verify.test.ts — sha256 mismatch on
  round-trip aborts the migration.
- tests/integration/storage/proxy-route.test.ts — happy path, wrong
  HMAC secret, expired token, replay rejection.

Phase 6a ships zero file-bearing tables — TABLES_WITH_STORAGE_KEYS is
intentionally empty. berth_pdf_versions and brochure_versions land in
Phase 6b and join the list there. Existing s3_key columns: only
gdpr_export_jobs.storage_key, already named correctly — no rename needed.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-05 03:15:59 +02:00
Matt Ciaccio
e00e812199 feat(eoi): multi-berth EOI generation + berth-range formatter 
Plan §4.6 + §1: a render function that compresses every berth marked
is_in_eoi_bundle=true on an interest into a compact range string
("A1-A3, B5-B7"), wired into both EOI generation paths (the Documenso
template-generate call and the in-app pdf-lib AcroForm fill).

- src/lib/templates/berth-range.ts: pure formatBerthRange() with the
  full edge-case set from §4.6 - empty, single, run, gap, multiple
  prefixes, sort/dedup, multi-letter prefixes, non-canonical
  passthrough, long ranges. Sorts by (prefix, number); dedupes; passes
  non-canonical inputs through with a logger warning.
- src/lib/templates/merge-fields.ts: new {{eoi.berthRange}} token
  added to VALID_MERGE_TOKENS allow-list under a fresh `eoi` scope so
  unknown-token validation at template creation time still rejects
  typos.
- src/lib/services/eoi-context.ts: EoiContext gains eoiBerthRange.
  Resolved by joining interest_berths (is_in_eoi_bundle=true) →
  berths and feeding the mooring numbers through formatBerthRange.
- src/lib/services/documenso-payload.ts: formValues now includes
  "Berth Range" alongside the legacy "Berth Number". Multi-berth EOIs
  surface here; single-berth EOIs duplicate the primary.
- src/lib/pdf/fill-eoi-form.ts: in-app AcroForm fill mirrors the
  Documenso payload by populating "Berth Range". Falls back silently
  when older PDFs don't have the field (setText is no-op-on-missing).

15 unit tests on the formatter; existing EoiContext + Documenso
payload tests updated to assert the new field. 1022 -> 1037 passing.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-05 03:03:29 +02:00
Matt Ciaccio
b1e787e55c feat(recommender): SQL ranking + tier ladder + heat scoring 
Plan §4.4 + §13: pure SQL recommender, no AI. Single CTE chain
(feasible -> aggregates) + JS-side tier classification, fall-through
cooldown filter, heat scoring, and fit ranking. Per-port settings via
system_settings layered over global + DEFAULT_RECOMMENDER_SETTINGS.

Tier ladder (default):
  A : no interest history
  B : lost-only history (still recommendable + boosted by heat)
  C : active interest in early stage (open..eoi_signed)
  D : active interest at deposit_10pct or beyond (hidden by default)

Heat (only for tier B):
  recency        weight 30  full @ <=30 days, decays to 0 @ 365 days
  furthest stage weight 40  full when prior reached deposit
  interest count weight 15  saturates at 5+
  EOI count      weight 15  saturates at 3+

Multi-port isolation enforced (§14.10 critical): the SQL filters by
port_id AND the entry-point function rejects cross-port interest
lookups with an explicit error. Fall-through policy supports
immediate_with_heat (default), cooldown, and never_auto_recommend.

15 unit tests covering tier classification, heat saturation, weight
tuning, zero-weight guard. Smoke-tested end-to-end via
scripts/dev-recommender-smoke.ts.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-05 02:58:34 +02:00
Matt Ciaccio
fb1116f1d4 feat(berths): public berths API + health env-match endpoint 
Adds the read-only public-website data feed promised by plan §4.5 and
§7.3. The marketing site's `getBerths()` swap is now a one-line URL
change against the existing 5-min TTL behaviour.

- src/app/api/public/berths/route.ts: GET / unauth, returns the full
  port-nimara berth list as { list, pageInfo } in the verbatim NocoDB
  shape ("Mooring Number", "Side Pontoon", quoted-key fields). Cache:
  s-maxage=300 + stale-while-revalidate=60. portSlug query param lets
  future ports opt in.
- src/app/api/public/berths/[mooringNumber]/route.ts: GET single. Up-
  front regex validation (^[A-Z]+\\d+$) rejects malformed lookups with
  400 + cache-control:no-store before hitting the DB. 404 + no-store
  when not found.
- src/app/api/public/health/route.ts: returns { status, env, appUrl,
  timestamp } so the marketing site can refuse to start when its
  CRM_PUBLIC_URL points at a different deployment env (§14.8 critical
  env-mismatch protection).
- src/lib/services/public-berths.ts: pure mapper with derivePublicStatus
  ("sold" wins; otherwise specific-interest junction OR
  status='under_offer' -> "Under Offer"; else "Available").
- 11 unit tests covering numeric coercion, status derivation,
  archived-berth handling, missing-map-data omission, and the
  status-precedence rule that "sold" trumps the specific-interest
  signal.

Smoke-tested: /api/public/berths -> 117 rows, A1 correctly shows
"Under Offer" (has interest_berths.is_specific_interest=true link),
INVALID -> 400, Z99 -> 404. Total tests: 996 -> 1007.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-05 02:52:44 +02:00
Matt Ciaccio
57cbc9a506 fix(tests): cascade interest_berths in global teardown 
The Phase 2b refactor (commit 6e3d910) added a junction table whose
berth_id has onDelete: 'restrict'. The vitest global teardown deletes
test-port berths but never explicitly clears interest_berths first,
so any test leaking junction rows (e.g. via the new createInterest
write path) leaves berths un-deletable and ports stranded.

Adds DELETE FROM interest_berths WHERE berth_id IN (test berths) to
the WITH-chain so cascading teardown completes cleanly.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-05 02:45:45 +02:00
Matt Ciaccio
6e3d910c76 refactor(interests): migrate callers to interest_berths junction + drop berth_id 
Phase 2b of the berth-recommender refactor (plan §3.4). Every caller of
the legacy `interests.berth_id` column now reads / writes through the
`interest_berths` junction via the helper service introduced in Phase 2a;
the column itself is dropped in a final migration.

Service-layer changes
- interests.service: filter `?berthId=X` becomes EXISTS-against-junction;
  list enrichment uses `getPrimaryBerthsForInterests`; create/update/
  linkBerth/unlinkBerth all dispatch through the junction helpers, with
  createInterest's row insert + junction write sharing a single transaction.
- clients / dashboard / report-generators / search: leftJoin chains pivot
  through `interest_berths` filtered by `is_primary=true`.
- eoi-context / document-templates / berth-rules-engine / portal /
  record-export / queue worker: read primary via `getPrimaryBerth(...)`.
- interest-scoring: berthLinked is now derived from any junction row count.
- dedup/migration-apply + public interest route: write a primary junction
  row alongside the interest insert when a berth is provided.

API contract preserved: list/detail responses still emit `berthId` and
`berthMooringNumber`, derived from the primary junction row, so frontend
consumers (interest-form, interest-detail-header) need no changes.

Schema + migration
- Drop `interestsRelations.berth` and `idx_interests_berth`.
- Replace `berthsRelations.interests` with `interestBerths`.
- Migration 0029_puzzling_romulus drops `interests.berth_id` + the index.
- Tests that previously inserted `interests.berthId` now seed a primary
  junction row alongside the interest.

Verified: vitest 995 passing (1 unrelated pre-existing flake in
maintenance-cleanup.test.ts), tsc clean.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-05 02:41:52 +02:00
Matt Ciaccio
18119644ae feat(berths): nocodb berth import script + helpers + unit tests 
Idempotent NocoDB Berths -> CRM `berths` import script with full
re-run safety. Re-running picks up NocoDB additions/edits without
clobbering CRM-side overrides (compares updated_at vs last_imported_at,
1-second tolerance for sub-second clock drift). --force overrides the
edit guard.

Mitigates the §14.1 critical/high cases:
- Mooring collisions: unique (port_id, mooring_number) on the table.
- Concurrent runs: pg_advisory_xact_lock on a stable BIGINT key.
- Numeric-with-units inputs: parseDecimalWithUnit() strips trailing
  ft/m/kw/v/usd/$ markers before parsing.
- Metric drift: NocoDB's metric formula columns are ignored; metric
  values recomputed from imperial via 0.3048 + round-to-2-decimals to
  match NocoDB's `precision: 2` columns and avoid spurious diffs.
- Map Data shape: zod-validated; failures are skipped rather than
  aborting the import.
- Status enum mapping: NocoDB display strings -> CRM snake_case.
- NocoDB row deleted: reported as "orphaned in CRM"; never auto-
  deleted (rep decides via admin UI in a future phase).

Pure helpers (parseDecimalWithUnit, mapStatus, parseMapData,
extractNumerics, mapRow, buildPlan) live in
src/lib/services/berth-import.ts so vitest can exercise the mapping
logic without triggering the script's top-level db connection.

40 new unit tests (956 -> 996 passing).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-05 02:07:58 +02:00
Matt Ciaccio
05be89ec6f feat(berths): normalize mooring numbers to canonical form 
Sweep CRM mooring numbers from the legacy hyphen+padded form ("A-01")
to the canonical bare form ("A1") used by NocoDB, the public website,
the per-berth PDFs, and the Documenso EOI templates. Drift was
introduced by the original load-berths-to-port-nimara.ts seed; this
gates the Phase 3 public-website cutover where /berths/A1 URLs would
404 against a CRM still storing "A-01".

- 0024 data migration: idempotent regexp_replace + post-update sanity
  check that surfaces any non-conforming rows for manual triage.
- Invert normalizeLegacyMooring in dedup/migration-apply: it now
  canonicalizes ("D-32" -> "D32") instead of legacy-izing.
- Update tiptap-to-pdfme example tokens, EOI fixture moorings, and
  smoke-test seed moorings.
- Refresh seed-data/berths.json to canonical form; drop the now-
  redundant legacyMooringNumber field.
- Delete scripts/load-berths-to-port-nimara.ts (superseded in 0c).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-05 01:59:26 +02:00
Matt Ciaccio
d62822c284 fix(migration): NocoDB import safety + dedup helpers + lead-source backfill 
migration-apply: residential client + interest inserts now wrap in
db.transaction so a partial failure can't leave an orphan client
row without its interest (or vice versa).

migration-transform: buildPlannedDocument returns null when there
are no signers so the apply pass doesn't try to send a Documenso
envelope without recipients. mapDocumentStatus gets an explicit
"Awaiting Further Details" branch that no longer auto-promotes via
stale sign-time fields. parseFlexibleDate handles ISO and DD-MM-YYYY
inputs uniformly.

backfill-legacy-lead-source: chunk UPDATE WHERE clause now
isNull(source) on top of the inArray match, so a re-run can't
overwrite a more accurate source written between batches.

Adds 235 lines of vitest coverage on migration-transform.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-04 22:56:18 +02:00
Matt Ciaccio
49d34e00c8 feat(website-intake): dual-write endpoint + migration chain repair 
Adds website_submissions table + shared-secret POST endpoint so the
marketing site can dual-write inquiries alongside its NocoDB write.
Race-safe via INSERT ... ON CONFLICT, idempotent on submission_id,
refuses every request when WEBSITE_INTAKE_SECRET is unset. Also
repairs pre-existing 0020/0021/0022 prevId collision (renumbered +
journal re-sorted) so db:generate works again. 11 unit tests.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-04 22:52:33 +02:00
Matt Ciaccio
872c75f1a1 fix(safety): plug 3 EMAIL_REDIRECT_TO leaks + 10 unit tests + live smoke
Some checks failed
Build & Push Docker Images / lint (push) Failing after 1m10s
Details
Build & Push Docker Images / build-and-push (push) Has been skipped
Details 
A pre-import audit caught three places where outbound comms could escape
even with EMAIL_REDIRECT_TO set. Plugged each, added unit tests so the
behavior can't silently regress, and shipped a live smoke script the
operator can run before any production data import.

Leak 1: email-compose.service.ts (per-account user composer)
  Built its own nodemailer transporter and called sendMail() directly,
  bypassing the centralized sendEmail()'s redirect. Now mirrors the same
  redirect: when EMAIL_REDIRECT_TO is set, "to" is rewritten, "cc" is
  dropped, and the subject is prefixed with "[redirected from <orig>]".

Leak 2: documenso-client.sendDocument()
  Tells Documenso to actually email the document. Recipient emails were
  rerouted at create-time (in pass-3) but a document created BEFORE the
  redirect was turned on could still trigger a real-client email. Now
  short-circuited when the redirect is set — returns the existing doc
  shape so downstream code doesn't see an unexpected null.

Leak 3: documenso-client.sendReminder()
  Same shape as sendDocument: emails a stored recipient address that may
  predate the redirect. Now short-circuits with a warn-level log.

Tests (tests/unit/comms-safety.test.ts):
  - createDocument rewrites recipients
  - generateDocumentFromTemplate rewrites both v1.13 formValues.*Email
    keys AND v2.x recipients[] arrays
  - sendDocument is short-circuited (no /send call)
  - sendReminder is short-circuited (no /remind call)
  - createDocument passes through unchanged when redirect unset
  - sendEmail rewrites to + subject for single recipient
  - sendEmail handles array of recipients (joined into subject prefix)
  - sendEmail passes through unchanged when redirect unset
  - Webhook worker reads process.env.EMAIL_REDIRECT_TO at dispatch time
    (no module-level caching that could miss a runtime flip)

Live smoke (scripts/smoke-test-redirect.ts):
  Monkey-patches nodemailer.createTransport, calls the real sendEmail()
  with a fake real-client address, verifies the captured outbound has
  the right "to" + subject. Run: `pnpm tsx scripts/smoke-test-redirect.ts`.
  Exits non-zero if the redirect failed for any reason — drop-in for a
  pre-deploy check.

Verification:
  pnpm exec tsc --noEmit       — 0 errors
  pnpm exec vitest run         — 936/936 (was 926, +10 new safety tests)
  pnpm tsx scripts/smoke-test-redirect.ts — PASS

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-03 20:55:53 +02:00
Matt Ciaccio
cb83b09b2d Merge feat/dedup-migration: client dedup library + NocoDB migration script + admin queue 
# Conflicts:
#	.gitignore
#	src/lib/db/migrations/meta/_journal.json
 2026-05-03 16:24:13 +02:00
Matt Ciaccio
49d92234dd fix(test): align stage names with consolidated pipeline enum 
Followup to 886119c (refactor(sales): consolidate pipeline stages) — the
runtime enum was renamed but a few test fixtures and PDF report templates
still referenced the legacy names, leaving them broken at the type level
(36 tsc errors before this fix).

Renames in this commit:
  visited        -> in_communication (alerts test) / removed (PDF reports)
  signed_eoi_nda -> eoi_signed
  contract       -> contract_signed (interests test) / contract_sent (factory)

Affected files: pipeline-report, revenue-report, makeCreateInterestInput
factory, alerts-engine, pipeline-transitions, interest-scoring.

Verification: tsc clean, 858/858 vitest passing.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-03 16:14:04 +02:00
Matt Ciaccio
4bcc7f8be6 feat(dedup): runtime surfaces — merge service, at-create suggestion, admin queue (P2) 
Adds the live dedup pipeline on top of the P1 library + P3 migration
script. The new `client/interest` model now actively prevents duplicate
client records at creation time and gives admins a queue to triage
the borderline pairs the at-create check missed.

Three layers, per design §7:

Layer 1 — At-create suggestion
==============================

`GET /api/v1/clients/match-candidates`
  Accepts free-text email / phone / name from the in-flight client
  form, normalizes them via the dedup library, and returns scored
  matches against the port's live client pool. Filters out
  low-confidence noise (the background scoring queue picks those up
  separately). Strict port scoping; never leaks across tenants.

`<DedupSuggestionPanel>` (`src/components/clients/dedup-suggestion-panel.tsx`)
  Debounced React Query hook. Renders nothing for short inputs or
  no useful match. On a high-confidence match it interrupts visually
  with an amber-tinted card and a "Use this client" primary button.
  Medium confidence falls back to a softer "possible match — check
  before creating" treatment.

`<ClientForm>`
  Renders the panel above the form (create path only — skipped on
  edit). New `onUseExistingClient` callback fires when the user
  picks the existing client; the form closes and the parent decides
  what to do (typically: navigate to that client's detail page or
  open the create-interest dialog pre-filled).

Layer 2 — Merge service
=======================

`mergeClients` (`src/lib/services/client-merge.service.ts`)
  The atomic merge primitive that everything else calls. Single
  transaction. Per §6 of the design:

  - Locks both rows (FOR UPDATE) so concurrent merges of the same
    loser fail with a clear error rather than racing.
  - Snapshots the full loser state (contacts / addresses / notes /
    tags / interest+reservation IDs / relationship rows) into the
    `client_merge_log.merge_details` JSONB column for the eventual
    undo flow.
  - Reattaches every loser-side row to the winner: interests,
    reservations, contacts (skipping duplicates by `(channel, value)`),
    addresses, notes, tags (deduped), relationships.
  - Optional `fieldChoices` — per-scalar overrides letting the user
    keep the loser's value for fullName / nationality / preferences /
    timezone / source.
  - Marks the loser archived with `mergedIntoClientId` set (a redirect
    pointer for stragglers; never hard-deleted within the undo window).
  - Resolves any matching `client_merge_candidates` row to status='merged'.
  - Writes audit log entry.

Schema additions:
  - `clients.merged_into_client_id` (nullable text, indexed) — the
    redirect pointer set on archive.

Tests: 6 cases against a real DB — happy path moves rows + writes log;
self-merge / cross-port / already-merged refused; duplicate-contact
deduped on reattach; fieldChoices copies loser values to winner.

Layer 3 — Admin review queue
============================

`GET /api/v1/admin/duplicates`
  Pending merge candidates (status='pending') for the current port,
  with both client summaries hydrated for side-by-side rendering.
  Skips pairs where one side is already archived/merged.

`POST /api/v1/admin/duplicates/[id]/merge`
  Confirms a candidate. Body picks the winner; the other side
  becomes the loser. Calls into `mergeClients` — the only path that
  writes `client_merge_log`.

`POST /api/v1/admin/duplicates/[id]/dismiss`
  Marks the candidate dismissed. Future scoring runs skip the same
  pair until a score change recreates the row.

`<DuplicatesReviewQueue>` (`/admin/duplicates`)
  Side-by-side card UI for each pending pair. Click a card to pick
  the winner; the other side is automatically the loser. Toolbar:
  "Merge into selected" + "Dismiss". No per-field merge editor in
  this PR — that's a future polish; the simple "pick the better row"
  flow handles ~80% of cases.

Test coverage
=============

11 new integration tests (76 added in this branch total):
  - 6 mergeClients (atomicity, refusal cases, contact dedup,
    fieldChoices)
  - 5 match-candidates API (shape, port scoping, confidence tiers,
    Pattern F false-positive guard)

Full vitest: 926/926 passing (was 858 before the dedup branch).
Lint: clean. tsc: clean for new files (only pre-existing errors in
unrelated `tests/integration/` files remain, same as before this PR).

Out of scope, deferred
======================

- Background scoring cron that populates `client_merge_candidates`
  (the queue is empty until this lands; manual seeding works for
  now via the at-create flow).
- Side-by-side per-field merge editor with checkboxes (the simple
  "pick the winner" UX shipped here covers ~80% of real cases).
- Admin settings UI for tuning the dedup thresholds. Defaults from
  the design (90 / 50) are baked in for now.
- `unmergeClients` (the snapshot is captured in client_merge_log;
  the undo endpoint just hasn't been wired yet).

These are all natural follow-up PRs that don't block shipping the
runtime UX.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-03 14:59:04 +02:00
Matt Ciaccio
18e5c124b0 feat(dedup): NocoDB migration script + tables (P3 dry-run) 
Lands the one-shot migration pipeline from the legacy NocoDB Interests
base into the new client/interest schema. Dry-run mode is fully
operational: pulls the live snapshot, runs the dedup library, and
writes a CSV + Markdown report under .migration/<timestamp>/. The
--apply phase is stubbed for a follow-up PR per the design's P3
implementation sequence.

Schema additions
================

- `client_merge_candidates` — pairs flagged by the background scoring
  job for the /admin/duplicates review queue. Status enum: pending /
  dismissed / merged. Unique-(portId, clientAId, clientBId) so the
  same pair can't surface twice. Empty until P2 lands the cron.
- `migration_source_links` — idempotency ledger. Maps source-system
  rows (NocoDB Interest #624 → new client UUID) so re-running --apply
  against the same dry-run report skips already-imported entities.

Both tables ship with the migration `0020_unusual_azazel.sql` —
already applied to the local dev DB during this commit's preparation.

Library
=======

src/lib/dedup/nocodb-source.ts
  Read-only adapter for the legacy NocoDB v2 API. xc-token auth,
  auto-paginates until isLastPage, captures the table IDs from the
  2026-05-03 audit. `fetchSnapshot()` pulls every relevant table in
  parallel into one in-memory object the transform layer consumes.

src/lib/dedup/migration-transform.ts
  Pure function: NocoDB snapshot in, MigrationPlan out. Per row:
    - normalizes name / email / phone / country via the dedup library
    - parses the legacy DD-MM-YYYY / DD/MM/YYYY / ISO date formats
    - maps the 8-stage `Sales Process Level` enum to the new 9-stage
      pipelineStage
    - filters yacht-name placeholders ('TBC', 'Na', etc.)
    - merges Internal Notes + Extra Comments + Berth Size Desired into
      a single notes blob
  Then runs `findClientMatches` pairwise (with blocking) and
  union-finds clusters of rows whose score crosses the auto-link
  threshold (90). Lower-scoring pairs (50–89) become 'needs review'.
  Each cluster's "lead" row is picked by completeness score with
  recency tie-break.

src/lib/dedup/migration-report.ts
  Writes three artifacts to .migration/<timestamp>/:
    - report.csv  — one row per planned op, RFC-4180 escaped
    - summary.md  — human-skimmable overview
    - plan.json   — full structured plan for the --apply phase
  CSV cells with comma / quote / newline are quoted; internal quotes
  are doubled. No external CSV dep.

src/lib/dedup/phone-parse.ts
  Script-safe wrapper around libphonenumber-js's `core` entry that
  loads `metadata.min.json` directly. The default `index.cjs.js`
  bundled by libphonenumber hits a metadata-shape interop bug under
  Node 25 + tsx (`{ default }` wrapping); core+JSON sidesteps it.
  The dedup `normalizePhone` and `find-matches` both use this wrapper
  now so the same code path runs in vitest, Next.js, and the migration
  CLI without surprises.

src/lib/dedup/normalize.ts
  Tightened country resolution: added Caribbean short-form aliases
  ('antigua' → AG, 'st kitts' → KN, etc.) and a city map covering the
  US locations seen in the NocoDB dump (Boston, Tampa, Fort
  Lauderdale, Port Jefferson, Nantucket). Also relaxed phone parsing
  to drop the `isValid()` strict check — the libphonenumber min build
  rejects many real NANP-territory numbers, and dedup only needs a
  canonical E.164 to compare.

CLI
===

scripts/migrate-from-nocodb.ts
  pnpm tsx scripts/migrate-from-nocodb.ts --dry-run
    → Pulls the live NocoDB base (NOCODB_URL + NOCODB_TOKEN env vars),
       runs the transform, writes report. No DB writes.
  pnpm tsx scripts/migrate-from-nocodb.ts --apply --report .migration/<dir>/
    → Stubbed; exits with `not yet implemented` and a pointer to the
       design doc. Apply phase ships in a follow-up.

Tests
=====

tests/unit/dedup/migration-transform.test.ts (7 cases)
  Fixture-based regression. A frozen 12-row NocoDB snapshot covers
  every duplicate pattern in the design (§1.2). The test asserts:
    - 12 input rows → 7 unique clients (cluster math is right)
    - Patterns A / B / C / E auto-link
    - Pattern F (Etiennette Clamouze) does NOT auto-link
    - Every interest preserved as its own row even when clients merge
    - 8-stage → 9-stage enum mapping is correct per spec
    - Multi-yacht merge (Constanzo CALYPSO + Costanzo GEMINI under one
      client) — the design's signature win
    - Output is deterministic (run twice, identical)

Validation against real data
============================

Ran `pnpm tsx scripts/migrate-from-nocodb.ts --dry-run` against the
live NocoDB. Result on 252 Interests rows:
  - 237 clients (15 merged into 13 clusters)
  - 252 interests (one per source row)
  - 406 contacts, 52 addresses
  - 13 auto-linked clusters (every confirmed cluster from §1.2 audit)
  - 3 pairs flagged for review (Camazou, Zasso, one new)
  - 1 phone placeholder flagged

Total dedup test count: 57 (50 from P1 + 7 fixture tests).
Lint: clean. Tsc: clean for new files.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-03 14:50:01 +02:00
Matt Ciaccio
8b077e1999 feat(dedup): normalization + match-finding library (P1) 
The pure-logic spine of the client deduplication system spec'd in
docs/superpowers/specs/2026-05-03-dedup-and-migration-design.md.
Two modules, JSX-free, vitest-tested against fixtures drawn directly
from real dirty values observed in the legacy NocoDB Interests audit.

src/lib/dedup/normalize.ts
- normalizeName: trims whitespace, replaces \r/\n/\t, intelligently
  title-cases ALL-CAPS surnames while keeping particles (van / de /
  dalla / etc.) lowercase mid-name. Preserves Irish O' surnames and
  the "slash-with-company" structure ("Daniel Wainstein / 7 Knots,
  LLC") seen in production. Returns a surnameToken (lowercased last
  non-particle token) for use as a dedup blocking key.
- normalizeEmail: trim + lowercase + zod email validation. Plus-aliases
  preserved; null on invalid.
- normalizePhone: pre-cleans the input (strips spreadsheet apostrophes,
  carriage returns, dots/dashes/parens, converts 00 prefix to +) then
  delegates to libphonenumber-js. Detects multi-number fields ("a/b",
  "a;b") and placeholder fakes (8+ consecutive zeros, e.g.
  +447000000000). Flags every quirk so the migration report and runtime
  audit log can surface it.
- resolveCountry: maps free-text country/region input to ISO-3166-1
  alpha-2 via alias → exact (vs. Intl-derived names) → city → fuzzy
  (Levenshtein ≤ 2). Fuzzy is gated by length so 4-char inputs ("Mars")
  don't false-positive against short country names.
- levenshtein: standard iterative implementation, exported for reuse
  by find-matches.

src/lib/dedup/find-matches.ts
- findClientMatches: builds three blocking indexes off the pool (email
  / phone / surname-token), gathers the comparison set via union, and
  scores each candidate via the rule set in design §4.2:
    Email match            +60
    Phone E.164 match      +50  (≥ 8 digits, excludes placeholder zeros)
    Name exact match       +20
    Surname + given fuzzy  +15  (Levenshtein ≤ 1)
    Negative: shared email but different phone country  −15
    Negative: name match but no shared contact          −20
  Score is clamped to [0,100]. Confidence tier ('high' / 'medium' /
  'low') is derived from configurable thresholds passed in by the
  caller — defaults are highScore=90, mediumScore=50.

tests/unit/dedup/normalize.test.ts (38 cases)
Every dirty-data pattern from design §1.3 has a fixture: carriage
returns in names, ALL-CAPS surnames, lowercase entries, particles,
slash-with-company, plus-aliases, capitalized email localparts,
spreadsheet-apostrophe phones, multi-number phones, placeholder
phones, 00-prefix phones, French/UK local-format phones,
Saint-Barthélemy diacritic variants, Kansas City fallback.

tests/unit/dedup/find-matches.test.ts (12 cases)
Each duplicate cluster from design §1.2 has a test:
- Pattern A (Deepak Ramchandani — pure double-submit) → high
- Pattern B (Howard Wiarda — phone format variance) → high
- Pattern C (Nicolas Ruiz — name capitalization) → high
- Pattern D (Chris/Christopher Allen — name shortening) → high
- Pattern E (Christopher Camazou — typo on resubmit) → high or medium
- Pattern E (Constanzo/Costanzo — surname typo, multi-yacht) → high
- Pattern F (Etiennette Clamouze — same name, different country) →
  must NOT auto-merge
- Pattern F (Bruno+Bruce — shared household contact) → no match
- Negative evidence (same email, different phone country) → medium
- Blocking (no shared keys → 0 matches)
- Sort order (high before low)
- Empty pool

Total: 50 new tests, all green. Zero changes to runtime behavior or
schema; unblocks P2 (runtime surfaces) and P3 (NocoDB migration).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-03 14:28:59 +02:00
Matt Ciaccio
e2398099c4 test(audit-fixes): cover the new permission and webhook surfaces 
Adds integration coverage for the routes / handlers shipped in the
preceding audit-fix commits, plus refactors two route files to expose
inner handlers from a sibling `handlers.ts` (the pattern used elsewhere
in `src/app/api/v1`) so tests can call them without the
`withAuth(withPermission(…))` wrapper.

New tests (18 cases across 4 files):
- `tests/integration/portal-auth.test.ts` (6) — verifyPortalToken
  rejects tokens missing `aud: 'portal'` or `iss: 'pn-crm'`, with the
  wrong audience (CRM-session-replay shape) or wrong issuer, plus a
  round-trip happy path. Locks in the portal-vs-CRM token isolation.
- `tests/integration/api/saved-views-ownership.test.ts` (6) — patch
  and delete handlers return 403 for a different user, 404 for an
  unknown id or cross-port id, and 200 for the owner. Ownership is
  enforced at the route layer regardless of the service's internal
  filtering.
- `tests/integration/api/berth-reservations-list.test.ts` (3) — the
  new global list returns rows for the current port only and honors
  pagination params. A reservation in a different port never leaks.
- `tests/integration/documents-expired-webhook.test.ts` (3) —
  handleDocumentExpired flips the document to `expired`, also flips
  the linked interest's `eoiStatus`, writes a `documentEvents` row,
  and is a no-op (not a throw) when the documensoId is unknown.

Refactors:
- `src/app/api/v1/saved-views/[id]/route.ts` extracts `patchHandler` /
  `deleteHandler` (and the shared `assertViewOwner`) into
  `handlers.ts`. The route file is now a 4-line `withAuth(handler)`
  wrapper.
- `src/app/api/v1/berth-reservations/route.ts` extracts `listHandler`
  similarly. Tests import directly from `handlers.ts`.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-02 23:17:08 +02:00
Matt Ciaccio
d364b09885 fix(realtime): keep socket through reconnects, stop re-subscribe storm 
Two correctness bugs in the real-time stack — both silent failures, both
session-wide once they trigger.

(1) `SocketProvider` was setting the React context to null on every
`disconnect` event. socket.io's built-in reconnection re-establishes the
underlying transport and replays handlers, but the React tree had
already lost its reference to the socket — so every `useSocket()`
consumer saw null until a session/port change forced a remount. Effect:
after the first transient drop (laptop sleep, wifi blip, server
restart), realtime invalidation and toasts went dead session-wide with
no user-visible signal.

Fix: keep the socket reference stable for the lifetime of the
session+port, and surface a separate `isConnected` boolean for any UI
that wants to render an offline indicator. Exposed as a new
`useIsSocketConnected()` hook; `useSocket()` signature is unchanged.

(2) `useRealtimeInvalidation` captured `eventMap` as a useEffect
dependency. Every caller passes a fresh `{ ... }` object literal on each
render, so the effect re-ran every render → `socket.off`/`socket.on`
storm on pages with many subscribed events.

Fix: extract the subscription logic into a pure helper
(`realtime-invalidation-core.ts`, JSX-free for vitest). The hook now
keeps the latest map in a ref and only re-subscribes when the SET of
event names changes (joined-keys signature, not object identity). The
handler reads `ref.current` at fire time, so callers still see fresh
queryKey lists without re-binding.

Helper is unit-tested with a stub socket: registration count,
fire-time map lookup, cleanup deregistration, missing-event safety.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-02 23:11:52 +02:00
Matt Ciaccio
d197f8b321 feat(eoi): align prerequisites with EOI document structure 
Match the gate to the actual EOI's structure (Section 2 vs Section 3) so
the rep can generate the document the moment they have what they need —
and not before.

  Required (Section 2 — top paragraph):
    - Client name
    - Client primary email
    - Client primary address

  Optional (Section 3 — left blank when absent):
    - Linked yacht (name, dimensions)
    - Linked berth (mooring number)

Previously the dialog blocked generation unless yacht AND berth were both
linked, which was overzealous — early-stage EOIs are routinely sent before
a specific berth is pinned down.

  - eoi-context.ts: yacht and berth are now nullable in the returned
    context. The hard ValidationError is now driven by the EOI's Section
    2 fields (name/email/address) rather than yacht/berth presence. The
    owner block falls back to the interest's client when no yacht is
    linked, so signing parties remain resolvable.

  - documenso-payload.ts + fill-eoi-form.ts: Section 3 form values
    render as empty strings when yacht or berth are absent, so the
    rendered PDF leaves those template inputs blank.

  - document-templates.ts: yacht.* and berth.* tokens fall back to
    empty strings; the legacy-fallback catch handler also recognises
    the new "missing required client details" error.

  - interests.service.ts: getInterestById now also returns
    `clientPrimaryEmail` and `clientHasAddress` so the Documents tab
    can compute the EOI prerequisites checklist client-side without an
    extra fetch.

  - eoi-generate-dialog.tsx: prereqs split into two groups visually —
    Required (with red ✗ when missing) and Optional (with grey – when
    absent). The Generate button only requires the Required block to
    pass. A small amber banner surfaces when Required is incomplete so
    the rep knows where to add the missing data.

Tests: 835/835 pass. Replaces the obsolete "throws on missing yacht/
berth" tests with parity coverage for the new behaviour ("builds a
valid context when yacht/berth missing", "throws when client email/
address missing"). Adds a payload test for the empty-Section-3 case.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-02 03:11:14 +02:00
Matt Ciaccio
886119cbde refactor(sales): consolidate pipeline stages + wire EOI auto-advance 
The 8→9 stage refresh from earlier today only updated constants.ts and the DB —
20 component/service files still hardcoded the old enum, leaving labels blank,
filter dropdowns wrong, kanban columns mismatched, and the analytics funnel
silently dropping new-stage rows. The platform also never advanced
pipelineStage on EOI lifecycle events: documents.service.ts wrote eoiStatus
but left the user-visible stage stuck.

This commit closes both gaps:

  1. Single source of truth in src/lib/constants.ts — adds STAGE_LABELS,
     STAGE_BADGE, STAGE_DOT, STAGE_WEIGHTS, STAGE_TRANSITIONS plus
     stageLabel / stageBadgeClass / stageDotClass / safeStage /
     canTransitionStage helpers. components/clients/pipeline-constants.ts
     becomes a re-export shim so existing imports keep working.

  2. 18 stale-enum surfaces migrated — interest list (table, card, filters,
     form, stage picker), pipeline board, client card, berth interests tab,
     portal client interests page, dashboard pipeline / funnel / revenue-
     forecast charts, settings pipeline_weights default, dashboard.service
     weights, analytics.service funnel stages, alert-rules stale-interest
     filter, interest-scoring stage rank.

  3. Documents tab wired into interest detail — replaced the placeholder in
     interest-tabs.tsx with InterestDocumentsTab + InterestFilesTab so the
     EOI launcher is back where salespeople work.

  4. Auto-advance — new advanceStageIfBehind() in interests.service.ts
     (forward-only, no-op if interest is already past the target). Called
     from documents.service.ts on send (→ eoi_sent), Documenso completed
     webhook (→ eoi_signed), and manual signed-EOI upload (→ eoi_signed).

  5. Transition guard — canTransitionStage() blocks egregious skips
     (e.g. completed → open, open → contract_signed). Enforced in
     changeInterestStage before the DB write.

Tests updated to reflect the 9-stage model. tsc clean, vitest 832/832,
ESLint clean on every file touched.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-01 23:33:53 +02:00
Matt Ciaccio
5d44f3cfa4 fix(test): raise mobile-audit timeout to 30min for 4-viewport runs 
Task 24 audit run hit the 10-minute test.setTimeout ceiling after capturing
2 of 4 viewport passes (iphone-se complete, iphone-16 complete-ish, 16-pro
partial, pro-max not started). 4 viewports × ~45 routes × slowMo: 200 needs
more headroom than 600s gave. 30min is comfortable headroom; the per-test
project timeout is matched.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-04-29 15:15:26 +02:00
Matt Ciaccio
d0540dca55 fix(build): extract route.ts handlers to handlers.ts (CLAUDE.md convention) 
8 API route files were exporting handler functions directly from route.ts,
which Next.js 15 rejects with "$NAME is not a valid Route export field".
Per CLAUDE.md convention, service-tested handler functions live in sibling
handlers.ts files and route.ts only re-exports the GET/POST/etc. wrapped
in withAuth(withPermission(...)).

Discovered during the mobile-foundation Task 24 build validation; the route
files predate this branch but the build was never re-run on data-model.

Files:
- berth-reservations/[id], companies/autocomplete, companies/[id]/members
  + nested mid/set-primary, yachts/autocomplete, yachts/[id]/transfer,
  yachts/[id]/ownership-history
- Integration tests updated to import from handlers.ts (companies,
  memberships, reservations, yachts-detail)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-04-29 15:14:40 +02:00
Matt Ciaccio
0e9c24e222 test(visual): add mobile shell snapshot baselines (dashboard + more sheet) 
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-04-29 14:47:26 +02:00
Matt Ciaccio
3aba2181dc feat(test): extract anchor iPhone device descriptors to shared fixture 
Move the four iPhone viewport descriptors (SE, 15/16, 16/17 Pro, Pro Max)
into tests/e2e/fixtures/devices.ts so the upcoming visual spec (Task 23)
can share the same anchors. The mobile-audit spec now spreads each
descriptor and adds a slug `name` plus a human `label` for the run header.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-04-29 14:31:51 +02:00
Matt Ciaccio
9f786fbcf3 feat(mobile): set data-form-factor body attr from User-Agent in root layout 2026-04-29 13:59:03 +02:00
Matt Ciaccio
fbb1f1f366 scaffold(mobile): branch setup — audit harness, spec, plan, gitignore + client-portal cleanup 
Pre-execution baseline for the mobile foundation PR:

- Mobile audit harness (tests/e2e/audit/mobile.spec.ts + mobile-audit Playwright project) — visits every page at four anchor iPhone viewports (375/393/402/440), screenshots full-page to .audit/mobile/, generates index.md
- Design spec (docs/superpowers/specs/2026-04-29-mobile-optimization-design.md) — adaptive shell + responsive content; full active-iPhone-range coverage; foundation + per-page migration phases
- Implementation plan (docs/superpowers/plans/2026-04-29-mobile-foundation.md) — 24 TDD tasks for the foundation PR
- .gitignore: ignore /client-portal/ (legacy nested Nuxt repo) and /.audit/ (regenerable screenshots)
- Remove phantom client-portal gitlink (mode 160000 with no .gitmodules)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-04-29 13:49:38 +02:00
Matt Ciaccio
ba89b61b3f fix(security): port-scope clientId/berthId/yachtId on interests + clientRelationships
All checks were successful
Build & Push Docker Images / lint (pull_request) Successful in 1m17s
Details
Build & Push Docker Images / build-and-push (pull_request) Has been skipped
Details 
Pass-6 findings — both MEDIUM cross-tenant FK injection.

- interests.service: createInterest/updateInterest/linkBerth accepted
  clientId/berthId/yachtId from the request body without verifying the
  referenced row belongs to the caller's port. getInterestById joins
  clients/berths/yachtTags on these FKs without a port filter, so a
  port-A caller could splice a foreign-port id and surface that
  tenant's clientName, mooringNumber, or yacht ownership on read.
  New assertInterestFksInPort helper guards all three surfaces.

- clients.service.createRelationship: accepted clientBId from the
  body without a port check; the relationship list endpoint joins
  clients without filtering by port, so the foreign client's name
  + email would render in the relationships tab. Now verifies
  clientBId belongs to portId and rejects self-relationships.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-04-29 04:14:09 +02:00
Matt Ciaccio
4eea19a85b sec: lock down 5 cross-tenant FK gaps from fifth-pass review 
1. HIGH — reminders.create/updateReminder accepted clientId/interestId/
   berthId from the body and persisted them with no port check; getReminder
   then hydrated the row via Drizzle relations (no port filter on the
   join), so a port-A user with reminders:create could exfiltrate any
   port-B client/interest/berth row by guessing its UUID. New
   assertReminderFksInPort gates create + update.

2. HIGH — listRecommendations(interestId, _portId) discarded portId
   entirely; the route GET /api/v1/interests/[id]/recommendations
   forwarded the URL id straight through. A port-A user with
   interests:view could read any other tenant's recommended berths
   (mooring numbers, dimensions, status). Service now verifies the
   interest belongs to portId and joins berths filtered by port.

3. HIGH — Berth waiting list. The PATCH route did not pre-check that
   the berth belonged to ctx.portId — a port-A user with
   manage_waiting_list could reorder a port-B berth's queue. Separately,
   updateWaitingList accepted arbitrary entries[].clientId and inserted
   them without verifying tenancy, polluting the table with foreign-port
   FKs. Both gaps closed.

4. MEDIUM — setEntityTags (clients/companies/yachts/interests/berths)
   accepted any tagId and inserted into the join table. The tags table
   is per-port but the join only carries a single-column FK. The
   downstream getById join `tags ON join.tag_id = tags.id` has no port
   filter, so a foreign tag's name + color render in the requesting port.
   Helper now batch-validates tagIds belong to portId before insert.

5. MEDIUM — /api/v1/custom-fields/[entityId] PUT had no withPermission
   gate (any role, including viewer, could write) and didn't validate
   that the URL entityId pointed at a port-scoped entity of the field
   definition's entityType. Route now uses
   withPermission('clients','view'/'edit',…); service validates the
   entityId per resolved entityType (client/interest/berth/yacht/company)
   against portId.

Test mocks updated to cover the new entity-port-scope check.
818 vitest tests pass.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-04-29 03:28:31 +02:00
Matt Ciaccio
47a1a51832 sec: webhook SSRF guard, IMAP-sync owner check, watcher port membership 
Three findings from a fourth-pass review:

1. MEDIUM — webhook URL SSRF. The validator only enforced HTTPS+URL
   parse; it accepted private/loopback/link-local/.internal hosts. The
   delivery worker fetched arbitrary URLs and persisted up to 1KB of
   response body into webhook_deliveries.response_body, which is then
   surfaced via the deliveries listing endpoint — a port admin could
   register a webhook to an internal HTTPS endpoint, hit the test
   endpoint to force immediate dispatch, and read the response back.
   Validator now rejects RFC-1918/loopback/link-local/CGNAT/ULA IPs
   (v4 + v6) and .internal/.local/.localhost/.lan/.intranet/.corp
   suffixes; the worker re-resolves the hostname at dispatch time and
   blocks before fetch (DNS rebinding defense). 21-case unit test
   covers the matrix.

2. MEDIUM — POST /api/v1/email/accounts/[id]/sync had no owner check.
   Any user with email:view could enqueue an inbox-sync job for any
   accountId, which the worker would honour using the foreign user's
   decrypted IMAP credentials and advance the account's lastSyncAt
   (data-loss risk on the legitimate owner's next sync). Route now
   asserts account.userId === ctx.userId before enqueueing, matching
   the toggle/disconnect endpoints.

3. MEDIUM — addDocumentWatcher (and the wizard / upload watcher
   inserts) didn't validate the watcher's userId belonged to the
   document's port. notifyDocumentEvent then emitted a real-time
   socket toast + email containing the document title to the foreign
   user. New assertWatchersInPort helper verifies each candidate has
   a userPortRoles row for the port (super-admin bypass).

818 vitest tests pass.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-04-29 03:15:39 +02:00
Matt Ciaccio
e06fb9545b sec: lock down 5 cross-tenant IDORs uncovered in second-pass review 
1. HIGH — /api/v1/admin/ports/[id] PATCH+GET let any port-admin
   (manage_settings) mutate any other tenant's port row by passing the
   foreign id in the path. Now non-super-admins must target their own
   ctx.portId; listPorts and createPort are super-admin only.

2. HIGH — Invoice create/update accepted arbitrary expenseIds and
   linked them into invoice_expenses with no port check; the GET
   response then re-emitted those foreign expense rows via the
   linkedExpenses join. assertExpensesInPort now validates each id
   belongs to the caller's portId before insert; getInvoiceById's
   join filters by expenses.portId as defense-in-depth.

3. HIGH — Document creation paths (createDocument, createFromWizard,
   createFromUpload) persisted user-supplied clientId/interestId/
   companyId/yachtId/reservationId without verifying those FKs were
   in-port. sendForSigning then loaded the foreign client/interest by
   id alone and pushed their PII into the Documenso payload. New
   assertSubjectFksInPort helper rejects out-of-port FKs at create
   time; sendForSigning's interest+client lookups now also filter by
   portId.

4. MEDIUM — calculateInterestScore read its redis cache before
   verifying portId, and the cache key was interestId-only — a
   foreign-port caller could observe a cached score breakdown.
   Cache key now includes portId, and the port-scope DB lookup runs
   before any cache.get.

5. MEDIUM — AI email-draft job results were retrievable by anyone who
   could guess the BullMQ jobId (default sequential integers). Job
   ids are now random UUIDs, requestEmailDraft validates interestId/
   clientId belong to ctx.portId before enqueueing, the worker's
   client lookup is port-scoped, and getEmailDraftResult requires
   the caller to match the original requester's userId+portId before
   returning the drafted subject/body.

The interest-scoring unit test that asserted "DB is bypassed on cache
hit" is updated to reflect the new (security-correct) ordering.
Two new regression test files cover the email-draft binding (5 tests).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-04-29 02:48:43 +02:00
Matt Ciaccio
4c5334d471 sec: gate super-admin invite minting, OCR settings, and alert mutations 
Three findings from the branch security review:

1. HIGH — Privilege escalation via super-admin invite. POST
   /api/v1/admin/invitations was gated only by manage_users (held by the
   port-scoped director role). The body schema accepted isSuperAdmin
   from the request, createCrmInvite persisted it verbatim, and
   consumeCrmInvite copied it into userProfiles.isSuperAdmin — granting
   the new account cross-tenant access. Now the route rejects
   isSuperAdmin=true unless ctx.isSuperAdmin, and createCrmInvite
   requires invitedBy.isSuperAdmin as defense-in-depth.

2. HIGH — Receipt-image exfiltration via OCR settings. The route
   /api/v1/admin/ocr-settings (and the sibling /test) were wrapped only
   in withAuth — any port role including viewer could PUT a swapped
   provider apiKey + flip aiEnabled, redirecting every subsequent
   receipt scan to attacker infrastructure. Both are now wrapped in
   withPermission('admin','manage_settings',…) matching the sibling
   admin routes (ai-budget, settings).

3. MEDIUM — Cross-tenant alert IDOR. dismissAlert / acknowledgeAlert
   issued UPDATE … WHERE id=? with no portId predicate. Any
   authenticated user with a foreign alert UUID could mutate it. Both
   service functions now require portId and add it to the WHERE; the
   route handlers pass ctx.portId.

The dev-trigger-crm-invite script passes a synthetic super-admin caller
identity since it runs out-of-band.

The two public-form tests randomize their IP prefix per run so a fresh
test process doesn't collide with leftover redis sliding-window entries
from a prior run (publicForm limiter pexpires after 1h).

Two new regression test files cover the fixes (6 tests).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-04-29 02:27:01 +02:00
Matt Ciaccio
61e40b5e76 chore(ops): split /api/health (liveness) from /api/ready (readiness) 
Previously /api/health did deep dependency probes (postgres + redis +
minio) and 503'd on any failure. That's readiness behavior, not
liveness — a transient Redis/MinIO blip would tell the orchestrator to
restart the pod when it should only be dropped from the load balancer.

Make /api/health a thin liveness check (returns 200 unconditionally if
the process is responding) and move the deep checks to a new
/api/ready endpoint with the canonical Kubernetes-style 200/503
contract. Docker-compose healthchecks keep pointing at /api/health,
which is now more conservative (no false-positive container restarts).

Documenso/SMTP are intentionally not probed in /api/ready: each tenant
configures its own credentials and a tenant misconfiguration shouldn't
deadline the entire shared CRM.

Also tighten the gdpr-bundle-builder casts: replace the scattered
`as unknown as Record<string, unknown>` double-casts with a small
`toJsonRow<T>()` helper that does the widen narrow→wide in one place
with one cast hop instead of two.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-04-29 02:03:10 +02:00
Matt Ciaccio
43f68ca093 chore(hardening): maintenance jobs, defense-in-depth, redis-backed public rate limit 
- maintenance worker now expires GDPR export bundles (db row + MinIO object)
  on the gdpr_exports.expires_at boundary, plus 90-day retention sweep on
  ai_usage_ledger; both jobs scheduled daily.
- portId scoping added to listClientRelationships and listClientExports
  (defense-in-depth — parent-resource gates already prevent cross-tenant
  reads, but service layer should enforce on its own).
- SELECT FOR UPDATE on parent client/company row inside add/update address
  transactions to serialize concurrent isPrimary toggles.
- public /interests + /residential-inquiries endpoints swap their
  in-memory ipHits maps for the redis sliding-window limiter via the
  new rateLimiters.publicForm config (5/hr/IP), so the cap survives
  restarts and is shared across worker processes.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-04-29 01:52:41 +02:00
Matt Ciaccio
a3305a94f3 feat(gdpr): staff-triggered client-data export bundle (Article 15) 
Adds a full GDPR Article 15 (right of access) workflow. Staff trigger
an export from the client detail; a BullMQ worker assembles every row
keyed to that client (profile, contacts, addresses, notes, tags,
yachts, company memberships, interests, reservations, invoices,
documents, last 500 audit events) into JSON + a self-contained HTML
report, ZIPs them, uploads to MinIO, and optionally emails the client
a 7-day signed download link.

- New table gdpr_exports tracks lifecycle (pending → building → ready
  → sent / failed) with a 30-day cleanup target
- Bundle builder (gdpr-bundle-builder.ts) — pure read-side, tenant-
  scoped, with HTML escaping to block injection from rogue field values
- Worker hook in export queue dispatches on job name 'gdpr-export'
- New audit actions: 'request_gdpr_export', 'send_gdpr_export'
- API: POST/GET /api/v1/clients/:id/gdpr-export (admin-gated, exports
  rate-limit, Article-15 audit on POST); GET /:exportId returns a
  fresh signed URL
- UI: <GdprExportButton> dialog on client detail header — admin-only,
  shows recent exports, supports email-to-client + override recipient,
  polls every 5s while open
- Validation: refuses email-to-client when no primary email + no
  override (rather than silently dropping the send)

Tests: 778/778 vitest (was 771) — +7 covering builder happy path,
HTML escaping, tenant isolation, empty client, request-flow validation,
and audit / queue interaction.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-04-28 20:06:31 +02:00
Matt Ciaccio
9dfa04094b feat(rate-limit): per-user limiters for OCR, AI, and exports 
Adds three named rate limiters to the existing Redis sliding-window
catalog and a withRateLimit wrapper that composes inside withAuth.
Wires the OCR limiter into the receipt-scan endpoint so a runaway
client can't burn through the AI budget in a tight loop.

- ocr: 10/min/user
- ai: 60/min/user (reserved for future server-side AI surfaces)
- exports: 30/hour/user (reserved for GDPR bundle, PDF, CSV exports)

429 responses include X-RateLimit-* headers and a Retry-After hint.

Tests: 771/771 vitest (was 766) — +5 rate-limit tests covering catalog
shape, sliding window, cross-prefix isolation, cross-user isolation,
and resetAt timestamp.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-04-28 19:56:01 +02:00
Matt Ciaccio
e7d23b254c feat(ai): per-port token budgets + usage ledger for AI features 
Adds a token-denominated guardrail in front of every server-side AI call
so a misconfigured port can't run up an unbounded bill. Soft caps surface
a banner; hard caps refuse new requests until the period rolls over.
Usage flows into a feature-typed ledger so future AI surfaces (summary,
embeddings, reply-draft) can drop in without schema changes.

- New table ai_usage_ledger (port, user, feature, provider, model,
  input/output/total tokens, request id) with two indexes for rollup
- New service ai-budget.service.ts: getAiBudget/setAiBudget,
  checkBudget (pre-flight gate), recordAiUsage, currentPeriodTokens,
  periodBreakdown — all token-based, period boundaries in UTC
- runOcr now returns provider usage so the route can record the actual
  spend instead of estimating
- Scan-receipt route gates on checkBudget before invoking AI; returns
  source: manual / reason: budget-exceeded when blocked, surfaces
  softCapWarning on the success path
- Admin UI: new AiBudgetCard on the OCR settings page — shows current
  spend, per-feature breakdown, soft/hard cap inputs, period selector
- Permission: admin.manage_settings on both routes

Tests: 766/766 vitest (was 756) — +10 budget tests covering enforce/
disabled/cap-exceed/estimate-exceed/soft-warn/period boundaries/
cross-port isolation/silent ledger failure.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-04-28 19:53:09 +02:00
Matt Ciaccio
2cf1bd9754 feat(ocr): Tesseract.js as default scanner, AI as opt-in per port 
The mobile receipt scanner now runs Tesseract.js in-browser by default —
on-device, free, and image bytes never leave the device. AI providers
(OpenAI / Claude) become a per-port opt-in for higher accuracy on
hard-to-read receipts.

- Lazy-load Tesseract WASM in src/lib/ocr/tesseract-client.ts (5 MB
  bundle dynamic-imports on first scan, not in main chunk)
- Heuristic parser src/lib/ocr/parse-receipt-text.ts extracts vendor,
  date, amount, currency, and line items from raw OCR text
- New port-scoped aiEnabled flag on OcrConfig (defaults false). Resolved
  flag never inherits from the global row — each port admin opts in
  independently
- Scan endpoint short-circuits to manual-mode when aiEnabled=false so
  the AI provider is never invoked unless the admin has flipped the
  switch
- Scan UI runs Tesseract first, then asks the server whether AI is
  enabled — uses the AI result only when its confidence beats Tesseract;
  network failures degrade gracefully to the local parse
- Admin OCR-settings form gains the per-port aiEnabled checkbox

Tests: 756/756 vitest (was 747) — +7 parser unit tests, +2 aiEnabled
config tests.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-04-28 19:46:29 +02:00
Matt Ciaccio
46937bbcb9 feat(addresses): full CRUD UI for client + company multi-address 
Client and company detail pages each gain an Addresses tab with click-to-edit
fields wired to the existing CountryCombobox/SubdivisionCombobox primitives.
Adds a primary toggle that demotes the previous primary inside one transaction
so the partial unique index never trips.

- New service helpers: list/add/update/remove ClientAddress + CompanyAddress
- New routes: /api/v1/clients/[id]/addresses[/addressId], same under companies/
- New shared component: <AddressesEditor> reused by both detail surfaces
- Integration tests cover happy path, primary demotion, and tenant scoping

Tests: 747/747 vitest (was 741, +6 address tests).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-04-28 19:38:43 +02:00
Matt Ciaccio
27cdbcc695 chore(i18n): drop legacy free-text country/nationality columns 
Test-data only — no production migration needed (per earlier decision).
Schema is now ISO-only; readers convert ISO codes to localized names where
human-readable output is required (EOI documents, invoices, portal).

Migration 0016 drops:
  - clients.nationality
  - companies.incorporation_country
  - client_addresses.{state_province, country}
  - company_addresses.{state_province, country}

Code paths that previously read free-text values now read the ISO column
and pass through `getCountryName()` / `getSubdivisionName()` for rendering.
Document templates ({{client.nationality}}), portal client view, EOI/
reservation-agreement contexts, and invoice billing addresses all updated.

Public yacht-interest endpoint (/api/public/interests) drops the legacy
fields from its insert path and writes ISO codes only. The Zod validators
no longer accept the legacy fields — older website builds posting raw
'incorporationCountry' / 'country' / 'stateProvince' will get 400s.
Server-side phone normalization is unchanged.

Seed data updated to use ISO codes (GB/FR/ES/GR/SE/IT/GH/MC/PA), spread
across continents to keep test fixtures realistic.

Test assertions updated to match the new render shape (e.g.
'United States' not 'US', 'California' not 'CA').

Vitest: 741 -> 741 (unchanged count; assertions updated, no new tests).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-04-28 19:00:57 +02:00
Matt Ciaccio
31fa3d08ec chore(cleanup): Phase 1 — gap closure across audit, alerts, soft-delete, perms 
Multi-area cleanup pass closing partial-implementation gaps surfaced by the
post-i18n audit. No behavior changes for happy-path users; closes real
correctness/security holes.

PR1a Public yacht-interest endpoint i18n. /api/public/interests now accepts
     phoneE164/phoneCountry, nationalityIso, address.{countryIso, subdivisionIso},
     and company.{incorporationCountryIso, incorporationSubdivisionIso}.
     Server-side parsePhone() fallback for legacy raw phone strings.

PR1b Alert rule registry trim. Two rule slots ('document.expiring_soon',
     'audit.suspicious_login') were registered but evaluators returned [].
     Both required schema/instrumentation that hadn't landed. Removed from
     the registry; comments record the dependencies needed to revive them.
     Effective rule count: 8 active.

PR1c vi.mock hoist + flake fix. Hoisted vi.mock calls to top-level in 5
     integration test files; webhook-delivery uses vi.hoisted for the
     queue-add ref. Vitest no longer warns about non-top-level mocks.
     Deflaked the 'short value' assertion in security-encryption.test.ts
     by switching plaintext from 'ab' to 'XY' (non-hex chars). 5/5 runs green.

PR1d Soft-delete reference audit. listClientOptions and listYachtsForOwner
     now filter by isNull(archivedAt). Berths use status (no archivedAt).

PR1e Permission-matrix audit script + report. scripts/audit-permissions.ts
     walks every src/app/api/v1/**/route.ts and reports handlers without a
     withPermission() wrapper. Initial run found 33 violations.
     - Allow-listed 17 with explicit reasons (self-data, admin, alerts,
       search, currency, ai, custom-fields — some marked TODO).
     - Wrapped 7 routes with concrete permissions: clients/options
       (clients:view), berths/options (berths:view), dashboard/*
       (reports:view_dashboard), analytics (reports:view_analytics).
     Audit report at docs/runbooks/permission-audit.md. Script exits
     non-zero on any unallow-listed violation so it can become a CI gate.

Vitest: 741 -> 741 (no new tests; existing suite covers the changes).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-04-28 18:48:22 +02:00
Matt Ciaccio
16d98d630e feat(i18n): country/phone/timezone/subdivision primitives + form wiring 
Cross-cutting i18n polish for forms across the marina + residential + company
domains. Introduces a single source of truth for country/phone/timezone/
subdivision data and replaces every nationality-as-free-text and timezone-
as-string Input with a dedicated combobox.

PR1  Countries — ALL_COUNTRY_CODES (~250 ISO-3166-1 alpha-2), Intl.DisplayNames
     for localized labels, detectDefaultCountry() with navigator-region
     fallback to US, CountryCombobox with regional-indicator flag glyphs +
     compact mode for inline use.
PR2  Phone — libphonenumber-js wrapper (parsePhone / formatAsYouType /
     callingCodeFor), PhoneInput with flag dropdown + national-format
     AsYouType + paste-detect that flips the country dropdown for pasted
     international strings.
PR3  Timezones — country->IANA map (250 entries, multi-zone for AU/BR/CA/CD/
     ID/KZ/MN/MX/RU/US), formatTimezoneLabel ("Europe/London (UTC+1)"),
     TimezoneCombobox with Suggested/All grouping driven by countryHint.
PR4  Subdivisions — wraps the iso-3166-2 npm package (~5000 ISO 3166-2
     codes for every country), per-country cache, SubdivisionCombobox with
     "Pick a country first" / "No regions available" empty states.
PR5  Schema deltas (migration 0015) — clients.nationality_iso, clientContacts
     {value_e164, value_country}, clientAddresses {country_iso, subdivision_iso},
     residentialClients {phone_e164, phone_country, nationality_iso, timezone,
     place_of_residence_country_iso, subdivision_iso}, companies {incorporation_
     country_iso, incorporation_subdivision_iso}, companyAddresses {country_iso,
     subdivision_iso}. Plus shared zod validators (validators/i18n.ts) used
     by every entity validator + route handler.
PR6  ClientForm + ClientDetail — CountryCombobox replaces nationality Input,
     TimezoneCombobox replaces timezone Input (driven by nationalityIso hint),
     PhoneInput conditionally rendered for phone/whatsapp contacts. Inline
     editors (InlineCountryField / InlineTimezoneField / InlinePhoneField)
     for the detail-page overview rows + ContactsEditor.
PR7  Residential client form + detail — phone -> PhoneInput, nationality/
     timezone/place-of-residence-country/subdivision rows in both create
     sheet and inline-editable detail view. Subdivision wipes when country
     flips since codes are country-scoped.
PR8  Company form + detail — incorporation country -> CountryCombobox,
     incorporation region -> SubdivisionCombobox in both modes.
PR9  Public inquiry endpoint — accepts pre-normalized phoneE164/phoneCountry
     and i18n fields from newer website builds, server-side parsePhone()
     fallback for legacy raw-international submissions. Old Nuxt builds
     keep working unchanged.

Tests: 4 unit suites for the primitives (25 tests), 1 integration spec for
the public phone-normalization path (3 tests), 1 smoke spec asserting the
combobox triggers render in all three create sheets.

Test totals: vitest 713 -> 741 (+28).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-04-28 18:13:08 +02:00
Matt Ciaccio
f52d21df83 feat(phase-b): ship analytics dashboard, alerts, scanner PWA, dedup, audit view 
Phase B (Insights & Alerts) PR4-11 in one drop. Builds on the schema +
service skeletons committed in PRs 1-3.

PR4  Analytics dashboard — 4 chart types (funnel/timeline/breakdown/source),
     date-range picker (today/7d/30d/90d), CSV+PNG export per card.
PR5  Alert rail UI + /alerts page — topbar bell w/ live count, dashboard
     right-rail, three-tab page (active/dismissed/resolved), socket-driven
     invalidation. Bell lazy-loads list on popover open to keep cold pages
     fast in non-dashboard routes.
PR6  EOI queue tab on documents hub — filters to in-flight EOIs, count
     surfaces in tab label.
PR7  Interests-by-berth tab on berth detail — replaces the stub.
PR8  Expense duplicate detection — BullMQ job runs scan on create, yellow
     banner on detail w/ Merge / Not-a-duplicate, transactional merge
     consolidates receipts and archives the source.
PR9  Receipt scanner PWA + multi-provider AI — port-scoped /scan route in
     its own (scanner) group with no dashboard chrome, dynamic per-port
     manifest, OpenAI + Claude provider abstraction, admin OCR settings
     page (port-level + super-admin global default w/ opt-in fallback),
     test-connection endpoint, manual-entry fallback when no key is
     configured. Verify form always shown before save — no ghost rows.
PR10 Audit log read view — swap to tsvector full-text search on the
     existing GIN index, cursor pagination, filters for entity/action/user
     /date range, batched actor-email resolution.
PR11 Real-API tests — opt-in receipt-ocr.spec (admin save+test, optional
     real-receipt parse via REALAPI_RECEIPT_FIXTURE) and alert-engine
     socket-fanout spec gated behind RUN_ALERT_ENGINE_REALAPI. Both skip
     cleanly without their gate envs so CI stays green.

Test totals: vitest 690 -> 713, smoke 130 -> 138, realapi +2 opt-in.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-04-28 17:21:55 +02:00
Matt Ciaccio
01b201e1a2 feat(analytics): real computations + 15-min snapshot refresh job 
PR3 of Phase B. Replaces the no-op stubs in analytics.service.ts with
working drizzle queries and adds the recurring BullMQ job that warms
the cache.

Computations:
- computePipelineFunnel: groups interests by pipeline_stage filtered by
  port + range + not archived; emits 8-row stages array with conversion
  pct relative to 'open' as the funnel top.
- computeOccupancyTimeline: per day in range, counts berths covered by
  an active reservation (start_date ≤ day, end_date IS NULL OR ≥ day);
  emits {date, occupied, total, occupancyPct}.
- computeRevenueBreakdown: sums invoices.total grouped by status +
  currency; filters out archived rows.
- computeLeadSourceAttribution: counts interests by source descending;
  null source bucketed as 'unspecified'.

Public API (getPipelineFunnel, getOccupancyTimeline, etc.) reads
analytics_snapshots first; falls back to compute + writeSnapshot. TTL
15 minutes (matches the cron interval).

Cron:
- queue/scheduler.ts registers 'analytics-refresh' on maintenance with
  pattern '*/15 * * * *'.
- queue/workers/maintenance.ts dispatches to refreshSnapshotsForPort
  for every port; per-port try/catch so one bad port doesn't kill the
  sweep.

Tests: tests/integration/analytics-service.test.ts (9 cases). Pipeline
funnel math (incl. zero state), occupancy timeline shape/percentages
with seeded reservations, revenue grouped by status + currency, lead
source attribution incl. null bucketing, cache hit (mutate snapshot
directly → next read returns mutated value), refreshSnapshotsForPort
warms every metric×range combo.

Vitest 690/690 (+9). tsc + lint clean.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-04-28 14:54:46 +02:00
Matt Ciaccio
df495133b7 feat(alerts): rule engine, recurring evaluator, socket fanout 
PR2 of Phase B. Wires the alert framework end-to-end:

- alert-rules.ts: 10 rule evaluators implemented as pure async fns over
  the existing schema. reservation.no_agreement, interest.stale,
  document.signer_overdue, berth.under_offer_stalled, expense.duplicate,
  expense.unscanned, interest.high_value_silent, eoi.unsigned_long,
  audit.suspicious_login fire against real conditions.
  document.expiring_soon stays inert until the documents schema gets an
  expires_at column. audit.suspicious_login also stays inert until the
  auth layer logs 'login.failed' rows (TODO noted in the rule body).

- alert-engine.ts: runAlertEngine() walks every port × every rule and
  calls reconcileAlertsForPort. Errors per (port, rule) are collected
  in the summary, not thrown — one bad evaluator can't stop the sweep.

- alerts.service.ts: reconcileAlertsForPort now emits 'alert:created'
  socket events on insert and 'alert:resolved' on auto-resolve;
  dismissAlert emits 'alert:dismissed'. All scoped to port:{portId}
  rooms.

- socket/events.ts: adds the three Server→Client alert event types.

- queue/scheduler.ts: registers 'alerts-evaluate' on the maintenance
  queue with cron */5 * * * * (every 5 min, per spec risk register).

- queue/workers/maintenance.ts: dispatches 'alerts-evaluate' to
  runAlertEngine; logs sweep summary.

Tests:
- tests/integration/alerts-engine.test.ts (6 cases): seeds reservation
  → fires, runs twice → no dupe, adds agreement → auto-resolves; seeds
  stale interest → fires; hot lead silent → critical; engine summary
  shape on no-data port. Socket emit module is vi.mocked.

Vitest 681/681 (was 675; +6). tsc clean. Lint clean.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-04-28 14:50:55 +02:00