letsbe/pn-new-crm
1
0
Fork 0
Code Issues Pull Requests 1 Actions Packages Projects Releases Wiki Activity

feat(post-audit): Phase 4 polish + Phase 2 wiring + Phase 6 cron + CLAUDE.md

Browse Source 
Three of the master plan's "suggested execution order" items shipped this
session; Phase 3b (EOI dialog overrides) deferred — estimate exceeded the
remaining session time.

- Phase 4 polish: yachtId field on <ReminderForm> via the existing
  YachtPicker, Ship-icon subtitle on <ReminderCard>, listReminders filter
  by yachtId, getReminder joins the yacht relation.
- Phase 2 risk-signal data wiring: getInterestById derives the 3 dates
  (dateDocumentDeclined / dateReservationCancelled / dateBerthSoldToOther)
  from document_events / berth_reservations / cross-interest interest_berths
  in parallel — chosen over new schema columns to keep the master plan's
  "no new tables" promise. Threaded through to DealPulseChip.
- Phase 6 cron + UI: src/jobs/processors/imap-bounce-poller.ts polls the
  configured IMAP mailbox (IMAP_* env), matches NDRs to recent
  document_sends rows via recipient + 7-day window, idempotent via
  bounceDetectedAt, fires email_bounced notifications on hard/soft
  (skips OOO). State persisted to system_settings.bounce_poller_state.
  Wired into maintenance queue at */15 * * * *. Admin /admin/sends page
  surfaces the bounce badge + reason inline.
- CLAUDE.md: trimmed 27KB → ~19.5KB (~28% smaller bytes). Prose-heavy
  Documenso webhook / v1-v2 routing / Document folders sections rewritten
  as scannable bullets. Added a new "Working in this repo — skills, MCPs,
  agents" section promoting brainstorming/TDD/debugging/frontend-design
  skills, Context7/Playwright/Serena MCPs, and the Explore/feature-dev
  agents. Documented Phase 2 derivation choice in the data-model section.

Quality gates: 1374/1374 vitest pass, tsc --noEmit clean, lint 0 errors.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
Matt
2026-05-18 15:38:37 +02:00
parent a6e79231f3
commit 503207ef68
13 changed files with 561 additions and 123 deletions
Download Patch File Download Diff File Expand all files Collapse all files

255
CLAUDE.md
View File

@@ -1,18 +1,17 @@
# Port Nimara CRM
Multi-tenant CRM for marina/port management. Built with Next.js 15 App Router (standalone output), React 19, TypeScript (strict), Tailwind CSS 3, and Drizzle ORM on PostgreSQL.
Multi-tenant CRM for marina/port management. Next.js 15 App Router (standalone), React 19, TypeScript strict (`noUncheckedIndexedAccess`, no `any`), Drizzle ORM on PostgreSQL.
## Quick reference
```bash
pnpm dev # Start dev server
pnpm dev # Dev server
pnpm build # Production build
pnpm lint # ESLint
pnpm format # Prettier
pnpm lint / format # ESLint / Prettier
pnpm db:generate # Generate Drizzle migrations
pnpm db:push # Push schema to DB
pnpm db:studio # Drizzle Studio GUI
pnpm db:seed # Seed database (tsx src/lib/db/seed.ts)
pnpm db:seed # Seed (tsx src/lib/db/seed.ts)
# Tests
pnpm exec vitest run # Unit + integration (~3s)
@@ -26,25 +25,45 @@ pnpm exec playwright test --project=visual --update-snapshots # Regenerate base
# Dev helpers
pnpm tsx scripts/dev-trigger-portal-invite.ts # Send a portal activation email
pnpm tsx scripts/dev-imap-probe.ts # Dump recent IMAP inbox messages
# Schema migration (pnpm db:migrate is broken — apply via psql)
PGPASSWORD=changeme psql -h localhost -p 5434 -U crm -d port_nimara_crm -f src/lib/db/migrations/0075_*.sql
```
## Tech stack
## Working in this repo — skills, MCPs, agents
- **Framework:** Next.js 15.1 App Router, `output: 'standalone'`, `experimental.typedRoutes`
- **Auth:** better-auth (session cookie: `pn-crm.session_token`)
- **Database:** PostgreSQL via `postgres` driver + Drizzle ORM
Reach for these before grinding through tasks manually:
- **Skills** (invoke with `Skill` tool):
- `superpowers:brainstorming` before any feature/component work — explores intent + design first
- `superpowers:test-driven-development` for any feature or bugfix
- `superpowers:systematic-debugging` for any bug / test failure / unexpected behavior
- `superpowers:verification-before-completion` before claiming "done" or committing
- `superpowers:writing-plans` / `executing-plans` for multi-step specs
- `superpowers:dispatching-parallel-agents` when 2+ tasks are independent
- `frontend-design:frontend-design` for new UI work (avoids generic AI aesthetics)
- `code-review:code-review` and `security-review` before merging
- **MCPs**:
- **Context7** (`mcp__plugin_context7_context7__*`) — pull current docs for Next 15, Drizzle, better-auth, BullMQ, Tailwind, Radix etc. Prefer over web search; our training data lags.
- **Playwright** (`mcp__plugin_playwright_playwright__*`) — verify UI changes in a real browser before reporting "done". Default viewport — do NOT call `browser_resize`.
- **Serena** (`mcp__plugin_serena_serena__*`) — symbol-level navigation (`find_symbol`, `find_referencing_symbols`, `replace_symbol_body`). Much faster than grep for "where is this called".
- **Postman** (`mcp__claude_ai_Postman__*`) — when designing or auditing API surfaces.
- **Agents** (via `Agent` tool, `subagent_type=`):
- `Explore` for any codebase search that would take > 3 queries
- `feature-dev:code-explorer` / `code-architect` / `code-reviewer` for new feature work
- **Doctrine**: skills override default behavior except user instructions in this file. If a CLAUDE.md rule conflicts with a skill, this file wins.
## Tech stack (non-obvious choices)
- **Auth:** better-auth — session cookie `pn-crm.session_token`
- **Queue:** BullMQ + Redis (ioredis)
- **Storage:** MinIO (S3-compatible)
- **Storage:** pluggable via `getStorageBackend()` — MinIO/S3 default; never import the S3 SDK directly
- **Realtime:** Socket.IO with Redis adapter
- **UI:** Radix UI primitives, shadcn/ui components (`src/components/ui/`), Lucide icons, CVA + tailwind-merge + clsx
- **UI:** Radix UI + shadcn/ui (`src/components/ui/`) + Lucide + CVA + tailwind-merge
- **Forms:** react-hook-form + zod resolvers
- **Tables:** TanStack Table
- **State:** Zustand stores (`src/stores/`), TanStack React Query
- **PDF:** pdfme
- **State:** Zustand (`src/stores/`) + TanStack React Query
- **PDF:** pdfme (templates) + pdf-lib (AcroForm fill)
- **Email:** nodemailer + imapflow + mailparser
- **AI:** OpenAI SDK (optional)
- **Testing:** Vitest (unit), Playwright (e2e)
- **Logging:** pino + pino-pretty
## Project structure
@@ -52,134 +71,144 @@ pnpm tsx scripts/dev-imap-probe.ts # Dump recent IMAP inbox m
src/
app/
(auth)/ # Login/auth pages
(dashboard)/ # Main app - route: /[portSlug]/...
(dashboard)/ # Main app — route: /[portSlug]/...
(portal)/ # Client portal
api/ # API routes
api/ # API routes (route.ts + sibling handlers.ts)
components/
ui/ # shadcn/ui base components
layout/ # Shell, sidebar, header
[domain]/ # Domain components (clients, invoices, berths, etc.)
shared/ # Cross-domain shared components
hooks/ # React hooks (use-auth, use-permissions, use-socket, etc.)
[domain]/ # clients, yachts, companies, reservations, berths, …
shared/ # Cross-domain (BrandedAuthShell, InlineEditableField, …)
hooks/ # use-auth, use-permissions, use-socket, …
lib/
api/ # API client utilities
api/ # Route helpers (parseBody, errorResponse, withAuth, …)
auth/ # better-auth config
db/
schema/ # Drizzle schema (one file per domain)
migrations/ # Generated Drizzle migrations
db/schema/ # Drizzle schema — one file per domain, re-exported from index.ts
db/migrations/ # Generated Drizzle migrations (apply via psql in dev)
env.ts # Zod env validation (SKIP_ENV_VALIDATION=1 bypasses)
services/ # Business logic services
validators/ # Zod schemas for API input validation
utils/ # Shared utilities
services/ # Business logic
storage/ # Pluggable storage backend
templates/ # Email/document merge fields, berth-range formatter
validators/ # Zod schemas for API input
middleware.ts # Auth middleware (cookie check, redirects)
providers/ # React context providers
stores/ # Zustand stores
types/ # Shared TypeScript types
stores/ # Zustand
```
## Conventions
## Conventions & gotchas
- **TypeScript:** Strict mode with `noUncheckedIndexedAccess`. No `any` (ESLint error).
- **Formatting:** Prettier - single quotes, semicolons, trailing commas, 2-space indent, 100 char line width.
- **Lint:** ESLint flat config extending `next/core-web-vitals`, `next/typescript`, `prettier`. Unused vars prefixed with `_` are allowed.
- **Imports:** Use `@/*` path alias (maps to `src/*`).
- **Components:** shadcn/ui pattern - base components in `src/components/ui/`, domain components in `src/components/[domain]/`. Yacht / company / reservation domains live in `components/yachts`, `components/companies`, `components/reservations` respectively.
- **DB schema:** One file per domain in `src/lib/db/schema/`, re-exported from `index.ts`. Relations in `relations.ts`. Domain files include `clients.ts`, `yachts.ts`, `companies.ts`, `reservations.ts`, `interests.ts`, `berths.ts`, `documents.ts`, `invoices.ts`, etc.
- **Polymorphic ownership:** Yachts and invoice billing-entities use `<entity>_type` + `<entity>_id` column pairs (`'client' | 'company'`). Resolve owner identity through `src/lib/services/yachts.service.ts` / `eoi-context.ts` rather than reading the columns ad hoc — those services apply the type discriminator.
- **EOI generation:** Two pathways share the same `EoiContext` (`src/lib/services/eoi-context.ts`). Documenso pathway calls the template-generate endpoint via `documenso-payload.ts`; in-app pathway fills the same source PDF (`assets/eoi-template.pdf`) via `src/lib/pdf/fill-eoi-form.ts` (pdf-lib AcroForm). Routed through `generateAndSign(...)` in `src/lib/services/document-templates.ts` with a `pathway` parameter.
- **Merge fields:** Token catalog lives in `src/lib/templates/merge-fields.ts`; the `createTemplateSchema` validator uses `VALID_MERGE_TOKENS` as an allow-list, so unknown tokens are rejected at template creation time.
- **Documenso webhooks:** Documenso (both v1.13 and 2.x) authenticates outbound webhooks by sending the configured secret in plaintext via the `X-Documenso-Secret` header — there is no HMAC. The receiver at `src/app/api/webhooks/documenso/route.ts` does a timing-safe equality check via `verifyDocumensoSecret`. Event names arrive as the uppercase Prisma enum on the wire (`DOCUMENT_SIGNED`, `DOCUMENT_COMPLETED`, etc.) even though the UI displays them as lowercase-dotted. The route also normalizes lowercase-dotted variants for forward-compat. `handleDocumentCompleted` is **idempotent** — early-returns when `doc.status === 'completed' && doc.signedFileId` so Documenso retries on 5xx don't insert duplicate file rows + orphan blobs. The switch handles `DOCUMENT_SIGNED|COMPLETED|REJECTED|DECLINED|OPENED|EXPIRED`, plus v2 aliases `RECIPIENT_VIEWED` / `RECIPIENT_SIGNED` (logged + routed to v1 equivalents).
- **Documenso API responses:** 2.x renamed `id` → `documentId` and recipient `id` → `recipientId`; v1.13 still uses `id`. `src/lib/services/documenso-client.ts` runs every response through `normalizeDocument()` which reads either field name and surfaces the legacy `id` form to downstream consumers.
- **Documenso v1 vs v2 endpoint routing:** `getPortDocumensoConfig(portId)` resolves the per-port `apiVersion` ('v1' | 'v2'). `documenso-client.ts` exports version-aware wrappers: `getDocument`, `createDocument`, `sendDocument`, `sendReminder`, `downloadSignedPdf`, `voidDocument`, `placeFields`. v2 → `/api/v2/envelope/*` (`create` is multipart with `{payload, files}`; `distribute` returns per-recipient `signingUrl` in one round-trip; `redistribute` for reminders; `field/create-many` for bulk placement with percent coords + `fieldMeta`). v1 → existing `/api/v1/documents/*` paths. **Template flow is intentionally still v1** (`/api/v1/templates/{id}/generate-document` with `formValues` keyed by name) — v2 instances accept it via backward compat. Full v2 `/template/use` migration with `prefillFields` by ID needs per-template field-ID capture in admin settings and is deferred. Two per-port v2 settings now wired through `buildDocumensoPayload` + `documensoCreate.meta`: `documenso_signing_order` (PARALLEL/SEQUENTIAL — v2-enforced) and `documenso_redirect_url` (post-sign redirect; both versions honour). `checkDocumensoHealth` returns the resolved `apiVersion` for the admin Test button.
- **Email templates:** Branded HTML lives in `src/lib/email/templates/`. The portal-auth flow uses `portal-auth.ts` (activation + reset). All templates use the legacy table-based layout with the Port Nimara logo + blurred overhead background, max-width 600px and `width:100%` for responsive shrink. The `<img>` URLs reference `s3.portnimara.com` directly (will move to `/public` later).
- **Portal auth pages:** `/portal/login`, `/portal/activate`, `/portal/reset-password` and the CRM `/login`, `/reset-password`, `/set-password` all wrap their content in `<BrandedAuthShell>` (`src/components/shared/branded-auth-shell.tsx`) which renders the same blurred background + logo + white card the email templates use, so the in-app and email surfaces look unified.
- **Sheet vs Drawer doctrine:** `<Sheet side="right">` (`src/components/ui/sheet.tsx`, Radix dialog) is the canonical side-panel for forms and previews on **both** desktop and mobile (`w-3/4 ... sm:max-w-sm` adapts naturally). Vaul `<Drawer>` (`src/components/shared/drawer.tsx`) is reserved for **mobile-only bottom-sheet UX** — currently just the `MoreSheet` nav (`src/components/layout/mobile/more-sheet.tsx`). If you need a side panel of any kind, use Sheet. Don't add new Vaul drawers without a mobile-bottom-sheet justification.
- **Inline editing pattern:** detail pages (clients, yachts, companies, interests, residential clients/interests) use `<InlineEditableField>` (`src/components/shared/inline-editable-field.tsx`) for click-to-edit text/select/textarea fields and `<InlineTagEditor>` (`src/components/shared/inline-tag-editor.tsx`) for tag chips. Each entity exposes a `PUT /api/v1/<entity>/[id]/tags` endpoint backed by a `set<Entity>Tags` service helper that wipes-and-rewrites the join table inside a single transaction. There are no separate "Edit" modal forms on detail pages — the entire overview tab is editable in place.
- **Notes (polymorphic across entity types):** `notes.service.ts` dispatches across `clientNotes`, `interestNotes`, `yachtNotes`, `companyNotes` based on an `entityType` discriminator. `<NotesList entityType="…" />` works for all four. `companyNotes` lacks an `updatedAt` column — the service substitutes `createdAt` so callers get a uniform shape.
- **Document folders:** Per-port nestable tree (`document_folders` self-FK on `parent_id`; null parent = root). Documents and files carry a nullable `folder_id` (null = root). Sibling-name uniqueness via `uniq_document_folders_sibling_name` on `(port_id, COALESCE(parent_id, '__root__'), LOWER(name))`. Folder delete is **soft rescue**: `deleteFolderSoftRescue` re-parents every child folder + document + file up to the deleted folder's parent (or to root) inside a transaction, then drops the folder row — never CASCADE. Cycle prevention in `moveFolder` walks the destination's ancestor chain.
### API shape
Three system roots (`Clients/`, `Companies/`, `Yachts/`) are auto-created on port init via `ensureSystemRoots`. Per-entity subfolders are created lazily on first auto-deposit / manual upload via `ensureEntityFolder` — concurrent callers race safely via the partial unique index `uniq_document_folders_entity` on `(port_id, entity_type, entity_id) WHERE entity_id IS NOT NULL`. The `chk_system_folder_shape` CHECK pins the shape of system rows. Rename/move/delete on `system_managed = true` folders is rejected by `assertNotSystemManaged` (service-level, not DB-level). Entity rename auto-syncs the folder name via `syncEntityFolderName`; archive applies a ` (archived)` suffix via `applyEntityArchivedSuffix`; hard-delete demotes (`system_managed = false`) + appends ` (deleted)` via `demoteSystemFolderOnEntityDelete`.
- **Envelope:** `{ data: <T> }` for any returned content (read OR write). Mutations returning nothing emit `204 No Content`. Don't use `{ success: true }` (legacy; normalized away 2026-05-07). Public portal-auth endpoints keep `{ success: true }` so the frontend can chain.
- **Lists:** `{ data: <T[]>, total?, hasMore? }` — see `/api/v1/clients`.
- **Errors:** always via `errorResponse(error)` from `@/lib/errors` (request-id propagation + audit-tier mapping).
- **Body parsing:** always `parseBody(req, schema)` from `@/lib/api/route-helpers`. Raw `req.json() + schema.parse()` produces a generic 500 instead of the field-level 400 the frontend's `toastError` hook expects.
- **Route handlers:** `route.ts` files can only export `GET|POST|…`. Service-tested handlers live in sibling `handlers.ts` (e.g. `src/app/api/v1/yachts/[id]/handlers.ts`) and are imported by `route.ts` with `withAuth(withPermission(...))`. Integration tests import from `handlers.ts` directly to bypass middleware.
Auto-deposit on signing completion: `handleDocumentCompleted` resolves the owner via the Owner-wins chain (`document.clientId ?? .companyId ?? .yachtId ?? interest.clientId`), ensures the matching entity subfolder, and sets `files.folder_id` + the matching entity FK on the signed file row. Falls back to root when no owner is resolvable. (Notes: `interests` table has no `companyId` column, hence the chain's interest fallback omits it; `interests.clientId` is NOT NULL so an `interest.yachtId` tail branch — if added — would be structurally unreachable.)
### Data model
Aggregated projection: `listFilesAggregatedByEntity` / `listInflightWorkflowsAggregatedByEntity` walk the relationship graph from the requested entity (symmetric reach: Client ↔ Company via `company_memberships` filtered to active rows via `isNull(end_date)`, ↔ Yacht via `yachts.current_owner_type/id`) and return results grouped by source (DIRECTLY ATTACHED / FROM COMPANY / FROM YACHT / FROM CLIENT). Each group caps at 20 rows with a total for `Show all (N)`. The files projection LEFT JOINs `documents` on `signed_file_id` to surface `signedFromDocumentId` per row — used by the UI's "view signing details" link. **File-FK snapshot is the source of truth** — historical files stay where they were filed even if the linked entity's relationships change. **Defense-in-depth `port_id` filter at every join** (per recommender precedent) — entry-point check alone is rejected. Completed workflows are hidden from folder views (`listDocuments` excludes `status='completed'` when `folderId` is set); the signed-PDF file surfaces in the Files section with a "view signing details" link to the workflow audit trail (via `GET /api/v1/documents/[id]/signing-details`).
Hub UI: rebuilt around three render modes — `HubRootView` (no folder), `EntityFolderView` (system-managed entity subfolder, renders Signing-in-progress + Files via the aggregated projection), `FlatFolderListing` (any other folder). Sidebar shows lock markers on system folders and mutes archived entity folders. The signing-status tabs strip (`in_progress` / `awaiting_them` / etc.) was removed; folders are now the primary navigation.
Permission gating: `documents.view` for read of folders + entity-aggregated listing; `documents.manage_folders` for create / rename / move / delete of user folders (system folders are immutable through the API entirely).
Deploy: schema migration `0051_documents_hub_split.sql` ships the columns; `pnpm db:backfill:doc-folders` (script `scripts/backfill-document-folders.ts`) runs after the migration and is idempotent (per-port `pg_advisory_xact_lock`).
- **Route handler exports:** Next.js App Router `route.ts` files only allow specific named exports (`GET|POST|…`). Service-tested handler functions live in sibling `handlers.ts` files (e.g. `src/app/api/v1/yachts/[id]/handlers.ts`) and are imported by the colocated `route.ts` for `withAuth(withPermission(...))` wrapping. Integration tests import from `handlers.ts` directly to bypass auth/permission middleware.
- **Multi-berth interest model:** `interest_berths` is the source of truth for which berths an interest is linked to; `interests.berth_id` does not exist (dropped in migration 0029). Three role flags: `is_primary` (≤1 row per interest, enforced by partial unique index — surfaces as "the berth for this deal" in templates / forms / list views), `is_specific_interest` (true → berth shows as "Under Offer" on the public map; false → legal/EOI-only link), `is_in_eoi_bundle` (covered by the interest's EOI signature). Read/write through `src/lib/services/interest-berths.service.ts` helpers (`getPrimaryBerth`, `getPrimaryBerthsForInterests`, `upsertInterestBerth`, `setPrimaryBerth`, `removeInterestBerth`); never query `interest_berths` from outside that service.
- **Mooring number canonical format:** `^[A-Z]+\d+$` (e.g. `A1`, `B12`, `E18`) — no hyphen, no leading zeros. Stored, displayed, URL-encoded, and rendered in EOIs in this exact form. Phase 0 normalized the entire CRM dataset; the mooring-pattern regex gates the public `/api/public/berths/[mooringNumber]` route before any DB hit.
- **Public berths API:** `/api/public/berths` (list) and `/api/public/berths/[mooringNumber]` (single) are the public-facing data feed for the marketing website. Output shape mirrors the legacy NocoDB Berths shape verbatim (`"Mooring Number"`, `"Side Pontoon"`, etc.) — see `src/lib/services/public-berths.ts`. Cache headers: `s-maxage=300, stale-while-revalidate=60`. Status mapping: `"Sold"` (berth.status=sold) > `"Under Offer"` (status=under_offer OR has any active `interest_berths.is_specific_interest=true` link with `interests.outcome IS NULL`) > `"Available"`. The companion `/api/public/health` endpoint is dual-mode: anonymous callers get `{status, timestamp}` (uptime monitors, never 503); requests carrying a timing-safe-matched `X-Intake-Secret` (compared against `WEBSITE_INTAKE_SECRET`) get the full `{status, env, appUrl, timestamp, checks: {db, redis}}` payload and a 503 if any dependency is down. The website uses the authenticated form on startup so it refuses to start when its `CRM_PUBLIC_URL` points at a different deployment env.
- **Berth recommender:** Pure SQL ranking (no AI). Lives in `src/lib/services/berth-recommender.service.ts`. Tier ladder A/B/C/D classifies each feasible berth based on its `interest_berths` aggregates. Heat scoring (recency / furthest stage / interest count / EOI count) only fires for tier B (lost/cancelled-only history); per-port admin tunes weights via `system_settings` keys (`heat_weight_*`, `recommender_max_oversize_pct`, `recommender_top_n_default`, `fallthrough_policy`, `fallthrough_cooldown_days`, `tier_ladder_hide_late_stage`). The recommender enforces multi-port isolation both at the entry point (rejects cross-port interest lookups) AND inside the SQL aggregates CTE (defense-in-depth `i.port_id` filter).
- **Berth rules engine:** Per-port `system_settings` rules in `src/lib/services/berth-rules-engine.ts`. Seven triggers, all wired: `eoi_sent`, `eoi_signed`, `deposit_received` (invoices.ts), `contract_signed` (documents.service.ts), `interest_archived` / `interest_completed` (interests.service.ts), `berth_unlinked` (interest-berths.service.ts). Service callers fire `evaluateRule(trigger, interestId, portId, meta)` via dynamic import to avoid circular deps. Default modes vary (`auto` for state changes, `suggest` for recommendations, `off` for `berth_unlinked`); admins tune via `berth_rules` system_settings key. Webhook auto-advance pairs the rule with `advanceStageIfBehind` so the pipeline stage and berth status move together.
- **EOI bundle / range formatter:** Multi-berth EOIs render the in-bundle berth set as a compact range string ("A1-A3, B5-B7") via `formatBerthRange()` in `src/lib/templates/berth-range.ts`. The output populates the existing `Berth Number` Documenso form field (single-berth output is byte-identical to the primary mooring, multi-berth shows the full range). CRM UI always shows berths as individual chips. The `{{eoi.berthRange}}` token is in `VALID_MERGE_TOKENS` for template body copy.
- **Pluggable storage backend:** Code never imports MinIO/S3 directly. All file I/O goes through `getStorageBackend()` from `src/lib/storage/`. The `StorageBackend` interface requires `put`, `get`, `head`, `delete`, `listByPrefix`, `presignUpload`, `presignDownload` — any new backend must implement all seven. Configured via `system_settings.storage_backend` ('s3' | 'filesystem'). Switching backends is a settings change + `pnpm tsx scripts/migrate-storage.ts` run (the migrator round-trips every blob in `files`, `berth_pdf_versions`, `brochure_versions`, `gdpr_exports` and verifies SHA-256 — `TABLES_WITH_STORAGE_KEYS` populated in 9a5ba87; was no-op before). MinIO ops are wrapped in a 30s `withTimeout` to prevent TCP-blackhole worker stalls. **Filesystem backend is single-node only**: refuses to start when `MULTI_NODE_DEPLOYMENT=true`. Multi-node deployments must use the s3-compatible backend.
- **Per-berth PDFs:** Versioned via `berth_pdf_versions`; `berths.current_pdf_version_id` always points to the latest active version. Storage key is UUID-based per upload (not version-numbered) so concurrent uploads can't collide on blob paths; `pg_advisory_xact_lock` per berth_id serializes the version-number allocation. 3-tier parser: AcroForm → OCR (Tesseract.js with positional heuristics) → optional AI (rep clicks "AI parse" only when OCR confidence is low). Magic-byte (`%PDF-`) check enforced on BOTH the in-server upload path AND the presigned-PUT path (the post-upload service streams the first 5 bytes via the storage backend). Mooring-number mismatch between PDF and target berth surfaces as a service-level `ConflictError` unless the apply call passes `confirmMooringMismatch: true`.
- **Brochures:** Per-port; default brochure marked via `is_default` (enforced by partial unique index on `(port_id) WHERE is_default=true AND archived_at IS NULL`). Archived brochures retain version history. Same upload flow as berth PDFs (presign + magic-byte verification on the post-upload register endpoint).
- **Send-from accounts (sales send-outs):** Configurable via `system_settings`; defaults to `sales@portnimara.com` for human-touch and `noreply@portnimara.com` for automation. SMTP/IMAP passwords are AES-256-GCM encrypted at rest; the API never returns decrypted secrets — only `*PassIsSet` boolean markers. Send-out audit goes to `document_sends` (separate from `audit_logs` because of volume + binary refs). Body markdown is XSS-safe via `renderEmailBody()` (escape-then-allowlist; tested against the standard XSS vector list). Rate limit: 50 sends/user/hour individual. Pre-send size threshold: files > `email_attach_threshold_mb` ship as a 24h signed-URL link rather than an attachment (avoids the duplicate-send race from async bounces). The download-link fallback HTML-escapes the filename to prevent injection from admin-supplied brochure names. Bounce monitoring requires IMAP credentials in addition to SMTP — without them, the size-rejection banner stays disabled.
- **NocoDB berth import:** `pnpm tsx scripts/import-berths-from-nocodb.ts --apply --port-slug port-nimara` re-imports from the legacy NocoDB Berths table. Idempotent: rows where `updated_at > last_imported_at` (the "human edited this since last import" guard) are skipped unless `--force`. Adds `--update-snapshot` to also rewrite `src/lib/db/seed-data/berths.json`. Uses `pg_advisory_xact_lock` so two simultaneous runs serialize. Pure helpers in `src/lib/services/berth-import.ts` are unit-tested.
- **Polymorphic ownership:** Yachts and invoice billing-entities use `<entity>_type` + `<entity>_id` pairs (`'client' | 'company'`). Resolve via `src/lib/services/yachts.service.ts` / `eoi-context.ts` — never read the columns ad hoc.
- **Deal-pulse risk signals:** `dateDocumentDeclined`, `dateReservationCancelled`, `dateBerthSoldToOther` are NOT columns on `interests` — they're **derived at read time** inside `getInterestById` from `document_events` (eventType in `('rejected','declined')`), `berth_reservations` (status='cancelled'), and other won interests sharing the same berth via `interest_berths`. The Phase 2 design call: derive vs. denormalize — derivation kept the master plan's "no new tables" promise. Cost: 3 extra SELECTs on the detail endpoint (run in parallel); list views don't render the chip so they're unaffected.
- **Multi-berth interest model:** `interest_berths` is the source of truth — `interests.berth_id` does not exist (dropped in 0029). Three flags: `is_primary` (≤1 per interest, partial unique index — "the berth for this deal"), `is_specific_interest` (true → public map shows "Under Offer"), `is_in_eoi_bundle` (covered by EOI signature). Read/write only via `src/lib/services/interest-berths.service.ts` helpers.
- **Notes (polymorphic):** `notes.service.ts` dispatches across `clientNotes`/`interestNotes`/`yachtNotes`/`companyNotes` via an `entityType` discriminator. `<NotesList entityType="…" />` works for all four. `companyNotes` lacks `updatedAt` — service substitutes `createdAt` for shape uniformity.
- **Mooring number canonical format:** `^[A-Z]+\d+$` (e.g. `A1`, `B12`, `E18`) — no hyphen, no leading zeros. Stored, displayed, URL-encoded, EOI-rendered in this exact form. Regex gates the public `/api/public/berths/[mooringNumber]` route before any DB hit.
- **Routes:** Multi-tenant via `[portSlug]` dynamic segment. Typed routes enabled.
- **API response shapes:** Conventional envelope is `{ data: <T> }` for any endpoint that returns content (read OR write). Mutations that return nothing emit `204 No Content` (`new NextResponse(null, { status: 204 })`). Don't use `{ success: true }` for CRM mutations — it was a legacy pattern, normalized away in 2026-05-07. Public portal-auth endpoints are an exception: they return `{ success: true }` because the frontend needs a non-error JSON body to chain on. List/paginated reads return `{ data: <T[]>, total?, hasMore? }` (see `/api/v1/clients` for the shape). Errors always go through `errorResponse(error)` from `@/lib/errors` so request-id propagation and the audit-tier mapping stay uniform.
- **Body parsing:** Always use `parseBody(req, schema)` from `@/lib/api/route-helpers` instead of `await req.json(); schema.parse(body)`. The helper returns a uniform 400 with field-level errors that the frontend's `toastError` hook recognizes; raw `req.json` + `schema.parse` produces a generic 500 because the ZodError isn't caught in the same shape.
- **Pre-commit:** Husky + lint-staged runs ESLint fix + Prettier on staged `.ts`/`.tsx` files. The hook also blocks `.env*` files (including `.env.example`) from being committed; pass them via a separate workflow if needed.
## Schema migrations during dev
### Schema migrations during dev
When you run a `db:push` or apply a migration via `psql` against a running dev server, **restart the dev server afterwards**. Drizzle/postgres.js keeps connection-level prepared statements that can hold stale column lists; a stale pool causes `column X does not exist` errors on pages that touch the migrated table even though the column is present in the DB. Symptom: pages return 500 with `errorMissingColumn`/`42703` after a successful migration. Fix: kill `next dev` and restart it.
After `db:push` or applying a migration via `psql` against a running dev server, **restart `next dev`**. Drizzle/postgres.js prepared statements cache stale column lists; symptom is `42703 column X does not exist` 500s on migrated tables.
### Documenso
- **Webhooks:** plaintext secret in `X-Documenso-Secret` (no HMAC) — timing-safe equality via `verifyDocumensoSecret`. Event names arrive uppercase-enum (`DOCUMENT_SIGNED`, `DOCUMENT_COMPLETED` …); the receiver also normalizes lowercase-dotted for forward-compat. `handleDocumentCompleted` is **idempotent** (early-return when `status='completed' && signedFileId`) so 5xx retries don't double-write. Switch handles SIGNED|COMPLETED|REJECTED|DECLINED|OPENED|EXPIRED + v2 aliases RECIPIENT_VIEWED/SIGNED. Detail: `docs/documenso-integration-audit.md`.
- **v1 vs v2 routing:** `getPortDocumensoConfig(portId)` resolves per-port `apiVersion`. `documenso-client.ts` exports version-aware wrappers (`getDocument`, `createDocument`, `sendDocument`, `sendReminder`, `downloadSignedPdf`, `voidDocument`, `placeFields`). v2 → `/api/v2/envelope/*` (multipart create, `distribute` returns per-recipient signingUrl, `redistribute` for reminders, `field/create-many` for bulk placement). v1 → `/api/v1/documents/*`. **Template flow stays v1** (`/api/v1/templates/{id}/generate-document` with name-keyed `formValues`) — v2 instances accept via backcompat. v2-only settings honoured: `documenso_signing_order` (PARALLEL/SEQUENTIAL) + `documenso_redirect_url`.
- **Response normalization:** 2.x uses `documentId` / `recipientId`; v1.13 uses `id`. `normalizeDocument()` surfaces the legacy `id` form to downstream consumers.
- **`DOCUMENSO_API_URL`:** bare host only — never include `/api/v1`. Client appends versioned paths based on `DOCUMENSO_API_VERSION`. Double-pathing returns 404 with no useful diagnostic.
### EOI generation
- Two pathways share `EoiContext` (`src/lib/services/eoi-context.ts`). Documenso pathway uses `documenso-payload.ts` → template-generate endpoint; in-app pathway fills `assets/eoi-template.pdf` via `src/lib/pdf/fill-eoi-form.ts`. Routed through `generateAndSign(...)` in `document-templates.ts` with a `pathway` parameter.
- **Merge fields:** Catalog in `src/lib/templates/merge-fields.ts`; `createTemplateSchema` uses `VALID_MERGE_TOKENS` as an allow-list, rejecting unknown tokens at template creation.
- **Berth range formatter:** Multi-berth EOIs render the in-bundle berth set as a compact range ("A1-A3, B5-B7") via `formatBerthRange()` (`src/lib/templates/berth-range.ts`). Output populates the existing `Berth Number` Documenso field (single-berth = primary mooring verbatim; multi-berth = range). CRM UI always shows berths as chips. `{{eoi.berthRange}}` token available for template body copy.
- Detail: `docs/eoi-documenso-field-mapping.md`, `assets/README.md`.
### UI patterns
- **Sheet vs Drawer:** `<Sheet side="right">` (`src/components/ui/sheet.tsx`, Radix dialog) is the canonical side-panel for both desktop and mobile (`w-3/4 sm:max-w-sm`). Vaul `<Drawer>` (`src/components/shared/drawer.tsx`) is mobile-bottom-sheet only — currently just `MoreSheet`. Need a side panel? Use Sheet. Don't add Vaul without a mobile-bottom-sheet justification.
- **Inline editing:** Detail pages use `<InlineEditableField>` for text/select/textarea and `<InlineTagEditor>` for tag chips. Each entity exposes `PUT /api/v1/<entity>/[id]/tags` backed by a `set<Entity>Tags` service helper (single-transaction wipe-and-rewrite). No separate "Edit" modals — overview tab is editable in place.
- **Email + auth surfaces:** Branded HTML in `src/lib/email/templates/`; portal-auth uses `portal-auth.ts`. All templates: table-based, max-width 600, logo + blurred overhead background (`s3.portnimara.com`). CRM `/login`, `/reset-password`, `/set-password` and portal `/portal/login`, `/portal/activate`, `/portal/reset-password` all wrap content in `<BrandedAuthShell>` for visual continuity.
### Document folders
- Per-port nestable tree (`document_folders.parent_id` self-FK; null parent = root). Documents and files carry nullable `folder_id`. Sibling-name uniqueness via `uniq_document_folders_sibling_name` on `(port_id, COALESCE(parent_id,'__root__'), LOWER(name))`. Folder delete is **soft rescue** (`deleteFolderSoftRescue`) — re-parents children up, drops folder; never CASCADE. `moveFolder` walks ancestor chain to prevent cycles.
- Three system roots (`Clients/`, `Companies/`, `Yachts/`) auto-created via `ensureSystemRoots`. Entity subfolders are lazy via `ensureEntityFolder` — race-safe via partial unique index `uniq_document_folders_entity` on `(port_id, entity_type, entity_id) WHERE entity_id IS NOT NULL`. System rows mutated only by entity rename/archive/hard-delete (auto-sync via service helpers); `assertNotSystemManaged` rejects direct API mutation.
- **Auto-deposit on signing completion:** `handleDocumentCompleted` resolves owner via the Owner-wins chain (`document.clientId ?? .companyId ?? .yachtId ?? interest.clientId`), ensures the entity folder, and sets `files.folder_id` + entity FK. Falls back to root when unresolvable.
- **Aggregated projection:** `listFilesAggregatedByEntity` / `listInflightWorkflowsAggregatedByEntity` walk symmetric reach (Client ↔ Company via `company_memberships` active rows, ↔ Yacht via `yachts.current_owner_type/id`), group by source (DIRECTLY ATTACHED / FROM COMPANY / FROM YACHT / FROM CLIENT), cap 20 per group. **Defense-in-depth `port_id` at every join.** **File-FK snapshot is source of truth** — historical files stay filed even if relationships change.
- Permission gating: `documents.view` reads; `documents.manage_folders` for create/rename/move/delete (system folders immutable via API).
- Deploy: migration `0051_documents_hub_split.sql` + `pnpm db:backfill:doc-folders` (idempotent via per-port advisory lock).
### Berths
- **Public API:** `/api/public/berths` (list) + `/api/public/berths/[mooringNumber]` (single) feed the marketing site. Output mirrors legacy NocoDB shape verbatim. Status precedence: `"Sold"` > `"Under Offer"` (status OR active `is_specific_interest=true` link with open outcome) > `"Available"`. Cache `s-maxage=300, stale-while-revalidate=60`.
- **Public health:** `/api/public/health` dual-mode — anonymous gets `{status, timestamp}` (never 503); requests with timing-safe `X-Intake-Secret` matching `WEBSITE_INTAKE_SECRET` get full `{checks: {db, redis}}` + 503 on failure. The website uses the authenticated form on startup so it refuses to start when pointed at the wrong env.
- **Recommender:** Pure SQL (no AI). `src/lib/services/berth-recommender.service.ts`. Tier ladder A/B/C/D from `interest_berths` aggregates. Heat scoring fires only for tier B; weights tuned via `system_settings` (`heat_weight_*`, `recommender_*`, `fallthrough_*`, `tier_ladder_hide_late_stage`). Multi-port isolation enforced at entry point AND in the SQL aggregates CTE.
- **Rules engine:** `src/lib/services/berth-rules-engine.ts`. Seven triggers, all wired: `eoi_sent`, `eoi_signed`, `deposit_received`, `contract_signed`, `interest_archived`, `interest_completed`, `berth_unlinked`. Callers fire `evaluateRule(...)` via dynamic import (circular-dep avoidance). Defaults vary; admins tune via `berth_rules` setting. Pairs with `advanceStageIfBehind` to keep pipeline stage in sync.
- **Per-berth PDFs:** Versioned via `berth_pdf_versions`; `berths.current_pdf_version_id` is current. Storage key is UUID per upload (no collisions on concurrent uploads); `pg_advisory_xact_lock` per berth_id serializes version-number allocation. 3-tier parse: AcroForm → OCR (Tesseract.js) → optional AI on low confidence. Magic-byte (`%PDF-`) check on BOTH in-server and presigned-PUT paths. Mooring mismatch → service-level `ConflictError` unless `confirmMooringMismatch: true`.
- **Brochures:** Per-port, `is_default` enforced by partial unique index `(port_id) WHERE is_default=true AND archived_at IS NULL`. Same upload flow as berth PDFs.
- **NocoDB re-import:** `pnpm tsx scripts/import-berths-from-nocodb.ts --apply --port-slug port-nimara`. Idempotent (skips rows where `updated_at > last_imported_at` unless `--force`); add `--update-snapshot` to rewrite the seed JSON. Helpers in `src/lib/services/berth-import.ts` are unit-tested.
- Plan-of-record: `docs/berth-recommender-and-pdf-plan.md`.
### Storage
- All file I/O through `getStorageBackend()` (`src/lib/storage/`). Interface: `put`, `get`, `head`, `delete`, `listByPrefix`, `presignUpload`, `presignDownload`. Selected via `system_settings.storage_backend` (`'s3' | 'filesystem'`). Switching backends = settings change + `pnpm tsx scripts/migrate-storage.ts` (round-trips every blob in `files`, `berth_pdf_versions`, `brochure_versions`, `gdpr_exports`, verifies SHA-256).
- MinIO calls wrapped in 30s `withTimeout` to prevent TCP-blackhole stalls. **Filesystem backend is single-node only** — refuses to start when `MULTI_NODE_DEPLOYMENT=true`.
### Send-from accounts (sales send-outs)
- Configurable via `system_settings`; defaults to `sales@portnimara.com` (human) + `noreply@portnimara.com` (automation). SMTP/IMAP passwords AES-256-GCM at rest; API returns only `*PassIsSet` markers.
- Audit → `document_sends` (separate from `audit_logs` for volume + binary refs). Body markdown rendered via `renderEmailBody()` (escape-then-allowlist; XSS-tested). Rate limit 50 sends/user/hour. Files > `email_attach_threshold_mb` ship as 24h signed-URL link (filename HTML-escaped against injection). Bounce monitoring needs IMAP creds in addition to SMTP — without them, the size-rejection banner is disabled.
### Pre-commit
Husky + lint-staged runs ESLint fix + Prettier on staged `.ts`/`.tsx`. **Blocks all `.env*` files** (including `.env.example`) — pass them via a separate workflow if needed.
## Environment
Copy `.env.example` to `.env` for local dev. See `src/lib/env.ts` for the full schema. Set `SKIP_ENV_VALIDATION=1` to bypass validation (used in Docker build).
Copy `.env.example` to `.env`. See `src/lib/env.ts` for the full Zod schema. `SKIP_ENV_VALIDATION=1` bypasses validation (Docker build).
Required env gotchas:
Dev/test-only env (not in `.env.example`):
- `DOCUMENSO_API_URL` — **bare host only**, never include `/api/v1`. The client appends versioned paths based on `DOCUMENSO_API_VERSION` (`v1` for 1.13.x, `v2` for 2.x). A double-pathed URL returns 404 on every call with no useful diagnostic.
Optional dev/test-only env vars (not in `.env.example`):
- `EMAIL_REDIRECT_TO=<address>` — when set, every outbound email is rerouted to this address regardless of the requested recipient and the subject is prefixed with `[redirected from <original>]`. Dev safety net so seeded fake-client emails don't escape; **must be unset in production**.
- `IMAP_HOST` / `IMAP_PORT` / `IMAP_USER` / `IMAP_PASS` — read by `tests/e2e/realapi/portal-imap-activation.spec.ts` to fetch the activation email from a real mailbox during the IMAP round-trip test. The spec skips when any are missing.
- `EMAIL_REDIRECT_TO=<address>` — reroutes every outbound email to this address, prefixes subject `[redirected from <original>]`. Dev safety net; **must be unset in production**.
- `IMAP_HOST` / `IMAP_PORT` / `IMAP_USER` / `IMAP_PASS` — used by `tests/e2e/realapi/portal-imap-activation.spec.ts`; the spec skips when any are missing.
## Testing
Five Playwright projects, defined in `playwright.config.ts`:
Six Playwright projects (`playwright.config.ts`):
- `setup` — global setup (seeds users, port, berths, system settings).
- `smoke` — fast click-through over every major flow. Run on every change (~10 min, 125 specs).
- `exhaustive` — deeper UI coverage that takes longer.
- `destructive` — archive/delete/cancel paths against throwaway entities.
- `realapi` — opt-in suite that hits real external services (Documenso send-side + IMAP round-trip). Requires `DOCUMENSO_API_*`, `SMTP_*`, `IMAP_*` env. Cloudflared tunnel needs to be running so Documenso can call the local webhook receiver.
- `visual` — pixel-diff baselines for stable list/landing pages. Snapshots committed under `tests/e2e/visual/snapshots.spec.ts-snapshots/`. Regenerate with `--update-snapshots` after intentional UI changes.
- `setup` — global setup (seeds users, port, berths, system settings)
- `smoke` — fast click-through, run on every change (~10 min, 125 specs)
- `exhaustive` — deeper UI coverage
- `destructive` — archive/delete/cancel paths against throwaway entities
- `realapi` — opt-in real Documenso send-side + IMAP round-trip. Needs `DOCUMENSO_API_*`, `SMTP_*`, `IMAP_*` env + cloudflared tunnel running for the local webhook receiver
- `visual` — pixel-diff baselines (`tests/e2e/visual/snapshots.spec.ts-snapshots/`); regenerate with `--update-snapshots`
Vitest covers unit + integration with mocked external services (`tests/unit/`, `tests/integration/`).
Vitest covers unit + integration with mocked externals (`tests/unit/`, `tests/integration/`).
## Docker
- `Dockerfile` - Production multi-stage build (deps -> build -> runner)
- `Dockerfile.dev` - Dev with bind-mounted source
- `Dockerfile.worker` - BullMQ worker process
- `docker-compose.yml` / `docker-compose.dev.yml` / `docker-compose.prod.yml`
- `Dockerfile` — production multi-stage (deps → build → runner)
- `Dockerfile.dev` — dev with bind-mounted source
- `Dockerfile.worker` — BullMQ worker process
- `docker-compose.yml` / `.dev.yml` / `.prod.yml`
## Architecture docs
Numbered spec files in repo root (`01-CONSOLIDATED-SYSTEM-SPEC.md` through `15-DESIGN-TOKENS.md`) contain detailed architecture decisions, feature specs, DB schema docs, API catalog, and implementation sequence.
Numbered specs (`01-CONSOLIDATED-SYSTEM-SPEC.md` … `15-DESIGN-TOKENS.md`) in repo root carry the detailed architecture decisions, schema docs, API catalog, and sequence.
Domain-specific references:
Active plans of record under `docs/`:
- `docs/eoi-documenso-field-mapping.md` — canonical mapping from `EoiContext`
paths to the Documenso template's `formValues` keys, with the matching
AcroForm field names used by the in-app pathway. The `Berth Number`
field carries the `formatBerthRange()` output — single-berth EOIs
populate it with just the primary mooring (e.g. `A1`), multi-berth
EOIs with the compact range (`A1-A3, B5`). No separate `Berth Range`
template field is needed (the dedicated field was retired 2026-05-14).
- `assets/README.md` — what the in-app EOI source PDF must contain and how
to override its path in dev/test.
- `docs/berth-recommender-and-pdf-plan.md` — the comprehensive plan for the
Phase 0–8 berth-recommender + PDF + send-outs work bundle. Single source
of truth for the multi-berth interest model, recommender tier ladder,
pluggable storage, per-berth PDF parser, and sales send-out flows.
- `docs/MASTER-PLAN-2026-05-18.md` — current 7-phase post-audit plan
- `docs/BACKLOG.md` — single entry point for everything outstanding
- `docs/berth-recommender-and-pdf-plan.md` — berths + PDF + send-outs bundle
- `docs/eoi-documenso-field-mapping.md` — canonical EoiContext ↔ Documenso/AcroForm mapping
- `docs/documenso-integration-audit.md` — full Documenso v1/v2 quirks reference
- `assets/README.md` — in-app EOI source PDF requirements