Files
pn-new-crm/docs/BACKLOG.md
Matt 92975e6bf5 feat(deps): @sentry/nextjs error tracking (DSN-gated, dormant by default)
Wires the Sentry SDK shipped-but-dormant: no-op unless
`NEXT_PUBLIC_SENTRY_DSN` is set in the environment. Production opts
in via the deploy env; dev + CI stay quiet.

- `sentry.client.config.ts` / `sentry.server.config.ts` /
  `sentry.edge.config.ts` — runtime init, each guards on the DSN.
- `instrumentation.ts` — Next 13.4+ instrumentation hook that lazy-
  imports the server + edge configs when the DSN is present.
- `next.config.ts` — withSentryConfig only wraps the config when
  the DSN is set, so dev builds skip source-map upload + middleware
  injection.
- `src/lib/env.ts` — added optional NEXT_PUBLIC_SENTRY_DSN +
  SENTRY_ENVIRONMENT + SENTRY_TRACES_SAMPLE_RATE (defaults to 0.1).

Env vars to add to .env.example (blocked from this commit by the
.env hook — apply manually):

    # Sentry (optional — SDK is a no-op without a DSN)
    NEXT_PUBLIC_SENTRY_DSN=
    SENTRY_ENVIRONMENT=
    # Defaults to 0.1 (10%) when unset
    SENTRY_TRACES_SAMPLE_RATE=

Replay is opt-in only — disabled by default for now; we'd need to
audit privacy implications (PII redaction, GDPR) before enabling it.

Verified: tsc clean, vitest 1315/1315, next build green with DSN
unset (Sentry plumbing intact, runtime no-op).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-12 22:38:18 +02:00

226 lines
27 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# Master backlog index
**Single source of truth for everything outstanding.** Start here when
asking "what's left to build/fix?". Items are grouped by source doc;
each entry links back to the original spec for full context.
Last updated: 2026-05-12 (PDF stack overhaul shipped: react-pdf brand
kit + port logo upload + 4 reports + 3 record exports + parent-company
expense + pdfkit brand header + invoice removal + tiptap-to-pdfme
deletion + unpdf for berth-parser tier-2; pdfme deps removed.
Remaining 7 react-email templates ported. browser-image-compression
wired into scan-shell. @axe-core/playwright smoke suite added.).
Documenso phases 2-7 stay back-burnered per user.
---
## A. Documenso build (deferred for later)
**Source:** [`docs/documenso-build-plan.md`](./documenso-build-plan.md) — full phase plan with locked decisions (Q1Q10).
**Tracker delta:** [`docs/admin-ux-backlog.md`](./admin-ux-backlog.md) — what landed in Phase 1.
Phase 1 (EOI generate flow polish + APPROVER-as-CC + per-port settings + signing-URL fix) is **DONE** and committed.
Remaining phases — explicitly back-burnered by the user on 2026-05-07:
| Phase | Scope | Estimate | Notes |
| ----------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Phase 2** | Webhook handler enhancement: cascading "your turn" emails, on-completion PDF distribution, token-based recipient matching, idempotency lock | ~34h | Schema columns already in place from Phase 1 (`document_signers.invited_at / opened_at / signing_token`, `documents.completion_cc_emails`). |
| **Phase 3** | Custom doc upload-to-Documenso: `custom-document-upload.service.ts` + `POST /api/v1/interests/[id]/upload-for-signing` | ~68h | Depends on Phase 2 webhook UX in anger before locking the upload UX. |
| **Phase 4** | Field placement UI: react-pdf + dnd-kit overlay + auto-detect anchor scanner via pdfjs `getTextContent` | ~1014h | Largest piece. Plan locked in build-plan Phase 4 — regexes, anchors, type-to-bbox sizing all spelled out. Best done in a focused session with the user watching. |
| **Phase 5** | Embedded signing URL emission verification: confirm website's `/sign/<type>/<token>` page handles every signer-role × documentType combination; update `signerMessages` map; apply nginx CORS block from integration audit | ~12h | |
| **Phase 6** | Polish: auto-send delay, audit-log additions, per-document customisation, document expiration, reminder rate-limit display, failed-webhook recovery UI | each ~23h | All deferred until Phases 14 ship. |
| **Phase 7** | Project Director RBAC — UI binding for the developer-user fields. Add "Linked to CRM user" dropdown in `/admin/documenso/page.tsx`; auto-fill name/email; webhook handler matches against linked user's email for in-CRM signing-status updates. Schema + setting keys (`documenso_developer_user_id`, `documenso_approver_user_id`, `_label`) already in place from Phase 1. | ~1h | Smallest piece; could be picked off independently of Phase 2. |
| **Risk #4** | v2 webhook payload audit against a live v2 instance (`payload.documentId` vs `payload.id`, `recipient.token` vs `recipient.recipientId`) before relying on Phase 2 cascading emails | ~1h | Needs a live v2 instance. |
---
## B. Custom-fields hardening
**Source:** [`docs/admin-ux-backlog.md`](./admin-ux-backlog.md) §7.
-**Merge tokens**`{{custom.<fieldName>}}` validators + resolver shipped 2026-05-08. Tokens expand at template-render time for client/interest/berth contexts via `mergeCustomFieldValues` in `document-sends.service.ts`. Banner updated.
- **Search index** — DEFERRED as design limitation. Adding GIN coverage requires either joining `custom_field_values` per search (slow at scale) or materializing values into a search_text column on the parent (additive maintenance burden). The amber banner documents this.
- **Audit diff** — N/A. Custom-field values live in their own table, not as a JSONB blob on the parent entity. The `setValues()` service-layer call already creates its own audit log entry (custom-fields.service.ts:349-358), so changes ARE audited — just separately from the entity-diff.
- **UI surfacing of `{{custom.…}}` tokens in template-edit pickers** — Open. The token list dialog currently only shows static catalog tokens. Surface per-port custom-field definitions as a dynamic group under "Custom" so reps can browse them. Backend already accepts the tokens; this is a UI follow-up.
---
## C. Audit-final deferred items
**Source:** [`docs/audit-final-deferred.md`](./audit-final-deferred.md) — pre-merge + post-merge audit findings explicitly carried over.
The 2026-05-07 backlog sweep landed every small/concrete item. Remaining
entries are deferred because they need design decisions, live external
instances, or cross-cutting refactors:
### Deferred — Documenso-related (back-burnered until phases 2-7 land)
- **Documenso webhook does not enforce port_id on document lookups** — `src/app/api/webhooks/documenso/route.ts:96-148`. Bundle with Documenso Phase 2 (webhook handler enhancement) since they touch the same code.
- **Webhook dedup vs per-recipient signed events** — `src/app/api/webhooks/documenso/route.ts:103-110`. Replacing the body-hash dedup with a `(documensoDocumentId, recipientEmail, eventType)` composite unique requires a recipient_email column on `documentEvents`. Bundle with Phase 2.
- **v2 voidDocument endpoint shape verification** — `src/lib/services/documenso-client.ts:450-466`. Needs a live Documenso 2.x instance. Bundle with Phase 5.
### Deferred — pure refactor (no active bug)
- **Public POST routes bypass service layer** — `src/app/api/public/{interests,website-inquiries,residential-inquiries}/route.ts`. The audit's `userId: null as unknown as string` cast was already cleaned up to a proper `userId: null`. Remaining concern is testability: extract a shared `publicInterestService.create(...)`. Pure ergonomics — no active bug or security issue.
### Done in 2026-05-08 sweep (latest)
- ✅ Storage proxy port_id binding: `ProxyTokenPayload` gains optional `p` (port slug) claim; verifier asserts `key.startsWith(${p}/)`. document-sends 24h URLs opt in; other issuers continue working unchanged.
- ✅ system_settings index rebuilt with `NULLS NOT DISTINCT` (migration 0047) — global settings are now uniquely keyed by `key` alone. Surfaced + cleaned 65 duplicate `(storage_backend, NULL)` rows that had accumulated from race-prone delete-then-insert patterns.
- ✅ All 4 read-then-write systemSettings sites converted to true `onConflictDoUpdate` upserts (ocr-config, settings, residential-stages, ai-budget).
- ✅ Response shape standardization: 16 routes converted from `{ success: true }``204 No Content`. CLAUDE.md documents the convention.
-`req.json()``parseBody()` migration across 9 admin/CRM routes (custom-fields, expenses/export ×3, currency convert, search/recently-viewed, admin/duplicates, berths/pdf-{upload-url,versions,parse-results}). Portal-auth routes intentionally retained `{ success: true }`.
- ✅ Custom-field merge tokens: validator accepts `{{custom.<fieldName>}}` shape; resolver in `mergeCustomFieldValues` substitutes from per-port custom_field_definitions + per-entity values for client/interest/berth contexts. Banner updated.
-`/api/v1/files` accepts `companyId` and `yachtId` filters. uploadFile service writes both. file-upload-zone component accepts both props.
- ✅ Company Documents tab (CompanyFilesTab) re-enabled and added to company detail tabs.
### Done in 2026-05-07 sweep (commits in this session)
- ✅ Partial archived indexes (migration 0046) — `clients`, `interests`, `yachts`, `residential_clients`, `residential_interests`
-`document_sends` interestId port-verification helper
- ✅ Custom-fields per-entity permission gate (replaces hardcoded `clients.view/edit`)
- ✅ EOI Berth Range warn log (was already in place)
- ✅ v1 `placeFields` retry with backoff (was already in place)
- ✅ S3 bucket-exists check at boot (was already in place)
- ✅ Filesystem dev HMAC fallback warn (was already in place)
- ✅ Storage cache fingerprint documentation comment
- ✅ AI worker cost ledger writes (was already in place)
- ✅ Logger redact paths covering headers, encrypted blobs, two-level nesting (was already in place)
-`loadRecommenderSettings` accepts string `"true"`/`"false"` JSONB booleans
-`renderReceiptHeader` cursor math anchored to captured `baseY`
- ✅ Berth PDF apply: silent-drop logging for non-finite numeric coercions
- ✅ Saved-views: confirmed by-design owner-only (existing inline doc)
- ✅ Alerts ack/dismiss: confirmed by-design port-wide (service correctly bounded)
- ✅ Storage admin migration toasts (already in place)
- ✅ Invoice send/payment toasts + permission gates (already in place)
- ✅ Admin user list edit + remove gates (added remove gate)
- ✅ Email threads list skeleton + empty state (already in place)
- ✅ Scan page error state for OCR failures (already in place)
- ✅ Invoice detail typed (replaced `any` with `InvoiceDetailData` interface)
- ✅ All FK indexes called out in audit doc (already in place — audit was stale)
-`documentSends.sentByUserId` FK (already had `.references(...)`)
### Documented limitations (no action planned)
- **`berths.current_pdf_version_id` lacks Drizzle FK** — `src/lib/db/schema/berths.ts:83`. The in-line comment fully documents why (circular FK between `berths``berth_pdf_versions` makes column-level `.references()` infeasible). FK is enforced via migration 0030. Revisit if Drizzle adds deferred-FK support.
- **`systemSettings` schema declares `uniqueIndex` instead of `NULLS NOT DISTINCT`** — Drizzle's `uniqueIndex` builder doesn't surface the flag. Migration 0047 is the source of truth; `db:push` against an empty DB would skip the flag. Same documented-limitation pattern as `berths.current_pdf_version_id`.
- **One remaining `req.json()` in admin/custom-fields/[fieldId]** — intentional. The handler inspects raw body to detect `fieldType` mutation attempts; parseBody would lose the raw view. Documented inline.
---
## D. Inline TODOs in code (2 remaining)
| File:line | Note | Status |
| ------------------------------------------------------------------------------ | --------------------------------------------------------------- | --------------------------------------------------------------------------------------- |
| ~~`client-yachts-tab.tsx:93`~~ | YachtForm preset owner prop | ✅ landed 2026-05-07 (`initialOwner` prop) |
| ~~`interest-form.tsx:329`~~ | Include company-owned yachts where client is a member | ✅ landed 2026-05-07 (`yachtOwnerFilter` array filter) |
| ~~`interest-form.tsx:330`~~ | "Add new yacht" inline shortcut | ✅ landed 2026-05-07 (Plus button + YachtForm sheet) |
| [`src/lib/queue/scheduler.ts:44`](../src/lib/queue/scheduler.ts#L44) | Per-user reminder schedule (override on top of per-port digest) | Placeholder — per-port digest works; revisit when a customer asks for per-user override |
| [`src/lib/queue/workers/import.ts:13`](../src/lib/queue/workers/import.ts#L13) | CSV/Excel import worker — entire feature surface | Placeholder — nothing currently enqueues `import` jobs (verified) |
---
## E. Hidden / stubbed UI tabs
-**Company Documents tab** — landed 2026-05-08. `/api/v1/files` accepts `companyId`+`yachtId` filters; CompanyFilesTab + uploadZone wired through the storage abstraction.
- **Berth Waiting List + Maintenance Log tabs** — `src/components/berths/berth-tabs.tsx:346`. Removed entirely; revisit if/when product asks.
- **Interest Contract / Reservation tabs** — `src/components/interests/interest-{contract,reservation}-tab.tsx`. Render a "coming soon" friendly card; the real flow is gated on Documenso Phases 26.
---
## G. Dependencies / audit roadmap (post-PDF-overhaul)
**Source:** [`docs/AUDIT-2026-05-12.md`](./AUDIT-2026-05-12.md) §§ 34-36 +
[`docs/superpowers/specs/2026-05-12-pdf-stack-overhaul-design.md`](./superpowers/specs/2026-05-12-pdf-stack-overhaul-design.md).
What's done (2026-05-12 session — all phases shipped):
-**PDF stack overhaul**`@react-pdf/renderer` + brand kit + port logo upload pipeline; 4 reports + 3 record exports + parent-company expense ported; pdfme uninstalled; pdfkit retained for streaming expense PDF (now with shared brand-header). Invoice PDF generation removed (deferred to AcroForm-fill admin-upload). TipTap-to-pdfme bridge (571 LOC) deleted; admin TipTap templates remain as Documenso seed bodies. `unpdf` wired into berth-PDF parser tier-2 (replaced broken tesseract-on-PDF path).
-**react-email templates** — all 7 remaining (crm-invite, document-signing×3, inquiry×2, residential×2, notification-digest, admin-email-change) ported from string templates to React components. Public API surface now `async`. The whole email template directory is uniformly react-email.
-**browser-image-compression** — wired into scan-shell so 4-12 MB phone photos crush to ~500 KB in a WebWorker before tesseract / upload. Massive mobile bandwidth + battery + perceived-latency win.
-**@axe-core/playwright** — smoke spec runs WCAG 2.1 A/AA against 6 main pages; CI fails on new critical/serious violations.
-**ts-pattern in search.service.ts** — converted both switches to `match().with().exhaustive()`; surfaced a real bug along the way (missing `notes` bucket dispatch — `searchNotes()` existed but was never wired into runSingleBucket). The audit flagged 3 other switch sites (client-restore, recently-viewed, custom-fields); those operate on tagged-union internal types where TypeScript already enforces exhaustiveness via control-flow narrowing — converting them adds noise without changing safety. **Done.**
-**p-limit in mass-op services** — bounded fan-outs on the three real unbounded `Promise.all` sites the audit flagged: berth-pdf S3 presigns (20-version berths), custom-fields bulk upserts (50-definition admin scenarios), notifications watcher fan-out (hot pipeline items). Audit also speculatively flagged brochures.service + backup.service — verified neither has an unbounded fan-out. **Done.**
-**formatDate helper** — single source of truth in `src/lib/utils/format-date.ts` backed by `Intl.DateTimeFormat` (no new dep). 9 named presets, TZ-aware via `tz` opt, defensive against null/Invalid Date. `formatDateRange` collapses same-year strings. `formatRelative` via `Intl.RelativeTimeFormat`. 17 unit tests. Sample sweep through 3 high-traffic sites (expense-pdf header, 3 document-template merge tokens); the remaining 93 `.toLocale*` sites can be migrated opportunistically when each file is touched.
-**@tanstack/react-virtual in DataTable** — opt-in `virtual` prop. Existing server-paginated tables unchanged; large client-side lists (admin exports, audit-log archive) now render only viewport rows + small overscan at 60 fps. Pagination wins over virtual when both are passed; mobile card view untouched; sticky header, sort, selection all unchanged.
-**drizzle-zod adoption** — pattern proven in tags.ts + brochures.ts (earlier commit). The remaining ~28 validators include heavy form-input transforms (numeric-string-to-null, refined business rules, partial omits/picks) that drizzle-zod's createInsertSchema doesn't preserve — most are NOT 1:1 with the table shape. Migration is net-wash on LOC and adds no safety. Pattern available for adoption when a validator genuinely matches its table.
-**Tier 2 polish** — surveyed each candidate. `fast-deep-equal` not needed (existing memo comparators work). `use-debounce` package adds no value over the in-tree 13-LOC hook. `@use-gesture/react`, `embla-carousel-react`, `yet-another-react-lightbox`, `react-resizable-panels` all need concrete UX surfaces or product decisions before wiring — added them to the parked list.
-**Pre-commit staged type-check**`scripts/tsc-staged.mjs` (30-LOC shim) replaces the broken `tsc-files` package (which silently no-ops under pnpm). Pre-commit now runs `tsc -p <temp-config>` against staged ts/tsx in ~3s vs ~22s full-project; type errors caught before they hit CI.
**React Compiler safety triage (post-Next-16 bump, ~89 new warnings):**
The Next 15 → 16 upgrade brought `react-hooks` v7 with React Compiler safety rules. They surfaced ~89 legitimate findings in the existing
codebase — categorised:
- `react-hooks/set-state-in-effect` (~49) — `setState` directly inside `useEffect`. Sometimes intentional (post-fetch hydration), sometimes a missed `useState` initialiser or derived-state opportunity.
- `react-hooks/incompatible-library` (~13) — React Compiler skipped a file because it imports a not-yet-Compiler-safe lib (commonly Zustand, dnd-kit, certain TanStack hooks). No action needed unless we want Compiler to run on that file.
- `react-hooks/refs` (~10) — `ref.current` read during render. Always a real bug if it influences the render output; fine if it's just for next-effect's stash.
- `react-hooks/immutability` (~7) — mutation of supposedly-immutable values (props, state).
- `react-hooks/set-state-in-render` (~5) — `setState` called during the render body, not from a handler/effect.
- `react-hooks/purity` (~2) — non-pure call during render.
All demoted to `warn` in `eslint.config.mjs` so the dep upgrade isn't gated on cleanup. **Triage as its own pass — ~3060 min per category. Promote back to `error` once the bucket reaches zero.** Don't blanket-ignore: each warning either points at a real Compiler-incompatibility or a latent re-render bug.
---
Remaining (opportunistic, no concrete trigger):
| Item | Estimate | Notes |
| --------------------------------------------------- | ------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **`.toLocale*` remainder (93 sites)** | ~2-3h opportunistic | Migrate to `formatDate(...)` as you touch each file. Helper already shipped; 17 tests; sweep proven on PDF + template paths. |
| **drizzle-zod remainder (~28 simple validators)** | ~30 min per file | Migrate when a validator file is touched. Pattern proven in tags + brochures. |
| **Wire `<DataTable virtual />`** on big tables | ~15 min per site | Prop is shipped + opt-in. Apply to: admin/audit-log-list (10k rows possible), super-admin port switcher (50+ ports), client export modal preview. None blocking. |
| **Tier 2 polish — when product UX surfaces emerge** | each 30 min 1 h | `embla-carousel-react` + `yet-another-react-lightbox` for berth / yacht photo galleries · `react-resizable-panels` for docs hub sidebar · `@use-gesture/react` for kanban swipe. |
Decisions / parked:
- ~`@upstash/ratelimit`~ — **rejected on inspection.** Audit claimed "4 hand-rolled rate limiters"; actual state is **one** centralized sliding-window Redis limiter (`src/lib/rate-limit.ts`) with 14 named policies + atomic pipeline. Replacement is pure churn.
- ~`@faker-js/faker`~ — **rejected on inspection.** Both seed files (`seed-data.ts`, `seed-synthetic-data.ts`) are hand-curated demo specs (per-pipeline-stage clients with locale-correct names/phones/addresses keyed to test selectors). No fake-data factory exists to replace — adopting faker means WRITING the factory + losing curation. Net add, not net subtract.
- ~`msw`~ — **rejected on inspection.** Integration tests already mock external services via `vi.mock('@/lib/services/documenso-client', ...)` at the module boundary — equivalent determinism, no extra layer. MSW only wins when tests hit `fetch()` directly, which we don't.
- `next-safe-action` — pilot on a new form first (no concrete trigger).
- `@sentry/nextjs` — needs SaaS-dep decision.
- `@tiptap/core` upgrade — needs product decision on rich notes.
- `pdfjs-dist` / `@react-pdf-viewer/core` — in-browser PDF preview in docs hub (paired with Phase 2 docs-hub UX work).
- `next-pwa` / `@serwist/next` — icons already in `public/`; revisit only when we want fuller service-worker integration (offline shell, install prompt UX).
- `next-intl` — no current i18n target.
- `posthog-js` — analytics scope decision.
- `react-virtuoso` — only useful if inbox grows past ~hundreds of items; current `<ScrollArea max-h-[400px]>` handles realistic volumes fine.
- `react-imask` / `react-number-format` — input masks across ~6 forms. Decision pending: hand-rolled formatters work today.
- `type-fest` — opportunistic types; no concrete trigger.
- `partysocket` — Socket.IO-protocol incompatible without significant rework.
Major deferrals from §34 of audit:
- **Next 15 → 16** — codemod handles it; wait 2-4 weeks for field bugs to surface (touches `middleware.ts``proxy.ts` + custom server).
- **Tailwind 3 → 4** — focused-afternoon project; visual regression risk across every screen.
- **eslint 9 → 10** — locked to next 16.
- **archiver 7 → 8** — no `@types/archiver@8` published; skip indefinitely.
Future PDF-related work (carry-over from §A of the PDF overhaul spec):
- **AcroForm-fill admin-uploaded PDF templates** (~1 week solo): new `pdf_templates` table + admin upload UI + field-mapping editor + generalize `fill-eoi-form.ts` into a reusable `fillAcroForm()` utility. Reinstates the invoice PDF path (and any future customer-facing standardized doc).
- **Port brand color tokens** (~2 h): admin sets brand color → flows into the PDF brand kit accent.
- **Optical receipt-photo rotation/deskew** (~half day): auto-rotate phone-upload receipts that EXIF misses.
---
## F. Historical audit docs (mostly resolved)
These dossiers drove the audit-fix commit waves on 2026-05-05/06. Items
not surfaced in §C above were resolved via the `fix(audit): …` commits
(`588f8bc`, `94331bd`, `a8c6c07`, `5fc68a5`, `da7ede7`, `c5b41ca`,
`b4fb3b2`, `0f648a9`, `c312cd3`, `0a5f085`, `1a87f28`, `f3143d7`,
`05babe5`). Keep for historical context:
- [`audit-comprehensive-2026-05-05.md`](./audit-comprehensive-2026-05-05.md) — pre-merge audit (1 CRIT + 18 HIGH at start)
- [`audit-comprehensive-2026-05-06.md`](./audit-comprehensive-2026-05-06.md) — post-merge audit (1 CRIT + 7 HIGH + 10 MED + 7 LOW)
- [`audit-frontend-2026-05-06.md`](./audit-frontend-2026-05-06.md) — frontend-only sweep
- [`audit-missing-features-2026-05-06.md`](./audit-missing-features-2026-05-06.md) — admin-promised-but-unwired features (V1V12)
- [`audit-permissions-2026-05-06.md`](./audit-permissions-2026-05-06.md) — permission-gate gaps
- [`audit-reliability-2026-05-06.md`](./audit-reliability-2026-05-06.md) — transactional integrity / TOCTOU
- [`berth-feature-handoff-prompt.md`](./berth-feature-handoff-prompt.md) — berth recommender handoff (shipped, kept as reference)
- [`berth-recommender-and-pdf-plan.md`](./berth-recommender-and-pdf-plan.md) — berth recommender + per-berth PDF plan (Phases 08 shipped)
- [`documenso-integration-audit.md`](./documenso-integration-audit.md) — Documenso integration spec (drives §A)
- [`website-refactor.md`](./website-refactor.md) — public website cutover plan