letsbe/pn-new-crm
1
0
Fork 0
Code Issues Pull Requests 1 Actions Packages Projects Releases Wiki Activity

fix(audit): non-Documenso backlog sweep — port-binding, NULLS NOT DISTINCT, custom merge tokens, company docs
Some checks failed
Build & Push Docker Images / lint (push) Successful in 1m36s
Details
Build & Push Docker Images / build-and-push (push) Failing after 4m27s
Details

Browse Source 
Wave through the remaining audit-final-deferred items that aren't blocked
on the back-burnered Documenso work.

Multi-tenant isolation:
- Storage proxy ProxyTokenPayload gains optional `p` (port slug) claim;
  verifier asserts `key.startsWith(${p}/)`. Defense-in-depth against a
  buggy issuer in some future code path that mixes port scopes — every
  storage key generated by generateStorageKey() already prefixes the
  slug. document-sends opts in for 24h emailed download links; other
  callers continue working unchanged via the optional field.

DB schema reconciliation:
- Migration 0047 rebuilds system_settings unique index with NULLS NOT
  DISTINCT (Postgres 15+) so global settings (port_id IS NULL) are
  uniquely keyed by `key` alone. Surfaced + dedupe'd 65 duplicate
  (storage_backend, NULL) rows that had accumulated from race-prone
  delete-then-insert patterns in ocr-config / settings / residential-
  stages / ai-budget services. All four services converted to true
  onConflictDoUpdate upserts so the race window is closed.

API uniformity:
- Response shape standardization: 16 routes converted from
  `{ success: true }` to 204 No Content. CLAUDE.md documents the
  convention (`{ data: <T> }` for content, 204 for empty mutations,
  portal-auth retains `{ success: true }` for the frontend's auth chain).
- req.json() → parseBody() migration across 9 admin/CRM routes
  (custom-fields, expenses/export ×3, currency convert,
  search/recently-viewed, admin/duplicates, berths/pdf-{upload-url,
  versions, parse-results}). Uniform 400 error shapes for
  ZodError-flagged bodies.

Custom-fields merge tokens (shipped end-to-end):
- merge-fields.ts gains CUSTOM_MERGE_TOKEN_RE + helpers for the
  `{{custom.<fieldName>}}` shape.
- document-templates validator accepts the dynamic shape alongside
  the static catalog tokens.
- document-sends.service mergeCustomFieldValues resolver fetches
  per-port custom_field_definitions for client/interest/berth contexts
  and substitutes stored values keyed by `{{custom.fieldName}}`.
- custom-fields-manager amber banner updated to reflect that merge
  tokens now expand (search index + entity-diff remain documented
  design limitations).

/api/v1/files cross-entity filtering:
- Validator + listFiles + uploadFile accept companyId AND yachtId
  alongside clientId. file-upload-zone propagates both.
- New CompanyFilesTab component mirrors ClientFilesTab; restored as a
  visible Documents tab in company-tabs.tsx (was a hidden stub).

Inline TODOs:
- Reviewed remaining two TODOs (per-user reminder schedule, import
  worker handlers). Both are placeholders for future feature surfaces,
  not bugs — per-port digest works for every customer; nothing
  currently enqueues import jobs (verified). Annotated in BACKLOG.

BACKLOG.md updated to reflect what landed and what's still pending
(Documenso-related items still bundled with the back-burnered phases).

Tests: 1185/1185 vitest, tsc clean.
This commit is contained in:
Matt
2026-05-08 02:20:27 +02:00
parent 60365dc3de
commit 8dc16dcd2e
49 changed files with 578 additions and 254 deletions
Download Patch File Download Diff File Expand all files Collapse all files

74
docs/BACKLOG.md
View File

@@ -4,13 +4,12 @@
asking "what's left to build/fix?". Items are grouped by source doc;
each entry links back to the original spec for full context.
Last updated: 2026-05-07 (after the audit-final-deferred sweep — partial
archived indexes, document_sends interestId port-verify, custom-fields
per-entity permission gate, recommender bool parsing, expense PDF cursor
math, berth PDF silent-drop logging, YachtForm preset-owner + interest
form member-company yacht filter + add-new shortcut, invoice detail
typed). Many older items in §C and §F were already resolved by earlier
fix-audit commit waves; the audit doc was stale.
Last updated: 2026-05-08 (second non-Documenso sweep — storage-proxy
port-binding, system_settings NULLS NOT DISTINCT + dedup migration,
response-shape standardization, parseBody migration, custom-field merge
tokens, /api/v1/files companyId+yachtId filter, Company Documents tab,
file-upload zone wired for company/yacht targeting). Documenso phases
2-7 stay back-burnered per user.
---
@@ -35,15 +34,14 @@ Remaining phases — explicitly back-burnered by the user on 2026-05-07:
---
## B. Custom-fields hardening (~ongoing, deferred)
## B. Custom-fields hardening
**Source:** [`docs/admin-ux-backlog.md`](./admin-ux-backlog.md) §7.
Custom Settings page already shows the amber warning banner. Remediation work:
- **Search index** — extend the GIN tsvector to include `customFieldValues` content
- **Audit diff** — extend `diffEntity` to walk the `customFieldValues` blob
- **Merge tokens** — add `{{custom.<fieldName>}}` handling at template-render time, plus surface them in the merge-tokens UI
- ✅ **Merge tokens** — `{{custom.<fieldName>}}` validators + resolver shipped 2026-05-08. Tokens expand at template-render time for client/interest/berth contexts via `mergeCustomFieldValues` in `document-sends.service.ts`. Banner updated.
- **Search index** — DEFERRED as design limitation. Adding GIN coverage requires either joining `custom_field_values` per search (slow at scale) or materializing values into a search_text column on the parent (additive maintenance burden). The amber banner documents this.
- **Audit diff** — N/A. Custom-field values live in their own table, not as a JSONB blob on the parent entity. The `setValues()` service-layer call already creates its own audit log entry (custom-fields.service.ts:349-358), so changes ARE audited — just separately from the entity-diff.
- **UI surfacing of `{{custom.…}}` tokens in template-edit pickers** — Open. The token list dialog currently only shows static catalog tokens. Surface per-port custom-field definitions as a dynamic group under "Custom" so reps can browse them. Backend already accepts the tokens; this is a UI follow-up.
---
@@ -55,15 +53,26 @@ The 2026-05-07 backlog sweep landed every small/concrete item. Remaining
entries are deferred because they need design decisions, live external
instances, or cross-cutting refactors:
### Deferred — needs design or larger refactor
### Deferred — Documenso-related (back-burnered until phases 2-7 land)
- **Storage proxy token does not bind to port_id** — `src/lib/storage/filesystem.ts:73-84`. Adding a `p` (portId) claim is mechanical; the meaningful security gain requires the proxy verifier to look up the file's owning row + assert `owner.portId === payload.p`. That requires either a routing prefix in the key (currently `${portSlug}/...` already, so a prefix check is plausible) or a per-table lookup across all owners. Decide which approach before implementing — current state ships with `validateStorageKey` + per-issuer port scoping, so this is defense-in-depth rather than an open hole.
- **Documenso webhook does not enforce port_id on document lookups** — `src/app/api/webhooks/documenso/route.ts:96-148`. Adding port scope requires either including the originating Documenso instance/team id in the lookup (Documenso doesn't surface that on the webhook payload today) OR proving `documents(documenso_id)` is globally unique with a DB constraint and a backfill check. Pick the strategy with the audit doc open.
- **Webhook dedup vs per-recipient signed events** — `src/app/api/webhooks/documenso/route.ts:103-110`. Replacing the body-hash dedup with a `(documensoDocumentId, recipientEmail, eventType)` composite unique requires schema column for recipient_email on `documentEvents`. Right place to do this is alongside Documenso Phase 2 (webhook handler enhancement) since they touch the same code.
- **v2 voidDocument endpoint shape verification** — `src/lib/services/documenso-client.ts:450-466`. Needs a live Documenso 2.x instance to confirm `POST /api/v2/envelope/delete` body shape. Bundle with Documenso Phase 5.
- **Public POST routes bypass service layer** — `src/app/api/public/{interests,website-inquiries,residential-inquiries}/route.ts`. Multi-route refactor extracting a shared `publicInterestService.create(...)`. Worth doing but big enough to deserve its own session.
- **Inconsistent response shapes** — most endpoints return `{ data: ... }`, but `notifications/[notificationId]` returns `{ success: true }`, `website-inquiries` returns `{ id, deduped }`. Codebase-wide migration; document a convention in CLAUDE.md first.
- **`systemSettings` PK / unique-index drift** — `src/lib/db/schema/system.ts:119-133`. Schema declares `uniqueIndex` on `(key, port_id)`, migration uses `key` as PK. `port_id` is nullable so `(key, port_id)` cannot serve as a PK with default NULLs-not-equal semantics. Reconcile by either making `portId` non-null with a sentinel ("**global**") and declaring composite PK, OR by dropping the schema-level unique index and using partial unique indexes for global vs per-port. Either path is a data migration.
- **Documenso webhook does not enforce port_id on document lookups** — `src/app/api/webhooks/documenso/route.ts:96-148`. Bundle with Documenso Phase 2 (webhook handler enhancement) since they touch the same code.
- **Webhook dedup vs per-recipient signed events** — `src/app/api/webhooks/documenso/route.ts:103-110`. Replacing the body-hash dedup with a `(documensoDocumentId, recipientEmail, eventType)` composite unique requires a recipient_email column on `documentEvents`. Bundle with Phase 2.
- **v2 voidDocument endpoint shape verification** — `src/lib/services/documenso-client.ts:450-466`. Needs a live Documenso 2.x instance. Bundle with Phase 5.
### Deferred — pure refactor (no active bug)
- **Public POST routes bypass service layer** — `src/app/api/public/{interests,website-inquiries,residential-inquiries}/route.ts`. The audit's `userId: null as unknown as string` cast was already cleaned up to a proper `userId: null`. Remaining concern is testability: extract a shared `publicInterestService.create(...)`. Pure ergonomics — no active bug or security issue.
### Done in 2026-05-08 sweep (latest)
- ✅ Storage proxy port_id binding: `ProxyTokenPayload` gains optional `p` (port slug) claim; verifier asserts `key.startsWith(${p}/)`. document-sends 24h URLs opt in; other issuers continue working unchanged.
- ✅ system_settings index rebuilt with `NULLS NOT DISTINCT` (migration 0047) — global settings are now uniquely keyed by `key` alone. Surfaced + cleaned 65 duplicate `(storage_backend, NULL)` rows that had accumulated from race-prone delete-then-insert patterns.
- ✅ All 4 read-then-write systemSettings sites converted to true `onConflictDoUpdate` upserts (ocr-config, settings, residential-stages, ai-budget).
- ✅ Response shape standardization: 16 routes converted from `{ success: true }` → `204 No Content`. CLAUDE.md documents the convention.
- ✅ `req.json()` → `parseBody()` migration across 9 admin/CRM routes (custom-fields, expenses/export ×3, currency convert, search/recently-viewed, admin/duplicates, berths/pdf-{upload-url,versions,parse-results}). Portal-auth routes intentionally retained `{ success: true }`.
- ✅ Custom-field merge tokens: validator accepts `{{custom.<fieldName>}}` shape; resolver in `mergeCustomFieldValues` substitutes from per-port custom_field_definitions + per-entity values for client/interest/berth contexts. Banner updated.
- ✅ `/api/v1/files` accepts `companyId` and `yachtId` filters. uploadFile service writes both. file-upload-zone component accepts both props.
- ✅ Company Documents tab (CompanyFilesTab) re-enabled and added to company detail tabs.
### Done in 2026-05-07 sweep (commits in this session)
@@ -91,28 +100,29 @@ instances, or cross-cutting refactors:
- ✅ All FK indexes called out in audit doc (already in place — audit was stale)
- ✅ `documentSends.sentByUserId` FK (already had `.references(...)`)
### Still open — small enough to bundle next time
### Documented limitations (no action planned)
- **`berths.current_pdf_version_id` lacks Drizzle FK** — `src/lib/db/schema/berths.ts:83`. The in-line comment fully documents why (circular FK between `berths` ↔ `berth_pdf_versions` makes column-level `.references()` infeasible). FK is enforced via migration 0030. Treat as documented limitation; revisit if Drizzle adds deferred-FK support.
- **`req.json()` without `parseBody` helper** — admin custom-fields routes use `await req.json(); schema.parse(body)` directly. Migrate for uniform 400 error shapes when the surface area calms down.
- **`berths.current_pdf_version_id` lacks Drizzle FK** — `src/lib/db/schema/berths.ts:83`. The in-line comment fully documents why (circular FK between `berths` ↔ `berth_pdf_versions` makes column-level `.references()` infeasible). FK is enforced via migration 0030. Revisit if Drizzle adds deferred-FK support.
- **`systemSettings` schema declares `uniqueIndex` instead of `NULLS NOT DISTINCT`** — Drizzle's `uniqueIndex` builder doesn't surface the flag. Migration 0047 is the source of truth; `db:push` against an empty DB would skip the flag. Same documented-limitation pattern as `berths.current_pdf_version_id`.
- **One remaining `req.json()` in admin/custom-fields/[fieldId]** — intentional. The handler inspects raw body to detect `fieldType` mutation attempts; parseBody would lose the raw view. Documented inline.
---
## D. Inline TODOs in code (2 remaining)
| File:line | Note | Status |
| ------------------------------------------------------------------------------ | ------------------------------------------------------------ | ------------------------------------------------------ |
| ~~`client-yachts-tab.tsx:93`~~ | YachtForm preset owner prop | ✅ landed 2026-05-07 (`initialOwner` prop) |
| ~~`interest-form.tsx:329`~~ | Include company-owned yachts where client is a member | ✅ landed 2026-05-07 (`yachtOwnerFilter` array filter) |
| ~~`interest-form.tsx:330`~~ | "Add new yacht" inline shortcut | ✅ landed 2026-05-07 (Plus button + YachtForm sheet) |
| [`src/lib/queue/scheduler.ts:44`](../src/lib/queue/scheduler.ts#L44) | Per-user reminder schedule configurable from `user_settings` | Open — needs `user_settings` UI surface |
| [`src/lib/queue/workers/import.ts:13`](../src/lib/queue/workers/import.ts#L13) | Import job handlers — worker is a stub | Open — entire feature surface |
| File:line | Note | Status |
| ------------------------------------------------------------------------------ | --------------------------------------------------------------- | --------------------------------------------------------------------------------------- |
| ~~`client-yachts-tab.tsx:93`~~ | YachtForm preset owner prop | ✅ landed 2026-05-07 (`initialOwner` prop) |
| ~~`interest-form.tsx:329`~~ | Include company-owned yachts where client is a member | ✅ landed 2026-05-07 (`yachtOwnerFilter` array filter) |
| ~~`interest-form.tsx:330`~~ | "Add new yacht" inline shortcut | ✅ landed 2026-05-07 (Plus button + YachtForm sheet) |
| [`src/lib/queue/scheduler.ts:44`](../src/lib/queue/scheduler.ts#L44) | Per-user reminder schedule (override on top of per-port digest) | Placeholder — per-port digest works; revisit when a customer asks for per-user override |
| [`src/lib/queue/workers/import.ts:13`](../src/lib/queue/workers/import.ts#L13) | CSV/Excel import worker — entire feature surface | Placeholder — nothing currently enqueues `import` jobs (verified) |
---
## E. Hidden / stubbed UI tabs
- **Company Documents tab** — `src/components/companies/company-tabs.tsx:229`. Hidden until `/api/v1/files` accepts a `companyId` filter (schema supports it, validator doesn't).
- ✅ **Company Documents tab** — landed 2026-05-08. `/api/v1/files` accepts `companyId`+`yachtId` filters; CompanyFilesTab + uploadZone wired through the storage abstraction.
- **Berth Waiting List + Maintenance Log tabs** — `src/components/berths/berth-tabs.tsx:346`. Removed entirely; revisit if/when product asks.
- **Interest Contract / Reservation tabs** — `src/components/interests/interest-{contract,reservation}-tab.tsx`. Render a "coming soon" friendly card; the real flow is gated on Documenso Phases 2–6.