2026-04-14 12:44:11 -04:00
|
|
|
{
|
|
|
|
|
"version": "7",
|
|
|
|
|
"dialect": "postgresql",
|
|
|
|
|
"entries": [
|
|
|
|
|
{
|
|
|
|
|
"idx": 0,
|
|
|
|
|
"version": "7",
|
|
|
|
|
"when": 1776185027494,
|
|
|
|
|
"tag": "0000_narrow_longshot",
|
|
|
|
|
"breakpoints": true
|
2026-04-14 12:51:45 -04:00
|
|
|
},
|
|
|
|
|
{
|
|
|
|
|
"idx": 1,
|
|
|
|
|
"version": "7",
|
|
|
|
|
"when": 1776185487775,
|
|
|
|
|
"tag": "0001_soft_ender_wiggin",
|
|
|
|
|
"breakpoints": true
|
2026-04-23 17:51:19 +02:00
|
|
|
},
|
|
|
|
|
{
|
|
|
|
|
"idx": 2,
|
|
|
|
|
"version": "7",
|
|
|
|
|
"when": 1776958500747,
|
|
|
|
|
"tag": "0002_groovy_excalibur",
|
|
|
|
|
"breakpoints": true
|
2026-04-23 17:54:02 +02:00
|
|
|
},
|
|
|
|
|
{
|
|
|
|
|
"idx": 3,
|
|
|
|
|
"version": "7",
|
|
|
|
|
"when": 1776959610819,
|
|
|
|
|
"tag": "0003_opposite_lucky_pierre",
|
|
|
|
|
"breakpoints": true
|
2026-04-23 17:55:53 +02:00
|
|
|
},
|
|
|
|
|
{
|
|
|
|
|
"idx": 4,
|
|
|
|
|
"version": "7",
|
|
|
|
|
"when": 1776959707066,
|
|
|
|
|
"tag": "0004_nasty_warstar",
|
|
|
|
|
"breakpoints": true
|
2026-04-23 17:57:29 +02:00
|
|
|
},
|
|
|
|
|
{
|
|
|
|
|
"idx": 5,
|
|
|
|
|
"version": "7",
|
|
|
|
|
"when": 1776959832091,
|
|
|
|
|
"tag": "0005_stale_kronos",
|
|
|
|
|
"breakpoints": true
|
2026-04-23 17:58:52 +02:00
|
|
|
},
|
|
|
|
|
{
|
|
|
|
|
"idx": 6,
|
|
|
|
|
"version": "7",
|
|
|
|
|
"when": 1776959911400,
|
|
|
|
|
"tag": "0006_great_pixie",
|
|
|
|
|
"breakpoints": true
|
2026-04-23 18:00:12 +02:00
|
|
|
},
|
|
|
|
|
{
|
|
|
|
|
"idx": 7,
|
|
|
|
|
"version": "7",
|
|
|
|
|
"when": 1776959993173,
|
|
|
|
|
"tag": "0007_brainy_felicia_hardy",
|
|
|
|
|
"breakpoints": true
|
refactor(clients): drop deprecated yacht/company/proxy columns
PR 13: now that all reads are migrated to the dedicated yacht / company
/ membership entities, drop the columns that mirrored them on `clients`:
companyName, isProxy, proxyType, actualOwnerName, relationshipNotes,
yachtName, yachtLength{Ft,M}, yachtWidth{Ft,M}, yachtDraft{Ft,M},
berthSizeDesired.
Migration `0008_loud_ikaris.sql` issues the destructive ALTER TABLE
DROP COLUMN statements. Run `pnpm db:push` (or the migration runner) to
apply.
Caller cleanup (zero behavioral change to remaining flows):
- Drops the legacy `generateEoi` flow entirely (route, service function,
pdfme template, validator schema). The dual-path generate-and-sign
service from PR 11 has fully replaced it; the route was no longer
wired to the UI.
- `clients.service`: company-name search column / WHERE / audit value
removed; search now ranks by full name only.
- `interests.service`: `resolveLeadCategory` reads dimensions from
`yachts` via `interest.yachtId` instead of the dropped
`client.yachtLength{Ft,M}`.
- `record-export`: client-summary now lists yachts via owner-side
lookup (direct + active company memberships); interest-summary fetches
yacht via `interest.yachtId`. Both PDF templates updated to read
yacht details from the new entity.
- `client-detail-header`, `client-picker`, `command-search`,
`search-result-item`, `use-search` hook, `types/domain.ts`,
`search.service` — drop the companyName badge / sub-label / typed
field everywhere it was rendered or fetched.
- `ai.ts` worker: drop the company / yacht context lines from the
prompt (will be re-added later sourced from the new entities).
- `validators/interests.ts`: remove the deprecated public-form flat
yacht/company fields. The route already ignores them.
- `factories.ts`: drop the `isProxy: false` default.
Tests: 652/652 green; type-check clean. The
`security-sensitive-data` tests use `companyName` / `isProxy` as
arbitrary record keys for a generic util — left unchanged.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-26 13:57:54 +02:00
|
|
|
},
|
|
|
|
|
{
|
|
|
|
|
"idx": 8,
|
|
|
|
|
"version": "7",
|
|
|
|
|
"when": 1777204563579,
|
|
|
|
|
"tag": "0008_loud_ikaris",
|
|
|
|
|
"breakpoints": true
|
feat(portal): replace magic-link with email/password + admin-initiated activation
The client portal no longer uses passwordless / magic-link sign-in. Each
client now has a `portal_users` row with a scrypt-hashed password,
created by an admin from the client detail page; the admin's invite
mails an activation link that the client uses to set their own password.
Forgot-password is wired through the same token mechanism.
Schema (migration `0009_outgoing_rumiko_fujikawa.sql`):
- `portal_users` — one per client account, separate from the CRM
`users` table (better-auth) so the auth realms stay isolated. Email
is globally unique, password is null until activation.
- `portal_auth_tokens` — single-use activation / reset tokens. Stores
only the SHA-256 hash so a DB compromise never leaks live tokens.
Services:
- `src/lib/portal/passwords.ts` — scrypt hash/verify (no new deps;
uses node:crypto), token mint+hash helpers.
- `src/lib/services/portal-auth.service.ts` — createPortalUser,
resendActivation, activateAccount, signIn (timing-safe),
requestPasswordReset, resetPassword. Auth failures throw the new
UnauthorizedError (401); enumeration-safe behaviour everywhere.
Routes:
- POST /api/portal/auth/sign-in — sets the existing portal JWT cookie.
- POST /api/portal/auth/forgot-password — always 200.
- POST /api/portal/auth/reset-password — token + new password.
- POST /api/portal/auth/activate — token + initial password.
- POST /api/v1/clients/:id/portal-user — admin invite (and `?action=resend`).
- Removed: /api/portal/auth/request, /api/portal/auth/verify (magic link).
UI:
- /portal/login — replaced email-only magic-link form with email +
password + "forgot password" link.
- /portal/forgot-password, /portal/reset-password, /portal/activate — new.
- New shared `PasswordSetForm` component used by activate + reset.
- New `PortalInviteButton` rendered on the client detail header.
Email send:
- `createTransporter` now wires SMTP auth when SMTP_USER+SMTP_PASS are
set (gmail app-password or marina-server creds, configured via env).
- `SMTP_FROM` env var lets the sender address be overridden without
pinning it to `noreply@${SMTP_HOST}`.
Tests:
- Smoke spec 17 (client-portal) updated to the new flow: 7/7 green.
- Smoke specs 02-crud-spine, 05-invoices, 20-critical-path updated to
match the post-refactor client + invoice forms (drop companyName,
use OwnerPicker + billingEmail).
- Vitest 652/652 still green; type-check clean.
Drops the dead `requestMagicLink` from portal.service.ts.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-26 15:34:02 +02:00
|
|
|
},
|
|
|
|
|
{
|
|
|
|
|
"idx": 9,
|
|
|
|
|
"version": "7",
|
|
|
|
|
"when": 1777210206070,
|
|
|
|
|
"tag": "0009_outgoing_rumiko_fujikawa",
|
|
|
|
|
"breakpoints": true
|
feat(platform): residential module + admin UI + reliability fixes
Residential platform
- New schema: residentialClients, residentialInterests (separate from
marina/yacht clients) with migration 0010
- Service layer with CRUD + audit + sockets + per-port portal toggle
- v1 + public API routes (/api/v1/residential/*, /api/public/residential-inquiries)
- List + detail pages with inline editing for clients and interests
- Per-user residentialAccess toggle on userPortRoles (migration 0011)
- Permission keys: residential_clients, residential_interests
- Sidebar nav + role form integration
- Smoke spec covering page loads, UI create flow, public endpoint
Admin & shared UI
- Admin → Forms (form templates CRUD) with validators + service
- Notification preferences page (in-app + email per type)
- Email composition + accounts list + threads view
- Branded auth shell shared across CRM + portal auth surfaces
- Inline editing extended to yacht/company/interest detail pages
- InlineTagEditor + per-entity tags endpoints (yachts, companies)
- Notes service polymorphic across clients/interests/yachts/companies
- Client list columns: yachtCount + companyCount badges
- Reservation file-download via presigned URL (replaces stale <a href>)
Route handler refactor
- Extracted yachts/companies/berths reservation handlers to sibling
handlers.ts files (Next.js 15 route.ts only allows specific exports)
Reliability fixes
- apiFetch double-stringify bug fixed across 13 components
(apiFetch already JSON.stringifies its body; passing a stringified
body produced double-encoded JSON which failed zod validation)
- SocketProvider gated behind useSyncExternalStore-based mount check
to avoid useSession() SSR crashes under React 19 + Next 15
- apiFetch falls back to URL-pathname → port-id resolution when the
Zustand store hasn't hydrated yet (fresh contexts, e2e tests)
- CRM invite flow (schema, service, route, email, dev script)
- Dashboard route → [portSlug]/dashboard/page.tsx + redirect
- Document the dev-server restart-after-migration gotcha in CLAUDE.md
Tests
- 5-case residential smoke spec
- Integration test updates for new service signatures
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-27 21:54:32 +02:00
|
|
|
},
|
|
|
|
|
{
|
|
|
|
|
"idx": 10,
|
|
|
|
|
"version": "7",
|
|
|
|
|
"when": 1777303428222,
|
|
|
|
|
"tag": "0010_brave_joshua_kane",
|
|
|
|
|
"breakpoints": true
|
|
|
|
|
},
|
|
|
|
|
{
|
|
|
|
|
"idx": 11,
|
|
|
|
|
"version": "7",
|
|
|
|
|
"when": 1777307410311,
|
|
|
|
|
"tag": "0011_red_cargill",
|
|
|
|
|
"breakpoints": true
|
|
|
|
|
},
|
|
|
|
|
{
|
|
|
|
|
"idx": 12,
|
|
|
|
|
"version": "7",
|
|
|
|
|
"when": 1777308900666,
|
|
|
|
|
"tag": "0012_large_zarda",
|
|
|
|
|
"breakpoints": true
|
2026-04-28 02:12:05 +02:00
|
|
|
},
|
|
|
|
|
{
|
|
|
|
|
"idx": 13,
|
|
|
|
|
"version": "7",
|
|
|
|
|
"when": 1777334766194,
|
|
|
|
|
"tag": "0013_abnormal_thundra",
|
|
|
|
|
"breakpoints": true
|
feat(insights): Phase B schema + service skeletons
PR1 of Phase B per docs/superpowers/specs/2026-04-28-phase-b-insights-alerts-design.md.
Lays the foundation that PRs 2-10 will fill in with behaviour.
Schema (migration 0014):
- alerts table with rule-engine fields (rule_id, severity, link,
entity_type/id, fingerprint, fired/dismissed/acknowledged/resolved
timestamps, jsonb metadata). Partial-unique fingerprint index keeps
one open row per (port, rule, entity); separate indexes power
severity-filtered and time-ordered queries.
- analytics_snapshots (port_id, metric_id) -> jsonb cache + computedAt
for the 15-min recurring refresh.
- expenses: duplicate_of self-FK, dedup_scanned_at, ocr_status/raw/
confidence; partial index on (port, vendor, amount, date) where
duplicate_of IS NULL drives the dedup heuristic.
- audit_logs.search_text: GENERATED ALWAYS tsvector over
action+entity_type+entity_id+user_id, GIN-indexed (drizzle can't
model GENERATED ALWAYS in TS yet, so the migration appends manual
ALTER + the GIN index).
Service skeletons in src/lib/services/:
- alerts.service.ts: fingerprintFor, reconcileAlertsForPort (upsert +
auto-resolve), dismiss, acknowledge, listAlertsForPort.
- alert-rules.ts: RULE_REGISTRY of 10 rule evaluators (currently no-op);
PR2 fills in the bodies.
- analytics.service.ts: readSnapshot/writeSnapshot with 15-min TTL +
no-op compute* stubs for the four chart series; PR3 fills behavior.
- expense-dedup.service.ts: scanForDuplicates + markBestDuplicate
using the partial dedup index. PR8 wires the BullMQ trigger.
- expense-ocr.service.ts: OcrResult/OcrLineItem types + ocrReceipt
stub. PR9 wires Claude Vision (Haiku 4.5 + ephemeral system-prompt
cache).
- audit-search.service.ts: tsvector @@ plainto_tsquery + cursor
pagination on (createdAt, id). PR10 wires the admin UI.
tsc clean, lint clean, vitest 675/675 (one unrelated AES random-output
flake passes solo).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-28 14:43:01 +02:00
|
|
|
},
|
|
|
|
|
{
|
|
|
|
|
"idx": 14,
|
|
|
|
|
"version": "7",
|
|
|
|
|
"when": 1777379952283,
|
|
|
|
|
"tag": "0014_black_banshee",
|
|
|
|
|
"breakpoints": true
|
feat(i18n): country/phone/timezone/subdivision primitives + form wiring
Cross-cutting i18n polish for forms across the marina + residential + company
domains. Introduces a single source of truth for country/phone/timezone/
subdivision data and replaces every nationality-as-free-text and timezone-
as-string Input with a dedicated combobox.
PR1 Countries — ALL_COUNTRY_CODES (~250 ISO-3166-1 alpha-2), Intl.DisplayNames
for localized labels, detectDefaultCountry() with navigator-region
fallback to US, CountryCombobox with regional-indicator flag glyphs +
compact mode for inline use.
PR2 Phone — libphonenumber-js wrapper (parsePhone / formatAsYouType /
callingCodeFor), PhoneInput with flag dropdown + national-format
AsYouType + paste-detect that flips the country dropdown for pasted
international strings.
PR3 Timezones — country->IANA map (250 entries, multi-zone for AU/BR/CA/CD/
ID/KZ/MN/MX/RU/US), formatTimezoneLabel ("Europe/London (UTC+1)"),
TimezoneCombobox with Suggested/All grouping driven by countryHint.
PR4 Subdivisions — wraps the iso-3166-2 npm package (~5000 ISO 3166-2
codes for every country), per-country cache, SubdivisionCombobox with
"Pick a country first" / "No regions available" empty states.
PR5 Schema deltas (migration 0015) — clients.nationality_iso, clientContacts
{value_e164, value_country}, clientAddresses {country_iso, subdivision_iso},
residentialClients {phone_e164, phone_country, nationality_iso, timezone,
place_of_residence_country_iso, subdivision_iso}, companies {incorporation_
country_iso, incorporation_subdivision_iso}, companyAddresses {country_iso,
subdivision_iso}. Plus shared zod validators (validators/i18n.ts) used
by every entity validator + route handler.
PR6 ClientForm + ClientDetail — CountryCombobox replaces nationality Input,
TimezoneCombobox replaces timezone Input (driven by nationalityIso hint),
PhoneInput conditionally rendered for phone/whatsapp contacts. Inline
editors (InlineCountryField / InlineTimezoneField / InlinePhoneField)
for the detail-page overview rows + ContactsEditor.
PR7 Residential client form + detail — phone -> PhoneInput, nationality/
timezone/place-of-residence-country/subdivision rows in both create
sheet and inline-editable detail view. Subdivision wipes when country
flips since codes are country-scoped.
PR8 Company form + detail — incorporation country -> CountryCombobox,
incorporation region -> SubdivisionCombobox in both modes.
PR9 Public inquiry endpoint — accepts pre-normalized phoneE164/phoneCountry
and i18n fields from newer website builds, server-side parsePhone()
fallback for legacy raw-international submissions. Old Nuxt builds
keep working unchanged.
Tests: 4 unit suites for the primitives (25 tests), 1 integration spec for
the public phone-normalization path (3 tests), 1 smoke spec asserting the
combobox triggers render in all three create sheets.
Test totals: vitest 713 -> 741 (+28).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-28 18:13:08 +02:00
|
|
|
},
|
|
|
|
|
{
|
|
|
|
|
"idx": 15,
|
|
|
|
|
"version": "7",
|
|
|
|
|
"when": 1777391373291,
|
|
|
|
|
"tag": "0015_i18n_columns",
|
|
|
|
|
"breakpoints": true
|
chore(i18n): drop legacy free-text country/nationality columns
Test-data only — no production migration needed (per earlier decision).
Schema is now ISO-only; readers convert ISO codes to localized names where
human-readable output is required (EOI documents, invoices, portal).
Migration 0016 drops:
- clients.nationality
- companies.incorporation_country
- client_addresses.{state_province, country}
- company_addresses.{state_province, country}
Code paths that previously read free-text values now read the ISO column
and pass through `getCountryName()` / `getSubdivisionName()` for rendering.
Document templates ({{client.nationality}}), portal client view, EOI/
reservation-agreement contexts, and invoice billing addresses all updated.
Public yacht-interest endpoint (/api/public/interests) drops the legacy
fields from its insert path and writes ISO codes only. The Zod validators
no longer accept the legacy fields — older website builds posting raw
'incorporationCountry' / 'country' / 'stateProvince' will get 400s.
Server-side phone normalization is unchanged.
Seed data updated to use ISO codes (GB/FR/ES/GR/SE/IT/GH/MC/PA), spread
across continents to keep test fixtures realistic.
Test assertions updated to match the new render shape (e.g.
'United States' not 'US', 'California' not 'CA').
Vitest: 741 -> 741 (unchanged count; assertions updated, no new tests).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-28 19:00:57 +02:00
|
|
|
},
|
|
|
|
|
{
|
|
|
|
|
"idx": 16,
|
|
|
|
|
"version": "7",
|
|
|
|
|
"when": 1777395538988,
|
|
|
|
|
"tag": "0016_magical_spyke",
|
|
|
|
|
"breakpoints": true
|
feat(ai): per-port token budgets + usage ledger for AI features
Adds a token-denominated guardrail in front of every server-side AI call
so a misconfigured port can't run up an unbounded bill. Soft caps surface
a banner; hard caps refuse new requests until the period rolls over.
Usage flows into a feature-typed ledger so future AI surfaces (summary,
embeddings, reply-draft) can drop in without schema changes.
- New table ai_usage_ledger (port, user, feature, provider, model,
input/output/total tokens, request id) with two indexes for rollup
- New service ai-budget.service.ts: getAiBudget/setAiBudget,
checkBudget (pre-flight gate), recordAiUsage, currentPeriodTokens,
periodBreakdown — all token-based, period boundaries in UTC
- runOcr now returns provider usage so the route can record the actual
spend instead of estimating
- Scan-receipt route gates on checkBudget before invoking AI; returns
source: manual / reason: budget-exceeded when blocked, surfaces
softCapWarning on the success path
- Admin UI: new AiBudgetCard on the OCR settings page — shows current
spend, per-feature breakdown, soft/hard cap inputs, period selector
- Permission: admin.manage_settings on both routes
Tests: 766/766 vitest (was 756) — +10 budget tests covering enforce/
disabled/cap-exceed/estimate-exceed/soft-warn/period boundaries/
cross-port isolation/silent ledger failure.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-28 19:53:09 +02:00
|
|
|
},
|
|
|
|
|
{
|
|
|
|
|
"idx": 17,
|
|
|
|
|
"version": "7",
|
|
|
|
|
"when": 1777398450555,
|
|
|
|
|
"tag": "0017_tiny_mercury",
|
|
|
|
|
"breakpoints": true
|
feat(gdpr): staff-triggered client-data export bundle (Article 15)
Adds a full GDPR Article 15 (right of access) workflow. Staff trigger
an export from the client detail; a BullMQ worker assembles every row
keyed to that client (profile, contacts, addresses, notes, tags,
yachts, company memberships, interests, reservations, invoices,
documents, last 500 audit events) into JSON + a self-contained HTML
report, ZIPs them, uploads to MinIO, and optionally emails the client
a 7-day signed download link.
- New table gdpr_exports tracks lifecycle (pending → building → ready
→ sent / failed) with a 30-day cleanup target
- Bundle builder (gdpr-bundle-builder.ts) — pure read-side, tenant-
scoped, with HTML escaping to block injection from rogue field values
- Worker hook in export queue dispatches on job name 'gdpr-export'
- New audit actions: 'request_gdpr_export', 'send_gdpr_export'
- API: POST/GET /api/v1/clients/:id/gdpr-export (admin-gated, exports
rate-limit, Article-15 audit on POST); GET /:exportId returns a
fresh signed URL
- UI: <GdprExportButton> dialog on client detail header — admin-only,
shows recent exports, supports email-to-client + override recipient,
polls every 5s while open
- Validation: refuses email-to-client when no primary email + no
override (rather than silently dropping the send)
Tests: 778/778 vitest (was 771) — +7 covering builder happy path,
HTML escaping, tenant isolation, empty client, request-flow validation,
and audit / queue interaction.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-28 20:06:31 +02:00
|
|
|
},
|
|
|
|
|
{
|
|
|
|
|
"idx": 18,
|
|
|
|
|
"version": "7",
|
|
|
|
|
"when": 1777399135032,
|
|
|
|
|
"tag": "0018_stormy_spencer_smythe",
|
|
|
|
|
"breakpoints": true
|
2026-05-02 00:01:33 +02:00
|
|
|
},
|
|
|
|
|
{
|
|
|
|
|
"idx": 19,
|
|
|
|
|
"version": "7",
|
|
|
|
|
"when": 1777671562738,
|
|
|
|
|
"tag": "0019_lazy_vampiro",
|
|
|
|
|
"breakpoints": true
|
feat(dedup): NocoDB migration script + tables (P3 dry-run)
Lands the one-shot migration pipeline from the legacy NocoDB Interests
base into the new client/interest schema. Dry-run mode is fully
operational: pulls the live snapshot, runs the dedup library, and
writes a CSV + Markdown report under .migration/<timestamp>/. The
--apply phase is stubbed for a follow-up PR per the design's P3
implementation sequence.
Schema additions
================
- `client_merge_candidates` — pairs flagged by the background scoring
job for the /admin/duplicates review queue. Status enum: pending /
dismissed / merged. Unique-(portId, clientAId, clientBId) so the
same pair can't surface twice. Empty until P2 lands the cron.
- `migration_source_links` — idempotency ledger. Maps source-system
rows (NocoDB Interest #624 → new client UUID) so re-running --apply
against the same dry-run report skips already-imported entities.
Both tables ship with the migration `0020_unusual_azazel.sql` —
already applied to the local dev DB during this commit's preparation.
Library
=======
src/lib/dedup/nocodb-source.ts
Read-only adapter for the legacy NocoDB v2 API. xc-token auth,
auto-paginates until isLastPage, captures the table IDs from the
2026-05-03 audit. `fetchSnapshot()` pulls every relevant table in
parallel into one in-memory object the transform layer consumes.
src/lib/dedup/migration-transform.ts
Pure function: NocoDB snapshot in, MigrationPlan out. Per row:
- normalizes name / email / phone / country via the dedup library
- parses the legacy DD-MM-YYYY / DD/MM/YYYY / ISO date formats
- maps the 8-stage `Sales Process Level` enum to the new 9-stage
pipelineStage
- filters yacht-name placeholders ('TBC', 'Na', etc.)
- merges Internal Notes + Extra Comments + Berth Size Desired into
a single notes blob
Then runs `findClientMatches` pairwise (with blocking) and
union-finds clusters of rows whose score crosses the auto-link
threshold (90). Lower-scoring pairs (50–89) become 'needs review'.
Each cluster's "lead" row is picked by completeness score with
recency tie-break.
src/lib/dedup/migration-report.ts
Writes three artifacts to .migration/<timestamp>/:
- report.csv — one row per planned op, RFC-4180 escaped
- summary.md — human-skimmable overview
- plan.json — full structured plan for the --apply phase
CSV cells with comma / quote / newline are quoted; internal quotes
are doubled. No external CSV dep.
src/lib/dedup/phone-parse.ts
Script-safe wrapper around libphonenumber-js's `core` entry that
loads `metadata.min.json` directly. The default `index.cjs.js`
bundled by libphonenumber hits a metadata-shape interop bug under
Node 25 + tsx (`{ default }` wrapping); core+JSON sidesteps it.
The dedup `normalizePhone` and `find-matches` both use this wrapper
now so the same code path runs in vitest, Next.js, and the migration
CLI without surprises.
src/lib/dedup/normalize.ts
Tightened country resolution: added Caribbean short-form aliases
('antigua' → AG, 'st kitts' → KN, etc.) and a city map covering the
US locations seen in the NocoDB dump (Boston, Tampa, Fort
Lauderdale, Port Jefferson, Nantucket). Also relaxed phone parsing
to drop the `isValid()` strict check — the libphonenumber min build
rejects many real NANP-territory numbers, and dedup only needs a
canonical E.164 to compare.
CLI
===
scripts/migrate-from-nocodb.ts
pnpm tsx scripts/migrate-from-nocodb.ts --dry-run
→ Pulls the live NocoDB base (NOCODB_URL + NOCODB_TOKEN env vars),
runs the transform, writes report. No DB writes.
pnpm tsx scripts/migrate-from-nocodb.ts --apply --report .migration/<dir>/
→ Stubbed; exits with `not yet implemented` and a pointer to the
design doc. Apply phase ships in a follow-up.
Tests
=====
tests/unit/dedup/migration-transform.test.ts (7 cases)
Fixture-based regression. A frozen 12-row NocoDB snapshot covers
every duplicate pattern in the design (§1.2). The test asserts:
- 12 input rows → 7 unique clients (cluster math is right)
- Patterns A / B / C / E auto-link
- Pattern F (Etiennette Clamouze) does NOT auto-link
- Every interest preserved as its own row even when clients merge
- 8-stage → 9-stage enum mapping is correct per spec
- Multi-yacht merge (Constanzo CALYPSO + Costanzo GEMINI under one
client) — the design's signature win
- Output is deterministic (run twice, identical)
Validation against real data
============================
Ran `pnpm tsx scripts/migrate-from-nocodb.ts --dry-run` against the
live NocoDB. Result on 252 Interests rows:
- 237 clients (15 merged into 13 clusters)
- 252 interests (one per source row)
- 406 contacts, 52 addresses
- 13 auto-linked clusters (every confirmed cluster from §1.2 audit)
- 3 pairs flagged for review (Camazou, Zasso, one new)
- 1 phone placeholder flagged
Total dedup test count: 57 (50 from P1 + 7 fixture tests).
Lint: clean. Tsc: clean for new files.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-03 14:50:01 +02:00
|
|
|
},
|
|
|
|
|
{
|
|
|
|
|
"idx": 20,
|
|
|
|
|
"version": "7",
|
|
|
|
|
"when": 1777811835982,
|
|
|
|
|
"tag": "0020_unusual_azazel",
|
|
|
|
|
"breakpoints": true
|
feat(dedup): runtime surfaces — merge service, at-create suggestion, admin queue (P2)
Adds the live dedup pipeline on top of the P1 library + P3 migration
script. The new `client/interest` model now actively prevents duplicate
client records at creation time and gives admins a queue to triage
the borderline pairs the at-create check missed.
Three layers, per design §7:
Layer 1 — At-create suggestion
==============================
`GET /api/v1/clients/match-candidates`
Accepts free-text email / phone / name from the in-flight client
form, normalizes them via the dedup library, and returns scored
matches against the port's live client pool. Filters out
low-confidence noise (the background scoring queue picks those up
separately). Strict port scoping; never leaks across tenants.
`<DedupSuggestionPanel>` (`src/components/clients/dedup-suggestion-panel.tsx`)
Debounced React Query hook. Renders nothing for short inputs or
no useful match. On a high-confidence match it interrupts visually
with an amber-tinted card and a "Use this client" primary button.
Medium confidence falls back to a softer "possible match — check
before creating" treatment.
`<ClientForm>`
Renders the panel above the form (create path only — skipped on
edit). New `onUseExistingClient` callback fires when the user
picks the existing client; the form closes and the parent decides
what to do (typically: navigate to that client's detail page or
open the create-interest dialog pre-filled).
Layer 2 — Merge service
=======================
`mergeClients` (`src/lib/services/client-merge.service.ts`)
The atomic merge primitive that everything else calls. Single
transaction. Per §6 of the design:
- Locks both rows (FOR UPDATE) so concurrent merges of the same
loser fail with a clear error rather than racing.
- Snapshots the full loser state (contacts / addresses / notes /
tags / interest+reservation IDs / relationship rows) into the
`client_merge_log.merge_details` JSONB column for the eventual
undo flow.
- Reattaches every loser-side row to the winner: interests,
reservations, contacts (skipping duplicates by `(channel, value)`),
addresses, notes, tags (deduped), relationships.
- Optional `fieldChoices` — per-scalar overrides letting the user
keep the loser's value for fullName / nationality / preferences /
timezone / source.
- Marks the loser archived with `mergedIntoClientId` set (a redirect
pointer for stragglers; never hard-deleted within the undo window).
- Resolves any matching `client_merge_candidates` row to status='merged'.
- Writes audit log entry.
Schema additions:
- `clients.merged_into_client_id` (nullable text, indexed) — the
redirect pointer set on archive.
Tests: 6 cases against a real DB — happy path moves rows + writes log;
self-merge / cross-port / already-merged refused; duplicate-contact
deduped on reattach; fieldChoices copies loser values to winner.
Layer 3 — Admin review queue
============================
`GET /api/v1/admin/duplicates`
Pending merge candidates (status='pending') for the current port,
with both client summaries hydrated for side-by-side rendering.
Skips pairs where one side is already archived/merged.
`POST /api/v1/admin/duplicates/[id]/merge`
Confirms a candidate. Body picks the winner; the other side
becomes the loser. Calls into `mergeClients` — the only path that
writes `client_merge_log`.
`POST /api/v1/admin/duplicates/[id]/dismiss`
Marks the candidate dismissed. Future scoring runs skip the same
pair until a score change recreates the row.
`<DuplicatesReviewQueue>` (`/admin/duplicates`)
Side-by-side card UI for each pending pair. Click a card to pick
the winner; the other side is automatically the loser. Toolbar:
"Merge into selected" + "Dismiss". No per-field merge editor in
this PR — that's a future polish; the simple "pick the better row"
flow handles ~80% of cases.
Test coverage
=============
11 new integration tests (76 added in this branch total):
- 6 mergeClients (atomicity, refusal cases, contact dedup,
fieldChoices)
- 5 match-candidates API (shape, port scoping, confidence tiers,
Pattern F false-positive guard)
Full vitest: 926/926 passing (was 858 before the dedup branch).
Lint: clean. tsc: clean for new files (only pre-existing errors in
unrelated `tests/integration/` files remain, same as before this PR).
Out of scope, deferred
======================
- Background scoring cron that populates `client_merge_candidates`
(the queue is empty until this lands; manual seeding works for
now via the at-create flow).
- Side-by-side per-field merge editor with checkboxes (the simple
"pick the winner" UX shipped here covers ~80% of real cases).
- Admin settings UI for tuning the dedup thresholds. Defaults from
the design (90 / 50) are baked in for now.
- `unmergeClients` (the snapshot is captured in client_merge_log;
the undo endpoint just hasn't been wired yet).
These are all natural follow-up PRs that don't block shipping the
runtime UX.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-03 14:59:04 +02:00
|
|
|
},
|
|
|
|
|
{
|
|
|
|
|
"idx": 21,
|
|
|
|
|
"version": "7",
|
|
|
|
|
"when": 1777812671833,
|
|
|
|
|
"tag": "0021_magenta_madame_hydra",
|
|
|
|
|
"breakpoints": true
|
2026-04-14 12:44:11 -04:00
|
|
|
}
|
|
|
|
|
]
|
|
|
|
|
}
|
