Files
pn-new-crm/src/app/api/v1/clients/match-candidates/handlers.ts

161 lines
5.9 KiB
TypeScript
Raw Normal View History

feat(dedup): runtime surfaces — merge service, at-create suggestion, admin queue (P2) Adds the live dedup pipeline on top of the P1 library + P3 migration script. The new `client/interest` model now actively prevents duplicate client records at creation time and gives admins a queue to triage the borderline pairs the at-create check missed. Three layers, per design §7: Layer 1 — At-create suggestion ============================== `GET /api/v1/clients/match-candidates` Accepts free-text email / phone / name from the in-flight client form, normalizes them via the dedup library, and returns scored matches against the port's live client pool. Filters out low-confidence noise (the background scoring queue picks those up separately). Strict port scoping; never leaks across tenants. `<DedupSuggestionPanel>` (`src/components/clients/dedup-suggestion-panel.tsx`) Debounced React Query hook. Renders nothing for short inputs or no useful match. On a high-confidence match it interrupts visually with an amber-tinted card and a "Use this client" primary button. Medium confidence falls back to a softer "possible match — check before creating" treatment. `<ClientForm>` Renders the panel above the form (create path only — skipped on edit). New `onUseExistingClient` callback fires when the user picks the existing client; the form closes and the parent decides what to do (typically: navigate to that client's detail page or open the create-interest dialog pre-filled). Layer 2 — Merge service ======================= `mergeClients` (`src/lib/services/client-merge.service.ts`) The atomic merge primitive that everything else calls. Single transaction. Per §6 of the design: - Locks both rows (FOR UPDATE) so concurrent merges of the same loser fail with a clear error rather than racing. - Snapshots the full loser state (contacts / addresses / notes / tags / interest+reservation IDs / relationship rows) into the `client_merge_log.merge_details` JSONB column for the eventual undo flow. - Reattaches every loser-side row to the winner: interests, reservations, contacts (skipping duplicates by `(channel, value)`), addresses, notes, tags (deduped), relationships. - Optional `fieldChoices` — per-scalar overrides letting the user keep the loser's value for fullName / nationality / preferences / timezone / source. - Marks the loser archived with `mergedIntoClientId` set (a redirect pointer for stragglers; never hard-deleted within the undo window). - Resolves any matching `client_merge_candidates` row to status='merged'. - Writes audit log entry. Schema additions: - `clients.merged_into_client_id` (nullable text, indexed) — the redirect pointer set on archive. Tests: 6 cases against a real DB — happy path moves rows + writes log; self-merge / cross-port / already-merged refused; duplicate-contact deduped on reattach; fieldChoices copies loser values to winner. Layer 3 — Admin review queue ============================ `GET /api/v1/admin/duplicates` Pending merge candidates (status='pending') for the current port, with both client summaries hydrated for side-by-side rendering. Skips pairs where one side is already archived/merged. `POST /api/v1/admin/duplicates/[id]/merge` Confirms a candidate. Body picks the winner; the other side becomes the loser. Calls into `mergeClients` — the only path that writes `client_merge_log`. `POST /api/v1/admin/duplicates/[id]/dismiss` Marks the candidate dismissed. Future scoring runs skip the same pair until a score change recreates the row. `<DuplicatesReviewQueue>` (`/admin/duplicates`) Side-by-side card UI for each pending pair. Click a card to pick the winner; the other side is automatically the loser. Toolbar: "Merge into selected" + "Dismiss". No per-field merge editor in this PR — that's a future polish; the simple "pick the better row" flow handles ~80% of cases. Test coverage ============= 11 new integration tests (76 added in this branch total): - 6 mergeClients (atomicity, refusal cases, contact dedup, fieldChoices) - 5 match-candidates API (shape, port scoping, confidence tiers, Pattern F false-positive guard) Full vitest: 926/926 passing (was 858 before the dedup branch). Lint: clean. tsc: clean for new files (only pre-existing errors in unrelated `tests/integration/` files remain, same as before this PR). Out of scope, deferred ====================== - Background scoring cron that populates `client_merge_candidates` (the queue is empty until this lands; manual seeding works for now via the at-create flow). - Side-by-side per-field merge editor with checkboxes (the simple "pick the winner" UX shipped here covers ~80% of real cases). - Admin settings UI for tuning the dedup thresholds. Defaults from the design (90 / 50) are baked in for now. - `unmergeClients` (the snapshot is captured in client_merge_log; the undo endpoint just hasn't been wired yet). These are all natural follow-up PRs that don't block shipping the runtime UX. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-03 14:59:04 +02:00
import { NextResponse } from 'next/server';
import { and, eq, inArray } from 'drizzle-orm';
import type { AuthContext } from '@/lib/api/helpers';
import { db } from '@/lib/db';
import { clients, clientContacts } from '@/lib/db/schema/clients';
import { interests } from '@/lib/db/schema/interests';
import { errorResponse } from '@/lib/errors';
import { findClientMatches, type MatchCandidate } from '@/lib/dedup/find-matches';
import { normalizeEmail, normalizeName, normalizePhone } from '@/lib/dedup/normalize';
import type { CountryCode } from '@/lib/i18n/countries';
/**
* GET /api/v1/clients/match-candidates
*
* Query parameters (any combination):
* email Free-text email; gets normalized server-side.
* phone Free-text phone; gets normalized to E.164 server-side.
* name Free-text full name; used for surname-token blocking.
* country Optional ISO country hint (default: AI for Port Nimara).
*
* Returns the top candidates that scored above the soft-warn threshold,
* each with a small client summary the form's suggestion card can
* render. Confidence tiers and rules are applied server-side from the
* port's `system_settings` (when wired) or sensible defaults otherwise.
*
* Used by `useDedupSuggestion` in the new-client form. Debounced on
* the client; this endpoint must be cheap (single port pool fetch +
* an in-memory dedup pass).
*/
export async function getMatchCandidatesHandler(
req: Request,
ctx: AuthContext,
): Promise<NextResponse> {
try {
const url = new URL(req.url);
const rawEmail = url.searchParams.get('email');
const rawPhone = url.searchParams.get('phone');
const rawName = url.searchParams.get('name');
const country = (url.searchParams.get('country') ?? 'AI') as CountryCode;
const email = rawEmail ? normalizeEmail(rawEmail) : null;
const phoneResult = rawPhone ? normalizePhone(rawPhone, country) : null;
const nameResult = rawName ? normalizeName(rawName) : null;
// If the caller didn't give us anything useful to match on, return empty
// — short-circuit rather than scan every client for nothing.
if (!email && !phoneResult?.e164 && !nameResult?.surnameToken) {
return NextResponse.json({ data: [] });
}
// Build the input candidate.
const input: MatchCandidate = {
id: '__incoming__',
fullName: nameResult?.display ?? null,
surnameToken: nameResult?.surnameToken ?? null,
emails: email ? [email] : [],
phonesE164: phoneResult?.e164 ? [phoneResult.e164] : [],
countryIso: country,
};
// Fetch the live pool for this port. We keep this O(N) over clients
// since the dedup library does its own blocking; for ports with
// thousands of clients we can later restrict by surname-token /
// contact lookups, but for current scale the simple full-pool fetch
// is fine.
const liveClients = await db
.select({
id: clients.id,
fullName: clients.fullName,
nationalityIso: clients.nationalityIso,
})
.from(clients)
.where(and(eq(clients.portId, ctx.portId)));
if (liveClients.length === 0) {
return NextResponse.json({ data: [] });
}
const clientIds = liveClients.map((c) => c.id);
const contactRows = await db
.select({
clientId: clientContacts.clientId,
channel: clientContacts.channel,
value: clientContacts.value,
valueE164: clientContacts.valueE164,
})
.from(clientContacts)
.where(inArray(clientContacts.clientId, clientIds));
// Group contacts by client for the candidate map.
const emailsByClient = new Map<string, string[]>();
const phonesByClient = new Map<string, string[]>();
for (const c of contactRows) {
if (c.channel === 'email') {
const arr = emailsByClient.get(c.clientId) ?? [];
arr.push(c.value.toLowerCase());
emailsByClient.set(c.clientId, arr);
} else if (c.channel === 'phone' || c.channel === 'whatsapp') {
if (c.valueE164) {
const arr = phonesByClient.get(c.clientId) ?? [];
arr.push(c.valueE164);
phonesByClient.set(c.clientId, arr);
}
}
}
const pool: MatchCandidate[] = liveClients.map((c) => {
const named = normalizeName(c.fullName);
return {
id: c.id,
fullName: c.fullName,
surnameToken: named.surnameToken ?? null,
emails: emailsByClient.get(c.id) ?? [],
phonesE164: phonesByClient.get(c.id) ?? [],
countryIso: (c.nationalityIso as CountryCode | null) ?? null,
};
});
const matches = findClientMatches(input, pool, {
highScore: 90,
mediumScore: 50,
});
// Only return medium+ — low-confidence noise isn't useful at the
// create-form layer (background scoring queue picks those up).
const useful = matches.filter((m) => m.confidence !== 'low');
if (useful.length === 0) {
return NextResponse.json({ data: [] });
}
// Pull a quick summary for each surfaced candidate so the suggestion
// card has enough to render ("Marcus Laurent · 2 interests · last
// contact 9d ago").
const summarizedIds = useful.map((m) => m.candidate.id);
const interestCounts = await db
.select({ clientId: interests.clientId })
.from(interests)
.where(inArray(interests.clientId, summarizedIds));
const interestsByClient = new Map<string, number>();
for (const r of interestCounts) {
interestsByClient.set(r.clientId, (interestsByClient.get(r.clientId) ?? 0) + 1);
}
const data = useful.map((m) => ({
clientId: m.candidate.id,
fullName: m.candidate.fullName,
score: m.score,
confidence: m.confidence,
reasons: m.reasons,
interestCount: interestsByClient.get(m.candidate.id) ?? 0,
emails: m.candidate.emails,
phonesE164: m.candidate.phonesE164,
}));
return NextResponse.json({ data });
} catch (error) {
return errorResponse(error);
}
}