letsbe/pn-new-crm
1
0
Fork 0
Code Issues Pull Requests 1 Actions Packages Projects Releases Wiki Activity

feat(audit-cleanup): finish all 15 outstanding items from verified backlog

Browse Source 
Audit cleanup completion plan, all tiers shipped:

Tier 1 (security + data integrity)
- A.7 RTBF true wipe: redact email_messages body/subject/addresses for
  threads owned by deleted client; redact document_sends.recipient_email;
  collect file storage keys + delete blobs post-commit.
- A.8 user_permission_overrides FK: documented inline why cascade is
  correct (not set-null as audit suggested) — overrides have no value
  without their user.
- W2.14 PII redaction: camelCase normalization in audit.ts +
  error-events.service.ts isSensitiveKey; added city/postal/country/
  birth fragments. firstName/lastName/dateOfBirth/postalCode etc. now
  caught in BOTH masker paths. 12 new test cases lock the coverage.

Tier 2 (Documenso completion + refactor)
- C.2: documentEvents.recipient_email column + partial unique index for
  per-recipient webhook dedup (migration 0075). handleDocumentSigned
  now sets recipient_email on insert.
- Phase 2: completion_cc_emails distribution. handleDocumentCompleted
  reads documents.completionCcEmails, filters out signer-duplicates
  case-insensitively, fans signed PDF out to non-signer recipients.
- C.4: extracted createPublicInterest() service from the 346-line
  api/public/interests route. Route becomes a thin shell (rate-limit,
  port resolution, audit log, email fan-out). The trio creation logic
  is now unit-testable without an HTTP fixture.
- Phase 4: POST /api/v1/document-templates/[id]/detect-fields wired
  to document-field-detector.detectFields(). Sparkles "Auto-detect"
  button added to template-editor.tsx — maps DetectedField → marker
  with best-guess merge token (DATE / NAME / EMAIL); user retags.

Tier 3 (reporting + recommender snapshot lockfiles)
- W7.reports: extracted rollupStageRevenue / rollupStageCounts /
  computeTotalForecast / computeOccupancyRate / rollupBerthStatusCounts
  into src/lib/services/report-math.ts (pure functions). 16 new tests
  including an inline-snapshot lockfile on a representative 7-stage
  forecast. report-generators.ts now delegates.
- W7.recommender: 18 new toMatchSnapshot tripwires on classifyTier
  boundaries + computeHeat at canonical input points.

Tier 4 (rolling)
- W6.attach: fixed outdated CLAUDE.md claim — threshold banner is
  informational and never depended on IMAP; bounce monitoring (the
  IMAP poller) is separate.
- D.1 + D.2: documented deferral inline with full why-not-build-it
  reasoning so a future engineer sees the rationale.
- G.1: representative formatDate sweep (audit-log-list, user-list,
  document-templates merge tokens, document-signing email). Rest of
  the ~100 sites stay rolling.

Quality gates: 1420/1420 vitest (46 new tests above baseline of 1374),
tsc clean, 0 lint errors.

Plan: docs/superpowers/plans/2026-05-18-audit-cleanup-completion.md
Migration: 0075_c2_document_events_recipient_email.sql (applied to dev DB).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
Matt
2026-05-18 18:22:36 +02:00
parent ef0dc5abc4
commit b3f87563c6
25 changed files with 2569 additions and 350 deletions
Download Patch File Download Diff File Expand all files Collapse all files

281
src/app/api/public/interests/route.ts
View File

@@ -1,23 +1,15 @@
import { NextRequest, NextResponse } from 'next/server';
import { and, eq, isNull, sql } from 'drizzle-orm';
import type { z } from 'zod';
import { eq } from 'drizzle-orm';
import { db } from '@/lib/db';
import { withTransaction } from '@/lib/db/utils';
import { interests, interestBerths } from '@/lib/db/schema/interests';
import { clients, clientContacts, clientAddresses } from '@/lib/db/schema/clients';
import { berths } from '@/lib/db/schema/berths';
import { ports } from '@/lib/db/schema/ports';
import { yachts, yachtOwnershipHistory } from '@/lib/db/schema/yachts';
import { companies, companyMemberships } from '@/lib/db/schema/companies';
import { createAuditLog } from '@/lib/audit';
import { parseBody } from '@/lib/api/route-helpers';
import { errorResponse, RateLimitError, ValidationError } from '@/lib/errors';
import { checkRateLimit, rateLimiters } from '@/lib/rate-limit';
import { publicInterestSchema } from '@/lib/validators/interests';
import { sendInquiryNotifications } from '@/lib/services/inquiry-notifications.service';
import { parsePhone } from '@/lib/i18n/phone';
import type { CountryCode } from '@/lib/i18n/countries';
import { createPublicInterest } from '@/lib/services/public-interest.service';
/**
* Throws RateLimitError if the IP has exceeded the public-form quota.
@@ -32,14 +24,11 @@ async function gateRateLimit(ip: string): Promise<void> {
}
}
type PublicInterestData = z.infer<typeof publicInterestSchema>;
// `withTransaction` exposes its tx argument as `typeof db` (see lib/db/utils.ts).
// Keep the helper aligned with that.
type Tx = typeof db;
// POST /api/public/interests - unauthenticated public interest registration.
// Creates the trio (client + yacht + interest) plus an optional company +
// membership, all inside a single transaction.
// POST /api/public/interests — unauthenticated public interest registration.
// The transactional trio creation (client + yacht + interest, plus optional
// company + membership) lives in `createPublicInterest()` so it's testable
// without an HTTP fixture. This handler is the thin HTTP shell: rate-limit,
// port resolution, body parsing, then post-commit audit log + email fan-out.
export async function POST(req: NextRequest) {
try {
const ip = req.headers.get('x-forwarded-for')?.split(',')[0]?.trim() ?? 'unknown';
@@ -51,213 +40,12 @@ export async function POST(req: NextRequest) {
const portId = req.nextUrl.searchParams.get('portId') ?? req.headers.get('X-Port-Id');
if (!portId) throw new ValidationError('Port context required');
// Server-side phone normalization for older website builds that post raw
// international/national strings. Newer builds may pre-fill phoneE164/Country.
let phoneE164 = data.phoneE164 ?? null;
let phoneCountry: CountryCode | null = (data.phoneCountry as CountryCode | null) ?? null;
if (!phoneE164) {
const parsed = parsePhone(data.phone, phoneCountry ?? undefined);
phoneE164 = parsed.e164;
phoneCountry = parsed.country ?? phoneCountry;
}
const fullName =
data.firstName && data.lastName
? `${data.firstName} ${data.lastName}`
: (data.fullName ?? 'Unknown');
const firstName = data.firstName ?? fullName.split(/\s+/)[0] ?? 'Valued Guest';
// Resolve berth by mooring number (if provided). Read-only lookup - safe
// to do outside the transaction.
let berthId: string | null = null;
let resolvedMooringNumber: string | null = data.mooringNumber ?? null;
if (data.mooringNumber) {
const berth = await db.query.berths.findFirst({
where: and(eq(berths.mooringNumber, data.mooringNumber), eq(berths.portId, portId)),
});
if (berth) {
berthId = berth.id;
resolvedMooringNumber = berth.mooringNumber;
}
}
// ─── Transactional trio creation ────────────────────────────────────────
const result = await withTransaction(async (tx) => {
// 1. Find or create client by email. The inquiry-funnel audit
// flagged that the previous exact match was case-sensitive —
// capital-letter resubmissions spawned duplicate client+yacht+
// interest rows. Match LOWER(value) instead so foo@x.com and
// Foo@X.COM dedupe to the same client.
let clientId: string;
const normalizedEmail = data.email.trim().toLowerCase();
const existingContact = await tx.query.clientContacts.findFirst({
where: and(
eq(clientContacts.channel, 'email'),
sql`LOWER(${clientContacts.value}) = ${normalizedEmail}`,
),
});
if (existingContact) {
const existingClient = await tx.query.clients.findFirst({
where: eq(clients.id, existingContact.clientId),
});
if (existingClient && existingClient.portId === portId) {
clientId = existingClient.id;
const updates: Partial<typeof clients.$inferInsert> = {};
if (data.preferredContactMethod) {
updates.preferredContactMethod = data.preferredContactMethod;
}
if (data.nationalityIso && !existingClient.nationalityIso) {
updates.nationalityIso = data.nationalityIso;
}
if (Object.keys(updates).length > 0) {
await tx.update(clients).set(updates).where(eq(clients.id, clientId));
}
} else {
clientId = await createClientInTx(tx, portId, fullName, data, phoneE164, phoneCountry);
}
} else {
clientId = await createClientInTx(tx, portId, fullName, data, phoneE164, phoneCountry);
}
// 2. Optional: upsert company + add membership
let companyId: string | null = null;
if (data.company) {
const existingCompany = await tx.query.companies.findFirst({
where: and(
eq(companies.portId, portId),
sql`lower(${companies.name}) = lower(${data.company.name})`,
),
});
if (existingCompany) {
companyId = existingCompany.id;
} else {
const [newCompany] = await tx
.insert(companies)
.values({
portId,
name: data.company.name,
legalName: data.company.legalName ?? null,
taxId: data.company.taxId ?? null,
incorporationCountryIso: data.company.incorporationCountryIso ?? null,
incorporationSubdivisionIso: data.company.incorporationSubdivisionIso ?? null,
status: 'active',
})
.returning();
companyId = newCompany!.id;
}
// Add active membership only if one doesn't already exist (open row).
const existingMembership = await tx.query.companyMemberships.findFirst({
where: and(
eq(companyMemberships.companyId, companyId),
eq(companyMemberships.clientId, clientId),
isNull(companyMemberships.endDate),
),
});
if (!existingMembership) {
await tx.insert(companyMemberships).values({
companyId,
clientId,
role: data.company.role ?? 'representative',
startDate: new Date(),
isPrimary: false,
});
}
}
// 3. Create yacht. Owner is the company when provided, else the client.
const ownerType: 'client' | 'company' = companyId ? 'company' : 'client';
const ownerId = companyId ?? clientId;
const [newYacht] = await tx
.insert(yachts)
.values({
portId,
name: data.yacht.name,
hullNumber: data.yacht.hullNumber ?? null,
registration: data.yacht.registration ?? null,
flag: data.yacht.flag ?? null,
yearBuilt: data.yacht.yearBuilt ?? null,
lengthFt: data.yacht.lengthFt != null ? String(data.yacht.lengthFt) : null,
widthFt: data.yacht.widthFt != null ? String(data.yacht.widthFt) : null,
draftFt: data.yacht.draftFt != null ? String(data.yacht.draftFt) : null,
currentOwnerType: ownerType,
currentOwnerId: ownerId,
status: 'active',
})
.returning();
const yachtId = newYacht!.id;
// 3a. Open ownership_history row for the new yacht.
await tx.insert(yachtOwnershipHistory).values({
yachtId,
ownerType,
ownerId,
startDate: new Date(),
endDate: null,
createdBy: 'public-submission',
});
// 4. Store address if provided AND no primary address exists yet.
if (data.address && Object.values(data.address).some(Boolean)) {
const existingAddr = await tx.query.clientAddresses.findFirst({
where: and(eq(clientAddresses.clientId, clientId), eq(clientAddresses.isPrimary, true)),
});
if (!existingAddr) {
await tx.insert(clientAddresses).values({
clientId,
portId,
label: 'Primary',
streetAddress: data.address.street ?? null,
city: data.address.city ?? null,
subdivisionIso: data.address.subdivisionIso ?? null,
postalCode: data.address.postalCode ?? null,
countryIso: data.address.countryIso ?? null,
isPrimary: true,
});
}
}
// 5. Create interest with yachtId wired up. The legacy
// interests.berth_id column has been replaced by the
// interest_berths junction (plan §3.4); when the public form
// resolves to a known berth we materialise it as a primary,
// specific-interest junction row in the same transaction so it
// rolls back together with the parent interest insert.
const [newInterest] = await tx
.insert(interests)
.values({
portId,
clientId,
yachtId,
source: 'website',
pipelineStage: 'open',
})
.returning();
if (berthId) {
await tx.insert(interestBerths).values({
interestId: newInterest!.id,
berthId,
isPrimary: true,
isSpecificInterest: true,
isInEoiBundle: false,
});
}
return {
interestId: newInterest!.id,
clientId,
yachtId,
companyId,
};
});
const result = await createPublicInterest({ portId, data });
// ─── Post-commit side-effects (fire-and-forget) ─────────────────────────
// `AuditLogParams.userId` is `string | null`; null is the documented
// "system-generated" sentinel and matches `audit_logs.user_id` being
// nullable in the schema. The earlier `null as unknown as string`
// cast was a relic from before the type was widened.
// nullable in the schema.
void createAuditLog({
userId: null,
portId,
@@ -270,7 +58,7 @@ export async function POST(req: NextRequest) {
companyId: result.companyId,
source: 'website',
pipelineStage: 'open',
berthId,
berthId: result.berthId,
},
metadata: { type: 'public_registration', ip },
ipAddress: ip,
@@ -286,11 +74,11 @@ export async function POST(req: NextRequest) {
portId,
portSlug: port?.slug ?? portId,
interestId: result.interestId,
clientFullName: fullName,
clientFullName: result.fullName,
clientEmail: data.email,
clientPhone: data.phone,
mooringNumber: resolvedMooringNumber,
firstName,
mooringNumber: result.resolvedMooringNumber,
firstName: result.firstName,
});
return NextResponse.json(
@@ -301,46 +89,3 @@ export async function POST(req: NextRequest) {
return errorResponse(error);
}
}
// ─── Helpers ─────────────────────────────────────────────────────────────────
async function createClientInTx(
tx: Tx,
portId: string,
fullName: string,
data: Pick<PublicInterestData, 'email' | 'phone' | 'preferredContactMethod' | 'nationalityIso'>,
phoneE164: string | null,
phoneCountry: CountryCode | null,
): Promise<string> {
const [newClient] = await tx
.insert(clients)
.values({
portId,
fullName,
preferredContactMethod: data.preferredContactMethod,
nationalityIso: data.nationalityIso ?? null,
source: 'website',
})
.returning();
const clientId = newClient!.id;
await tx.insert(clientContacts).values({
clientId,
channel: 'email',
// Store lowercased so the case-insensitive dedup match above always
// hits on subsequent submissions.
value: data.email.trim().toLowerCase(),
isPrimary: true,
});
await tx.insert(clientContacts).values({
clientId,
channel: 'phone',
value: data.phone,
valueE164: phoneE164,
valueCountry: phoneCountry,
isPrimary: false,
});
return clientId;
}

66
src/app/api/v1/document-templates/[id]/detect-fields/route.ts Normal file
View File

@@ -0,0 +1,66 @@
import { NextResponse } from 'next/server';
import { and, eq } from 'drizzle-orm';
import { withAuth, withPermission } from '@/lib/api/helpers';
import { db } from '@/lib/db';
import { documentTemplates, files } from '@/lib/db/schema/documents';
import { errorResponse, NotFoundError, ValidationError } from '@/lib/errors';
import { getStorageBackend } from '@/lib/storage';
import { detectFields } from '@/lib/services/document-field-detector';
/**
* Phase 4 — Auto-detect signature/date/initials/name/email anchors in the
* template's current source PDF and return suggested field placements.
*
* The detector (`src/lib/services/document-field-detector.ts`) scans each
* page's text content via pdfjs-dist `getTextContent()` and matches anchors
* against a regex pattern table. Returned `DetectedField[]` is in percent-
* coords (0..100 of page dimensions), which the editor converts to its
* own 0..1 marker coords before adding to the field map.
*
* Permission: `admin.manage_settings` — same gate as the editor itself.
*/
export const POST = withAuth(
withPermission('admin', 'manage_settings', async (_req, ctx, params) => {
try {
const template = await db.query.documentTemplates.findFirst({
where: and(eq(documentTemplates.id, params.id!), eq(documentTemplates.portId, ctx.portId)),
});
if (!template) throw new NotFoundError('Template');
if (!template.sourceFileId) {
throw new ValidationError(
'Template has no source PDF — upload one first via the Replace PDF button',
);
}
const sourceFile = await db.query.files.findFirst({
where: and(eq(files.id, template.sourceFileId), eq(files.portId, ctx.portId)),
});
if (!sourceFile) {
throw new NotFoundError('Source PDF file row missing');
}
// Read the PDF blob from storage. Buffer the whole stream — the
// detector needs a contiguous Buffer for pdfjs-dist, and template
// source PDFs are capped at 10MB by the source-pdf upload route.
const backend = await getStorageBackend();
const stream = await backend.get(sourceFile.storagePath);
const chunks: Buffer[] = [];
for await (const chunk of stream) {
chunks.push(typeof chunk === 'string' ? Buffer.from(chunk) : Buffer.from(chunk));
}
const pdfBuffer = Buffer.concat(chunks);
const detected = await detectFields(pdfBuffer);
return NextResponse.json({
data: {
fields: detected,
totalAnchors: detected.length,
},
});
} catch (error) {
return errorResponse(error);
}
}),
);