letsbe/pn-new-crm
1
0
Fork 0
Code Issues Pull Requests 1 Actions Packages Projects Releases Wiki Activity

feat(audit-cleanup): finish all 15 outstanding items from verified backlog

Browse Source 
Audit cleanup completion plan, all tiers shipped:

Tier 1 (security + data integrity)
- A.7 RTBF true wipe: redact email_messages body/subject/addresses for
  threads owned by deleted client; redact document_sends.recipient_email;
  collect file storage keys + delete blobs post-commit.
- A.8 user_permission_overrides FK: documented inline why cascade is
  correct (not set-null as audit suggested) — overrides have no value
  without their user.
- W2.14 PII redaction: camelCase normalization in audit.ts +
  error-events.service.ts isSensitiveKey; added city/postal/country/
  birth fragments. firstName/lastName/dateOfBirth/postalCode etc. now
  caught in BOTH masker paths. 12 new test cases lock the coverage.

Tier 2 (Documenso completion + refactor)
- C.2: documentEvents.recipient_email column + partial unique index for
  per-recipient webhook dedup (migration 0075). handleDocumentSigned
  now sets recipient_email on insert.
- Phase 2: completion_cc_emails distribution. handleDocumentCompleted
  reads documents.completionCcEmails, filters out signer-duplicates
  case-insensitively, fans signed PDF out to non-signer recipients.
- C.4: extracted createPublicInterest() service from the 346-line
  api/public/interests route. Route becomes a thin shell (rate-limit,
  port resolution, audit log, email fan-out). The trio creation logic
  is now unit-testable without an HTTP fixture.
- Phase 4: POST /api/v1/document-templates/[id]/detect-fields wired
  to document-field-detector.detectFields(). Sparkles "Auto-detect"
  button added to template-editor.tsx — maps DetectedField → marker
  with best-guess merge token (DATE / NAME / EMAIL); user retags.

Tier 3 (reporting + recommender snapshot lockfiles)
- W7.reports: extracted rollupStageRevenue / rollupStageCounts /
  computeTotalForecast / computeOccupancyRate / rollupBerthStatusCounts
  into src/lib/services/report-math.ts (pure functions). 16 new tests
  including an inline-snapshot lockfile on a representative 7-stage
  forecast. report-generators.ts now delegates.
- W7.recommender: 18 new toMatchSnapshot tripwires on classifyTier
  boundaries + computeHeat at canonical input points.

Tier 4 (rolling)
- W6.attach: fixed outdated CLAUDE.md claim — threshold banner is
  informational and never depended on IMAP; bounce monitoring (the
  IMAP poller) is separate.
- D.1 + D.2: documented deferral inline with full why-not-build-it
  reasoning so a future engineer sees the rationale.
- G.1: representative formatDate sweep (audit-log-list, user-list,
  document-templates merge tokens, document-signing email). Rest of
  the ~100 sites stay rolling.

Quality gates: 1420/1420 vitest (46 new tests above baseline of 1374),
tsc clean, 0 lint errors.

Plan: docs/superpowers/plans/2026-05-18-audit-cleanup-completion.md
Migration: 0075_c2_document_events_recipient_email.sql (applied to dev DB).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
Matt
2026-05-18 18:22:36 +02:00
parent ef0dc5abc4
commit b3f87563c6
25 changed files with 2569 additions and 350 deletions
Download Patch File Download Diff File Expand all files Collapse all files

44
tests/unit/audit.test.ts
View File

@@ -3,12 +3,18 @@ import { diffFields, maskSensitiveFields } from '@/lib/audit';
describe('diffFields', () => {
it('returns empty array when records are identical', () => {
const result = diffFields({ name: 'Alice', status: 'active' }, { name: 'Alice', status: 'active' });
const result = diffFields(
{ name: 'Alice', status: 'active' },
{ name: 'Alice', status: 'active' },
);
expect(result).toEqual([]);
});
it('detects a single field change with correct field/old/new', () => {
const result = diffFields({ name: 'Alice', status: 'active' }, { name: 'Alice', status: 'inactive' });
const result = diffFields(
{ name: 'Alice', status: 'active' },
{ name: 'Alice', status: 'inactive' },
);
expect(result).toHaveLength(1);
expect(result[0]).toEqual({ field: 'status', oldValue: 'active', newValue: 'inactive' });
});
@@ -117,4 +123,38 @@ describe('maskSensitiveFields', () => {
maskSensitiveFields(original);
expect(original.email).toBe('alice@example.com');
});
describe('camelCase + PII coverage (W2.14 fix)', () => {
it.each([
['firstName', 'Alice'],
['lastName', 'Smith'],
['fullName', 'Alice Smith'],
['dateOfBirth', '1990-01-01'],
['addressLine1', '10 Downing St'],
['addressLine2', 'Flat 3'],
['city', 'London'],
['postalCode', 'SW1A 2AA'],
['country', 'United Kingdom'],
['recipientEmail', 'bob@example.com'],
['phoneNumber', '+44 1234 567890'],
])('masks %s (camelCase PII key)', (key, value) => {
const result = maskSensitiveFields({ [key]: value });
expect(result?.[key]).not.toBe(value);
expect(typeof result?.[key]).toBe('string');
expect(result?.[key] as string).toMatch(/\*\*\*/);
});
it('does not over-mask innocuous "name" fields without PII context', () => {
// 'name' alone (port name, tag name, column name) — must NOT be redacted
// unless it's part of first_name / last_name / full_name etc.
const result = maskSensitiveFields({
port_name: 'Port Nimara',
tag_name: 'VIP',
column_name: 'created_at',
});
expect(result?.port_name).toBe('Port Nimara');
expect(result?.tag_name).toBe('VIP');
expect(result?.column_name).toBe('created_at');
});
});
});

192
tests/unit/services/__snapshots__/berth-recommender.test.ts.snap Normal file
View File

@@ -0,0 +1,192 @@
// Vitest Snapshot v1, https://vitest.dev/guide/snapshot.html
exports[`W7 snapshots — heat at canonical inputs > heat: cold (no history) 1`] = `
{
"eoiCount": 0,
"furthestStage": 0,
"interestCount": 0,
"recency": 0,
"total": 0,
}
`;
exports[`W7 snapshots — heat at canonical inputs > heat: no fallthrough but many interests 1`] = `
{
"eoiCount": 15,
"furthestStage": 0,
"interestCount": 15,
"recency": 0,
"total": 30,
}
`;
exports[`W7 snapshots — heat at canonical inputs > heat: old fallthrough at deposit stage (recency decayed) 1`] = `
{
"eoiCount": 15,
"furthestStage": 40,
"interestCount": 15,
"recency": 21.94,
"total": 91.94,
}
`;
exports[`W7 snapshots — heat at canonical inputs > heat: recent fallthrough at deposit stage (deepest hurt) 1`] = `
{
"eoiCount": 15,
"furthestStage": 40,
"interestCount": 15,
"recency": 30,
"total": 100,
}
`;
exports[`W7 snapshots — heat at canonical inputs > heat: recent fallthrough at enquiry stage 1`] = `
{
"eoiCount": 0,
"furthestStage": 0,
"interestCount": 3,
"recency": 30,
"total": 33,
}
`;
exports[`W7 snapshots — heat at canonical inputs > heat: recent fallthrough at eoi stage 1`] = `
{
"eoiCount": 5,
"furthestStage": 20,
"interestCount": 6,
"recency": 30,
"total": 61,
}
`;
exports[`W7 snapshots — heat at canonical inputs > heat: typical mid-funnel hot lead 1`] = `
{
"eoiCount": 10,
"furthestStage": 30,
"interestCount": 9,
"recency": 30,
"total": 79,
}
`;
exports[`W7 snapshots — tier-ladder boundaries > tier(active=0, lost=0, stage=0) is stable 1`] = `
{
"in": {
"activeInterestCount": 0,
"lostCount": 0,
"maxActiveStage": 0,
},
"out": "A",
}
`;
exports[`W7 snapshots — tier-ladder boundaries > tier(active=0, lost=1, stage=0) is stable 1`] = `
{
"in": {
"activeInterestCount": 0,
"lostCount": 1,
"maxActiveStage": 0,
},
"out": "B",
}
`;
exports[`W7 snapshots — tier-ladder boundaries > tier(active=0, lost=5, stage=0) is stable 1`] = `
{
"in": {
"activeInterestCount": 0,
"lostCount": 5,
"maxActiveStage": 0,
},
"out": "B",
}
`;
exports[`W7 snapshots — tier-ladder boundaries > tier(active=1, lost=0, stage=1) is stable 1`] = `
{
"in": {
"activeInterestCount": 1,
"lostCount": 0,
"maxActiveStage": 1,
},
"out": "C",
}
`;
exports[`W7 snapshots — tier-ladder boundaries > tier(active=1, lost=0, stage=3) is stable 1`] = `
{
"in": {
"activeInterestCount": 1,
"lostCount": 0,
"maxActiveStage": 3,
},
"out": "C",
}
`;
exports[`W7 snapshots — tier-ladder boundaries > tier(active=1, lost=0, stage=4) is stable 1`] = `
{
"in": {
"activeInterestCount": 1,
"lostCount": 0,
"maxActiveStage": 4,
},
"out": "C",
}
`;
exports[`W7 snapshots — tier-ladder boundaries > tier(active=1, lost=0, stage=5) is stable 1`] = `
{
"in": {
"activeInterestCount": 1,
"lostCount": 0,
"maxActiveStage": 5,
},
"out": "D",
}
`;
exports[`W7 snapshots — tier-ladder boundaries > tier(active=1, lost=0, stage=6) is stable 1`] = `
{
"in": {
"activeInterestCount": 1,
"lostCount": 0,
"maxActiveStage": 6,
},
"out": "D",
}
`;
exports[`W7 snapshots — tier-ladder boundaries > tier(active=1, lost=5, stage=6) is stable 1`] = `
{
"in": {
"activeInterestCount": 1,
"lostCount": 5,
"maxActiveStage": 6,
},
"out": "D",
}
`;
exports[`W7 snapshots — tier-ladder boundaries > tier(active=2, lost=0, stage=5) is stable 1`] = `
{
"in": {
"activeInterestCount": 2,
"lostCount": 0,
"maxActiveStage": 5,
},
"out": "D",
}
`;
exports[`W7 snapshots — tier-ladder boundaries > tier(active=3, lost=2, stage=4) is stable 1`] = `
{
"in": {
"activeInterestCount": 3,
"lostCount": 2,
"maxActiveStage": 4,
},
"out": "C",
}
`;

62
tests/unit/services/berth-recommender.test.ts
View File

@@ -187,3 +187,65 @@ describe('computeHeat', () => {
expect(h.total).toBe(0);
});
});
// ─── W7 snapshot lockfile — locks current tier-ladder boundaries and heat
// ordering so weight-tuning changes can't silently shift outputs. The
// existing toBe / toBeCloseTo tests above cover correctness; these
// inline snapshots are the regression-catching tripwires.
describe('W7 snapshots — tier-ladder boundaries', () => {
it.each([
[0, 0, 0],
[0, 1, 0],
[0, 5, 0],
[1, 0, 1],
[1, 0, 3],
[1, 0, 4],
[1, 0, 5],
[1, 0, 6],
[1, 5, 6],
[2, 0, 5],
[3, 2, 4],
])(
'tier(active=%i, lost=%i, stage=%i) is stable',
(activeInterestCount, lostCount, maxActiveStage) => {
expect({
in: { activeInterestCount, lostCount, maxActiveStage },
out: classifyTier({ activeInterestCount, lostCount, maxActiveStage }),
}).toMatchSnapshot();
},
);
});
describe('W7 snapshots — heat at canonical inputs', () => {
const NOW = new Date('2026-05-05T00:00:00Z');
const w = DEFAULT_RECOMMENDER_SETTINGS;
it.each([
// [label, fallthroughDaysAgo|null, totalInterestCount, eoiSignedCount, fallthroughMaxStage]
['cold (no history)', null, 0, 0, 0],
['recent fallthrough at enquiry stage', 5, 1, 0, 1],
['recent fallthrough at eoi stage', 5, 2, 1, 3],
['recent fallthrough at deposit stage (deepest hurt)', 5, 5, 3, 5],
['old fallthrough at deposit stage (recency decayed)', 120, 5, 3, 5],
['no fallthrough but many interests', null, 8, 4, 0],
['typical mid-funnel hot lead', 14, 3, 2, 4],
])('heat: %s', (_label, daysAgo, totalInterestCount, eoiSignedCount, fallthroughMaxStage) => {
const latestFallthroughAt =
daysAgo === null ? null : new Date(NOW.getTime() - daysAgo * 86400 * 1000);
const h = computeHeat(
{ latestFallthroughAt, totalInterestCount, eoiSignedCount, fallthroughMaxStage },
w,
NOW,
);
// Snapshot the rounded breakdown — exact float math (toBeCloseTo)
// is covered above; this locks the relative ordering + magnitude.
expect({
total: Math.round(h.total * 1000) / 1000,
recency: Math.round(h.recency * 1000) / 1000,
furthestStage: Math.round(h.furthestStage * 1000) / 1000,
interestCount: Math.round(h.interestCount * 1000) / 1000,
eoiCount: Math.round(h.eoiCount * 1000) / 1000,
}).toMatchSnapshot();
});
});

186
tests/unit/services/report-math.test.ts Normal file
View File

@@ -0,0 +1,186 @@
import { describe, expect, it } from 'vitest';
import {
rollupStageRevenue,
rollupStageCounts,
computeTotalForecast,
computeOccupancyRate,
rollupBerthStatusCounts,
} from '@/lib/services/report-math';
// Canonical 7-stage pipeline (see PIPELINE_STAGES in src/lib/constants.ts):
// enquiry, qualified, nurturing, eoi, reservation, deposit_paid, contract.
// Non-canonical input stage strings go through canonicalizeStage which
// maps legacy 9-stage values back into the canonical bucket.
describe('rollupStageRevenue', () => {
it('sums revenue per canonicalized stage', () => {
const out = rollupStageRevenue([
{ stage: 'enquiry', revenue: '100.50' },
{ stage: 'eoi', revenue: '200.00' },
{ stage: 'enquiry', revenue: '50.25' },
]);
expect(out['enquiry']).toBe('150.75');
expect(out['eoi']).toBe('200');
});
it('canonicalizes legacy 9-stage keys into modern buckets without dropping rows', () => {
// 'deposit_10pct' → 'deposit_paid', 'contract_sent' → 'contract'.
const out = rollupStageRevenue([
{ stage: 'deposit_10pct', revenue: '1000' },
{ stage: 'contract_sent', revenue: '2000' },
]);
expect(out['deposit_paid']).toBe('1000');
expect(out['contract']).toBe('2000');
const total = Object.values(out).reduce((a, v) => a + parseFloat(v), 0);
expect(total).toBe(3000);
});
it("'open' legacy alias canonicalizes to 'enquiry'", () => {
const out = rollupStageRevenue([
{ stage: 'open', revenue: '500' },
{ stage: 'enquiry', revenue: '100' },
]);
// Both rows land in 'enquiry' since 'open' is the legacy alias.
expect(out['enquiry']).toBe('600');
expect(out['open']).toBeUndefined();
});
it('treats null revenue as 0', () => {
const out = rollupStageRevenue([
{ stage: 'enquiry', revenue: null },
{ stage: 'enquiry', revenue: '500' },
]);
expect(out['enquiry']).toBe('500');
});
});
describe('rollupStageCounts', () => {
it('sums counts per stage with canonicalization', () => {
const out = rollupStageCounts([
{ stage: 'enquiry', count: 5 },
{ stage: 'eoi', count: 3 },
{ stage: 'enquiry', count: 2 },
]);
expect(out['enquiry']).toBe(7);
expect(out['eoi']).toBe(3);
});
it('returns empty object for empty input', () => {
expect(rollupStageCounts([])).toEqual({});
});
});
describe('computeTotalForecast', () => {
it('applies stage weights and returns 2-decimal-fixed string', () => {
const forecast = computeTotalForecast(
[
{ stage: 'enquiry', revenue: '1000' },
{ stage: 'eoi', revenue: '2000' },
],
{ enquiry: 0.2, eoi: 0.5 },
);
// 1000*0.2 + 2000*0.5 = 200 + 1000 = 1200.00
expect(forecast).toBe('1200.00');
});
it('treats stages missing from weight map as 0 (no silent default)', () => {
const forecast = computeTotalForecast(
[
{ stage: 'enquiry', revenue: '1000' },
{ stage: 'unknown_stage', revenue: '2000' },
],
{ enquiry: 0.3 },
);
// 'unknown_stage' canonicalizes to 'enquiry' (fallback) — so it ALSO
// gets the enquiry weight. Verifies canonicalization stays consistent
// between rollup and forecast so the totals reconcile.
// 1000*0.3 + 2000*0.3 = 300 + 600 = 900
expect(forecast).toBe('900.00');
});
it('returns "0.00" for empty input', () => {
expect(computeTotalForecast([], {})).toBe('0.00');
});
it('skips rows with null revenue', () => {
const forecast = computeTotalForecast(
[
{ stage: 'enquiry', revenue: null },
{ stage: 'enquiry', revenue: '500' },
],
{ enquiry: 1.0 },
);
expect(forecast).toBe('500.00');
});
it('matches expected forecast snapshot for a representative pipeline', () => {
// Deterministic fixture across all 7 canonical stages. Locks the
// math against weight-tuning regressions.
const rows = [
{ stage: 'enquiry', revenue: '50000' }, // 50000 * 0.05 = 2500
{ stage: 'qualified', revenue: '40000' }, // 40000 * 0.10 = 4000
{ stage: 'nurturing', revenue: '30000' }, // 30000 * 0.15 = 4500
{ stage: 'eoi', revenue: '120000' }, // 120000 * 0.25 = 30000
{ stage: 'reservation', revenue: '80000' }, // 80000 * 0.50 = 40000
{ stage: 'deposit_paid', revenue: '60000' }, // 60000 * 0.75 = 45000
{ stage: 'contract', revenue: '40000' }, // 40000 * 0.90 = 36000
];
const weights = {
enquiry: 0.05,
qualified: 0.1,
nurturing: 0.15,
eoi: 0.25,
reservation: 0.5,
deposit_paid: 0.75,
contract: 0.9,
};
// 2500 + 4000 + 4500 + 30000 + 40000 + 45000 + 36000 = 162000
expect(computeTotalForecast(rows, weights)).toMatchInlineSnapshot('"162000.00"');
});
});
describe('computeOccupancyRate', () => {
it('counts only "sold" as occupied (under_offer is a hold, not occupied)', () => {
const result = computeOccupancyRate({
sold: 30,
under_offer: 10,
available: 60,
});
// 30 sold of 100 total = 30%
expect(result.occupancyRate).toBe(30);
expect(result.totalBerths).toBe(100);
});
it('rounds to 1 decimal place', () => {
const result = computeOccupancyRate({
sold: 1,
available: 2,
});
// 1/3 = 33.333... → 33.3
expect(result.occupancyRate).toBe(33.3);
});
it('returns 0/0 not NaN when there are no berths', () => {
const result = computeOccupancyRate({});
expect(result.occupancyRate).toBe(0);
expect(result.totalBerths).toBe(0);
});
it('returns 100 when every berth is sold', () => {
const result = computeOccupancyRate({ sold: 50 });
expect(result.occupancyRate).toBe(100);
expect(result.totalBerths).toBe(50);
});
});
describe('rollupBerthStatusCounts', () => {
it('maps per-status counts + computes total', () => {
const out = rollupBerthStatusCounts([
{ status: 'sold', count: 30 },
{ status: 'available', count: 60 },
{ status: 'under_offer', count: 10 },
]);
expect(out.statusCounts).toEqual({ sold: 30, available: 60, under_offer: 10 });
expect(out.totalBerths).toBe(100);
});
});