perf(audit-tier-3): bulk-fetch the five hot N+1 loops

Replaces per-row fan-out with grouped queries / inArray pre-fetches
across the five dashboard + cron hotspots flagged in the audit
(MED §13 / HIGH §11–14):

* reminders.processFollowUpReminders — was 3 round trips per
  enabled-and-due interest.  Now: filter in JS, single clients
  bulk-fetch, single reminders bulk-insert, single interests
  bulk-update, one summary socket emit.  1k due interests: 6 round
  trips total instead of 3000+.
* portal.getClientInvoices — was a full-table scan filtered in JS.
  Now an inArray push-down on lower(billingEmail) + defensive
  limit(100).  After 12mo this would have been the worst portal
  endpoint.
* interest-scoring.calculateBulkScores — was 6N round trips
  (1 redis + 1 findFirst + 4 counts per interest).  Now 4 grouped
  count queries on the port's interest set + a single redis pipeline
  to refresh the cache.  1k interests: ~7 round trips.
* document-reminders.processReminderQueue — was 5N round trips per
  cron tick (port + template + lastReminder + pendingSigners + send
  per doc).  Now hoists port + per-type template map + grouped
  lastReminder + bulk pendingSigners; per-row work collapses to a
  Map.get and the documenso send.  500 docs: ~7 round trips.
* inquiry-notifications.sendInquiryNotifications — was sequential
  createNotification + emailQueue.add per recipient inside a public
  POST.  Now Promise.all'd; a 20-user port stops blocking the public
  inquiry POST on ~80 round trips.

Test status: 1168/1168 vitest, tsc clean.

Refs: docs/audit-comprehensive-2026-05-05.md HIGH §§11–14 (auditor-I
Issues 1–4) + MED §13 (auditor-I Issue 5).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
Matt Ciaccio
2026-05-05 20:41:23 +02:00
parent d3a6a9beef
commit 7854cbabe4
5 changed files with 351 additions and 64 deletions

View File

@@ -185,10 +185,26 @@ export async function sendReminderIfAllowed(
* (override or template) is set, then attempts auto-fire on each.
* `interests.reminderEnabled` is no longer part of the gating - per-doc
* `remindersDisabled` is the kill switch instead.
*
* Performance: the pre-bulk version called `sendReminderIfAllowed` per
* doc, which re-fetched the port row (invariant), the template-by-type
* map (repeats heavily), the last reminder event, and the pending
* signers — 5×N round trips per cron tick. This implementation hoists
* the invariants out of the loop and turns the per-row queries into
* grouped scans (one per dimension), so a port with 500 in-flight docs
* is now ~7 round trips total instead of ~2500.
*/
export async function processReminderQueue(portId: string): Promise<void> {
const activeDocs = await db
.select({ id: documents.id })
.select({
id: documents.id,
documentType: documents.documentType,
documensoId: documents.documensoId,
status: documents.status,
remindersDisabled: documents.remindersDisabled,
reminderCadenceOverride: documents.reminderCadenceOverride,
fileId: documents.fileId,
})
.from(documents)
.leftJoin(documentTemplates, eq(documentTemplates.templateType, documents.documentType))
.where(
@@ -201,9 +217,95 @@ export async function processReminderQueue(portId: string): Promise<void> {
),
);
if (activeDocs.length === 0) return;
// Hoist invariants out of the per-doc loop ────────────────────────────────
// (1) Port row (timezone) — invariant across the whole batch.
const port = await db.query.ports.findFirst({ where: eq(ports.id, portId) });
const timezone = port?.timezone ?? 'UTC';
const currentHour = getCurrentHourInTimezone(timezone);
if (currentHour < 9 || currentHour >= 16) {
// Outside the 9-16 window — nothing to do this tick.
return;
}
// (2) Per-type template cadence map — repeats per documentType.
const distinctTypes = Array.from(new Set(activeDocs.map((d) => d.documentType)));
const templateRows = await db
.select({
templateType: documentTemplates.templateType,
reminderCadenceDays: documentTemplates.reminderCadenceDays,
})
.from(documentTemplates)
.where(
and(
eq(documentTemplates.portId, portId),
inArray(documentTemplates.templateType, distinctTypes),
),
);
const templateCadenceByType = new Map(
templateRows.map((r) => [r.templateType, r.reminderCadenceDays ?? null]),
);
// (3) Latest reminder_sent event per doc — one grouped query.
const docIds = activeDocs.map((d) => d.id);
const lastReminderRows = await db
.select({
documentId: documentEvents.documentId,
lastAt: sql<Date>`max(${documentEvents.createdAt})`,
})
.from(documentEvents)
.where(
and(
inArray(documentEvents.documentId, docIds),
eq(documentEvents.eventType, 'reminder_sent'),
),
)
.groupBy(documentEvents.documentId);
const lastReminderByDoc = new Map(lastReminderRows.map((r) => [r.documentId, r.lastAt]));
// (4) Pending signers per doc — one inArray scan.
const pendingSignerRows = await db
.select()
.from(documentSigners)
.where(and(inArray(documentSigners.documentId, docIds), eq(documentSigners.status, 'pending')))
.orderBy(sql`${documentSigners.signingOrder} ASC`);
const pendingByDoc = new Map<string, typeof pendingSignerRows>();
for (const row of pendingSignerRows) {
const arr = pendingByDoc.get(row.documentId) ?? [];
arr.push(row);
pendingByDoc.set(row.documentId, arr);
}
// Per-doc fire — at this point every per-row query is a Map.get.
for (const doc of activeDocs) {
try {
await sendReminderIfAllowed(doc.id, portId, { auto: true });
const due = isReminderDue({
status: doc.status,
documensoId: doc.documensoId,
remindersDisabled: doc.remindersDisabled,
reminderCadenceOverride: doc.reminderCadenceOverride,
templateCadenceDays: templateCadenceByType.get(doc.documentType) ?? null,
lastReminderAt: lastReminderByDoc.get(doc.id) ?? null,
});
if (!due) continue;
const pending = pendingByDoc.get(doc.id) ?? [];
const target = pending[0];
if (!target || !doc.documensoId) continue;
await documensoRemind(doc.documensoId, target.id, portId);
await db.insert(documentEvents).values({
documentId: doc.id,
eventType: 'reminder_sent',
signerId: target.id,
eventData: {
signerEmail: target.signerEmail,
signerRole: target.signerRole,
auto: true,
},
});
} catch (err) {
logger.error({ err, documentId: doc.id, portId }, 'Reminder processing failed');
}