Audit cleanup completion plan, all tiers shipped: Tier 1 (security + data integrity) - A.7 RTBF true wipe: redact email_messages body/subject/addresses for threads owned by deleted client; redact document_sends.recipient_email; collect file storage keys + delete blobs post-commit. - A.8 user_permission_overrides FK: documented inline why cascade is correct (not set-null as audit suggested) — overrides have no value without their user. - W2.14 PII redaction: camelCase normalization in audit.ts + error-events.service.ts isSensitiveKey; added city/postal/country/ birth fragments. firstName/lastName/dateOfBirth/postalCode etc. now caught in BOTH masker paths. 12 new test cases lock the coverage. Tier 2 (Documenso completion + refactor) - C.2: documentEvents.recipient_email column + partial unique index for per-recipient webhook dedup (migration 0075). handleDocumentSigned now sets recipient_email on insert. - Phase 2: completion_cc_emails distribution. handleDocumentCompleted reads documents.completionCcEmails, filters out signer-duplicates case-insensitively, fans signed PDF out to non-signer recipients. - C.4: extracted createPublicInterest() service from the 346-line api/public/interests route. Route becomes a thin shell (rate-limit, port resolution, audit log, email fan-out). The trio creation logic is now unit-testable without an HTTP fixture. - Phase 4: POST /api/v1/document-templates/[id]/detect-fields wired to document-field-detector.detectFields(). Sparkles "Auto-detect" button added to template-editor.tsx — maps DetectedField → marker with best-guess merge token (DATE / NAME / EMAIL); user retags. Tier 3 (reporting + recommender snapshot lockfiles) - W7.reports: extracted rollupStageRevenue / rollupStageCounts / computeTotalForecast / computeOccupancyRate / rollupBerthStatusCounts into src/lib/services/report-math.ts (pure functions). 16 new tests including an inline-snapshot lockfile on a representative 7-stage forecast. report-generators.ts now delegates. - W7.recommender: 18 new toMatchSnapshot tripwires on classifyTier boundaries + computeHeat at canonical input points. Tier 4 (rolling) - W6.attach: fixed outdated CLAUDE.md claim — threshold banner is informational and never depended on IMAP; bounce monitoring (the IMAP poller) is separate. - D.1 + D.2: documented deferral inline with full why-not-build-it reasoning so a future engineer sees the rationale. - G.1: representative formatDate sweep (audit-log-list, user-list, document-templates merge tokens, document-signing email). Rest of the ~100 sites stay rolling. Quality gates: 1420/1420 vitest (46 new tests above baseline of 1374), tsc clean, 0 lint errors. Plan: docs/superpowers/plans/2026-05-18-audit-cleanup-completion.md Migration: 0075_c2_document_events_recipient_email.sql (applied to dev DB). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
121 lines
5.6 KiB
TypeScript
121 lines
5.6 KiB
TypeScript
import { getQueue, type QueueName } from './index';
|
|
import { logger } from '@/lib/logger';
|
|
|
|
interface RecurringJobDef {
|
|
queue: QueueName;
|
|
name: string;
|
|
pattern: string;
|
|
}
|
|
|
|
/**
|
|
* Register all recurring jobs from 11-REALTIME-AND-BACKGROUND-JOBS.md Section 3.2.
|
|
* Called once on server startup.
|
|
*/
|
|
export async function registerRecurringJobs(): Promise<void> {
|
|
const recurring: RecurringJobDef[] = [
|
|
// Documenso signature fallback poll - primary is webhooks, this is the
|
|
// safety net for any missed delivery (cloudflared tunnel hiccup, transient
|
|
// 5xx on our receiver, Documenso quirk). Tightened from 6h to 5m so the
|
|
// user-facing "stuck on partially_signed" symptom only persists for the
|
|
// 5-min window between polls. Cheap query: ~1 GET per in-flight doc.
|
|
{ queue: 'documents', name: 'signature-poll', pattern: '*/5 * * * *' },
|
|
|
|
// Reminder checks
|
|
{ queue: 'notifications', name: 'reminder-check', pattern: '0 * * * *' },
|
|
{ queue: 'notifications', name: 'reminder-overdue-check', pattern: '*/15 * * * *' },
|
|
|
|
// Google Calendar background sync
|
|
{ queue: 'maintenance', name: 'calendar-sync', pattern: '*/30 * * * *' },
|
|
|
|
// Daily checks at 08:00
|
|
{ queue: 'notifications', name: 'invoice-overdue-check', pattern: '0 8 * * *' },
|
|
{ queue: 'notifications', name: 'tenure-expiry-check', pattern: '0 8 * * *' },
|
|
|
|
// Exchange rate refresh every 6 hours
|
|
{ queue: 'maintenance', name: 'currency-refresh', pattern: '0 */6 * * *' },
|
|
|
|
// Database backup / cleanup
|
|
{ queue: 'maintenance', name: 'database-backup', pattern: '0 2 * * *' },
|
|
{ queue: 'maintenance', name: 'backup-cleanup', pattern: '0 3 * * 0' }, // Sunday 03:00
|
|
|
|
// Session cleanup
|
|
{ queue: 'maintenance', name: 'session-cleanup', pattern: '0 4 * * *' },
|
|
|
|
// Report scheduler - checks every minute for reports due to run
|
|
{ queue: 'reports', name: 'report-scheduler', pattern: '* * * * *' },
|
|
|
|
// Notification digest — fires hourly globally; the worker checks each
|
|
// user's `notification_digest_paused_until` and unread-count threshold
|
|
// before composing a digest, so most ticks are no-ops. Per-user time-
|
|
// of-day scheduling is DEFERRED — implementing it requires a product
|
|
// decision on UX (slider? time picker? per-channel toggles?) and adds
|
|
// a per-user cron path that doesn't pay off until enough users are
|
|
// actively customizing it. The hourly bucket aligns with how reps
|
|
// already check inboxes ("on the hour") so the current behavior is
|
|
// operationally acceptable without per-user override. Revisit when
|
|
// a customer asks for digest-time control.
|
|
{ queue: 'email', name: 'notification-digest', pattern: '0 * * * *' },
|
|
|
|
// Cleanup jobs
|
|
{ queue: 'maintenance', name: 'temp-file-cleanup', pattern: '0 5 * * *' },
|
|
{ queue: 'maintenance', name: 'form-expiry-check', pattern: '0 * * * *' },
|
|
|
|
// Phase B: alert rule engine sweep
|
|
{ queue: 'maintenance', name: 'alerts-evaluate', pattern: '*/5 * * * *' },
|
|
// Phase 6: IMAP bounce poller — matches NDRs to document_sends rows
|
|
// and fires email_bounced notifications. No-op when IMAP_* env unset.
|
|
{ queue: 'maintenance', name: 'bounce-poll', pattern: '*/15 * * * *' },
|
|
// Phase B: analytics snapshot warm
|
|
{ queue: 'maintenance', name: 'analytics-refresh', pattern: '*/15 * * * *' },
|
|
|
|
// Phase 3d: GDPR Article 17 - actually delete expired export bundles
|
|
{ queue: 'maintenance', name: 'gdpr-export-cleanup', pattern: '0 4 * * *' },
|
|
// Phase 3b: AI usage ledger retention (90-day rolling window)
|
|
{ queue: 'maintenance', name: 'ai-usage-retention', pattern: '0 5 * * *' },
|
|
// Migration 0040 contract: error_events older than 90 days get pruned.
|
|
{ queue: 'maintenance', name: 'error-events-retention', pattern: '0 6 * * *' },
|
|
// 90-day retention for audit_logs — mirrors error_events. Metadata
|
|
// is masked at insert time but old rows still represent stale PII
|
|
// exposure that has no operational value past the window.
|
|
{ queue: 'maintenance', name: 'audit-logs-retention', pattern: '15 6 * * *' },
|
|
// Raw website inquiry payloads — 180-day retention.
|
|
{ queue: 'maintenance', name: 'website-submissions-retention', pattern: '0 7 * * *' },
|
|
];
|
|
|
|
// BullMQ defaults `tz` to UTC. The cron patterns above are spelled
|
|
// in port-local time (e.g. "0 8 * * *" = 8 AM local), so without an
|
|
// explicit `tz` the jobs fire in UTC and silently drift across DST
|
|
// — twice a year the local firing time shifts by an hour and admin
|
|
// docs ("daily check at 8 AM") break. datetime-auditor C2.
|
|
//
|
|
// The CRM is single-port today (Port Nimara, Europe/Warsaw); when
|
|
// multi-port admin schedules ship, this should resolve per-job from
|
|
// the owning port's `ports.timezone`. SCHEDULER_TZ overrides for
|
|
// ops debugging without a redeploy.
|
|
const schedulerTz = process.env.SCHEDULER_TZ ?? 'Europe/Warsaw';
|
|
|
|
for (const job of recurring) {
|
|
const queue = getQueue(job.queue);
|
|
// Sub-hourly / hourly patterns are TZ-invariant; skip the `tz`
|
|
// option for them so a misconfigured SCHEDULER_TZ can't perturb
|
|
// them.
|
|
const tzInvariant = /^\*|\*\/\d+ \*|^0 \*/.test(job.pattern);
|
|
await queue.upsertJobScheduler(
|
|
job.name,
|
|
{ pattern: job.pattern, ...(tzInvariant ? {} : { tz: schedulerTz }) },
|
|
{ data: {}, name: job.name },
|
|
);
|
|
logger.info(
|
|
{
|
|
queue: job.queue,
|
|
job: job.name,
|
|
pattern: job.pattern,
|
|
tz: tzInvariant ? 'UTC (invariant)' : schedulerTz,
|
|
},
|
|
'Registered recurring job',
|
|
);
|
|
}
|
|
|
|
logger.info({ count: recurring.length }, 'All recurring jobs registered');
|
|
}
|