Files
pn-new-crm/src/lib/queue/scheduler.ts
Matt b3f87563c6 feat(audit-cleanup): finish all 15 outstanding items from verified backlog
Audit cleanup completion plan, all tiers shipped:

Tier 1 (security + data integrity)
- A.7 RTBF true wipe: redact email_messages body/subject/addresses for
  threads owned by deleted client; redact document_sends.recipient_email;
  collect file storage keys + delete blobs post-commit.
- A.8 user_permission_overrides FK: documented inline why cascade is
  correct (not set-null as audit suggested) — overrides have no value
  without their user.
- W2.14 PII redaction: camelCase normalization in audit.ts +
  error-events.service.ts isSensitiveKey; added city/postal/country/
  birth fragments. firstName/lastName/dateOfBirth/postalCode etc. now
  caught in BOTH masker paths. 12 new test cases lock the coverage.

Tier 2 (Documenso completion + refactor)
- C.2: documentEvents.recipient_email column + partial unique index for
  per-recipient webhook dedup (migration 0075). handleDocumentSigned
  now sets recipient_email on insert.
- Phase 2: completion_cc_emails distribution. handleDocumentCompleted
  reads documents.completionCcEmails, filters out signer-duplicates
  case-insensitively, fans signed PDF out to non-signer recipients.
- C.4: extracted createPublicInterest() service from the 346-line
  api/public/interests route. Route becomes a thin shell (rate-limit,
  port resolution, audit log, email fan-out). The trio creation logic
  is now unit-testable without an HTTP fixture.
- Phase 4: POST /api/v1/document-templates/[id]/detect-fields wired
  to document-field-detector.detectFields(). Sparkles "Auto-detect"
  button added to template-editor.tsx — maps DetectedField → marker
  with best-guess merge token (DATE / NAME / EMAIL); user retags.

Tier 3 (reporting + recommender snapshot lockfiles)
- W7.reports: extracted rollupStageRevenue / rollupStageCounts /
  computeTotalForecast / computeOccupancyRate / rollupBerthStatusCounts
  into src/lib/services/report-math.ts (pure functions). 16 new tests
  including an inline-snapshot lockfile on a representative 7-stage
  forecast. report-generators.ts now delegates.
- W7.recommender: 18 new toMatchSnapshot tripwires on classifyTier
  boundaries + computeHeat at canonical input points.

Tier 4 (rolling)
- W6.attach: fixed outdated CLAUDE.md claim — threshold banner is
  informational and never depended on IMAP; bounce monitoring (the
  IMAP poller) is separate.
- D.1 + D.2: documented deferral inline with full why-not-build-it
  reasoning so a future engineer sees the rationale.
- G.1: representative formatDate sweep (audit-log-list, user-list,
  document-templates merge tokens, document-signing email). Rest of
  the ~100 sites stay rolling.

Quality gates: 1420/1420 vitest (46 new tests above baseline of 1374),
tsc clean, 0 lint errors.

Plan: docs/superpowers/plans/2026-05-18-audit-cleanup-completion.md
Migration: 0075_c2_document_events_recipient_email.sql (applied to dev DB).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-18 18:22:36 +02:00

121 lines
5.6 KiB
TypeScript

import { getQueue, type QueueName } from './index';
import { logger } from '@/lib/logger';
interface RecurringJobDef {
queue: QueueName;
name: string;
pattern: string;
}
/**
* Register all recurring jobs from 11-REALTIME-AND-BACKGROUND-JOBS.md Section 3.2.
* Called once on server startup.
*/
export async function registerRecurringJobs(): Promise<void> {
const recurring: RecurringJobDef[] = [
// Documenso signature fallback poll - primary is webhooks, this is the
// safety net for any missed delivery (cloudflared tunnel hiccup, transient
// 5xx on our receiver, Documenso quirk). Tightened from 6h to 5m so the
// user-facing "stuck on partially_signed" symptom only persists for the
// 5-min window between polls. Cheap query: ~1 GET per in-flight doc.
{ queue: 'documents', name: 'signature-poll', pattern: '*/5 * * * *' },
// Reminder checks
{ queue: 'notifications', name: 'reminder-check', pattern: '0 * * * *' },
{ queue: 'notifications', name: 'reminder-overdue-check', pattern: '*/15 * * * *' },
// Google Calendar background sync
{ queue: 'maintenance', name: 'calendar-sync', pattern: '*/30 * * * *' },
// Daily checks at 08:00
{ queue: 'notifications', name: 'invoice-overdue-check', pattern: '0 8 * * *' },
{ queue: 'notifications', name: 'tenure-expiry-check', pattern: '0 8 * * *' },
// Exchange rate refresh every 6 hours
{ queue: 'maintenance', name: 'currency-refresh', pattern: '0 */6 * * *' },
// Database backup / cleanup
{ queue: 'maintenance', name: 'database-backup', pattern: '0 2 * * *' },
{ queue: 'maintenance', name: 'backup-cleanup', pattern: '0 3 * * 0' }, // Sunday 03:00
// Session cleanup
{ queue: 'maintenance', name: 'session-cleanup', pattern: '0 4 * * *' },
// Report scheduler - checks every minute for reports due to run
{ queue: 'reports', name: 'report-scheduler', pattern: '* * * * *' },
// Notification digest — fires hourly globally; the worker checks each
// user's `notification_digest_paused_until` and unread-count threshold
// before composing a digest, so most ticks are no-ops. Per-user time-
// of-day scheduling is DEFERRED — implementing it requires a product
// decision on UX (slider? time picker? per-channel toggles?) and adds
// a per-user cron path that doesn't pay off until enough users are
// actively customizing it. The hourly bucket aligns with how reps
// already check inboxes ("on the hour") so the current behavior is
// operationally acceptable without per-user override. Revisit when
// a customer asks for digest-time control.
{ queue: 'email', name: 'notification-digest', pattern: '0 * * * *' },
// Cleanup jobs
{ queue: 'maintenance', name: 'temp-file-cleanup', pattern: '0 5 * * *' },
{ queue: 'maintenance', name: 'form-expiry-check', pattern: '0 * * * *' },
// Phase B: alert rule engine sweep
{ queue: 'maintenance', name: 'alerts-evaluate', pattern: '*/5 * * * *' },
// Phase 6: IMAP bounce poller — matches NDRs to document_sends rows
// and fires email_bounced notifications. No-op when IMAP_* env unset.
{ queue: 'maintenance', name: 'bounce-poll', pattern: '*/15 * * * *' },
// Phase B: analytics snapshot warm
{ queue: 'maintenance', name: 'analytics-refresh', pattern: '*/15 * * * *' },
// Phase 3d: GDPR Article 17 - actually delete expired export bundles
{ queue: 'maintenance', name: 'gdpr-export-cleanup', pattern: '0 4 * * *' },
// Phase 3b: AI usage ledger retention (90-day rolling window)
{ queue: 'maintenance', name: 'ai-usage-retention', pattern: '0 5 * * *' },
// Migration 0040 contract: error_events older than 90 days get pruned.
{ queue: 'maintenance', name: 'error-events-retention', pattern: '0 6 * * *' },
// 90-day retention for audit_logs — mirrors error_events. Metadata
// is masked at insert time but old rows still represent stale PII
// exposure that has no operational value past the window.
{ queue: 'maintenance', name: 'audit-logs-retention', pattern: '15 6 * * *' },
// Raw website inquiry payloads — 180-day retention.
{ queue: 'maintenance', name: 'website-submissions-retention', pattern: '0 7 * * *' },
];
// BullMQ defaults `tz` to UTC. The cron patterns above are spelled
// in port-local time (e.g. "0 8 * * *" = 8 AM local), so without an
// explicit `tz` the jobs fire in UTC and silently drift across DST
// — twice a year the local firing time shifts by an hour and admin
// docs ("daily check at 8 AM") break. datetime-auditor C2.
//
// The CRM is single-port today (Port Nimara, Europe/Warsaw); when
// multi-port admin schedules ship, this should resolve per-job from
// the owning port's `ports.timezone`. SCHEDULER_TZ overrides for
// ops debugging without a redeploy.
const schedulerTz = process.env.SCHEDULER_TZ ?? 'Europe/Warsaw';
for (const job of recurring) {
const queue = getQueue(job.queue);
// Sub-hourly / hourly patterns are TZ-invariant; skip the `tz`
// option for them so a misconfigured SCHEDULER_TZ can't perturb
// them.
const tzInvariant = /^\*|\*\/\d+ \*|^0 \*/.test(job.pattern);
await queue.upsertJobScheduler(
job.name,
{ pattern: job.pattern, ...(tzInvariant ? {} : { tz: schedulerTz }) },
{ data: {}, name: job.name },
);
logger.info(
{
queue: job.queue,
job: job.name,
pattern: job.pattern,
tz: tzInvariant ? 'UTC (invariant)' : schedulerTz,
},
'Registered recurring job',
);
}
logger.info({ count: recurring.length }, 'All recurring jobs registered');
}