Files
pn-new-crm/src/lib/storage/index.ts
Matt ebdd8408bf fix(audit-wave-11): dossier sweep — error-ux + webhook + storage + search + maintainability
Final pass over the unaddressed AUDIT-2026-05-12 dossiers, taking the
tractable Critical/High items from each:

error-ux-auditor (5 items)
- C2: 17 toast.error(err.message) sites swept to toastError(err, …) so
  every user-visible failure carries a copy-paste Reference ID
- C3: apiFetch synthesizes a client-side correlation id when a 5xx
  comes back with a non-JSON body (reverse-proxy HTML pages); message
  becomes "The server is unreachable. Please try again." with code
  UPSTREAM_UNREACHABLE
- C4: checkRateLimit fails OPEN when Redis is unavailable so an outage
  no longer 500s login + portal sign-in; logged at warn so monitoring
  catches it
- H2: StorageTimeoutError (name='TimeoutError') replaces the plain
  Error throw in s3.ts withTimeout — error-classifier hints fire now
- H5: errorResponse() adopted across /api/storage/[token],
  /api/public/website-inquiries, and the Documenso webhook body (drops
  the "Invalid secret" reconnaissance string)

outbound-webhook-auditor (5 items)
- C1: signature is now HMAC(secret, `${ts}.${body}`) with the
  timestamp surfaced as X-Webhook-Timestamp so receivers can reject
  replays outside a freshness window
- C3: dead-letter with reason missing_signing_secret when secret is
  null (defence-in-depth against DB tampering / future migration
  mistakes)
- H2: webhooks queue bumped to maxAttempts=8 with 30 s base
  exponential backoff so a 30 s receiver blip during a deploy no
  longer dead-letters every in-flight event; per-queue
  backoffDelayMs added to QUEUE_CONFIGS
- M1: SSRF denylist gains Oracle Cloud metadata 192.0.0.192
- M2: dispatch-time https:// assertion before fetch, so a bad DB edit
  can't slip plaintext through

storage-pathing-auditor (2 items)
- H1: berth-PDF presigned-upload keys now `${portSlug}/berths/…/…`
  with portSlug threaded into backend.presignUpload — engages the
  filesystem-proxy port-binding `p` token verifier
- H2: presignDownloadUrl auto-derives portSlug from the key's first
  segment when callers don't pass it, so all 8 download sites engage
  the `p`-token guard without per-site plumbing

search-auditor (1 item)
- H3: removed dead void wantEmail; void wantPhone; pair plus the
  unused looksLikeEmail helper — the bucket-reorder it was scaffolded
  for was never wired

maintainability-auditor (1 item)
- M2: swept seven abandoned `void <symbol>` markers and their dead
  imports across clients/bulk, interests/bulk, admin/email-templates,
  admin/website-submissions, alert-rules, and notes.service

Deferred to future work (substantial refactors, schema migrations, or
multi-file UI work):
- error-ux M3-M8 (global-error.tsx, per-route loading.tsx coverage,
  ErrorBanner component, /api/ready route, worker DLQ admin surface)
- maintainability C1-C4 (documents/search/notes service splits,
  interest-tabs split — multi-hour refactors)
- currency C1-H5 (mixed-currency dashboard aggregation, FX history
  table, rounding policy) — wait for second non-USD port
- outbound-webhook C2 (deliveries reaper job), H1 (DNS-rebind TOCTOU
  with undici Agent), H3 (circuit-breaker), H5 (presigned-post-policy)
- storage-pathing C2 (orphan reaper), H3-H5 (streaming + content-type
  binding)

Tests: 1315/1315 vitest  ; tsc clean.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-13 13:27:32 +02:00

282 lines
10 KiB
TypeScript

/**
* Pluggable storage backend (Phase 6a — see docs/berth-recommender-and-pdf-plan.md §4.7a).
*
* The CRM stores files (per-berth PDFs, brochures, GDPR exports, etc.) through a
* single `StorageBackend` abstraction. The deployment chooses between an
* S3-compatible store (MinIO / AWS S3 / Backblaze B2 / Cloudflare R2 / Wasabi /
* Tigris) and a local filesystem at runtime via `system_settings.storage_backend`.
*
* Callers should always import from this barrel — never from `s3.ts` or
* `filesystem.ts` directly — so the factory wiring stays the single source of
* truth.
*/
import { and, eq, isNull } from 'drizzle-orm';
import { db } from '@/lib/db';
import { systemSettings } from '@/lib/db/schema/system';
import { logger } from '@/lib/logger';
import { FilesystemBackend } from './filesystem';
import { S3Backend } from './s3';
export type StorageBackendName = 's3' | 'filesystem';
export interface PutOpts {
contentType: string;
/** Optional pre-computed sha256 hex — if absent, the backend computes one. */
sha256?: string;
/** Bytes (for streams that don't expose .length); used for capacity pre-flight. */
sizeBytes?: number;
/** Optional content-disposition for downloads (filesystem proxy only). */
contentDisposition?: string;
}
export interface PresignOpts {
/** TTL in seconds. Default 900 (15min) per SECURITY-GUIDELINES §7.1. */
expirySeconds?: number;
contentType?: string;
/** Filename used in Content-Disposition for downloads. */
filename?: string;
/**
* Optional port slug to bind the token to. The filesystem proxy
* verifier asserts the storage key starts with `${portSlug}/` when
* present. S3 backend ignores this field (presigned S3 URLs carry
* their own signature scope). Pass it whenever the issuer is in a
* port-scoped request — `generateStorageKey()` already prefixes the
* slug, so this is the matching enforcement.
*/
portSlug?: string;
}
export interface StorageBackend {
/** Upload bytes. Returns the canonical key, size, and sha256 hex. */
put(
key: string,
body: Buffer | NodeJS.ReadableStream,
opts: PutOpts,
): Promise<{ key: string; sizeBytes: number; sha256: string }>;
/** Stream a file out. Throws NotFoundError if missing. */
get(key: string): Promise<NodeJS.ReadableStream>;
/** Existence + size check without reading the full body. Returns null when missing. */
head(key: string): Promise<{ sizeBytes: number; contentType: string } | null>;
/** Delete. Idempotent — missing keys must not throw. */
delete(key: string): Promise<void>;
/** Generate a short-lived URL the browser can PUT to. */
presignUpload(key: string, opts: PresignOpts): Promise<{ url: string; method: 'PUT' | 'POST' }>;
/** Generate a short-lived URL the browser can GET from. */
presignDownload(key: string, opts: PresignOpts): Promise<{ url: string; expiresAt: Date }>;
/**
* Recursively list keys under `prefix`. Returns the relative key for each
* object, sorted alphabetically. Empty prefix means "the entire bucket /
* storage root". Used by one-shot importers (e.g. organized-bucket
* document import) that need to walk a flat key namespace; not meant for
* runtime hot paths.
*/
listByPrefix(prefix: string): Promise<string[]>;
readonly name: StorageBackendName;
}
// ─── factory ────────────────────────────────────────────────────────────────
interface CachedFactory {
backend: StorageBackend;
/** Resolved at cache-time so we can re-fetch when settings change. */
configFingerprint: string;
}
let cached: CachedFactory | null = null;
/**
* Reset the per-process backend cache. Called after `system_settings` writes
* via the existing pub/sub invalidation hook, and exposed for tests.
*/
export function resetStorageBackendCache(): void {
cached = null;
}
interface StorageConfigSnapshot {
backend: StorageBackendName;
s3?: {
endpoint?: string;
region?: string;
bucket?: string;
accessKey?: string;
secretKeyEncrypted?: string;
forcePathStyle?: boolean;
};
filesystem?: {
root?: string;
proxyHmacSecretEncrypted?: string;
};
}
async function readGlobalSetting<T = unknown>(key: string): Promise<T | null> {
const [row] = await db
.select()
.from(systemSettings)
.where(and(eq(systemSettings.key, key), isNull(systemSettings.portId)));
return (row?.value as T | undefined) ?? null;
}
async function loadStorageConfig(): Promise<StorageConfigSnapshot> {
// Each setting key is a separate row. We read them in parallel.
const keys = [
'storage_backend',
'storage_s3_endpoint',
'storage_s3_region',
'storage_s3_bucket',
'storage_s3_access_key',
'storage_s3_secret_key_encrypted',
'storage_s3_force_path_style',
'storage_filesystem_root',
'storage_proxy_hmac_secret_encrypted',
] as const;
const [
backendRaw,
s3Endpoint,
s3Region,
s3Bucket,
s3AccessKey,
s3SecretKeyEncrypted,
s3ForcePathStyle,
fsRoot,
fsHmacSecretEncrypted,
] = await Promise.all(keys.map((k) => readGlobalSetting<unknown>(k)));
const backend: StorageBackendName = backendRaw === 'filesystem' ? 'filesystem' : 's3';
return {
backend,
s3: {
endpoint: typeof s3Endpoint === 'string' ? s3Endpoint : undefined,
region: typeof s3Region === 'string' ? s3Region : undefined,
bucket: typeof s3Bucket === 'string' ? s3Bucket : undefined,
accessKey: typeof s3AccessKey === 'string' ? s3AccessKey : undefined,
secretKeyEncrypted:
typeof s3SecretKeyEncrypted === 'string' ? s3SecretKeyEncrypted : undefined,
forcePathStyle:
typeof s3ForcePathStyle === 'boolean' ? s3ForcePathStyle : Boolean(s3ForcePathStyle),
},
filesystem: {
root: typeof fsRoot === 'string' ? fsRoot : undefined,
proxyHmacSecretEncrypted:
typeof fsHmacSecretEncrypted === 'string' ? fsHmacSecretEncrypted : undefined,
},
};
}
/**
* The fingerprint includes encrypted-secret material because rotating the
* secret should invalidate the cached client. After a key rotation the
* settings-write hook calls `resetStorageBackendCache()` explicitly, so
* this comparison is a defense-in-depth backstop rather than the primary
* invalidation path. If you ever change `loadStorageConfig` to read
* additional sensitive material, make sure the rotation flow keeps
* resetting the cache — relying on fingerprint diff alone means the old
* client is held in memory until the next mismatch.
*/
function fingerprint(cfg: StorageConfigSnapshot): string {
return JSON.stringify(cfg);
}
/**
* Resolve the active backend. Caches per-process; the cache is invalidated by
* `resetStorageBackendCache()` (called when `system_settings.storage_backend`
* changes via the migration flow).
*/
export async function getStorageBackend(): Promise<StorageBackend> {
const cfg = await loadStorageConfig();
const fp = fingerprint(cfg);
if (cached && cached.configFingerprint === fp) {
return cached.backend;
}
const backend = await buildBackend(cfg);
cached = { backend, configFingerprint: fp };
logger.info({ backend: backend.name }, 'Storage backend resolved');
return backend;
}
async function buildBackend(cfg: StorageConfigSnapshot): Promise<StorageBackend> {
if (cfg.backend === 'filesystem') {
return FilesystemBackend.create({
root: cfg.filesystem?.root ?? './storage',
proxyHmacSecretEncrypted: cfg.filesystem?.proxyHmacSecretEncrypted ?? null,
});
}
return S3Backend.create({
endpoint: cfg.s3?.endpoint,
region: cfg.s3?.region,
bucket: cfg.s3?.bucket,
accessKey: cfg.s3?.accessKey,
secretKeyEncrypted: cfg.s3?.secretKeyEncrypted,
forcePathStyle: cfg.s3?.forcePathStyle,
});
}
// ─── url helpers ────────────────────────────────────────────────────────────
/**
* Convenience wrapper that returns just the presigned download URL — the most
* common need at call sites that don't track expiry. Mirrors the legacy
* `getPresignedUrl(key)` helper in `@/lib/minio` but routes through the
* active backend so filesystem-mode deployments work too.
*
* storage-pathing-auditor H2: when `portSlug` is not passed explicitly,
* we attempt to infer it from the key's first path segment — every
* storage key minted via `buildStoragePath(slug, …)` starts with the
* slug, so the inference is correct for the overwhelming majority of
* callers. This engages the filesystem-proxy port-binding token (`p`)
* verifier so a stolen-token / cross-port replay attempt fails fast.
*/
export async function presignDownloadUrl(
key: string,
expirySeconds = 900,
filename?: string,
portSlug?: string,
): Promise<string> {
const backend = await getStorageBackend();
const inferredSlug = portSlug ?? inferPortSlugFromKey(key);
const { url } = await backend.presignDownload(key, {
expirySeconds,
filename,
portSlug: inferredSlug,
});
return url;
}
/**
* Best-effort recovery of the port slug from a storage key prefix.
* Returns undefined when the key doesn't look slug-prefixed (e.g. legacy
* keys minted before `buildStoragePath` was canonical) so the caller
* falls back to the no-binding path.
*
* A slug is conservatively defined as kebab/alphanumeric (the same
* shape `createPortSchema` enforces). Non-matching first segments
* include UUID-only keys like the legacy `berths/{id}/uploads/...`
* shape — those are still served but skip the binding gate.
*/
function inferPortSlugFromKey(key: string): string | undefined {
const slash = key.indexOf('/');
if (slash <= 0) return undefined;
const first = key.slice(0, slash);
if (!/^[a-z0-9-]+$/.test(first)) return undefined;
// Reserved namespaces that historically lived at the top level of the
// bucket and aren't port slugs.
if (first === 'berths' || first === 'backups' || first === 'tmp') return undefined;
return first;
}
// ─── re-exports ─────────────────────────────────────────────────────────────
export { S3Backend } from './s3';
export { FilesystemBackend, validateStorageKey } from './filesystem';