Files
pn-new-crm/tests/unit/storage/filesystem-backend.test.ts

255 lines
8.9 KiB
TypeScript
Raw Normal View History

feat(storage): pluggable s3-or-filesystem backend + migration CLI + admin UI Phase 6a from docs/berth-recommender-and-pdf-plan.md §4.7a + §14.9a. Lays the storage groundwork for Phase 6b/7 file-bearing schemas (per-berth PDFs, brochures) without touching those domains yet. New files: - src/lib/storage/index.ts StorageBackend interface + per-process factory keyed on system_settings. - src/lib/storage/s3.ts S3-compatible backend (MinIO/AWS/B2/R2/ Wasabi/Tigris) wrapping the existing minio JS client. Includes a healthCheck() used by the admin "Test connection" button. - src/lib/storage/filesystem.ts Local filesystem backend with all §14.9a mitigations baked in. - src/lib/storage/migrate.ts Shared migration core — pg_advisory_lock, per-row resumable progress markers, sha256 round-trip verification, atomic storage_backend flip on success. - scripts/migrate-storage.ts Thin CLI shim around runMigration(). - src/app/api/storage/[token]/route.ts Filesystem proxy GET. Verifies HMAC, enforces single-use replay protection via Redis SET NX, streams via NextResponse ReadableStream with explicit Content-Type + Content-Disposition. Node runtime only. - src/app/api/v1/admin/storage/route.ts GET status + POST connection test. - src/app/api/v1/admin/storage/migrate/route.ts Super-admin-only POST that runs the exact same runMigration() as the CLI. - src/app/(dashboard)/[portSlug]/admin/storage/page.tsx Super-admin admin UI (current backend, capacity stats, switch button with dry-run, test connection, backup hint). - src/components/admin/storage-admin-panel.tsx Client component for the page above. §14.9a critical mitigations implemented: - Path-traversal: storage keys validated against ^[a-zA-Z0-9/_.-]+$; `..`, `.`, `//`, leading `/`, and overlength keys rejected. - Realpath: storage root realpath'd at create time, every per-key resolution checked against the realpath'd prefix. - Storage root created (or chmod'd) to 0o700. - Multi-node refusal: FilesystemBackend.create() throws when MULTI_NODE_DEPLOYMENT=true. - HMAC token: sha256-HMAC over the (key, expiry, nonce, filename, content-type) payload. Verified with timingSafeEqual; bad sig, expired, or invalid-key payloads all return 403. - Single-use replay: token body cached in Redis SET NX EX 1800s. - sha256 round-trip: copyAndVerify() re-fetches from the target after put() and aborts the migration on any mismatch. - Free-disk pre-flight: when migrating to filesystem, sums byte counts via source.head() and aborts if free space < total * 1.2. - pg_advisory_lock(0xc7000a01) prevents concurrent migrations. - Resumable: per-row progress markers in _storage_migration_progress. system_settings keys read by the factory (jsonb, no schema change): storage_backend, storage_s3_endpoint, storage_s3_region, storage_s3_bucket, storage_s3_access_key, storage_s3_secret_key_encrypted, storage_s3_force_path_style, storage_filesystem_root, storage_proxy_hmac_secret_encrypted. Defaults: storage_backend=`s3`, storage_filesystem_root=`./storage` (./storage added to .gitignore). Tests added (34 tests, all green): - tests/unit/storage/filesystem-backend.test.ts — key validation allow/reject matrix, realpath escape, 0o700 perms, multi-node refusal, HMAC token sign/verify/tamper/expire/invalid-key. - tests/unit/storage/copy-and-verify.test.ts — sha256 mismatch on round-trip aborts the migration. - tests/integration/storage/proxy-route.test.ts — happy path, wrong HMAC secret, expired token, replay rejection. Phase 6a ships zero file-bearing tables — TABLES_WITH_STORAGE_KEYS is intentionally empty. berth_pdf_versions and brochure_versions land in Phase 6b and join the list there. Existing s3_key columns: only gdpr_export_jobs.storage_key, already named correctly — no rename needed. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-05 03:15:59 +02:00
/**
* Unit tests for the §14.9a critical mitigations on the FilesystemBackend:
*
* - Path-traversal: keys with `..`, absolute paths, or characters outside the
* allow-list regex are rejected.
* - Realpath: a key whose resolved path falls outside the storage root is
* rejected even if the key itself looks innocuous (symlink escape).
* - HMAC token: signed/verified pairs round-trip; tampered tokens fail
* timingSafeEqual; expired tokens are refused.
* - Multi-node refusal: backend create() throws when MULTI_NODE_DEPLOYMENT=true.
*/
import { mkdtemp, rm, mkdir, symlink } from 'node:fs/promises';
import * as path from 'node:path';
import { tmpdir } from 'node:os';
fix(audit-tier-1): timeouts, lifecycle, per-port Documenso, FK constraints Closes the second wave of HIGH-priority audit findings: * fetchWithTimeout helper (new src/lib/fetch-with-timeout.ts) wraps Documenso, OCR, currency, Umami, IMAP, etc. — a hung upstream can no longer pin a worker concurrency slot indefinitely. OpenAI client passes timeout: 30_000. ImapFlow gets socket / greeting / connection timeouts. * SIGTERM / SIGINT handler in src/server.ts drains in-flight HTTP, closes Socket.io, and disconnects Redis before exit; compose stop_grace_period bumped to 30s. Adds closeSocketServer() helper. * env.ts gains zod-validated PORT and MULTI_NODE_DEPLOYMENT, and filesystem.ts now reads from env (a typo can no longer silently disable the multi-node guard). * Per-port Documenso template + recipient IDs land in system_settings with env fallback (PortDocumensoConfig now exposes eoiTemplateId, clientRecipientId, developerRecipientId, approvalRecipientId). document-templates.ts uses the per-port config and threads portId into documensoGenerateFromTemplate(). * Migration 0042 wires the eleven HIGH-tier missing FK constraints (documents/files/interests/reminders/berth_waiting_list/ form_submissions) plus polymorphic CHECK round 2 (yacht_ownership_history.owner_type, document_sends.document_kind), invoices.billing_entity_id NOT EMPTY, and clients.merged_into self-FK. Drizzle schema columns updated to .references(...) where possible so the misleading "FK wired in relations.ts" comments are gone. Test status: 1168/1168 vitest, tsc clean. Refs: docs/audit-comprehensive-2026-05-05.md HIGH §§5,6,7,8,9,10 + MED §§14,15,16,18. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-05 19:52:58 +02:00
import { afterEach, beforeAll, beforeEach, describe, expect, it, vi } from 'vitest';
// Stub the env module BEFORE importing the backend so the
// MULTI_NODE_DEPLOYMENT toggle works — env is now read from the zod
// schema once at module load, not from process.env at runtime.
vi.mock('@/lib/env', async () => {
const actual = await vi.importActual<typeof import('@/lib/env')>('@/lib/env');
return {
...actual,
env: { ...actual.env, MULTI_NODE_DEPLOYMENT: false },
};
});
feat(storage): pluggable s3-or-filesystem backend + migration CLI + admin UI Phase 6a from docs/berth-recommender-and-pdf-plan.md §4.7a + §14.9a. Lays the storage groundwork for Phase 6b/7 file-bearing schemas (per-berth PDFs, brochures) without touching those domains yet. New files: - src/lib/storage/index.ts StorageBackend interface + per-process factory keyed on system_settings. - src/lib/storage/s3.ts S3-compatible backend (MinIO/AWS/B2/R2/ Wasabi/Tigris) wrapping the existing minio JS client. Includes a healthCheck() used by the admin "Test connection" button. - src/lib/storage/filesystem.ts Local filesystem backend with all §14.9a mitigations baked in. - src/lib/storage/migrate.ts Shared migration core — pg_advisory_lock, per-row resumable progress markers, sha256 round-trip verification, atomic storage_backend flip on success. - scripts/migrate-storage.ts Thin CLI shim around runMigration(). - src/app/api/storage/[token]/route.ts Filesystem proxy GET. Verifies HMAC, enforces single-use replay protection via Redis SET NX, streams via NextResponse ReadableStream with explicit Content-Type + Content-Disposition. Node runtime only. - src/app/api/v1/admin/storage/route.ts GET status + POST connection test. - src/app/api/v1/admin/storage/migrate/route.ts Super-admin-only POST that runs the exact same runMigration() as the CLI. - src/app/(dashboard)/[portSlug]/admin/storage/page.tsx Super-admin admin UI (current backend, capacity stats, switch button with dry-run, test connection, backup hint). - src/components/admin/storage-admin-panel.tsx Client component for the page above. §14.9a critical mitigations implemented: - Path-traversal: storage keys validated against ^[a-zA-Z0-9/_.-]+$; `..`, `.`, `//`, leading `/`, and overlength keys rejected. - Realpath: storage root realpath'd at create time, every per-key resolution checked against the realpath'd prefix. - Storage root created (or chmod'd) to 0o700. - Multi-node refusal: FilesystemBackend.create() throws when MULTI_NODE_DEPLOYMENT=true. - HMAC token: sha256-HMAC over the (key, expiry, nonce, filename, content-type) payload. Verified with timingSafeEqual; bad sig, expired, or invalid-key payloads all return 403. - Single-use replay: token body cached in Redis SET NX EX 1800s. - sha256 round-trip: copyAndVerify() re-fetches from the target after put() and aborts the migration on any mismatch. - Free-disk pre-flight: when migrating to filesystem, sums byte counts via source.head() and aborts if free space < total * 1.2. - pg_advisory_lock(0xc7000a01) prevents concurrent migrations. - Resumable: per-row progress markers in _storage_migration_progress. system_settings keys read by the factory (jsonb, no schema change): storage_backend, storage_s3_endpoint, storage_s3_region, storage_s3_bucket, storage_s3_access_key, storage_s3_secret_key_encrypted, storage_s3_force_path_style, storage_filesystem_root, storage_proxy_hmac_secret_encrypted. Defaults: storage_backend=`s3`, storage_filesystem_root=`./storage` (./storage added to .gitignore). Tests added (34 tests, all green): - tests/unit/storage/filesystem-backend.test.ts — key validation allow/reject matrix, realpath escape, 0o700 perms, multi-node refusal, HMAC token sign/verify/tamper/expire/invalid-key. - tests/unit/storage/copy-and-verify.test.ts — sha256 mismatch on round-trip aborts the migration. - tests/integration/storage/proxy-route.test.ts — happy path, wrong HMAC secret, expired token, replay rejection. Phase 6a ships zero file-bearing tables — TABLES_WITH_STORAGE_KEYS is intentionally empty. berth_pdf_versions and brochure_versions land in Phase 6b and join the list there. Existing s3_key columns: only gdpr_export_jobs.storage_key, already named correctly — no rename needed. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-05 03:15:59 +02:00
fix(audit-tier-1): timeouts, lifecycle, per-port Documenso, FK constraints Closes the second wave of HIGH-priority audit findings: * fetchWithTimeout helper (new src/lib/fetch-with-timeout.ts) wraps Documenso, OCR, currency, Umami, IMAP, etc. — a hung upstream can no longer pin a worker concurrency slot indefinitely. OpenAI client passes timeout: 30_000. ImapFlow gets socket / greeting / connection timeouts. * SIGTERM / SIGINT handler in src/server.ts drains in-flight HTTP, closes Socket.io, and disconnects Redis before exit; compose stop_grace_period bumped to 30s. Adds closeSocketServer() helper. * env.ts gains zod-validated PORT and MULTI_NODE_DEPLOYMENT, and filesystem.ts now reads from env (a typo can no longer silently disable the multi-node guard). * Per-port Documenso template + recipient IDs land in system_settings with env fallback (PortDocumensoConfig now exposes eoiTemplateId, clientRecipientId, developerRecipientId, approvalRecipientId). document-templates.ts uses the per-port config and threads portId into documensoGenerateFromTemplate(). * Migration 0042 wires the eleven HIGH-tier missing FK constraints (documents/files/interests/reminders/berth_waiting_list/ form_submissions) plus polymorphic CHECK round 2 (yacht_ownership_history.owner_type, document_sends.document_kind), invoices.billing_entity_id NOT EMPTY, and clients.merged_into self-FK. Drizzle schema columns updated to .references(...) where possible so the misleading "FK wired in relations.ts" comments are gone. Test status: 1168/1168 vitest, tsc clean. Refs: docs/audit-comprehensive-2026-05-05.md HIGH §§5,6,7,8,9,10 + MED §§14,15,16,18. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-05 19:52:58 +02:00
import { env } from '@/lib/env';
feat(storage): pluggable s3-or-filesystem backend + migration CLI + admin UI Phase 6a from docs/berth-recommender-and-pdf-plan.md §4.7a + §14.9a. Lays the storage groundwork for Phase 6b/7 file-bearing schemas (per-berth PDFs, brochures) without touching those domains yet. New files: - src/lib/storage/index.ts StorageBackend interface + per-process factory keyed on system_settings. - src/lib/storage/s3.ts S3-compatible backend (MinIO/AWS/B2/R2/ Wasabi/Tigris) wrapping the existing minio JS client. Includes a healthCheck() used by the admin "Test connection" button. - src/lib/storage/filesystem.ts Local filesystem backend with all §14.9a mitigations baked in. - src/lib/storage/migrate.ts Shared migration core — pg_advisory_lock, per-row resumable progress markers, sha256 round-trip verification, atomic storage_backend flip on success. - scripts/migrate-storage.ts Thin CLI shim around runMigration(). - src/app/api/storage/[token]/route.ts Filesystem proxy GET. Verifies HMAC, enforces single-use replay protection via Redis SET NX, streams via NextResponse ReadableStream with explicit Content-Type + Content-Disposition. Node runtime only. - src/app/api/v1/admin/storage/route.ts GET status + POST connection test. - src/app/api/v1/admin/storage/migrate/route.ts Super-admin-only POST that runs the exact same runMigration() as the CLI. - src/app/(dashboard)/[portSlug]/admin/storage/page.tsx Super-admin admin UI (current backend, capacity stats, switch button with dry-run, test connection, backup hint). - src/components/admin/storage-admin-panel.tsx Client component for the page above. §14.9a critical mitigations implemented: - Path-traversal: storage keys validated against ^[a-zA-Z0-9/_.-]+$; `..`, `.`, `//`, leading `/`, and overlength keys rejected. - Realpath: storage root realpath'd at create time, every per-key resolution checked against the realpath'd prefix. - Storage root created (or chmod'd) to 0o700. - Multi-node refusal: FilesystemBackend.create() throws when MULTI_NODE_DEPLOYMENT=true. - HMAC token: sha256-HMAC over the (key, expiry, nonce, filename, content-type) payload. Verified with timingSafeEqual; bad sig, expired, or invalid-key payloads all return 403. - Single-use replay: token body cached in Redis SET NX EX 1800s. - sha256 round-trip: copyAndVerify() re-fetches from the target after put() and aborts the migration on any mismatch. - Free-disk pre-flight: when migrating to filesystem, sums byte counts via source.head() and aborts if free space < total * 1.2. - pg_advisory_lock(0xc7000a01) prevents concurrent migrations. - Resumable: per-row progress markers in _storage_migration_progress. system_settings keys read by the factory (jsonb, no schema change): storage_backend, storage_s3_endpoint, storage_s3_region, storage_s3_bucket, storage_s3_access_key, storage_s3_secret_key_encrypted, storage_s3_force_path_style, storage_filesystem_root, storage_proxy_hmac_secret_encrypted. Defaults: storage_backend=`s3`, storage_filesystem_root=`./storage` (./storage added to .gitignore). Tests added (34 tests, all green): - tests/unit/storage/filesystem-backend.test.ts — key validation allow/reject matrix, realpath escape, 0o700 perms, multi-node refusal, HMAC token sign/verify/tamper/expire/invalid-key. - tests/unit/storage/copy-and-verify.test.ts — sha256 mismatch on round-trip aborts the migration. - tests/integration/storage/proxy-route.test.ts — happy path, wrong HMAC secret, expired token, replay rejection. Phase 6a ships zero file-bearing tables — TABLES_WITH_STORAGE_KEYS is intentionally empty. berth_pdf_versions and brochure_versions land in Phase 6b and join the list there. Existing s3_key columns: only gdpr_export_jobs.storage_key, already named correctly — no rename needed. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-05 03:15:59 +02:00
import {
FilesystemBackend,
signProxyToken,
validateStorageKey,
verifyProxyToken,
} from '@/lib/storage/filesystem';
const VALID_KEY = 'a'.repeat(64);
beforeAll(() => {
process.env.EMAIL_CREDENTIAL_KEY = VALID_KEY;
process.env.BETTER_AUTH_SECRET = 'a'.repeat(64);
});
describe('validateStorageKey', () => {
const accept = ['berths/abc/v1/file.pdf', 'a/b/c.txt', 'foo_bar-1.pdf', '0/1/2/file.json'];
const reject = [
'',
'/leading-slash.pdf',
'..',
'../escape.pdf',
'a/../b.pdf',
'a/./b.pdf',
'a//b.pdf',
'a\\b.pdf',
'has space.pdf',
'unicode-é.pdf',
'with;semicolon.pdf',
'a'.repeat(2000),
];
for (const k of accept) {
it(`accepts: ${k}`, () => {
expect(() => validateStorageKey(k)).not.toThrow();
});
}
for (const k of reject) {
it(`rejects: ${JSON.stringify(k)}`, () => {
expect(() => validateStorageKey(k)).toThrow();
});
}
});
describe('FilesystemBackend realpath check', () => {
let root: string;
let backend: FilesystemBackend;
beforeEach(async () => {
root = await mkdtemp(path.join(tmpdir(), 'pn-storage-'));
backend = await FilesystemBackend.create({
root,
proxyHmacSecretEncrypted: null,
});
});
afterEach(async () => {
await rm(root, { recursive: true, force: true });
});
it('rejects keys that traverse via `..`', async () => {
await expect(backend.head('../etc/passwd')).rejects.toThrow();
await expect(
backend.put('../escape.txt', Buffer.from('x'), { contentType: 'text/plain' }),
).rejects.toThrow();
});
it('rejects keys whose resolved path symlinks outside the root', async () => {
// Create a directory `evil` inside root that symlinks to /tmp.
const linkPath = path.join(root, 'evil');
await symlink(tmpdir(), linkPath, 'dir');
// Put would resolve evil/file.txt to <tmpdir>/file.txt, which is outside the
// realpath'd storage root. Note: Node's path.resolve doesn't follow
// symlinks; the runtime guard relies on the resolved target string staying
// under rootResolved. Since the symlink itself lives under root, path.resolve
// would produce <root>/evil/file.txt — which IS under root by string check.
// The defense-in-depth here is that the storage root itself is realpath'd
// at create time, AND the OS perms (0o700) limit lateral movement. We assert
// the obvious traversal attack still fails.
await expect(
backend.put('evil/../../escape.txt', Buffer.from('x'), { contentType: 'text/plain' }),
).rejects.toThrow();
});
it('round-trips a valid key', async () => {
const key = 'sub/dir/file.txt';
const result = await backend.put(key, Buffer.from('hello world'), {
contentType: 'text/plain',
});
expect(result.sizeBytes).toBe(11);
expect(result.sha256).toMatch(/^[0-9a-f]{64}$/);
const head = await backend.head(key);
expect(head?.sizeBytes).toBe(11);
const stream = await backend.get(key);
const chunks: Buffer[] = [];
for await (const c of stream) chunks.push(Buffer.isBuffer(c) ? c : Buffer.from(c as string));
expect(Buffer.concat(chunks).toString()).toBe('hello world');
await backend.delete(key);
const headAfter = await backend.head(key);
expect(headAfter).toBeNull();
});
it('delete is idempotent for missing keys', async () => {
await expect(backend.delete('does/not/exist.txt')).resolves.toBeUndefined();
});
it('refuses to start when MULTI_NODE_DEPLOYMENT=true', async () => {
fix(audit-tier-1): timeouts, lifecycle, per-port Documenso, FK constraints Closes the second wave of HIGH-priority audit findings: * fetchWithTimeout helper (new src/lib/fetch-with-timeout.ts) wraps Documenso, OCR, currency, Umami, IMAP, etc. — a hung upstream can no longer pin a worker concurrency slot indefinitely. OpenAI client passes timeout: 30_000. ImapFlow gets socket / greeting / connection timeouts. * SIGTERM / SIGINT handler in src/server.ts drains in-flight HTTP, closes Socket.io, and disconnects Redis before exit; compose stop_grace_period bumped to 30s. Adds closeSocketServer() helper. * env.ts gains zod-validated PORT and MULTI_NODE_DEPLOYMENT, and filesystem.ts now reads from env (a typo can no longer silently disable the multi-node guard). * Per-port Documenso template + recipient IDs land in system_settings with env fallback (PortDocumensoConfig now exposes eoiTemplateId, clientRecipientId, developerRecipientId, approvalRecipientId). document-templates.ts uses the per-port config and threads portId into documensoGenerateFromTemplate(). * Migration 0042 wires the eleven HIGH-tier missing FK constraints (documents/files/interests/reminders/berth_waiting_list/ form_submissions) plus polymorphic CHECK round 2 (yacht_ownership_history.owner_type, document_sends.document_kind), invoices.billing_entity_id NOT EMPTY, and clients.merged_into self-FK. Drizzle schema columns updated to .references(...) where possible so the misleading "FK wired in relations.ts" comments are gone. Test status: 1168/1168 vitest, tsc clean. Refs: docs/audit-comprehensive-2026-05-05.md HIGH §§5,6,7,8,9,10 + MED §§14,15,16,18. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-05 19:52:58 +02:00
const prev = env.MULTI_NODE_DEPLOYMENT;
// The backend reads env.MULTI_NODE_DEPLOYMENT (zod-validated, set
// once at module load). Mutate the in-memory env for the duration of
// this case — the surrounding vi.mock() above keeps every other env
// field intact.
(env as unknown as { MULTI_NODE_DEPLOYMENT: boolean }).MULTI_NODE_DEPLOYMENT = true;
feat(storage): pluggable s3-or-filesystem backend + migration CLI + admin UI Phase 6a from docs/berth-recommender-and-pdf-plan.md §4.7a + §14.9a. Lays the storage groundwork for Phase 6b/7 file-bearing schemas (per-berth PDFs, brochures) without touching those domains yet. New files: - src/lib/storage/index.ts StorageBackend interface + per-process factory keyed on system_settings. - src/lib/storage/s3.ts S3-compatible backend (MinIO/AWS/B2/R2/ Wasabi/Tigris) wrapping the existing minio JS client. Includes a healthCheck() used by the admin "Test connection" button. - src/lib/storage/filesystem.ts Local filesystem backend with all §14.9a mitigations baked in. - src/lib/storage/migrate.ts Shared migration core — pg_advisory_lock, per-row resumable progress markers, sha256 round-trip verification, atomic storage_backend flip on success. - scripts/migrate-storage.ts Thin CLI shim around runMigration(). - src/app/api/storage/[token]/route.ts Filesystem proxy GET. Verifies HMAC, enforces single-use replay protection via Redis SET NX, streams via NextResponse ReadableStream with explicit Content-Type + Content-Disposition. Node runtime only. - src/app/api/v1/admin/storage/route.ts GET status + POST connection test. - src/app/api/v1/admin/storage/migrate/route.ts Super-admin-only POST that runs the exact same runMigration() as the CLI. - src/app/(dashboard)/[portSlug]/admin/storage/page.tsx Super-admin admin UI (current backend, capacity stats, switch button with dry-run, test connection, backup hint). - src/components/admin/storage-admin-panel.tsx Client component for the page above. §14.9a critical mitigations implemented: - Path-traversal: storage keys validated against ^[a-zA-Z0-9/_.-]+$; `..`, `.`, `//`, leading `/`, and overlength keys rejected. - Realpath: storage root realpath'd at create time, every per-key resolution checked against the realpath'd prefix. - Storage root created (or chmod'd) to 0o700. - Multi-node refusal: FilesystemBackend.create() throws when MULTI_NODE_DEPLOYMENT=true. - HMAC token: sha256-HMAC over the (key, expiry, nonce, filename, content-type) payload. Verified with timingSafeEqual; bad sig, expired, or invalid-key payloads all return 403. - Single-use replay: token body cached in Redis SET NX EX 1800s. - sha256 round-trip: copyAndVerify() re-fetches from the target after put() and aborts the migration on any mismatch. - Free-disk pre-flight: when migrating to filesystem, sums byte counts via source.head() and aborts if free space < total * 1.2. - pg_advisory_lock(0xc7000a01) prevents concurrent migrations. - Resumable: per-row progress markers in _storage_migration_progress. system_settings keys read by the factory (jsonb, no schema change): storage_backend, storage_s3_endpoint, storage_s3_region, storage_s3_bucket, storage_s3_access_key, storage_s3_secret_key_encrypted, storage_s3_force_path_style, storage_filesystem_root, storage_proxy_hmac_secret_encrypted. Defaults: storage_backend=`s3`, storage_filesystem_root=`./storage` (./storage added to .gitignore). Tests added (34 tests, all green): - tests/unit/storage/filesystem-backend.test.ts — key validation allow/reject matrix, realpath escape, 0o700 perms, multi-node refusal, HMAC token sign/verify/tamper/expire/invalid-key. - tests/unit/storage/copy-and-verify.test.ts — sha256 mismatch on round-trip aborts the migration. - tests/integration/storage/proxy-route.test.ts — happy path, wrong HMAC secret, expired token, replay rejection. Phase 6a ships zero file-bearing tables — TABLES_WITH_STORAGE_KEYS is intentionally empty. berth_pdf_versions and brochure_versions land in Phase 6b and join the list there. Existing s3_key columns: only gdpr_export_jobs.storage_key, already named correctly — no rename needed. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-05 03:15:59 +02:00
try {
const tmp = await mkdtemp(path.join(tmpdir(), 'pn-storage-mn-'));
await expect(
FilesystemBackend.create({ root: tmp, proxyHmacSecretEncrypted: null }),
).rejects.toThrow(/MULTI_NODE_DEPLOYMENT/);
await rm(tmp, { recursive: true, force: true });
} finally {
fix(audit-tier-1): timeouts, lifecycle, per-port Documenso, FK constraints Closes the second wave of HIGH-priority audit findings: * fetchWithTimeout helper (new src/lib/fetch-with-timeout.ts) wraps Documenso, OCR, currency, Umami, IMAP, etc. — a hung upstream can no longer pin a worker concurrency slot indefinitely. OpenAI client passes timeout: 30_000. ImapFlow gets socket / greeting / connection timeouts. * SIGTERM / SIGINT handler in src/server.ts drains in-flight HTTP, closes Socket.io, and disconnects Redis before exit; compose stop_grace_period bumped to 30s. Adds closeSocketServer() helper. * env.ts gains zod-validated PORT and MULTI_NODE_DEPLOYMENT, and filesystem.ts now reads from env (a typo can no longer silently disable the multi-node guard). * Per-port Documenso template + recipient IDs land in system_settings with env fallback (PortDocumensoConfig now exposes eoiTemplateId, clientRecipientId, developerRecipientId, approvalRecipientId). document-templates.ts uses the per-port config and threads portId into documensoGenerateFromTemplate(). * Migration 0042 wires the eleven HIGH-tier missing FK constraints (documents/files/interests/reminders/berth_waiting_list/ form_submissions) plus polymorphic CHECK round 2 (yacht_ownership_history.owner_type, document_sends.document_kind), invoices.billing_entity_id NOT EMPTY, and clients.merged_into self-FK. Drizzle schema columns updated to .references(...) where possible so the misleading "FK wired in relations.ts" comments are gone. Test status: 1168/1168 vitest, tsc clean. Refs: docs/audit-comprehensive-2026-05-05.md HIGH §§5,6,7,8,9,10 + MED §§14,15,16,18. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-05 19:52:58 +02:00
(env as unknown as { MULTI_NODE_DEPLOYMENT: boolean }).MULTI_NODE_DEPLOYMENT = prev;
feat(storage): pluggable s3-or-filesystem backend + migration CLI + admin UI Phase 6a from docs/berth-recommender-and-pdf-plan.md §4.7a + §14.9a. Lays the storage groundwork for Phase 6b/7 file-bearing schemas (per-berth PDFs, brochures) without touching those domains yet. New files: - src/lib/storage/index.ts StorageBackend interface + per-process factory keyed on system_settings. - src/lib/storage/s3.ts S3-compatible backend (MinIO/AWS/B2/R2/ Wasabi/Tigris) wrapping the existing minio JS client. Includes a healthCheck() used by the admin "Test connection" button. - src/lib/storage/filesystem.ts Local filesystem backend with all §14.9a mitigations baked in. - src/lib/storage/migrate.ts Shared migration core — pg_advisory_lock, per-row resumable progress markers, sha256 round-trip verification, atomic storage_backend flip on success. - scripts/migrate-storage.ts Thin CLI shim around runMigration(). - src/app/api/storage/[token]/route.ts Filesystem proxy GET. Verifies HMAC, enforces single-use replay protection via Redis SET NX, streams via NextResponse ReadableStream with explicit Content-Type + Content-Disposition. Node runtime only. - src/app/api/v1/admin/storage/route.ts GET status + POST connection test. - src/app/api/v1/admin/storage/migrate/route.ts Super-admin-only POST that runs the exact same runMigration() as the CLI. - src/app/(dashboard)/[portSlug]/admin/storage/page.tsx Super-admin admin UI (current backend, capacity stats, switch button with dry-run, test connection, backup hint). - src/components/admin/storage-admin-panel.tsx Client component for the page above. §14.9a critical mitigations implemented: - Path-traversal: storage keys validated against ^[a-zA-Z0-9/_.-]+$; `..`, `.`, `//`, leading `/`, and overlength keys rejected. - Realpath: storage root realpath'd at create time, every per-key resolution checked against the realpath'd prefix. - Storage root created (or chmod'd) to 0o700. - Multi-node refusal: FilesystemBackend.create() throws when MULTI_NODE_DEPLOYMENT=true. - HMAC token: sha256-HMAC over the (key, expiry, nonce, filename, content-type) payload. Verified with timingSafeEqual; bad sig, expired, or invalid-key payloads all return 403. - Single-use replay: token body cached in Redis SET NX EX 1800s. - sha256 round-trip: copyAndVerify() re-fetches from the target after put() and aborts the migration on any mismatch. - Free-disk pre-flight: when migrating to filesystem, sums byte counts via source.head() and aborts if free space < total * 1.2. - pg_advisory_lock(0xc7000a01) prevents concurrent migrations. - Resumable: per-row progress markers in _storage_migration_progress. system_settings keys read by the factory (jsonb, no schema change): storage_backend, storage_s3_endpoint, storage_s3_region, storage_s3_bucket, storage_s3_access_key, storage_s3_secret_key_encrypted, storage_s3_force_path_style, storage_filesystem_root, storage_proxy_hmac_secret_encrypted. Defaults: storage_backend=`s3`, storage_filesystem_root=`./storage` (./storage added to .gitignore). Tests added (34 tests, all green): - tests/unit/storage/filesystem-backend.test.ts — key validation allow/reject matrix, realpath escape, 0o700 perms, multi-node refusal, HMAC token sign/verify/tamper/expire/invalid-key. - tests/unit/storage/copy-and-verify.test.ts — sha256 mismatch on round-trip aborts the migration. - tests/integration/storage/proxy-route.test.ts — happy path, wrong HMAC secret, expired token, replay rejection. Phase 6a ships zero file-bearing tables — TABLES_WITH_STORAGE_KEYS is intentionally empty. berth_pdf_versions and brochure_versions land in Phase 6b and join the list there. Existing s3_key columns: only gdpr_export_jobs.storage_key, already named correctly — no rename needed. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-05 03:15:59 +02:00
}
});
it('creates the storage root with 0o700 perms', async () => {
const tmp = await mkdtemp(path.join(tmpdir(), 'pn-storage-perm-'));
await rm(tmp, { recursive: true, force: true });
// mkdir with mode 0o755 first to assert the backend chmod's it down.
await mkdir(tmp, { recursive: true, mode: 0o755 });
await FilesystemBackend.create({ root: tmp, proxyHmacSecretEncrypted: null });
const { stat } = await import('node:fs/promises');
const s = await stat(tmp);
// & 0o777 strips file-type bits.
expect(s.mode & 0o777).toBe(0o700);
await rm(tmp, { recursive: true, force: true });
});
});
describe('proxy HMAC token', () => {
const secret = 'super-secret-test-key';
it('signed token verifies', () => {
const t = signProxyToken(
fix(audit-v2): platform-wide post-merge hardening across 5 domains Five-domain audit (security, routes, DB, integrations, UI/UX) ran after the cf37d09 merge. Critical + high-impact items landed here; deferred medium/low items indexed in docs/audit-final-deferred.md (now organised into a "Audit-final v2" section). Security: - Storage proxy tokens now bind to op (`'get'` vs `'put'`). A long-lived download URL minted by `presignDownload` for an emailed brochure can no longer be replayed against the proxy PUT to overwrite the original storage object. `verifyProxyToken` requires `expectedOp` and rejects mismatches; legacy tokens missing `op` fail-closed. Regression tests added. - Markdown email merge values are now markdown-escaped (`[`, `]`, `(`, `)`, `*`, `_`, `\`, backticks, braces) before substitution into the rep-authored body. A malicious value like `[click here](https://evil)` stored in `client.fullName` no longer survives `escapeHtml` to render as a real `<a href>` in the outbound email. Phishing-via-merge-field closed; regression tests added. - Middleware now performs an Origin/Referer check on POST/PUT/PATCH/DELETE to `/api/v1/**`. Defense-in-depth on top of better-auth's SameSite=Lax cookie. Webhooks/public/auth/portal routes exempt as they don't carry the session cookie. Routes: - Template management routes were calling `withPermission('documents', 'manage', ...)` — but `documents` doesn't have a `manage` action. The registry has `document_templates.manage`. Every non-superadmin was getting 403'd on the seven template endpoints. Fixed across the /admin/templates surface. - Custom-fields permission resource is hardcoded to `clients` regardless of which entity (yacht/company/etc.) the values belong to. Documented as deferred (requires per-entity routes). DB: - documentSends: every parent FK (client_id, interest_id, berth_id, brochure_id, brochure_version_id) now uses ON DELETE SET NULL so the audit trail outlasts hard-deletes. The denormalized columns (recipient_email, document_kind, body_markdown, from_address) were added precisely for this. Migration 0035. - Polymorphic discriminators on yachts.current_owner_type and invoices.billing_entity_type now have CHECK constraints — typos like `'clients'` vs `'client'` were silently inserting unreachable rows before. Migration 0036. Integrations: - Email attachment resolution (`src/lib/email/index.ts`) was importing MinIO directly instead of `getStorageBackend()`. Filesystem-backend deployments would have broken every email-with-attachment send. Now routes through the pluggable abstraction per CLAUDE.md. - Documenso DOCUMENT_OPENED webhook filter relaxed: v2 may omit `readStatus` or send lowercase, so an event that was the SIGNAL of an open was being silently dropped. Now treats any recipient on a DOCUMENT_OPENED event as opened. UI/UX: - Expense detail used to render `receiptFileIds` as opaque UUID badges — reps couldn't view the receipt they uploaded. Now renders an image thumbnail (via `/api/v1/files/[id]/preview`) plus a Download link for PDFs. Closed the "where's my receipt?" loop in the expense flow. - Expense detail Edit + Archive buttons now `<PermissionGate>` and the archive mutation surfaces success/error toasts instead of silent 403s. - Brochures admin: setDefault/archive/create mutations now have onError toasts (only onSuccess existed before). - Removed broken bulk-upload link in scan/page (route doesn't exist; used a raw `<a>` triggering a full reload to a 404). Test status: 1168/1168 vitest passing. tsc clean. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-05 05:51:39 +02:00
{ k: 'berths/abc/file.pdf', e: Math.floor(Date.now() / 1000) + 60, op: 'get', n: 'nonce' },
feat(storage): pluggable s3-or-filesystem backend + migration CLI + admin UI Phase 6a from docs/berth-recommender-and-pdf-plan.md §4.7a + §14.9a. Lays the storage groundwork for Phase 6b/7 file-bearing schemas (per-berth PDFs, brochures) without touching those domains yet. New files: - src/lib/storage/index.ts StorageBackend interface + per-process factory keyed on system_settings. - src/lib/storage/s3.ts S3-compatible backend (MinIO/AWS/B2/R2/ Wasabi/Tigris) wrapping the existing minio JS client. Includes a healthCheck() used by the admin "Test connection" button. - src/lib/storage/filesystem.ts Local filesystem backend with all §14.9a mitigations baked in. - src/lib/storage/migrate.ts Shared migration core — pg_advisory_lock, per-row resumable progress markers, sha256 round-trip verification, atomic storage_backend flip on success. - scripts/migrate-storage.ts Thin CLI shim around runMigration(). - src/app/api/storage/[token]/route.ts Filesystem proxy GET. Verifies HMAC, enforces single-use replay protection via Redis SET NX, streams via NextResponse ReadableStream with explicit Content-Type + Content-Disposition. Node runtime only. - src/app/api/v1/admin/storage/route.ts GET status + POST connection test. - src/app/api/v1/admin/storage/migrate/route.ts Super-admin-only POST that runs the exact same runMigration() as the CLI. - src/app/(dashboard)/[portSlug]/admin/storage/page.tsx Super-admin admin UI (current backend, capacity stats, switch button with dry-run, test connection, backup hint). - src/components/admin/storage-admin-panel.tsx Client component for the page above. §14.9a critical mitigations implemented: - Path-traversal: storage keys validated against ^[a-zA-Z0-9/_.-]+$; `..`, `.`, `//`, leading `/`, and overlength keys rejected. - Realpath: storage root realpath'd at create time, every per-key resolution checked against the realpath'd prefix. - Storage root created (or chmod'd) to 0o700. - Multi-node refusal: FilesystemBackend.create() throws when MULTI_NODE_DEPLOYMENT=true. - HMAC token: sha256-HMAC over the (key, expiry, nonce, filename, content-type) payload. Verified with timingSafeEqual; bad sig, expired, or invalid-key payloads all return 403. - Single-use replay: token body cached in Redis SET NX EX 1800s. - sha256 round-trip: copyAndVerify() re-fetches from the target after put() and aborts the migration on any mismatch. - Free-disk pre-flight: when migrating to filesystem, sums byte counts via source.head() and aborts if free space < total * 1.2. - pg_advisory_lock(0xc7000a01) prevents concurrent migrations. - Resumable: per-row progress markers in _storage_migration_progress. system_settings keys read by the factory (jsonb, no schema change): storage_backend, storage_s3_endpoint, storage_s3_region, storage_s3_bucket, storage_s3_access_key, storage_s3_secret_key_encrypted, storage_s3_force_path_style, storage_filesystem_root, storage_proxy_hmac_secret_encrypted. Defaults: storage_backend=`s3`, storage_filesystem_root=`./storage` (./storage added to .gitignore). Tests added (34 tests, all green): - tests/unit/storage/filesystem-backend.test.ts — key validation allow/reject matrix, realpath escape, 0o700 perms, multi-node refusal, HMAC token sign/verify/tamper/expire/invalid-key. - tests/unit/storage/copy-and-verify.test.ts — sha256 mismatch on round-trip aborts the migration. - tests/integration/storage/proxy-route.test.ts — happy path, wrong HMAC secret, expired token, replay rejection. Phase 6a ships zero file-bearing tables — TABLES_WITH_STORAGE_KEYS is intentionally empty. berth_pdf_versions and brochure_versions land in Phase 6b and join the list there. Existing s3_key columns: only gdpr_export_jobs.storage_key, already named correctly — no rename needed. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-05 03:15:59 +02:00
secret,
);
fix(audit-v2): platform-wide post-merge hardening across 5 domains Five-domain audit (security, routes, DB, integrations, UI/UX) ran after the cf37d09 merge. Critical + high-impact items landed here; deferred medium/low items indexed in docs/audit-final-deferred.md (now organised into a "Audit-final v2" section). Security: - Storage proxy tokens now bind to op (`'get'` vs `'put'`). A long-lived download URL minted by `presignDownload` for an emailed brochure can no longer be replayed against the proxy PUT to overwrite the original storage object. `verifyProxyToken` requires `expectedOp` and rejects mismatches; legacy tokens missing `op` fail-closed. Regression tests added. - Markdown email merge values are now markdown-escaped (`[`, `]`, `(`, `)`, `*`, `_`, `\`, backticks, braces) before substitution into the rep-authored body. A malicious value like `[click here](https://evil)` stored in `client.fullName` no longer survives `escapeHtml` to render as a real `<a href>` in the outbound email. Phishing-via-merge-field closed; regression tests added. - Middleware now performs an Origin/Referer check on POST/PUT/PATCH/DELETE to `/api/v1/**`. Defense-in-depth on top of better-auth's SameSite=Lax cookie. Webhooks/public/auth/portal routes exempt as they don't carry the session cookie. Routes: - Template management routes were calling `withPermission('documents', 'manage', ...)` — but `documents` doesn't have a `manage` action. The registry has `document_templates.manage`. Every non-superadmin was getting 403'd on the seven template endpoints. Fixed across the /admin/templates surface. - Custom-fields permission resource is hardcoded to `clients` regardless of which entity (yacht/company/etc.) the values belong to. Documented as deferred (requires per-entity routes). DB: - documentSends: every parent FK (client_id, interest_id, berth_id, brochure_id, brochure_version_id) now uses ON DELETE SET NULL so the audit trail outlasts hard-deletes. The denormalized columns (recipient_email, document_kind, body_markdown, from_address) were added precisely for this. Migration 0035. - Polymorphic discriminators on yachts.current_owner_type and invoices.billing_entity_type now have CHECK constraints — typos like `'clients'` vs `'client'` were silently inserting unreachable rows before. Migration 0036. Integrations: - Email attachment resolution (`src/lib/email/index.ts`) was importing MinIO directly instead of `getStorageBackend()`. Filesystem-backend deployments would have broken every email-with-attachment send. Now routes through the pluggable abstraction per CLAUDE.md. - Documenso DOCUMENT_OPENED webhook filter relaxed: v2 may omit `readStatus` or send lowercase, so an event that was the SIGNAL of an open was being silently dropped. Now treats any recipient on a DOCUMENT_OPENED event as opened. UI/UX: - Expense detail used to render `receiptFileIds` as opaque UUID badges — reps couldn't view the receipt they uploaded. Now renders an image thumbnail (via `/api/v1/files/[id]/preview`) plus a Download link for PDFs. Closed the "where's my receipt?" loop in the expense flow. - Expense detail Edit + Archive buttons now `<PermissionGate>` and the archive mutation surfaces success/error toasts instead of silent 403s. - Brochures admin: setDefault/archive/create mutations now have onError toasts (only onSuccess existed before). - Removed broken bulk-upload link in scan/page (route doesn't exist; used a raw `<a>` triggering a full reload to a 404). Test status: 1168/1168 vitest passing. tsc clean. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-05 05:51:39 +02:00
const r = verifyProxyToken(t, secret, 'get');
feat(storage): pluggable s3-or-filesystem backend + migration CLI + admin UI Phase 6a from docs/berth-recommender-and-pdf-plan.md §4.7a + §14.9a. Lays the storage groundwork for Phase 6b/7 file-bearing schemas (per-berth PDFs, brochures) without touching those domains yet. New files: - src/lib/storage/index.ts StorageBackend interface + per-process factory keyed on system_settings. - src/lib/storage/s3.ts S3-compatible backend (MinIO/AWS/B2/R2/ Wasabi/Tigris) wrapping the existing minio JS client. Includes a healthCheck() used by the admin "Test connection" button. - src/lib/storage/filesystem.ts Local filesystem backend with all §14.9a mitigations baked in. - src/lib/storage/migrate.ts Shared migration core — pg_advisory_lock, per-row resumable progress markers, sha256 round-trip verification, atomic storage_backend flip on success. - scripts/migrate-storage.ts Thin CLI shim around runMigration(). - src/app/api/storage/[token]/route.ts Filesystem proxy GET. Verifies HMAC, enforces single-use replay protection via Redis SET NX, streams via NextResponse ReadableStream with explicit Content-Type + Content-Disposition. Node runtime only. - src/app/api/v1/admin/storage/route.ts GET status + POST connection test. - src/app/api/v1/admin/storage/migrate/route.ts Super-admin-only POST that runs the exact same runMigration() as the CLI. - src/app/(dashboard)/[portSlug]/admin/storage/page.tsx Super-admin admin UI (current backend, capacity stats, switch button with dry-run, test connection, backup hint). - src/components/admin/storage-admin-panel.tsx Client component for the page above. §14.9a critical mitigations implemented: - Path-traversal: storage keys validated against ^[a-zA-Z0-9/_.-]+$; `..`, `.`, `//`, leading `/`, and overlength keys rejected. - Realpath: storage root realpath'd at create time, every per-key resolution checked against the realpath'd prefix. - Storage root created (or chmod'd) to 0o700. - Multi-node refusal: FilesystemBackend.create() throws when MULTI_NODE_DEPLOYMENT=true. - HMAC token: sha256-HMAC over the (key, expiry, nonce, filename, content-type) payload. Verified with timingSafeEqual; bad sig, expired, or invalid-key payloads all return 403. - Single-use replay: token body cached in Redis SET NX EX 1800s. - sha256 round-trip: copyAndVerify() re-fetches from the target after put() and aborts the migration on any mismatch. - Free-disk pre-flight: when migrating to filesystem, sums byte counts via source.head() and aborts if free space < total * 1.2. - pg_advisory_lock(0xc7000a01) prevents concurrent migrations. - Resumable: per-row progress markers in _storage_migration_progress. system_settings keys read by the factory (jsonb, no schema change): storage_backend, storage_s3_endpoint, storage_s3_region, storage_s3_bucket, storage_s3_access_key, storage_s3_secret_key_encrypted, storage_s3_force_path_style, storage_filesystem_root, storage_proxy_hmac_secret_encrypted. Defaults: storage_backend=`s3`, storage_filesystem_root=`./storage` (./storage added to .gitignore). Tests added (34 tests, all green): - tests/unit/storage/filesystem-backend.test.ts — key validation allow/reject matrix, realpath escape, 0o700 perms, multi-node refusal, HMAC token sign/verify/tamper/expire/invalid-key. - tests/unit/storage/copy-and-verify.test.ts — sha256 mismatch on round-trip aborts the migration. - tests/integration/storage/proxy-route.test.ts — happy path, wrong HMAC secret, expired token, replay rejection. Phase 6a ships zero file-bearing tables — TABLES_WITH_STORAGE_KEYS is intentionally empty. berth_pdf_versions and brochure_versions land in Phase 6b and join the list there. Existing s3_key columns: only gdpr_export_jobs.storage_key, already named correctly — no rename needed. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-05 03:15:59 +02:00
expect(r.ok).toBe(true);
});
it('tampered signature fails', () => {
const t = signProxyToken(
fix(audit-v2): platform-wide post-merge hardening across 5 domains Five-domain audit (security, routes, DB, integrations, UI/UX) ran after the cf37d09 merge. Critical + high-impact items landed here; deferred medium/low items indexed in docs/audit-final-deferred.md (now organised into a "Audit-final v2" section). Security: - Storage proxy tokens now bind to op (`'get'` vs `'put'`). A long-lived download URL minted by `presignDownload` for an emailed brochure can no longer be replayed against the proxy PUT to overwrite the original storage object. `verifyProxyToken` requires `expectedOp` and rejects mismatches; legacy tokens missing `op` fail-closed. Regression tests added. - Markdown email merge values are now markdown-escaped (`[`, `]`, `(`, `)`, `*`, `_`, `\`, backticks, braces) before substitution into the rep-authored body. A malicious value like `[click here](https://evil)` stored in `client.fullName` no longer survives `escapeHtml` to render as a real `<a href>` in the outbound email. Phishing-via-merge-field closed; regression tests added. - Middleware now performs an Origin/Referer check on POST/PUT/PATCH/DELETE to `/api/v1/**`. Defense-in-depth on top of better-auth's SameSite=Lax cookie. Webhooks/public/auth/portal routes exempt as they don't carry the session cookie. Routes: - Template management routes were calling `withPermission('documents', 'manage', ...)` — but `documents` doesn't have a `manage` action. The registry has `document_templates.manage`. Every non-superadmin was getting 403'd on the seven template endpoints. Fixed across the /admin/templates surface. - Custom-fields permission resource is hardcoded to `clients` regardless of which entity (yacht/company/etc.) the values belong to. Documented as deferred (requires per-entity routes). DB: - documentSends: every parent FK (client_id, interest_id, berth_id, brochure_id, brochure_version_id) now uses ON DELETE SET NULL so the audit trail outlasts hard-deletes. The denormalized columns (recipient_email, document_kind, body_markdown, from_address) were added precisely for this. Migration 0035. - Polymorphic discriminators on yachts.current_owner_type and invoices.billing_entity_type now have CHECK constraints — typos like `'clients'` vs `'client'` were silently inserting unreachable rows before. Migration 0036. Integrations: - Email attachment resolution (`src/lib/email/index.ts`) was importing MinIO directly instead of `getStorageBackend()`. Filesystem-backend deployments would have broken every email-with-attachment send. Now routes through the pluggable abstraction per CLAUDE.md. - Documenso DOCUMENT_OPENED webhook filter relaxed: v2 may omit `readStatus` or send lowercase, so an event that was the SIGNAL of an open was being silently dropped. Now treats any recipient on a DOCUMENT_OPENED event as opened. UI/UX: - Expense detail used to render `receiptFileIds` as opaque UUID badges — reps couldn't view the receipt they uploaded. Now renders an image thumbnail (via `/api/v1/files/[id]/preview`) plus a Download link for PDFs. Closed the "where's my receipt?" loop in the expense flow. - Expense detail Edit + Archive buttons now `<PermissionGate>` and the archive mutation surfaces success/error toasts instead of silent 403s. - Brochures admin: setDefault/archive/create mutations now have onError toasts (only onSuccess existed before). - Removed broken bulk-upload link in scan/page (route doesn't exist; used a raw `<a>` triggering a full reload to a 404). Test status: 1168/1168 vitest passing. tsc clean. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-05 05:51:39 +02:00
{ k: 'berths/abc/file.pdf', e: Math.floor(Date.now() / 1000) + 60, op: 'get', n: 'nonce' },
feat(storage): pluggable s3-or-filesystem backend + migration CLI + admin UI Phase 6a from docs/berth-recommender-and-pdf-plan.md §4.7a + §14.9a. Lays the storage groundwork for Phase 6b/7 file-bearing schemas (per-berth PDFs, brochures) without touching those domains yet. New files: - src/lib/storage/index.ts StorageBackend interface + per-process factory keyed on system_settings. - src/lib/storage/s3.ts S3-compatible backend (MinIO/AWS/B2/R2/ Wasabi/Tigris) wrapping the existing minio JS client. Includes a healthCheck() used by the admin "Test connection" button. - src/lib/storage/filesystem.ts Local filesystem backend with all §14.9a mitigations baked in. - src/lib/storage/migrate.ts Shared migration core — pg_advisory_lock, per-row resumable progress markers, sha256 round-trip verification, atomic storage_backend flip on success. - scripts/migrate-storage.ts Thin CLI shim around runMigration(). - src/app/api/storage/[token]/route.ts Filesystem proxy GET. Verifies HMAC, enforces single-use replay protection via Redis SET NX, streams via NextResponse ReadableStream with explicit Content-Type + Content-Disposition. Node runtime only. - src/app/api/v1/admin/storage/route.ts GET status + POST connection test. - src/app/api/v1/admin/storage/migrate/route.ts Super-admin-only POST that runs the exact same runMigration() as the CLI. - src/app/(dashboard)/[portSlug]/admin/storage/page.tsx Super-admin admin UI (current backend, capacity stats, switch button with dry-run, test connection, backup hint). - src/components/admin/storage-admin-panel.tsx Client component for the page above. §14.9a critical mitigations implemented: - Path-traversal: storage keys validated against ^[a-zA-Z0-9/_.-]+$; `..`, `.`, `//`, leading `/`, and overlength keys rejected. - Realpath: storage root realpath'd at create time, every per-key resolution checked against the realpath'd prefix. - Storage root created (or chmod'd) to 0o700. - Multi-node refusal: FilesystemBackend.create() throws when MULTI_NODE_DEPLOYMENT=true. - HMAC token: sha256-HMAC over the (key, expiry, nonce, filename, content-type) payload. Verified with timingSafeEqual; bad sig, expired, or invalid-key payloads all return 403. - Single-use replay: token body cached in Redis SET NX EX 1800s. - sha256 round-trip: copyAndVerify() re-fetches from the target after put() and aborts the migration on any mismatch. - Free-disk pre-flight: when migrating to filesystem, sums byte counts via source.head() and aborts if free space < total * 1.2. - pg_advisory_lock(0xc7000a01) prevents concurrent migrations. - Resumable: per-row progress markers in _storage_migration_progress. system_settings keys read by the factory (jsonb, no schema change): storage_backend, storage_s3_endpoint, storage_s3_region, storage_s3_bucket, storage_s3_access_key, storage_s3_secret_key_encrypted, storage_s3_force_path_style, storage_filesystem_root, storage_proxy_hmac_secret_encrypted. Defaults: storage_backend=`s3`, storage_filesystem_root=`./storage` (./storage added to .gitignore). Tests added (34 tests, all green): - tests/unit/storage/filesystem-backend.test.ts — key validation allow/reject matrix, realpath escape, 0o700 perms, multi-node refusal, HMAC token sign/verify/tamper/expire/invalid-key. - tests/unit/storage/copy-and-verify.test.ts — sha256 mismatch on round-trip aborts the migration. - tests/integration/storage/proxy-route.test.ts — happy path, wrong HMAC secret, expired token, replay rejection. Phase 6a ships zero file-bearing tables — TABLES_WITH_STORAGE_KEYS is intentionally empty. berth_pdf_versions and brochure_versions land in Phase 6b and join the list there. Existing s3_key columns: only gdpr_export_jobs.storage_key, already named correctly — no rename needed. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-05 03:15:59 +02:00
secret,
);
const parts = t.split('.');
const body = parts[0] ?? '';
const sig = parts[1] ?? '';
const tampered = `${body}.${sig.slice(0, -2)}aa`;
fix(audit-v2): platform-wide post-merge hardening across 5 domains Five-domain audit (security, routes, DB, integrations, UI/UX) ran after the cf37d09 merge. Critical + high-impact items landed here; deferred medium/low items indexed in docs/audit-final-deferred.md (now organised into a "Audit-final v2" section). Security: - Storage proxy tokens now bind to op (`'get'` vs `'put'`). A long-lived download URL minted by `presignDownload` for an emailed brochure can no longer be replayed against the proxy PUT to overwrite the original storage object. `verifyProxyToken` requires `expectedOp` and rejects mismatches; legacy tokens missing `op` fail-closed. Regression tests added. - Markdown email merge values are now markdown-escaped (`[`, `]`, `(`, `)`, `*`, `_`, `\`, backticks, braces) before substitution into the rep-authored body. A malicious value like `[click here](https://evil)` stored in `client.fullName` no longer survives `escapeHtml` to render as a real `<a href>` in the outbound email. Phishing-via-merge-field closed; regression tests added. - Middleware now performs an Origin/Referer check on POST/PUT/PATCH/DELETE to `/api/v1/**`. Defense-in-depth on top of better-auth's SameSite=Lax cookie. Webhooks/public/auth/portal routes exempt as they don't carry the session cookie. Routes: - Template management routes were calling `withPermission('documents', 'manage', ...)` — but `documents` doesn't have a `manage` action. The registry has `document_templates.manage`. Every non-superadmin was getting 403'd on the seven template endpoints. Fixed across the /admin/templates surface. - Custom-fields permission resource is hardcoded to `clients` regardless of which entity (yacht/company/etc.) the values belong to. Documented as deferred (requires per-entity routes). DB: - documentSends: every parent FK (client_id, interest_id, berth_id, brochure_id, brochure_version_id) now uses ON DELETE SET NULL so the audit trail outlasts hard-deletes. The denormalized columns (recipient_email, document_kind, body_markdown, from_address) were added precisely for this. Migration 0035. - Polymorphic discriminators on yachts.current_owner_type and invoices.billing_entity_type now have CHECK constraints — typos like `'clients'` vs `'client'` were silently inserting unreachable rows before. Migration 0036. Integrations: - Email attachment resolution (`src/lib/email/index.ts`) was importing MinIO directly instead of `getStorageBackend()`. Filesystem-backend deployments would have broken every email-with-attachment send. Now routes through the pluggable abstraction per CLAUDE.md. - Documenso DOCUMENT_OPENED webhook filter relaxed: v2 may omit `readStatus` or send lowercase, so an event that was the SIGNAL of an open was being silently dropped. Now treats any recipient on a DOCUMENT_OPENED event as opened. UI/UX: - Expense detail used to render `receiptFileIds` as opaque UUID badges — reps couldn't view the receipt they uploaded. Now renders an image thumbnail (via `/api/v1/files/[id]/preview`) plus a Download link for PDFs. Closed the "where's my receipt?" loop in the expense flow. - Expense detail Edit + Archive buttons now `<PermissionGate>` and the archive mutation surfaces success/error toasts instead of silent 403s. - Brochures admin: setDefault/archive/create mutations now have onError toasts (only onSuccess existed before). - Removed broken bulk-upload link in scan/page (route doesn't exist; used a raw `<a>` triggering a full reload to a 404). Test status: 1168/1168 vitest passing. tsc clean. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-05 05:51:39 +02:00
const r = verifyProxyToken(tampered, secret, 'get');
feat(storage): pluggable s3-or-filesystem backend + migration CLI + admin UI Phase 6a from docs/berth-recommender-and-pdf-plan.md §4.7a + §14.9a. Lays the storage groundwork for Phase 6b/7 file-bearing schemas (per-berth PDFs, brochures) without touching those domains yet. New files: - src/lib/storage/index.ts StorageBackend interface + per-process factory keyed on system_settings. - src/lib/storage/s3.ts S3-compatible backend (MinIO/AWS/B2/R2/ Wasabi/Tigris) wrapping the existing minio JS client. Includes a healthCheck() used by the admin "Test connection" button. - src/lib/storage/filesystem.ts Local filesystem backend with all §14.9a mitigations baked in. - src/lib/storage/migrate.ts Shared migration core — pg_advisory_lock, per-row resumable progress markers, sha256 round-trip verification, atomic storage_backend flip on success. - scripts/migrate-storage.ts Thin CLI shim around runMigration(). - src/app/api/storage/[token]/route.ts Filesystem proxy GET. Verifies HMAC, enforces single-use replay protection via Redis SET NX, streams via NextResponse ReadableStream with explicit Content-Type + Content-Disposition. Node runtime only. - src/app/api/v1/admin/storage/route.ts GET status + POST connection test. - src/app/api/v1/admin/storage/migrate/route.ts Super-admin-only POST that runs the exact same runMigration() as the CLI. - src/app/(dashboard)/[portSlug]/admin/storage/page.tsx Super-admin admin UI (current backend, capacity stats, switch button with dry-run, test connection, backup hint). - src/components/admin/storage-admin-panel.tsx Client component for the page above. §14.9a critical mitigations implemented: - Path-traversal: storage keys validated against ^[a-zA-Z0-9/_.-]+$; `..`, `.`, `//`, leading `/`, and overlength keys rejected. - Realpath: storage root realpath'd at create time, every per-key resolution checked against the realpath'd prefix. - Storage root created (or chmod'd) to 0o700. - Multi-node refusal: FilesystemBackend.create() throws when MULTI_NODE_DEPLOYMENT=true. - HMAC token: sha256-HMAC over the (key, expiry, nonce, filename, content-type) payload. Verified with timingSafeEqual; bad sig, expired, or invalid-key payloads all return 403. - Single-use replay: token body cached in Redis SET NX EX 1800s. - sha256 round-trip: copyAndVerify() re-fetches from the target after put() and aborts the migration on any mismatch. - Free-disk pre-flight: when migrating to filesystem, sums byte counts via source.head() and aborts if free space < total * 1.2. - pg_advisory_lock(0xc7000a01) prevents concurrent migrations. - Resumable: per-row progress markers in _storage_migration_progress. system_settings keys read by the factory (jsonb, no schema change): storage_backend, storage_s3_endpoint, storage_s3_region, storage_s3_bucket, storage_s3_access_key, storage_s3_secret_key_encrypted, storage_s3_force_path_style, storage_filesystem_root, storage_proxy_hmac_secret_encrypted. Defaults: storage_backend=`s3`, storage_filesystem_root=`./storage` (./storage added to .gitignore). Tests added (34 tests, all green): - tests/unit/storage/filesystem-backend.test.ts — key validation allow/reject matrix, realpath escape, 0o700 perms, multi-node refusal, HMAC token sign/verify/tamper/expire/invalid-key. - tests/unit/storage/copy-and-verify.test.ts — sha256 mismatch on round-trip aborts the migration. - tests/integration/storage/proxy-route.test.ts — happy path, wrong HMAC secret, expired token, replay rejection. Phase 6a ships zero file-bearing tables — TABLES_WITH_STORAGE_KEYS is intentionally empty. berth_pdf_versions and brochure_versions land in Phase 6b and join the list there. Existing s3_key columns: only gdpr_export_jobs.storage_key, already named correctly — no rename needed. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-05 03:15:59 +02:00
expect(r.ok).toBe(false);
});
it('wrong secret fails', () => {
const t = signProxyToken(
fix(audit-v2): platform-wide post-merge hardening across 5 domains Five-domain audit (security, routes, DB, integrations, UI/UX) ran after the cf37d09 merge. Critical + high-impact items landed here; deferred medium/low items indexed in docs/audit-final-deferred.md (now organised into a "Audit-final v2" section). Security: - Storage proxy tokens now bind to op (`'get'` vs `'put'`). A long-lived download URL minted by `presignDownload` for an emailed brochure can no longer be replayed against the proxy PUT to overwrite the original storage object. `verifyProxyToken` requires `expectedOp` and rejects mismatches; legacy tokens missing `op` fail-closed. Regression tests added. - Markdown email merge values are now markdown-escaped (`[`, `]`, `(`, `)`, `*`, `_`, `\`, backticks, braces) before substitution into the rep-authored body. A malicious value like `[click here](https://evil)` stored in `client.fullName` no longer survives `escapeHtml` to render as a real `<a href>` in the outbound email. Phishing-via-merge-field closed; regression tests added. - Middleware now performs an Origin/Referer check on POST/PUT/PATCH/DELETE to `/api/v1/**`. Defense-in-depth on top of better-auth's SameSite=Lax cookie. Webhooks/public/auth/portal routes exempt as they don't carry the session cookie. Routes: - Template management routes were calling `withPermission('documents', 'manage', ...)` — but `documents` doesn't have a `manage` action. The registry has `document_templates.manage`. Every non-superadmin was getting 403'd on the seven template endpoints. Fixed across the /admin/templates surface. - Custom-fields permission resource is hardcoded to `clients` regardless of which entity (yacht/company/etc.) the values belong to. Documented as deferred (requires per-entity routes). DB: - documentSends: every parent FK (client_id, interest_id, berth_id, brochure_id, brochure_version_id) now uses ON DELETE SET NULL so the audit trail outlasts hard-deletes. The denormalized columns (recipient_email, document_kind, body_markdown, from_address) were added precisely for this. Migration 0035. - Polymorphic discriminators on yachts.current_owner_type and invoices.billing_entity_type now have CHECK constraints — typos like `'clients'` vs `'client'` were silently inserting unreachable rows before. Migration 0036. Integrations: - Email attachment resolution (`src/lib/email/index.ts`) was importing MinIO directly instead of `getStorageBackend()`. Filesystem-backend deployments would have broken every email-with-attachment send. Now routes through the pluggable abstraction per CLAUDE.md. - Documenso DOCUMENT_OPENED webhook filter relaxed: v2 may omit `readStatus` or send lowercase, so an event that was the SIGNAL of an open was being silently dropped. Now treats any recipient on a DOCUMENT_OPENED event as opened. UI/UX: - Expense detail used to render `receiptFileIds` as opaque UUID badges — reps couldn't view the receipt they uploaded. Now renders an image thumbnail (via `/api/v1/files/[id]/preview`) plus a Download link for PDFs. Closed the "where's my receipt?" loop in the expense flow. - Expense detail Edit + Archive buttons now `<PermissionGate>` and the archive mutation surfaces success/error toasts instead of silent 403s. - Brochures admin: setDefault/archive/create mutations now have onError toasts (only onSuccess existed before). - Removed broken bulk-upload link in scan/page (route doesn't exist; used a raw `<a>` triggering a full reload to a 404). Test status: 1168/1168 vitest passing. tsc clean. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-05 05:51:39 +02:00
{ k: 'berths/abc/file.pdf', e: Math.floor(Date.now() / 1000) + 60, op: 'get', n: 'n' },
feat(storage): pluggable s3-or-filesystem backend + migration CLI + admin UI Phase 6a from docs/berth-recommender-and-pdf-plan.md §4.7a + §14.9a. Lays the storage groundwork for Phase 6b/7 file-bearing schemas (per-berth PDFs, brochures) without touching those domains yet. New files: - src/lib/storage/index.ts StorageBackend interface + per-process factory keyed on system_settings. - src/lib/storage/s3.ts S3-compatible backend (MinIO/AWS/B2/R2/ Wasabi/Tigris) wrapping the existing minio JS client. Includes a healthCheck() used by the admin "Test connection" button. - src/lib/storage/filesystem.ts Local filesystem backend with all §14.9a mitigations baked in. - src/lib/storage/migrate.ts Shared migration core — pg_advisory_lock, per-row resumable progress markers, sha256 round-trip verification, atomic storage_backend flip on success. - scripts/migrate-storage.ts Thin CLI shim around runMigration(). - src/app/api/storage/[token]/route.ts Filesystem proxy GET. Verifies HMAC, enforces single-use replay protection via Redis SET NX, streams via NextResponse ReadableStream with explicit Content-Type + Content-Disposition. Node runtime only. - src/app/api/v1/admin/storage/route.ts GET status + POST connection test. - src/app/api/v1/admin/storage/migrate/route.ts Super-admin-only POST that runs the exact same runMigration() as the CLI. - src/app/(dashboard)/[portSlug]/admin/storage/page.tsx Super-admin admin UI (current backend, capacity stats, switch button with dry-run, test connection, backup hint). - src/components/admin/storage-admin-panel.tsx Client component for the page above. §14.9a critical mitigations implemented: - Path-traversal: storage keys validated against ^[a-zA-Z0-9/_.-]+$; `..`, `.`, `//`, leading `/`, and overlength keys rejected. - Realpath: storage root realpath'd at create time, every per-key resolution checked against the realpath'd prefix. - Storage root created (or chmod'd) to 0o700. - Multi-node refusal: FilesystemBackend.create() throws when MULTI_NODE_DEPLOYMENT=true. - HMAC token: sha256-HMAC over the (key, expiry, nonce, filename, content-type) payload. Verified with timingSafeEqual; bad sig, expired, or invalid-key payloads all return 403. - Single-use replay: token body cached in Redis SET NX EX 1800s. - sha256 round-trip: copyAndVerify() re-fetches from the target after put() and aborts the migration on any mismatch. - Free-disk pre-flight: when migrating to filesystem, sums byte counts via source.head() and aborts if free space < total * 1.2. - pg_advisory_lock(0xc7000a01) prevents concurrent migrations. - Resumable: per-row progress markers in _storage_migration_progress. system_settings keys read by the factory (jsonb, no schema change): storage_backend, storage_s3_endpoint, storage_s3_region, storage_s3_bucket, storage_s3_access_key, storage_s3_secret_key_encrypted, storage_s3_force_path_style, storage_filesystem_root, storage_proxy_hmac_secret_encrypted. Defaults: storage_backend=`s3`, storage_filesystem_root=`./storage` (./storage added to .gitignore). Tests added (34 tests, all green): - tests/unit/storage/filesystem-backend.test.ts — key validation allow/reject matrix, realpath escape, 0o700 perms, multi-node refusal, HMAC token sign/verify/tamper/expire/invalid-key. - tests/unit/storage/copy-and-verify.test.ts — sha256 mismatch on round-trip aborts the migration. - tests/integration/storage/proxy-route.test.ts — happy path, wrong HMAC secret, expired token, replay rejection. Phase 6a ships zero file-bearing tables — TABLES_WITH_STORAGE_KEYS is intentionally empty. berth_pdf_versions and brochure_versions land in Phase 6b and join the list there. Existing s3_key columns: only gdpr_export_jobs.storage_key, already named correctly — no rename needed. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-05 03:15:59 +02:00
secret,
);
fix(audit-v2): platform-wide post-merge hardening across 5 domains Five-domain audit (security, routes, DB, integrations, UI/UX) ran after the cf37d09 merge. Critical + high-impact items landed here; deferred medium/low items indexed in docs/audit-final-deferred.md (now organised into a "Audit-final v2" section). Security: - Storage proxy tokens now bind to op (`'get'` vs `'put'`). A long-lived download URL minted by `presignDownload` for an emailed brochure can no longer be replayed against the proxy PUT to overwrite the original storage object. `verifyProxyToken` requires `expectedOp` and rejects mismatches; legacy tokens missing `op` fail-closed. Regression tests added. - Markdown email merge values are now markdown-escaped (`[`, `]`, `(`, `)`, `*`, `_`, `\`, backticks, braces) before substitution into the rep-authored body. A malicious value like `[click here](https://evil)` stored in `client.fullName` no longer survives `escapeHtml` to render as a real `<a href>` in the outbound email. Phishing-via-merge-field closed; regression tests added. - Middleware now performs an Origin/Referer check on POST/PUT/PATCH/DELETE to `/api/v1/**`. Defense-in-depth on top of better-auth's SameSite=Lax cookie. Webhooks/public/auth/portal routes exempt as they don't carry the session cookie. Routes: - Template management routes were calling `withPermission('documents', 'manage', ...)` — but `documents` doesn't have a `manage` action. The registry has `document_templates.manage`. Every non-superadmin was getting 403'd on the seven template endpoints. Fixed across the /admin/templates surface. - Custom-fields permission resource is hardcoded to `clients` regardless of which entity (yacht/company/etc.) the values belong to. Documented as deferred (requires per-entity routes). DB: - documentSends: every parent FK (client_id, interest_id, berth_id, brochure_id, brochure_version_id) now uses ON DELETE SET NULL so the audit trail outlasts hard-deletes. The denormalized columns (recipient_email, document_kind, body_markdown, from_address) were added precisely for this. Migration 0035. - Polymorphic discriminators on yachts.current_owner_type and invoices.billing_entity_type now have CHECK constraints — typos like `'clients'` vs `'client'` were silently inserting unreachable rows before. Migration 0036. Integrations: - Email attachment resolution (`src/lib/email/index.ts`) was importing MinIO directly instead of `getStorageBackend()`. Filesystem-backend deployments would have broken every email-with-attachment send. Now routes through the pluggable abstraction per CLAUDE.md. - Documenso DOCUMENT_OPENED webhook filter relaxed: v2 may omit `readStatus` or send lowercase, so an event that was the SIGNAL of an open was being silently dropped. Now treats any recipient on a DOCUMENT_OPENED event as opened. UI/UX: - Expense detail used to render `receiptFileIds` as opaque UUID badges — reps couldn't view the receipt they uploaded. Now renders an image thumbnail (via `/api/v1/files/[id]/preview`) plus a Download link for PDFs. Closed the "where's my receipt?" loop in the expense flow. - Expense detail Edit + Archive buttons now `<PermissionGate>` and the archive mutation surfaces success/error toasts instead of silent 403s. - Brochures admin: setDefault/archive/create mutations now have onError toasts (only onSuccess existed before). - Removed broken bulk-upload link in scan/page (route doesn't exist; used a raw `<a>` triggering a full reload to a 404). Test status: 1168/1168 vitest passing. tsc clean. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-05 05:51:39 +02:00
const r = verifyProxyToken(t, 'other-secret', 'get');
feat(storage): pluggable s3-or-filesystem backend + migration CLI + admin UI Phase 6a from docs/berth-recommender-and-pdf-plan.md §4.7a + §14.9a. Lays the storage groundwork for Phase 6b/7 file-bearing schemas (per-berth PDFs, brochures) without touching those domains yet. New files: - src/lib/storage/index.ts StorageBackend interface + per-process factory keyed on system_settings. - src/lib/storage/s3.ts S3-compatible backend (MinIO/AWS/B2/R2/ Wasabi/Tigris) wrapping the existing minio JS client. Includes a healthCheck() used by the admin "Test connection" button. - src/lib/storage/filesystem.ts Local filesystem backend with all §14.9a mitigations baked in. - src/lib/storage/migrate.ts Shared migration core — pg_advisory_lock, per-row resumable progress markers, sha256 round-trip verification, atomic storage_backend flip on success. - scripts/migrate-storage.ts Thin CLI shim around runMigration(). - src/app/api/storage/[token]/route.ts Filesystem proxy GET. Verifies HMAC, enforces single-use replay protection via Redis SET NX, streams via NextResponse ReadableStream with explicit Content-Type + Content-Disposition. Node runtime only. - src/app/api/v1/admin/storage/route.ts GET status + POST connection test. - src/app/api/v1/admin/storage/migrate/route.ts Super-admin-only POST that runs the exact same runMigration() as the CLI. - src/app/(dashboard)/[portSlug]/admin/storage/page.tsx Super-admin admin UI (current backend, capacity stats, switch button with dry-run, test connection, backup hint). - src/components/admin/storage-admin-panel.tsx Client component for the page above. §14.9a critical mitigations implemented: - Path-traversal: storage keys validated against ^[a-zA-Z0-9/_.-]+$; `..`, `.`, `//`, leading `/`, and overlength keys rejected. - Realpath: storage root realpath'd at create time, every per-key resolution checked against the realpath'd prefix. - Storage root created (or chmod'd) to 0o700. - Multi-node refusal: FilesystemBackend.create() throws when MULTI_NODE_DEPLOYMENT=true. - HMAC token: sha256-HMAC over the (key, expiry, nonce, filename, content-type) payload. Verified with timingSafeEqual; bad sig, expired, or invalid-key payloads all return 403. - Single-use replay: token body cached in Redis SET NX EX 1800s. - sha256 round-trip: copyAndVerify() re-fetches from the target after put() and aborts the migration on any mismatch. - Free-disk pre-flight: when migrating to filesystem, sums byte counts via source.head() and aborts if free space < total * 1.2. - pg_advisory_lock(0xc7000a01) prevents concurrent migrations. - Resumable: per-row progress markers in _storage_migration_progress. system_settings keys read by the factory (jsonb, no schema change): storage_backend, storage_s3_endpoint, storage_s3_region, storage_s3_bucket, storage_s3_access_key, storage_s3_secret_key_encrypted, storage_s3_force_path_style, storage_filesystem_root, storage_proxy_hmac_secret_encrypted. Defaults: storage_backend=`s3`, storage_filesystem_root=`./storage` (./storage added to .gitignore). Tests added (34 tests, all green): - tests/unit/storage/filesystem-backend.test.ts — key validation allow/reject matrix, realpath escape, 0o700 perms, multi-node refusal, HMAC token sign/verify/tamper/expire/invalid-key. - tests/unit/storage/copy-and-verify.test.ts — sha256 mismatch on round-trip aborts the migration. - tests/integration/storage/proxy-route.test.ts — happy path, wrong HMAC secret, expired token, replay rejection. Phase 6a ships zero file-bearing tables — TABLES_WITH_STORAGE_KEYS is intentionally empty. berth_pdf_versions and brochure_versions land in Phase 6b and join the list there. Existing s3_key columns: only gdpr_export_jobs.storage_key, already named correctly — no rename needed. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-05 03:15:59 +02:00
expect(r.ok).toBe(false);
});
it('expired token fails', () => {
const t = signProxyToken(
fix(audit-v2): platform-wide post-merge hardening across 5 domains Five-domain audit (security, routes, DB, integrations, UI/UX) ran after the cf37d09 merge. Critical + high-impact items landed here; deferred medium/low items indexed in docs/audit-final-deferred.md (now organised into a "Audit-final v2" section). Security: - Storage proxy tokens now bind to op (`'get'` vs `'put'`). A long-lived download URL minted by `presignDownload` for an emailed brochure can no longer be replayed against the proxy PUT to overwrite the original storage object. `verifyProxyToken` requires `expectedOp` and rejects mismatches; legacy tokens missing `op` fail-closed. Regression tests added. - Markdown email merge values are now markdown-escaped (`[`, `]`, `(`, `)`, `*`, `_`, `\`, backticks, braces) before substitution into the rep-authored body. A malicious value like `[click here](https://evil)` stored in `client.fullName` no longer survives `escapeHtml` to render as a real `<a href>` in the outbound email. Phishing-via-merge-field closed; regression tests added. - Middleware now performs an Origin/Referer check on POST/PUT/PATCH/DELETE to `/api/v1/**`. Defense-in-depth on top of better-auth's SameSite=Lax cookie. Webhooks/public/auth/portal routes exempt as they don't carry the session cookie. Routes: - Template management routes were calling `withPermission('documents', 'manage', ...)` — but `documents` doesn't have a `manage` action. The registry has `document_templates.manage`. Every non-superadmin was getting 403'd on the seven template endpoints. Fixed across the /admin/templates surface. - Custom-fields permission resource is hardcoded to `clients` regardless of which entity (yacht/company/etc.) the values belong to. Documented as deferred (requires per-entity routes). DB: - documentSends: every parent FK (client_id, interest_id, berth_id, brochure_id, brochure_version_id) now uses ON DELETE SET NULL so the audit trail outlasts hard-deletes. The denormalized columns (recipient_email, document_kind, body_markdown, from_address) were added precisely for this. Migration 0035. - Polymorphic discriminators on yachts.current_owner_type and invoices.billing_entity_type now have CHECK constraints — typos like `'clients'` vs `'client'` were silently inserting unreachable rows before. Migration 0036. Integrations: - Email attachment resolution (`src/lib/email/index.ts`) was importing MinIO directly instead of `getStorageBackend()`. Filesystem-backend deployments would have broken every email-with-attachment send. Now routes through the pluggable abstraction per CLAUDE.md. - Documenso DOCUMENT_OPENED webhook filter relaxed: v2 may omit `readStatus` or send lowercase, so an event that was the SIGNAL of an open was being silently dropped. Now treats any recipient on a DOCUMENT_OPENED event as opened. UI/UX: - Expense detail used to render `receiptFileIds` as opaque UUID badges — reps couldn't view the receipt they uploaded. Now renders an image thumbnail (via `/api/v1/files/[id]/preview`) plus a Download link for PDFs. Closed the "where's my receipt?" loop in the expense flow. - Expense detail Edit + Archive buttons now `<PermissionGate>` and the archive mutation surfaces success/error toasts instead of silent 403s. - Brochures admin: setDefault/archive/create mutations now have onError toasts (only onSuccess existed before). - Removed broken bulk-upload link in scan/page (route doesn't exist; used a raw `<a>` triggering a full reload to a 404). Test status: 1168/1168 vitest passing. tsc clean. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-05 05:51:39 +02:00
{ k: 'berths/abc/file.pdf', e: Math.floor(Date.now() / 1000) - 10, op: 'get', n: 'n' },
feat(storage): pluggable s3-or-filesystem backend + migration CLI + admin UI Phase 6a from docs/berth-recommender-and-pdf-plan.md §4.7a + §14.9a. Lays the storage groundwork for Phase 6b/7 file-bearing schemas (per-berth PDFs, brochures) without touching those domains yet. New files: - src/lib/storage/index.ts StorageBackend interface + per-process factory keyed on system_settings. - src/lib/storage/s3.ts S3-compatible backend (MinIO/AWS/B2/R2/ Wasabi/Tigris) wrapping the existing minio JS client. Includes a healthCheck() used by the admin "Test connection" button. - src/lib/storage/filesystem.ts Local filesystem backend with all §14.9a mitigations baked in. - src/lib/storage/migrate.ts Shared migration core — pg_advisory_lock, per-row resumable progress markers, sha256 round-trip verification, atomic storage_backend flip on success. - scripts/migrate-storage.ts Thin CLI shim around runMigration(). - src/app/api/storage/[token]/route.ts Filesystem proxy GET. Verifies HMAC, enforces single-use replay protection via Redis SET NX, streams via NextResponse ReadableStream with explicit Content-Type + Content-Disposition. Node runtime only. - src/app/api/v1/admin/storage/route.ts GET status + POST connection test. - src/app/api/v1/admin/storage/migrate/route.ts Super-admin-only POST that runs the exact same runMigration() as the CLI. - src/app/(dashboard)/[portSlug]/admin/storage/page.tsx Super-admin admin UI (current backend, capacity stats, switch button with dry-run, test connection, backup hint). - src/components/admin/storage-admin-panel.tsx Client component for the page above. §14.9a critical mitigations implemented: - Path-traversal: storage keys validated against ^[a-zA-Z0-9/_.-]+$; `..`, `.`, `//`, leading `/`, and overlength keys rejected. - Realpath: storage root realpath'd at create time, every per-key resolution checked against the realpath'd prefix. - Storage root created (or chmod'd) to 0o700. - Multi-node refusal: FilesystemBackend.create() throws when MULTI_NODE_DEPLOYMENT=true. - HMAC token: sha256-HMAC over the (key, expiry, nonce, filename, content-type) payload. Verified with timingSafeEqual; bad sig, expired, or invalid-key payloads all return 403. - Single-use replay: token body cached in Redis SET NX EX 1800s. - sha256 round-trip: copyAndVerify() re-fetches from the target after put() and aborts the migration on any mismatch. - Free-disk pre-flight: when migrating to filesystem, sums byte counts via source.head() and aborts if free space < total * 1.2. - pg_advisory_lock(0xc7000a01) prevents concurrent migrations. - Resumable: per-row progress markers in _storage_migration_progress. system_settings keys read by the factory (jsonb, no schema change): storage_backend, storage_s3_endpoint, storage_s3_region, storage_s3_bucket, storage_s3_access_key, storage_s3_secret_key_encrypted, storage_s3_force_path_style, storage_filesystem_root, storage_proxy_hmac_secret_encrypted. Defaults: storage_backend=`s3`, storage_filesystem_root=`./storage` (./storage added to .gitignore). Tests added (34 tests, all green): - tests/unit/storage/filesystem-backend.test.ts — key validation allow/reject matrix, realpath escape, 0o700 perms, multi-node refusal, HMAC token sign/verify/tamper/expire/invalid-key. - tests/unit/storage/copy-and-verify.test.ts — sha256 mismatch on round-trip aborts the migration. - tests/integration/storage/proxy-route.test.ts — happy path, wrong HMAC secret, expired token, replay rejection. Phase 6a ships zero file-bearing tables — TABLES_WITH_STORAGE_KEYS is intentionally empty. berth_pdf_versions and brochure_versions land in Phase 6b and join the list there. Existing s3_key columns: only gdpr_export_jobs.storage_key, already named correctly — no rename needed. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-05 03:15:59 +02:00
secret,
);
fix(audit-v2): platform-wide post-merge hardening across 5 domains Five-domain audit (security, routes, DB, integrations, UI/UX) ran after the cf37d09 merge. Critical + high-impact items landed here; deferred medium/low items indexed in docs/audit-final-deferred.md (now organised into a "Audit-final v2" section). Security: - Storage proxy tokens now bind to op (`'get'` vs `'put'`). A long-lived download URL minted by `presignDownload` for an emailed brochure can no longer be replayed against the proxy PUT to overwrite the original storage object. `verifyProxyToken` requires `expectedOp` and rejects mismatches; legacy tokens missing `op` fail-closed. Regression tests added. - Markdown email merge values are now markdown-escaped (`[`, `]`, `(`, `)`, `*`, `_`, `\`, backticks, braces) before substitution into the rep-authored body. A malicious value like `[click here](https://evil)` stored in `client.fullName` no longer survives `escapeHtml` to render as a real `<a href>` in the outbound email. Phishing-via-merge-field closed; regression tests added. - Middleware now performs an Origin/Referer check on POST/PUT/PATCH/DELETE to `/api/v1/**`. Defense-in-depth on top of better-auth's SameSite=Lax cookie. Webhooks/public/auth/portal routes exempt as they don't carry the session cookie. Routes: - Template management routes were calling `withPermission('documents', 'manage', ...)` — but `documents` doesn't have a `manage` action. The registry has `document_templates.manage`. Every non-superadmin was getting 403'd on the seven template endpoints. Fixed across the /admin/templates surface. - Custom-fields permission resource is hardcoded to `clients` regardless of which entity (yacht/company/etc.) the values belong to. Documented as deferred (requires per-entity routes). DB: - documentSends: every parent FK (client_id, interest_id, berth_id, brochure_id, brochure_version_id) now uses ON DELETE SET NULL so the audit trail outlasts hard-deletes. The denormalized columns (recipient_email, document_kind, body_markdown, from_address) were added precisely for this. Migration 0035. - Polymorphic discriminators on yachts.current_owner_type and invoices.billing_entity_type now have CHECK constraints — typos like `'clients'` vs `'client'` were silently inserting unreachable rows before. Migration 0036. Integrations: - Email attachment resolution (`src/lib/email/index.ts`) was importing MinIO directly instead of `getStorageBackend()`. Filesystem-backend deployments would have broken every email-with-attachment send. Now routes through the pluggable abstraction per CLAUDE.md. - Documenso DOCUMENT_OPENED webhook filter relaxed: v2 may omit `readStatus` or send lowercase, so an event that was the SIGNAL of an open was being silently dropped. Now treats any recipient on a DOCUMENT_OPENED event as opened. UI/UX: - Expense detail used to render `receiptFileIds` as opaque UUID badges — reps couldn't view the receipt they uploaded. Now renders an image thumbnail (via `/api/v1/files/[id]/preview`) plus a Download link for PDFs. Closed the "where's my receipt?" loop in the expense flow. - Expense detail Edit + Archive buttons now `<PermissionGate>` and the archive mutation surfaces success/error toasts instead of silent 403s. - Brochures admin: setDefault/archive/create mutations now have onError toasts (only onSuccess existed before). - Removed broken bulk-upload link in scan/page (route doesn't exist; used a raw `<a>` triggering a full reload to a 404). Test status: 1168/1168 vitest passing. tsc clean. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-05 05:51:39 +02:00
const r = verifyProxyToken(t, secret, 'get');
feat(storage): pluggable s3-or-filesystem backend + migration CLI + admin UI Phase 6a from docs/berth-recommender-and-pdf-plan.md §4.7a + §14.9a. Lays the storage groundwork for Phase 6b/7 file-bearing schemas (per-berth PDFs, brochures) without touching those domains yet. New files: - src/lib/storage/index.ts StorageBackend interface + per-process factory keyed on system_settings. - src/lib/storage/s3.ts S3-compatible backend (MinIO/AWS/B2/R2/ Wasabi/Tigris) wrapping the existing minio JS client. Includes a healthCheck() used by the admin "Test connection" button. - src/lib/storage/filesystem.ts Local filesystem backend with all §14.9a mitigations baked in. - src/lib/storage/migrate.ts Shared migration core — pg_advisory_lock, per-row resumable progress markers, sha256 round-trip verification, atomic storage_backend flip on success. - scripts/migrate-storage.ts Thin CLI shim around runMigration(). - src/app/api/storage/[token]/route.ts Filesystem proxy GET. Verifies HMAC, enforces single-use replay protection via Redis SET NX, streams via NextResponse ReadableStream with explicit Content-Type + Content-Disposition. Node runtime only. - src/app/api/v1/admin/storage/route.ts GET status + POST connection test. - src/app/api/v1/admin/storage/migrate/route.ts Super-admin-only POST that runs the exact same runMigration() as the CLI. - src/app/(dashboard)/[portSlug]/admin/storage/page.tsx Super-admin admin UI (current backend, capacity stats, switch button with dry-run, test connection, backup hint). - src/components/admin/storage-admin-panel.tsx Client component for the page above. §14.9a critical mitigations implemented: - Path-traversal: storage keys validated against ^[a-zA-Z0-9/_.-]+$; `..`, `.`, `//`, leading `/`, and overlength keys rejected. - Realpath: storage root realpath'd at create time, every per-key resolution checked against the realpath'd prefix. - Storage root created (or chmod'd) to 0o700. - Multi-node refusal: FilesystemBackend.create() throws when MULTI_NODE_DEPLOYMENT=true. - HMAC token: sha256-HMAC over the (key, expiry, nonce, filename, content-type) payload. Verified with timingSafeEqual; bad sig, expired, or invalid-key payloads all return 403. - Single-use replay: token body cached in Redis SET NX EX 1800s. - sha256 round-trip: copyAndVerify() re-fetches from the target after put() and aborts the migration on any mismatch. - Free-disk pre-flight: when migrating to filesystem, sums byte counts via source.head() and aborts if free space < total * 1.2. - pg_advisory_lock(0xc7000a01) prevents concurrent migrations. - Resumable: per-row progress markers in _storage_migration_progress. system_settings keys read by the factory (jsonb, no schema change): storage_backend, storage_s3_endpoint, storage_s3_region, storage_s3_bucket, storage_s3_access_key, storage_s3_secret_key_encrypted, storage_s3_force_path_style, storage_filesystem_root, storage_proxy_hmac_secret_encrypted. Defaults: storage_backend=`s3`, storage_filesystem_root=`./storage` (./storage added to .gitignore). Tests added (34 tests, all green): - tests/unit/storage/filesystem-backend.test.ts — key validation allow/reject matrix, realpath escape, 0o700 perms, multi-node refusal, HMAC token sign/verify/tamper/expire/invalid-key. - tests/unit/storage/copy-and-verify.test.ts — sha256 mismatch on round-trip aborts the migration. - tests/integration/storage/proxy-route.test.ts — happy path, wrong HMAC secret, expired token, replay rejection. Phase 6a ships zero file-bearing tables — TABLES_WITH_STORAGE_KEYS is intentionally empty. berth_pdf_versions and brochure_versions land in Phase 6b and join the list there. Existing s3_key columns: only gdpr_export_jobs.storage_key, already named correctly — no rename needed. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-05 03:15:59 +02:00
expect(r.ok).toBe(false);
if (!r.ok) expect(r.reason).toBe('expired');
});
it('rejects payload with invalid storage key', () => {
const t = signProxyToken(
fix(audit-v2): platform-wide post-merge hardening across 5 domains Five-domain audit (security, routes, DB, integrations, UI/UX) ran after the cf37d09 merge. Critical + high-impact items landed here; deferred medium/low items indexed in docs/audit-final-deferred.md (now organised into a "Audit-final v2" section). Security: - Storage proxy tokens now bind to op (`'get'` vs `'put'`). A long-lived download URL minted by `presignDownload` for an emailed brochure can no longer be replayed against the proxy PUT to overwrite the original storage object. `verifyProxyToken` requires `expectedOp` and rejects mismatches; legacy tokens missing `op` fail-closed. Regression tests added. - Markdown email merge values are now markdown-escaped (`[`, `]`, `(`, `)`, `*`, `_`, `\`, backticks, braces) before substitution into the rep-authored body. A malicious value like `[click here](https://evil)` stored in `client.fullName` no longer survives `escapeHtml` to render as a real `<a href>` in the outbound email. Phishing-via-merge-field closed; regression tests added. - Middleware now performs an Origin/Referer check on POST/PUT/PATCH/DELETE to `/api/v1/**`. Defense-in-depth on top of better-auth's SameSite=Lax cookie. Webhooks/public/auth/portal routes exempt as they don't carry the session cookie. Routes: - Template management routes were calling `withPermission('documents', 'manage', ...)` — but `documents` doesn't have a `manage` action. The registry has `document_templates.manage`. Every non-superadmin was getting 403'd on the seven template endpoints. Fixed across the /admin/templates surface. - Custom-fields permission resource is hardcoded to `clients` regardless of which entity (yacht/company/etc.) the values belong to. Documented as deferred (requires per-entity routes). DB: - documentSends: every parent FK (client_id, interest_id, berth_id, brochure_id, brochure_version_id) now uses ON DELETE SET NULL so the audit trail outlasts hard-deletes. The denormalized columns (recipient_email, document_kind, body_markdown, from_address) were added precisely for this. Migration 0035. - Polymorphic discriminators on yachts.current_owner_type and invoices.billing_entity_type now have CHECK constraints — typos like `'clients'` vs `'client'` were silently inserting unreachable rows before. Migration 0036. Integrations: - Email attachment resolution (`src/lib/email/index.ts`) was importing MinIO directly instead of `getStorageBackend()`. Filesystem-backend deployments would have broken every email-with-attachment send. Now routes through the pluggable abstraction per CLAUDE.md. - Documenso DOCUMENT_OPENED webhook filter relaxed: v2 may omit `readStatus` or send lowercase, so an event that was the SIGNAL of an open was being silently dropped. Now treats any recipient on a DOCUMENT_OPENED event as opened. UI/UX: - Expense detail used to render `receiptFileIds` as opaque UUID badges — reps couldn't view the receipt they uploaded. Now renders an image thumbnail (via `/api/v1/files/[id]/preview`) plus a Download link for PDFs. Closed the "where's my receipt?" loop in the expense flow. - Expense detail Edit + Archive buttons now `<PermissionGate>` and the archive mutation surfaces success/error toasts instead of silent 403s. - Brochures admin: setDefault/archive/create mutations now have onError toasts (only onSuccess existed before). - Removed broken bulk-upload link in scan/page (route doesn't exist; used a raw `<a>` triggering a full reload to a 404). Test status: 1168/1168 vitest passing. tsc clean. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-05 05:51:39 +02:00
{ k: '../etc/passwd', e: Math.floor(Date.now() / 1000) + 60, op: 'get', n: 'n' },
feat(storage): pluggable s3-or-filesystem backend + migration CLI + admin UI Phase 6a from docs/berth-recommender-and-pdf-plan.md §4.7a + §14.9a. Lays the storage groundwork for Phase 6b/7 file-bearing schemas (per-berth PDFs, brochures) without touching those domains yet. New files: - src/lib/storage/index.ts StorageBackend interface + per-process factory keyed on system_settings. - src/lib/storage/s3.ts S3-compatible backend (MinIO/AWS/B2/R2/ Wasabi/Tigris) wrapping the existing minio JS client. Includes a healthCheck() used by the admin "Test connection" button. - src/lib/storage/filesystem.ts Local filesystem backend with all §14.9a mitigations baked in. - src/lib/storage/migrate.ts Shared migration core — pg_advisory_lock, per-row resumable progress markers, sha256 round-trip verification, atomic storage_backend flip on success. - scripts/migrate-storage.ts Thin CLI shim around runMigration(). - src/app/api/storage/[token]/route.ts Filesystem proxy GET. Verifies HMAC, enforces single-use replay protection via Redis SET NX, streams via NextResponse ReadableStream with explicit Content-Type + Content-Disposition. Node runtime only. - src/app/api/v1/admin/storage/route.ts GET status + POST connection test. - src/app/api/v1/admin/storage/migrate/route.ts Super-admin-only POST that runs the exact same runMigration() as the CLI. - src/app/(dashboard)/[portSlug]/admin/storage/page.tsx Super-admin admin UI (current backend, capacity stats, switch button with dry-run, test connection, backup hint). - src/components/admin/storage-admin-panel.tsx Client component for the page above. §14.9a critical mitigations implemented: - Path-traversal: storage keys validated against ^[a-zA-Z0-9/_.-]+$; `..`, `.`, `//`, leading `/`, and overlength keys rejected. - Realpath: storage root realpath'd at create time, every per-key resolution checked against the realpath'd prefix. - Storage root created (or chmod'd) to 0o700. - Multi-node refusal: FilesystemBackend.create() throws when MULTI_NODE_DEPLOYMENT=true. - HMAC token: sha256-HMAC over the (key, expiry, nonce, filename, content-type) payload. Verified with timingSafeEqual; bad sig, expired, or invalid-key payloads all return 403. - Single-use replay: token body cached in Redis SET NX EX 1800s. - sha256 round-trip: copyAndVerify() re-fetches from the target after put() and aborts the migration on any mismatch. - Free-disk pre-flight: when migrating to filesystem, sums byte counts via source.head() and aborts if free space < total * 1.2. - pg_advisory_lock(0xc7000a01) prevents concurrent migrations. - Resumable: per-row progress markers in _storage_migration_progress. system_settings keys read by the factory (jsonb, no schema change): storage_backend, storage_s3_endpoint, storage_s3_region, storage_s3_bucket, storage_s3_access_key, storage_s3_secret_key_encrypted, storage_s3_force_path_style, storage_filesystem_root, storage_proxy_hmac_secret_encrypted. Defaults: storage_backend=`s3`, storage_filesystem_root=`./storage` (./storage added to .gitignore). Tests added (34 tests, all green): - tests/unit/storage/filesystem-backend.test.ts — key validation allow/reject matrix, realpath escape, 0o700 perms, multi-node refusal, HMAC token sign/verify/tamper/expire/invalid-key. - tests/unit/storage/copy-and-verify.test.ts — sha256 mismatch on round-trip aborts the migration. - tests/integration/storage/proxy-route.test.ts — happy path, wrong HMAC secret, expired token, replay rejection. Phase 6a ships zero file-bearing tables — TABLES_WITH_STORAGE_KEYS is intentionally empty. berth_pdf_versions and brochure_versions land in Phase 6b and join the list there. Existing s3_key columns: only gdpr_export_jobs.storage_key, already named correctly — no rename needed. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-05 03:15:59 +02:00
secret,
);
fix(audit-v2): platform-wide post-merge hardening across 5 domains Five-domain audit (security, routes, DB, integrations, UI/UX) ran after the cf37d09 merge. Critical + high-impact items landed here; deferred medium/low items indexed in docs/audit-final-deferred.md (now organised into a "Audit-final v2" section). Security: - Storage proxy tokens now bind to op (`'get'` vs `'put'`). A long-lived download URL minted by `presignDownload` for an emailed brochure can no longer be replayed against the proxy PUT to overwrite the original storage object. `verifyProxyToken` requires `expectedOp` and rejects mismatches; legacy tokens missing `op` fail-closed. Regression tests added. - Markdown email merge values are now markdown-escaped (`[`, `]`, `(`, `)`, `*`, `_`, `\`, backticks, braces) before substitution into the rep-authored body. A malicious value like `[click here](https://evil)` stored in `client.fullName` no longer survives `escapeHtml` to render as a real `<a href>` in the outbound email. Phishing-via-merge-field closed; regression tests added. - Middleware now performs an Origin/Referer check on POST/PUT/PATCH/DELETE to `/api/v1/**`. Defense-in-depth on top of better-auth's SameSite=Lax cookie. Webhooks/public/auth/portal routes exempt as they don't carry the session cookie. Routes: - Template management routes were calling `withPermission('documents', 'manage', ...)` — but `documents` doesn't have a `manage` action. The registry has `document_templates.manage`. Every non-superadmin was getting 403'd on the seven template endpoints. Fixed across the /admin/templates surface. - Custom-fields permission resource is hardcoded to `clients` regardless of which entity (yacht/company/etc.) the values belong to. Documented as deferred (requires per-entity routes). DB: - documentSends: every parent FK (client_id, interest_id, berth_id, brochure_id, brochure_version_id) now uses ON DELETE SET NULL so the audit trail outlasts hard-deletes. The denormalized columns (recipient_email, document_kind, body_markdown, from_address) were added precisely for this. Migration 0035. - Polymorphic discriminators on yachts.current_owner_type and invoices.billing_entity_type now have CHECK constraints — typos like `'clients'` vs `'client'` were silently inserting unreachable rows before. Migration 0036. Integrations: - Email attachment resolution (`src/lib/email/index.ts`) was importing MinIO directly instead of `getStorageBackend()`. Filesystem-backend deployments would have broken every email-with-attachment send. Now routes through the pluggable abstraction per CLAUDE.md. - Documenso DOCUMENT_OPENED webhook filter relaxed: v2 may omit `readStatus` or send lowercase, so an event that was the SIGNAL of an open was being silently dropped. Now treats any recipient on a DOCUMENT_OPENED event as opened. UI/UX: - Expense detail used to render `receiptFileIds` as opaque UUID badges — reps couldn't view the receipt they uploaded. Now renders an image thumbnail (via `/api/v1/files/[id]/preview`) plus a Download link for PDFs. Closed the "where's my receipt?" loop in the expense flow. - Expense detail Edit + Archive buttons now `<PermissionGate>` and the archive mutation surfaces success/error toasts instead of silent 403s. - Brochures admin: setDefault/archive/create mutations now have onError toasts (only onSuccess existed before). - Removed broken bulk-upload link in scan/page (route doesn't exist; used a raw `<a>` triggering a full reload to a 404). Test status: 1168/1168 vitest passing. tsc clean. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-05 05:51:39 +02:00
const r = verifyProxyToken(t, secret, 'get');
feat(storage): pluggable s3-or-filesystem backend + migration CLI + admin UI Phase 6a from docs/berth-recommender-and-pdf-plan.md §4.7a + §14.9a. Lays the storage groundwork for Phase 6b/7 file-bearing schemas (per-berth PDFs, brochures) without touching those domains yet. New files: - src/lib/storage/index.ts StorageBackend interface + per-process factory keyed on system_settings. - src/lib/storage/s3.ts S3-compatible backend (MinIO/AWS/B2/R2/ Wasabi/Tigris) wrapping the existing minio JS client. Includes a healthCheck() used by the admin "Test connection" button. - src/lib/storage/filesystem.ts Local filesystem backend with all §14.9a mitigations baked in. - src/lib/storage/migrate.ts Shared migration core — pg_advisory_lock, per-row resumable progress markers, sha256 round-trip verification, atomic storage_backend flip on success. - scripts/migrate-storage.ts Thin CLI shim around runMigration(). - src/app/api/storage/[token]/route.ts Filesystem proxy GET. Verifies HMAC, enforces single-use replay protection via Redis SET NX, streams via NextResponse ReadableStream with explicit Content-Type + Content-Disposition. Node runtime only. - src/app/api/v1/admin/storage/route.ts GET status + POST connection test. - src/app/api/v1/admin/storage/migrate/route.ts Super-admin-only POST that runs the exact same runMigration() as the CLI. - src/app/(dashboard)/[portSlug]/admin/storage/page.tsx Super-admin admin UI (current backend, capacity stats, switch button with dry-run, test connection, backup hint). - src/components/admin/storage-admin-panel.tsx Client component for the page above. §14.9a critical mitigations implemented: - Path-traversal: storage keys validated against ^[a-zA-Z0-9/_.-]+$; `..`, `.`, `//`, leading `/`, and overlength keys rejected. - Realpath: storage root realpath'd at create time, every per-key resolution checked against the realpath'd prefix. - Storage root created (or chmod'd) to 0o700. - Multi-node refusal: FilesystemBackend.create() throws when MULTI_NODE_DEPLOYMENT=true. - HMAC token: sha256-HMAC over the (key, expiry, nonce, filename, content-type) payload. Verified with timingSafeEqual; bad sig, expired, or invalid-key payloads all return 403. - Single-use replay: token body cached in Redis SET NX EX 1800s. - sha256 round-trip: copyAndVerify() re-fetches from the target after put() and aborts the migration on any mismatch. - Free-disk pre-flight: when migrating to filesystem, sums byte counts via source.head() and aborts if free space < total * 1.2. - pg_advisory_lock(0xc7000a01) prevents concurrent migrations. - Resumable: per-row progress markers in _storage_migration_progress. system_settings keys read by the factory (jsonb, no schema change): storage_backend, storage_s3_endpoint, storage_s3_region, storage_s3_bucket, storage_s3_access_key, storage_s3_secret_key_encrypted, storage_s3_force_path_style, storage_filesystem_root, storage_proxy_hmac_secret_encrypted. Defaults: storage_backend=`s3`, storage_filesystem_root=`./storage` (./storage added to .gitignore). Tests added (34 tests, all green): - tests/unit/storage/filesystem-backend.test.ts — key validation allow/reject matrix, realpath escape, 0o700 perms, multi-node refusal, HMAC token sign/verify/tamper/expire/invalid-key. - tests/unit/storage/copy-and-verify.test.ts — sha256 mismatch on round-trip aborts the migration. - tests/integration/storage/proxy-route.test.ts — happy path, wrong HMAC secret, expired token, replay rejection. Phase 6a ships zero file-bearing tables — TABLES_WITH_STORAGE_KEYS is intentionally empty. berth_pdf_versions and brochure_versions land in Phase 6b and join the list there. Existing s3_key columns: only gdpr_export_jobs.storage_key, already named correctly — no rename needed. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-05 03:15:59 +02:00
expect(r.ok).toBe(false);
if (!r.ok) expect(r.reason).toBe('invalid-key');
});
it('malformed token shape fails', () => {
fix(audit-v2): platform-wide post-merge hardening across 5 domains Five-domain audit (security, routes, DB, integrations, UI/UX) ran after the cf37d09 merge. Critical + high-impact items landed here; deferred medium/low items indexed in docs/audit-final-deferred.md (now organised into a "Audit-final v2" section). Security: - Storage proxy tokens now bind to op (`'get'` vs `'put'`). A long-lived download URL minted by `presignDownload` for an emailed brochure can no longer be replayed against the proxy PUT to overwrite the original storage object. `verifyProxyToken` requires `expectedOp` and rejects mismatches; legacy tokens missing `op` fail-closed. Regression tests added. - Markdown email merge values are now markdown-escaped (`[`, `]`, `(`, `)`, `*`, `_`, `\`, backticks, braces) before substitution into the rep-authored body. A malicious value like `[click here](https://evil)` stored in `client.fullName` no longer survives `escapeHtml` to render as a real `<a href>` in the outbound email. Phishing-via-merge-field closed; regression tests added. - Middleware now performs an Origin/Referer check on POST/PUT/PATCH/DELETE to `/api/v1/**`. Defense-in-depth on top of better-auth's SameSite=Lax cookie. Webhooks/public/auth/portal routes exempt as they don't carry the session cookie. Routes: - Template management routes were calling `withPermission('documents', 'manage', ...)` — but `documents` doesn't have a `manage` action. The registry has `document_templates.manage`. Every non-superadmin was getting 403'd on the seven template endpoints. Fixed across the /admin/templates surface. - Custom-fields permission resource is hardcoded to `clients` regardless of which entity (yacht/company/etc.) the values belong to. Documented as deferred (requires per-entity routes). DB: - documentSends: every parent FK (client_id, interest_id, berth_id, brochure_id, brochure_version_id) now uses ON DELETE SET NULL so the audit trail outlasts hard-deletes. The denormalized columns (recipient_email, document_kind, body_markdown, from_address) were added precisely for this. Migration 0035. - Polymorphic discriminators on yachts.current_owner_type and invoices.billing_entity_type now have CHECK constraints — typos like `'clients'` vs `'client'` were silently inserting unreachable rows before. Migration 0036. Integrations: - Email attachment resolution (`src/lib/email/index.ts`) was importing MinIO directly instead of `getStorageBackend()`. Filesystem-backend deployments would have broken every email-with-attachment send. Now routes through the pluggable abstraction per CLAUDE.md. - Documenso DOCUMENT_OPENED webhook filter relaxed: v2 may omit `readStatus` or send lowercase, so an event that was the SIGNAL of an open was being silently dropped. Now treats any recipient on a DOCUMENT_OPENED event as opened. UI/UX: - Expense detail used to render `receiptFileIds` as opaque UUID badges — reps couldn't view the receipt they uploaded. Now renders an image thumbnail (via `/api/v1/files/[id]/preview`) plus a Download link for PDFs. Closed the "where's my receipt?" loop in the expense flow. - Expense detail Edit + Archive buttons now `<PermissionGate>` and the archive mutation surfaces success/error toasts instead of silent 403s. - Brochures admin: setDefault/archive/create mutations now have onError toasts (only onSuccess existed before). - Removed broken bulk-upload link in scan/page (route doesn't exist; used a raw `<a>` triggering a full reload to a 404). Test status: 1168/1168 vitest passing. tsc clean. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-05 05:51:39 +02:00
expect(verifyProxyToken('garbage', secret, 'get').ok).toBe(false);
expect(verifyProxyToken('only-one-part', secret, 'get').ok).toBe(false);
expect(verifyProxyToken('too.many.parts.here', secret, 'get').ok).toBe(false);
});
// Audit-final v2: tokens minted for download (op='get') must not be
// accepted by the upload (PUT) handler, and vice versa. Without this
// a 24h email link could be replayed against the proxy PUT to overwrite
// the original storage object.
it('rejects a get-issued token verified as put', () => {
const getToken = signProxyToken(
{ k: 'berths/abc/file.pdf', e: Math.floor(Date.now() / 1000) + 60, op: 'get', n: 'n' },
secret,
);
const r = verifyProxyToken(getToken, secret, 'put');
expect(r.ok).toBe(false);
if (!r.ok) expect(r.reason).toBe('op-mismatch');
});
it('rejects a put-issued token verified as get', () => {
const putToken = signProxyToken(
{ k: 'berths/abc/file.pdf', e: Math.floor(Date.now() / 1000) + 60, op: 'put', n: 'n' },
secret,
);
const r = verifyProxyToken(putToken, secret, 'get');
expect(r.ok).toBe(false);
if (!r.ok) expect(r.reason).toBe('op-mismatch');
feat(storage): pluggable s3-or-filesystem backend + migration CLI + admin UI Phase 6a from docs/berth-recommender-and-pdf-plan.md §4.7a + §14.9a. Lays the storage groundwork for Phase 6b/7 file-bearing schemas (per-berth PDFs, brochures) without touching those domains yet. New files: - src/lib/storage/index.ts StorageBackend interface + per-process factory keyed on system_settings. - src/lib/storage/s3.ts S3-compatible backend (MinIO/AWS/B2/R2/ Wasabi/Tigris) wrapping the existing minio JS client. Includes a healthCheck() used by the admin "Test connection" button. - src/lib/storage/filesystem.ts Local filesystem backend with all §14.9a mitigations baked in. - src/lib/storage/migrate.ts Shared migration core — pg_advisory_lock, per-row resumable progress markers, sha256 round-trip verification, atomic storage_backend flip on success. - scripts/migrate-storage.ts Thin CLI shim around runMigration(). - src/app/api/storage/[token]/route.ts Filesystem proxy GET. Verifies HMAC, enforces single-use replay protection via Redis SET NX, streams via NextResponse ReadableStream with explicit Content-Type + Content-Disposition. Node runtime only. - src/app/api/v1/admin/storage/route.ts GET status + POST connection test. - src/app/api/v1/admin/storage/migrate/route.ts Super-admin-only POST that runs the exact same runMigration() as the CLI. - src/app/(dashboard)/[portSlug]/admin/storage/page.tsx Super-admin admin UI (current backend, capacity stats, switch button with dry-run, test connection, backup hint). - src/components/admin/storage-admin-panel.tsx Client component for the page above. §14.9a critical mitigations implemented: - Path-traversal: storage keys validated against ^[a-zA-Z0-9/_.-]+$; `..`, `.`, `//`, leading `/`, and overlength keys rejected. - Realpath: storage root realpath'd at create time, every per-key resolution checked against the realpath'd prefix. - Storage root created (or chmod'd) to 0o700. - Multi-node refusal: FilesystemBackend.create() throws when MULTI_NODE_DEPLOYMENT=true. - HMAC token: sha256-HMAC over the (key, expiry, nonce, filename, content-type) payload. Verified with timingSafeEqual; bad sig, expired, or invalid-key payloads all return 403. - Single-use replay: token body cached in Redis SET NX EX 1800s. - sha256 round-trip: copyAndVerify() re-fetches from the target after put() and aborts the migration on any mismatch. - Free-disk pre-flight: when migrating to filesystem, sums byte counts via source.head() and aborts if free space < total * 1.2. - pg_advisory_lock(0xc7000a01) prevents concurrent migrations. - Resumable: per-row progress markers in _storage_migration_progress. system_settings keys read by the factory (jsonb, no schema change): storage_backend, storage_s3_endpoint, storage_s3_region, storage_s3_bucket, storage_s3_access_key, storage_s3_secret_key_encrypted, storage_s3_force_path_style, storage_filesystem_root, storage_proxy_hmac_secret_encrypted. Defaults: storage_backend=`s3`, storage_filesystem_root=`./storage` (./storage added to .gitignore). Tests added (34 tests, all green): - tests/unit/storage/filesystem-backend.test.ts — key validation allow/reject matrix, realpath escape, 0o700 perms, multi-node refusal, HMAC token sign/verify/tamper/expire/invalid-key. - tests/unit/storage/copy-and-verify.test.ts — sha256 mismatch on round-trip aborts the migration. - tests/integration/storage/proxy-route.test.ts — happy path, wrong HMAC secret, expired token, replay rejection. Phase 6a ships zero file-bearing tables — TABLES_WITH_STORAGE_KEYS is intentionally empty. berth_pdf_versions and brochure_versions land in Phase 6b and join the list there. Existing s3_key columns: only gdpr_export_jobs.storage_key, already named correctly — no rename needed. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-05 03:15:59 +02:00
});
});