Files
pn-new-crm/src/components/admin/storage-admin-panel.tsx

393 lines
15 KiB
TypeScript
Raw Normal View History

feat(storage): pluggable s3-or-filesystem backend + migration CLI + admin UI Phase 6a from docs/berth-recommender-and-pdf-plan.md §4.7a + §14.9a. Lays the storage groundwork for Phase 6b/7 file-bearing schemas (per-berth PDFs, brochures) without touching those domains yet. New files: - src/lib/storage/index.ts StorageBackend interface + per-process factory keyed on system_settings. - src/lib/storage/s3.ts S3-compatible backend (MinIO/AWS/B2/R2/ Wasabi/Tigris) wrapping the existing minio JS client. Includes a healthCheck() used by the admin "Test connection" button. - src/lib/storage/filesystem.ts Local filesystem backend with all §14.9a mitigations baked in. - src/lib/storage/migrate.ts Shared migration core — pg_advisory_lock, per-row resumable progress markers, sha256 round-trip verification, atomic storage_backend flip on success. - scripts/migrate-storage.ts Thin CLI shim around runMigration(). - src/app/api/storage/[token]/route.ts Filesystem proxy GET. Verifies HMAC, enforces single-use replay protection via Redis SET NX, streams via NextResponse ReadableStream with explicit Content-Type + Content-Disposition. Node runtime only. - src/app/api/v1/admin/storage/route.ts GET status + POST connection test. - src/app/api/v1/admin/storage/migrate/route.ts Super-admin-only POST that runs the exact same runMigration() as the CLI. - src/app/(dashboard)/[portSlug]/admin/storage/page.tsx Super-admin admin UI (current backend, capacity stats, switch button with dry-run, test connection, backup hint). - src/components/admin/storage-admin-panel.tsx Client component for the page above. §14.9a critical mitigations implemented: - Path-traversal: storage keys validated against ^[a-zA-Z0-9/_.-]+$; `..`, `.`, `//`, leading `/`, and overlength keys rejected. - Realpath: storage root realpath'd at create time, every per-key resolution checked against the realpath'd prefix. - Storage root created (or chmod'd) to 0o700. - Multi-node refusal: FilesystemBackend.create() throws when MULTI_NODE_DEPLOYMENT=true. - HMAC token: sha256-HMAC over the (key, expiry, nonce, filename, content-type) payload. Verified with timingSafeEqual; bad sig, expired, or invalid-key payloads all return 403. - Single-use replay: token body cached in Redis SET NX EX 1800s. - sha256 round-trip: copyAndVerify() re-fetches from the target after put() and aborts the migration on any mismatch. - Free-disk pre-flight: when migrating to filesystem, sums byte counts via source.head() and aborts if free space < total * 1.2. - pg_advisory_lock(0xc7000a01) prevents concurrent migrations. - Resumable: per-row progress markers in _storage_migration_progress. system_settings keys read by the factory (jsonb, no schema change): storage_backend, storage_s3_endpoint, storage_s3_region, storage_s3_bucket, storage_s3_access_key, storage_s3_secret_key_encrypted, storage_s3_force_path_style, storage_filesystem_root, storage_proxy_hmac_secret_encrypted. Defaults: storage_backend=`s3`, storage_filesystem_root=`./storage` (./storage added to .gitignore). Tests added (34 tests, all green): - tests/unit/storage/filesystem-backend.test.ts — key validation allow/reject matrix, realpath escape, 0o700 perms, multi-node refusal, HMAC token sign/verify/tamper/expire/invalid-key. - tests/unit/storage/copy-and-verify.test.ts — sha256 mismatch on round-trip aborts the migration. - tests/integration/storage/proxy-route.test.ts — happy path, wrong HMAC secret, expired token, replay rejection. Phase 6a ships zero file-bearing tables — TABLES_WITH_STORAGE_KEYS is intentionally empty. berth_pdf_versions and brochure_versions land in Phase 6b and join the list there. Existing s3_key columns: only gdpr_export_jobs.storage_key, already named correctly — no rename needed. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-05 03:15:59 +02:00
'use client';
import { useState } from 'react';
import { useMutation, useQuery, useQueryClient } from '@tanstack/react-query';
feat: autonomous backlog push — admin UX overhaul + storage parity + residential parity + Documenso Phase 1 Massive multi-area push driven by docs/admin-ux-backlog.md. Every byte path now goes through getStorageBackend() so signed EOIs, contracts, brochures, berth PDFs, files, avatars, branding logos, and DB backups all work identically on S3 and filesystem backends. USER SETTINGS (rebuild) - Country + Timezone selectors with cross-defaulting - Browser-detected timezone banner ("Looks like you're in Europe/Paris…") - Email change with verification flow (user_email_changes table, OLD-address cancel link + NEW-address confirm link) + EMAIL_CHANGE_INSTANT=true dev shortcut - Password reset triggered via better-auth requestPasswordReset - Profile photo upload + crop (square 256×256) via shared <ImageCropperDialog> + /api/v1/me/avatar BRANDING - Shared <ImageCropperDialog> using react-easy-crop - Logo upload + crop in /admin/branding (writes via /api/v1/admin/settings/image -> storage backend) - Email header/footer HTML defaults injectable via "Insert default" - SettingsFormCard new field types: timezone (combobox), image-upload STORAGE ADMIN OVERHAUL - S3 config form FIRST, swap action SECOND - Test connection before any switch - Two-button switch: "Switch + migrate" vs "Switch only" with warning modals - runMigration() honours skipMigration flag - /api/ready + system-monitoring health check use the active storage backend instead of always probing MinIO - Filesystem backend already had full feature parity — verified BACKUP MANAGEMENT (real) - New backup_jobs table (id / status / trigger / size / storage_path) - runBackup() service spawns pg_dump --format=custom, streams to active storage backend via getStorageBackend().put() - /admin/backup page: trigger, history, download .dump for restore - Super-admin gated AI ADMIN PANEL - /admin/ai consolidates master switch + monthly token cap + provider credentials - Per-feature settings (OCR, berth-PDF parser, recommender) linked from the same page ONBOARDING WIZARD - /admin/onboarding now real with auto-checked steps - Reads each setting key + lists endpoint (roles/users/tags) to decide completion - Manual checkboxes for steps without an auto-detect signal - Progress bar + Mark done/Mark incomplete buttons - State persisted in system_settings.onboarding_manual_status RESIDENTIAL PARITY (full) - New residential_client_notes + residential_interest_notes tables (mirror marina-side shape) - Polymorphic notes.service.ts extended (verifyParent, listForEntity, create, update, delete) for residential_clients/_interests - <NotesList> component accepts the new entity types - 4 new note endpoints (GET/POST/PATCH/DELETE for clients + interests) - 2 new activity endpoints (residential clients + interests) - residential-client-tabs.tsx + residential-interest-tabs.tsx use DetailLayout (Overview / Interests / Notes / Activity) - residential-client-detail-header.tsx mirrors marina-side strip - useBreadcrumbHint wired into both detail components - Configurable Assigned-to dropdown (residential_interests.view perm) CONFIGURABLE RESIDENTIAL STAGES - residential-stages.service.ts with list / save / orphan-check - /api/v1/residential/stages GET/PUT - /admin/residential-stages admin UI with reassign-on-remove modal - Validators relaxed from z.enum to z.string DOCUMENSO PHASE 1 - Schema: document_signers.invited_at / opened_at / last_reminder_sent_at / signing_token (+ idx_ds_signing_token) - Schema: documents.completion_cc_emails (text[]) + auto_reminder_interval_days (int) - transformSigningUrl() now maps SignerRole -> URL segment via ROLE_TO_URL_SEGMENT (approver->cc, witness->witness) — fixes Risk #5 where approver invites landed on /sign/error - POST /api/v1/documents/[id]/send-invitation with auto-pick of next pending signer - Per-port settings: documenso_developer_label / _approver_label + documenso_developer_user_id / _approver_user_id (Phase 7 Project Director RBAC binding fields) ADMIN UX RAPID-FIRE - Sidebar collapse removed (always-expanded design) - Audit log: input sizes (h-9), date pickers w-44, action cell sub-label so single-row entries aren't blank - Sales email config: token list <details> + tooltips on threshold + body fields - Custom Settings card: long-form description - Reminder digest timezone uses TimezoneCombobox - Port form: currency dropdown (10 common currencies) + timezone combobox + brand color picker - Permissions count badge opens modal with granted/denied per resource - Role names display-normalized via prettifyRoleName - Tag form: native input type=color - Custom Fields page: amber heads-up about non-integration - Settings manager: select field type + fallthrough_policy as dropdown - Storage admin S3 fields ship as proper password + boolean LIST PAGES - Residential client list: clickable email/phone (mailto/tel/wa.me) - Residential interests + Documents Hub search inputs sized h-9 CURRENCY API - scripts/test-currency-api.ts verifies live Frankfurter fetch -> DB upsert -> getRate -> convert. Inverse-rate drift <=0.001 TESTS - 1185/1185 vitest passing - tsc clean - eslint 0 errors (16 pre-existing warnings) Note: WEBSITE_INTAKE_SECRET added to .env.example but committed separately due to pre-commit hook policy on .env* files. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-07 21:02:12 +02:00
import {
AlertTriangle,
CheckCircle2,
HardDrive,
Loader2,
RefreshCw,
ServerCog,
XCircle,
} from 'lucide-react';
fix(audit-v3): platform-wide deferred-list cleanup (rounds 1-4) Working through the audit-v2 deferred backlog. Each round was tested (typecheck + 1168/1168 vitest) before moving on. Round 1 — DB performance + AI cost visibility: - Add missing FK indexes Postgres doesn't auto-create on berth_reservations.{interest_id, contract_file_id}, documents.{file_id, signed_file_id}, document_events.signer_id, document_templates.source_file_id, form_submissions.{form_template_id, client_id}, document_sends.{brochure_id, brochure_version_id, sent_by_user_id}. Without these, RESTRICT-checks on parent delete + reverse-lookups walk the child tables fully. Migration 0037. - AI worker now writes one ai_usage_ledger row per OpenAI call so admins can audit spend per port/user/feature and future per-port budgets have history to read from. Failure to write is logged-not-thrown so the user-facing email draft is unaffected. Round 2 — Boot-time + transport hardening: - S3 backend verifies the bucket exists at startup (or auto-creates when MINIO_AUTO_CREATE_BUCKET=true). A typo'd bucket name now surfaces with a clear boot error instead of a vague Minio error inside the first user-facing request. - Documenso v1 placeFields: 3-attempt exponential-backoff retry on 5xx + network errors, fail-fast on 4xx. Stops one transient flake from leaving a document with a partial field set. - FilesystemBackend logs a structured warn-once at boot when the dev HMAC fallback is in effect, so two processes started with different BETTER_AUTH_SECRET values are observable (random 401s on file downloads otherwise). - Logger redact paths extended to cover *.headers.{authorization, cookie}, *.config.headers.authorization, encrypted-credential blobs (secretKeyEncrypted, smtpPassEncrypted, etc.), the Documenso X-Documenso-Secret header, and 2-level nested forms. Round 3 — UI feedback + permission gates: - Storage admin migrate dialog: success toast with row count + error toast on both dryRun and migrate mutations. - Invoice detail Send + Record-payment buttons wrapped in PermissionGate (invoices.send / invoices.record_payment); both mutations now toast on success/error. - Admin user list Edit button wrapped in PermissionGate(admin.manage_users). - Scan-receipt page surfaces an amber warning when OCR fails so reps know they can fill the form manually instead of staring at a stalled spinner; the editable form now also opens on scanMutation.isError / uploadedFile, not only on success. - Email threads list now renders skeleton rows during load + shared EmptyState for the empty case (was a single "Loading…" line). Round 4 — Service / route correctness: - documentSends.sent_by_user_id was a free-text NOT NULL column with no FK. Now nullable + FK to user(id) ON DELETE SET NULL so the audit row survives a user being hard-deleted. Migration 0038 with a defensive null-out for any orphan ids before attaching the constraint. - Saved-views route: documented why withAuth alone is correct (the service strictly filters by (portId, userId) — owner-only by design). - Public-interests audit log: replaced "userId: null as unknown as string" cast with userId: null; AuditLogParams already accepts null for system-generated events. - EOI in-app PDF fill: extracted setBerthRange() that, when the AcroForm field is missing AND the context has a non-empty range string, logs a structured warn so the deployment gap (live Documenso template needs the field) is observable instead of silently dropping the multi-berth range. Test status: 1168/1168 vitest. tsc clean. Two new migrations (0037/0038) need pnpm db:push (or migration apply) on the dev DB. Deferred-doc updated with the remaining open items (bigger refactors). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-05 12:49:53 +02:00
import { toast } from 'sonner';
feat(storage): pluggable s3-or-filesystem backend + migration CLI + admin UI Phase 6a from docs/berth-recommender-and-pdf-plan.md §4.7a + §14.9a. Lays the storage groundwork for Phase 6b/7 file-bearing schemas (per-berth PDFs, brochures) without touching those domains yet. New files: - src/lib/storage/index.ts StorageBackend interface + per-process factory keyed on system_settings. - src/lib/storage/s3.ts S3-compatible backend (MinIO/AWS/B2/R2/ Wasabi/Tigris) wrapping the existing minio JS client. Includes a healthCheck() used by the admin "Test connection" button. - src/lib/storage/filesystem.ts Local filesystem backend with all §14.9a mitigations baked in. - src/lib/storage/migrate.ts Shared migration core — pg_advisory_lock, per-row resumable progress markers, sha256 round-trip verification, atomic storage_backend flip on success. - scripts/migrate-storage.ts Thin CLI shim around runMigration(). - src/app/api/storage/[token]/route.ts Filesystem proxy GET. Verifies HMAC, enforces single-use replay protection via Redis SET NX, streams via NextResponse ReadableStream with explicit Content-Type + Content-Disposition. Node runtime only. - src/app/api/v1/admin/storage/route.ts GET status + POST connection test. - src/app/api/v1/admin/storage/migrate/route.ts Super-admin-only POST that runs the exact same runMigration() as the CLI. - src/app/(dashboard)/[portSlug]/admin/storage/page.tsx Super-admin admin UI (current backend, capacity stats, switch button with dry-run, test connection, backup hint). - src/components/admin/storage-admin-panel.tsx Client component for the page above. §14.9a critical mitigations implemented: - Path-traversal: storage keys validated against ^[a-zA-Z0-9/_.-]+$; `..`, `.`, `//`, leading `/`, and overlength keys rejected. - Realpath: storage root realpath'd at create time, every per-key resolution checked against the realpath'd prefix. - Storage root created (or chmod'd) to 0o700. - Multi-node refusal: FilesystemBackend.create() throws when MULTI_NODE_DEPLOYMENT=true. - HMAC token: sha256-HMAC over the (key, expiry, nonce, filename, content-type) payload. Verified with timingSafeEqual; bad sig, expired, or invalid-key payloads all return 403. - Single-use replay: token body cached in Redis SET NX EX 1800s. - sha256 round-trip: copyAndVerify() re-fetches from the target after put() and aborts the migration on any mismatch. - Free-disk pre-flight: when migrating to filesystem, sums byte counts via source.head() and aborts if free space < total * 1.2. - pg_advisory_lock(0xc7000a01) prevents concurrent migrations. - Resumable: per-row progress markers in _storage_migration_progress. system_settings keys read by the factory (jsonb, no schema change): storage_backend, storage_s3_endpoint, storage_s3_region, storage_s3_bucket, storage_s3_access_key, storage_s3_secret_key_encrypted, storage_s3_force_path_style, storage_filesystem_root, storage_proxy_hmac_secret_encrypted. Defaults: storage_backend=`s3`, storage_filesystem_root=`./storage` (./storage added to .gitignore). Tests added (34 tests, all green): - tests/unit/storage/filesystem-backend.test.ts — key validation allow/reject matrix, realpath escape, 0o700 perms, multi-node refusal, HMAC token sign/verify/tamper/expire/invalid-key. - tests/unit/storage/copy-and-verify.test.ts — sha256 mismatch on round-trip aborts the migration. - tests/integration/storage/proxy-route.test.ts — happy path, wrong HMAC secret, expired token, replay rejection. Phase 6a ships zero file-bearing tables — TABLES_WITH_STORAGE_KEYS is intentionally empty. berth_pdf_versions and brochure_versions land in Phase 6b and join the list there. Existing s3_key columns: only gdpr_export_jobs.storage_key, already named correctly — no rename needed. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-05 03:15:59 +02:00
import { PageHeader } from '@/components/shared/page-header';
import { Button } from '@/components/ui/button';
import { Card, CardContent, CardDescription, CardHeader, CardTitle } from '@/components/ui/card';
import {
Dialog,
DialogContent,
DialogDescription,
DialogFooter,
DialogHeader,
DialogTitle,
} from '@/components/ui/dialog';
feat: autonomous backlog push — admin UX overhaul + storage parity + residential parity + Documenso Phase 1 Massive multi-area push driven by docs/admin-ux-backlog.md. Every byte path now goes through getStorageBackend() so signed EOIs, contracts, brochures, berth PDFs, files, avatars, branding logos, and DB backups all work identically on S3 and filesystem backends. USER SETTINGS (rebuild) - Country + Timezone selectors with cross-defaulting - Browser-detected timezone banner ("Looks like you're in Europe/Paris…") - Email change with verification flow (user_email_changes table, OLD-address cancel link + NEW-address confirm link) + EMAIL_CHANGE_INSTANT=true dev shortcut - Password reset triggered via better-auth requestPasswordReset - Profile photo upload + crop (square 256×256) via shared <ImageCropperDialog> + /api/v1/me/avatar BRANDING - Shared <ImageCropperDialog> using react-easy-crop - Logo upload + crop in /admin/branding (writes via /api/v1/admin/settings/image -> storage backend) - Email header/footer HTML defaults injectable via "Insert default" - SettingsFormCard new field types: timezone (combobox), image-upload STORAGE ADMIN OVERHAUL - S3 config form FIRST, swap action SECOND - Test connection before any switch - Two-button switch: "Switch + migrate" vs "Switch only" with warning modals - runMigration() honours skipMigration flag - /api/ready + system-monitoring health check use the active storage backend instead of always probing MinIO - Filesystem backend already had full feature parity — verified BACKUP MANAGEMENT (real) - New backup_jobs table (id / status / trigger / size / storage_path) - runBackup() service spawns pg_dump --format=custom, streams to active storage backend via getStorageBackend().put() - /admin/backup page: trigger, history, download .dump for restore - Super-admin gated AI ADMIN PANEL - /admin/ai consolidates master switch + monthly token cap + provider credentials - Per-feature settings (OCR, berth-PDF parser, recommender) linked from the same page ONBOARDING WIZARD - /admin/onboarding now real with auto-checked steps - Reads each setting key + lists endpoint (roles/users/tags) to decide completion - Manual checkboxes for steps without an auto-detect signal - Progress bar + Mark done/Mark incomplete buttons - State persisted in system_settings.onboarding_manual_status RESIDENTIAL PARITY (full) - New residential_client_notes + residential_interest_notes tables (mirror marina-side shape) - Polymorphic notes.service.ts extended (verifyParent, listForEntity, create, update, delete) for residential_clients/_interests - <NotesList> component accepts the new entity types - 4 new note endpoints (GET/POST/PATCH/DELETE for clients + interests) - 2 new activity endpoints (residential clients + interests) - residential-client-tabs.tsx + residential-interest-tabs.tsx use DetailLayout (Overview / Interests / Notes / Activity) - residential-client-detail-header.tsx mirrors marina-side strip - useBreadcrumbHint wired into both detail components - Configurable Assigned-to dropdown (residential_interests.view perm) CONFIGURABLE RESIDENTIAL STAGES - residential-stages.service.ts with list / save / orphan-check - /api/v1/residential/stages GET/PUT - /admin/residential-stages admin UI with reassign-on-remove modal - Validators relaxed from z.enum to z.string DOCUMENSO PHASE 1 - Schema: document_signers.invited_at / opened_at / last_reminder_sent_at / signing_token (+ idx_ds_signing_token) - Schema: documents.completion_cc_emails (text[]) + auto_reminder_interval_days (int) - transformSigningUrl() now maps SignerRole -> URL segment via ROLE_TO_URL_SEGMENT (approver->cc, witness->witness) — fixes Risk #5 where approver invites landed on /sign/error - POST /api/v1/documents/[id]/send-invitation with auto-pick of next pending signer - Per-port settings: documenso_developer_label / _approver_label + documenso_developer_user_id / _approver_user_id (Phase 7 Project Director RBAC binding fields) ADMIN UX RAPID-FIRE - Sidebar collapse removed (always-expanded design) - Audit log: input sizes (h-9), date pickers w-44, action cell sub-label so single-row entries aren't blank - Sales email config: token list <details> + tooltips on threshold + body fields - Custom Settings card: long-form description - Reminder digest timezone uses TimezoneCombobox - Port form: currency dropdown (10 common currencies) + timezone combobox + brand color picker - Permissions count badge opens modal with granted/denied per resource - Role names display-normalized via prettifyRoleName - Tag form: native input type=color - Custom Fields page: amber heads-up about non-integration - Settings manager: select field type + fallthrough_policy as dropdown - Storage admin S3 fields ship as proper password + boolean LIST PAGES - Residential client list: clickable email/phone (mailto/tel/wa.me) - Residential interests + Documents Hub search inputs sized h-9 CURRENCY API - scripts/test-currency-api.ts verifies live Frankfurter fetch -> DB upsert -> getRate -> convert. Inverse-rate drift <=0.001 TESTS - 1185/1185 vitest passing - tsc clean - eslint 0 errors (16 pre-existing warnings) Note: WEBSITE_INTAKE_SECRET added to .env.example but committed separately due to pre-commit hook policy on .env* files. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-07 21:02:12 +02:00
import {
SettingsFormCard,
type SettingFieldDef,
} from '@/components/admin/shared/settings-form-card';
feat(storage): pluggable s3-or-filesystem backend + migration CLI + admin UI Phase 6a from docs/berth-recommender-and-pdf-plan.md §4.7a + §14.9a. Lays the storage groundwork for Phase 6b/7 file-bearing schemas (per-berth PDFs, brochures) without touching those domains yet. New files: - src/lib/storage/index.ts StorageBackend interface + per-process factory keyed on system_settings. - src/lib/storage/s3.ts S3-compatible backend (MinIO/AWS/B2/R2/ Wasabi/Tigris) wrapping the existing minio JS client. Includes a healthCheck() used by the admin "Test connection" button. - src/lib/storage/filesystem.ts Local filesystem backend with all §14.9a mitigations baked in. - src/lib/storage/migrate.ts Shared migration core — pg_advisory_lock, per-row resumable progress markers, sha256 round-trip verification, atomic storage_backend flip on success. - scripts/migrate-storage.ts Thin CLI shim around runMigration(). - src/app/api/storage/[token]/route.ts Filesystem proxy GET. Verifies HMAC, enforces single-use replay protection via Redis SET NX, streams via NextResponse ReadableStream with explicit Content-Type + Content-Disposition. Node runtime only. - src/app/api/v1/admin/storage/route.ts GET status + POST connection test. - src/app/api/v1/admin/storage/migrate/route.ts Super-admin-only POST that runs the exact same runMigration() as the CLI. - src/app/(dashboard)/[portSlug]/admin/storage/page.tsx Super-admin admin UI (current backend, capacity stats, switch button with dry-run, test connection, backup hint). - src/components/admin/storage-admin-panel.tsx Client component for the page above. §14.9a critical mitigations implemented: - Path-traversal: storage keys validated against ^[a-zA-Z0-9/_.-]+$; `..`, `.`, `//`, leading `/`, and overlength keys rejected. - Realpath: storage root realpath'd at create time, every per-key resolution checked against the realpath'd prefix. - Storage root created (or chmod'd) to 0o700. - Multi-node refusal: FilesystemBackend.create() throws when MULTI_NODE_DEPLOYMENT=true. - HMAC token: sha256-HMAC over the (key, expiry, nonce, filename, content-type) payload. Verified with timingSafeEqual; bad sig, expired, or invalid-key payloads all return 403. - Single-use replay: token body cached in Redis SET NX EX 1800s. - sha256 round-trip: copyAndVerify() re-fetches from the target after put() and aborts the migration on any mismatch. - Free-disk pre-flight: when migrating to filesystem, sums byte counts via source.head() and aborts if free space < total * 1.2. - pg_advisory_lock(0xc7000a01) prevents concurrent migrations. - Resumable: per-row progress markers in _storage_migration_progress. system_settings keys read by the factory (jsonb, no schema change): storage_backend, storage_s3_endpoint, storage_s3_region, storage_s3_bucket, storage_s3_access_key, storage_s3_secret_key_encrypted, storage_s3_force_path_style, storage_filesystem_root, storage_proxy_hmac_secret_encrypted. Defaults: storage_backend=`s3`, storage_filesystem_root=`./storage` (./storage added to .gitignore). Tests added (34 tests, all green): - tests/unit/storage/filesystem-backend.test.ts — key validation allow/reject matrix, realpath escape, 0o700 perms, multi-node refusal, HMAC token sign/verify/tamper/expire/invalid-key. - tests/unit/storage/copy-and-verify.test.ts — sha256 mismatch on round-trip aborts the migration. - tests/integration/storage/proxy-route.test.ts — happy path, wrong HMAC secret, expired token, replay rejection. Phase 6a ships zero file-bearing tables — TABLES_WITH_STORAGE_KEYS is intentionally empty. berth_pdf_versions and brochure_versions land in Phase 6b and join the list there. Existing s3_key columns: only gdpr_export_jobs.storage_key, already named correctly — no rename needed. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-05 03:15:59 +02:00
import { apiFetch } from '@/lib/api/client';
import { toastError } from '@/lib/api/toast-error';
feat(storage): pluggable s3-or-filesystem backend + migration CLI + admin UI Phase 6a from docs/berth-recommender-and-pdf-plan.md §4.7a + §14.9a. Lays the storage groundwork for Phase 6b/7 file-bearing schemas (per-berth PDFs, brochures) without touching those domains yet. New files: - src/lib/storage/index.ts StorageBackend interface + per-process factory keyed on system_settings. - src/lib/storage/s3.ts S3-compatible backend (MinIO/AWS/B2/R2/ Wasabi/Tigris) wrapping the existing minio JS client. Includes a healthCheck() used by the admin "Test connection" button. - src/lib/storage/filesystem.ts Local filesystem backend with all §14.9a mitigations baked in. - src/lib/storage/migrate.ts Shared migration core — pg_advisory_lock, per-row resumable progress markers, sha256 round-trip verification, atomic storage_backend flip on success. - scripts/migrate-storage.ts Thin CLI shim around runMigration(). - src/app/api/storage/[token]/route.ts Filesystem proxy GET. Verifies HMAC, enforces single-use replay protection via Redis SET NX, streams via NextResponse ReadableStream with explicit Content-Type + Content-Disposition. Node runtime only. - src/app/api/v1/admin/storage/route.ts GET status + POST connection test. - src/app/api/v1/admin/storage/migrate/route.ts Super-admin-only POST that runs the exact same runMigration() as the CLI. - src/app/(dashboard)/[portSlug]/admin/storage/page.tsx Super-admin admin UI (current backend, capacity stats, switch button with dry-run, test connection, backup hint). - src/components/admin/storage-admin-panel.tsx Client component for the page above. §14.9a critical mitigations implemented: - Path-traversal: storage keys validated against ^[a-zA-Z0-9/_.-]+$; `..`, `.`, `//`, leading `/`, and overlength keys rejected. - Realpath: storage root realpath'd at create time, every per-key resolution checked against the realpath'd prefix. - Storage root created (or chmod'd) to 0o700. - Multi-node refusal: FilesystemBackend.create() throws when MULTI_NODE_DEPLOYMENT=true. - HMAC token: sha256-HMAC over the (key, expiry, nonce, filename, content-type) payload. Verified with timingSafeEqual; bad sig, expired, or invalid-key payloads all return 403. - Single-use replay: token body cached in Redis SET NX EX 1800s. - sha256 round-trip: copyAndVerify() re-fetches from the target after put() and aborts the migration on any mismatch. - Free-disk pre-flight: when migrating to filesystem, sums byte counts via source.head() and aborts if free space < total * 1.2. - pg_advisory_lock(0xc7000a01) prevents concurrent migrations. - Resumable: per-row progress markers in _storage_migration_progress. system_settings keys read by the factory (jsonb, no schema change): storage_backend, storage_s3_endpoint, storage_s3_region, storage_s3_bucket, storage_s3_access_key, storage_s3_secret_key_encrypted, storage_s3_force_path_style, storage_filesystem_root, storage_proxy_hmac_secret_encrypted. Defaults: storage_backend=`s3`, storage_filesystem_root=`./storage` (./storage added to .gitignore). Tests added (34 tests, all green): - tests/unit/storage/filesystem-backend.test.ts — key validation allow/reject matrix, realpath escape, 0o700 perms, multi-node refusal, HMAC token sign/verify/tamper/expire/invalid-key. - tests/unit/storage/copy-and-verify.test.ts — sha256 mismatch on round-trip aborts the migration. - tests/integration/storage/proxy-route.test.ts — happy path, wrong HMAC secret, expired token, replay rejection. Phase 6a ships zero file-bearing tables — TABLES_WITH_STORAGE_KEYS is intentionally empty. berth_pdf_versions and brochure_versions land in Phase 6b and join the list there. Existing s3_key columns: only gdpr_export_jobs.storage_key, already named correctly — no rename needed. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-05 03:15:59 +02:00
type BackendName = 's3' | 'filesystem';
interface StorageStatus {
backend: BackendName;
fileCount: number;
totalBytes: number;
tablesTracked: string[];
}
interface MigrationResult {
rowsConsidered: number;
rowsMigrated: number;
rowsSkippedAlreadyDone: number;
totalBytes: number;
flipped: boolean;
dryRun: boolean;
}
feat: autonomous backlog push — admin UX overhaul + storage parity + residential parity + Documenso Phase 1 Massive multi-area push driven by docs/admin-ux-backlog.md. Every byte path now goes through getStorageBackend() so signed EOIs, contracts, brochures, berth PDFs, files, avatars, branding logos, and DB backups all work identically on S3 and filesystem backends. USER SETTINGS (rebuild) - Country + Timezone selectors with cross-defaulting - Browser-detected timezone banner ("Looks like you're in Europe/Paris…") - Email change with verification flow (user_email_changes table, OLD-address cancel link + NEW-address confirm link) + EMAIL_CHANGE_INSTANT=true dev shortcut - Password reset triggered via better-auth requestPasswordReset - Profile photo upload + crop (square 256×256) via shared <ImageCropperDialog> + /api/v1/me/avatar BRANDING - Shared <ImageCropperDialog> using react-easy-crop - Logo upload + crop in /admin/branding (writes via /api/v1/admin/settings/image -> storage backend) - Email header/footer HTML defaults injectable via "Insert default" - SettingsFormCard new field types: timezone (combobox), image-upload STORAGE ADMIN OVERHAUL - S3 config form FIRST, swap action SECOND - Test connection before any switch - Two-button switch: "Switch + migrate" vs "Switch only" with warning modals - runMigration() honours skipMigration flag - /api/ready + system-monitoring health check use the active storage backend instead of always probing MinIO - Filesystem backend already had full feature parity — verified BACKUP MANAGEMENT (real) - New backup_jobs table (id / status / trigger / size / storage_path) - runBackup() service spawns pg_dump --format=custom, streams to active storage backend via getStorageBackend().put() - /admin/backup page: trigger, history, download .dump for restore - Super-admin gated AI ADMIN PANEL - /admin/ai consolidates master switch + monthly token cap + provider credentials - Per-feature settings (OCR, berth-PDF parser, recommender) linked from the same page ONBOARDING WIZARD - /admin/onboarding now real with auto-checked steps - Reads each setting key + lists endpoint (roles/users/tags) to decide completion - Manual checkboxes for steps without an auto-detect signal - Progress bar + Mark done/Mark incomplete buttons - State persisted in system_settings.onboarding_manual_status RESIDENTIAL PARITY (full) - New residential_client_notes + residential_interest_notes tables (mirror marina-side shape) - Polymorphic notes.service.ts extended (verifyParent, listForEntity, create, update, delete) for residential_clients/_interests - <NotesList> component accepts the new entity types - 4 new note endpoints (GET/POST/PATCH/DELETE for clients + interests) - 2 new activity endpoints (residential clients + interests) - residential-client-tabs.tsx + residential-interest-tabs.tsx use DetailLayout (Overview / Interests / Notes / Activity) - residential-client-detail-header.tsx mirrors marina-side strip - useBreadcrumbHint wired into both detail components - Configurable Assigned-to dropdown (residential_interests.view perm) CONFIGURABLE RESIDENTIAL STAGES - residential-stages.service.ts with list / save / orphan-check - /api/v1/residential/stages GET/PUT - /admin/residential-stages admin UI with reassign-on-remove modal - Validators relaxed from z.enum to z.string DOCUMENSO PHASE 1 - Schema: document_signers.invited_at / opened_at / last_reminder_sent_at / signing_token (+ idx_ds_signing_token) - Schema: documents.completion_cc_emails (text[]) + auto_reminder_interval_days (int) - transformSigningUrl() now maps SignerRole -> URL segment via ROLE_TO_URL_SEGMENT (approver->cc, witness->witness) — fixes Risk #5 where approver invites landed on /sign/error - POST /api/v1/documents/[id]/send-invitation with auto-pick of next pending signer - Per-port settings: documenso_developer_label / _approver_label + documenso_developer_user_id / _approver_user_id (Phase 7 Project Director RBAC binding fields) ADMIN UX RAPID-FIRE - Sidebar collapse removed (always-expanded design) - Audit log: input sizes (h-9), date pickers w-44, action cell sub-label so single-row entries aren't blank - Sales email config: token list <details> + tooltips on threshold + body fields - Custom Settings card: long-form description - Reminder digest timezone uses TimezoneCombobox - Port form: currency dropdown (10 common currencies) + timezone combobox + brand color picker - Permissions count badge opens modal with granted/denied per resource - Role names display-normalized via prettifyRoleName - Tag form: native input type=color - Custom Fields page: amber heads-up about non-integration - Settings manager: select field type + fallthrough_policy as dropdown - Storage admin S3 fields ship as proper password + boolean LIST PAGES - Residential client list: clickable email/phone (mailto/tel/wa.me) - Residential interests + Documents Hub search inputs sized h-9 CURRENCY API - scripts/test-currency-api.ts verifies live Frankfurter fetch -> DB upsert -> getRate -> convert. Inverse-rate drift <=0.001 TESTS - 1185/1185 vitest passing - tsc clean - eslint 0 errors (16 pre-existing warnings) Note: WEBSITE_INTAKE_SECRET added to .env.example but committed separately due to pre-commit hook policy on .env* files. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-07 21:02:12 +02:00
const S3_FIELDS: SettingFieldDef[] = [
{
key: 'storage_s3_endpoint',
label: 'S3 endpoint URL',
description:
'For AWS use https://s3.<region>.amazonaws.com. For MinIO/Backblaze/R2/Wasabi/Tigris paste the provider-supplied URL.',
type: 'string',
placeholder: 'https://s3.us-east-1.amazonaws.com',
defaultValue: '',
},
{
key: 'storage_s3_region',
label: 'S3 region',
description: 'AWS region code (e.g. us-east-1, eu-west-1). Use "auto" for Cloudflare R2.',
type: 'string',
placeholder: 'us-east-1',
defaultValue: 'us-east-1',
},
{
key: 'storage_s3_bucket',
label: 'S3 bucket name',
description: 'Existing bucket the CRM should read/write to. Must already exist.',
type: 'string',
placeholder: 'port-nimara-storage',
defaultValue: '',
},
{
key: 'storage_s3_access_key',
label: 'S3 access key',
description: 'IAM access key id (or provider equivalent).',
type: 'string',
placeholder: 'AKIA…',
defaultValue: '',
},
{
key: 'storage_s3_secret_key_encrypted',
label: 'S3 secret key',
description:
'Stored AES-encrypted at rest; the field shows blank after save and is replaced only when you type a new value.',
type: 'password',
placeholder: '(unchanged)',
defaultValue: '',
},
{
key: 'storage_s3_force_path_style',
label: 'Force path-style URLs',
description:
'On for MinIO and most self-hosted S3-compatible servers. Off for AWS S3 (which uses virtual-hosted-style by default).',
type: 'boolean',
defaultValue: false,
},
];
const FS_FIELDS: SettingFieldDef[] = [
{
key: 'storage_filesystem_root',
label: 'Filesystem root path',
description:
'Absolute path on the server where files are stored. Must be writable by the CRM process. Single-node deployments only.',
type: 'string',
placeholder: '/var/lib/port-nimara/storage',
defaultValue: '/var/lib/port-nimara/storage',
},
];
feat(storage): pluggable s3-or-filesystem backend + migration CLI + admin UI Phase 6a from docs/berth-recommender-and-pdf-plan.md §4.7a + §14.9a. Lays the storage groundwork for Phase 6b/7 file-bearing schemas (per-berth PDFs, brochures) without touching those domains yet. New files: - src/lib/storage/index.ts StorageBackend interface + per-process factory keyed on system_settings. - src/lib/storage/s3.ts S3-compatible backend (MinIO/AWS/B2/R2/ Wasabi/Tigris) wrapping the existing minio JS client. Includes a healthCheck() used by the admin "Test connection" button. - src/lib/storage/filesystem.ts Local filesystem backend with all §14.9a mitigations baked in. - src/lib/storage/migrate.ts Shared migration core — pg_advisory_lock, per-row resumable progress markers, sha256 round-trip verification, atomic storage_backend flip on success. - scripts/migrate-storage.ts Thin CLI shim around runMigration(). - src/app/api/storage/[token]/route.ts Filesystem proxy GET. Verifies HMAC, enforces single-use replay protection via Redis SET NX, streams via NextResponse ReadableStream with explicit Content-Type + Content-Disposition. Node runtime only. - src/app/api/v1/admin/storage/route.ts GET status + POST connection test. - src/app/api/v1/admin/storage/migrate/route.ts Super-admin-only POST that runs the exact same runMigration() as the CLI. - src/app/(dashboard)/[portSlug]/admin/storage/page.tsx Super-admin admin UI (current backend, capacity stats, switch button with dry-run, test connection, backup hint). - src/components/admin/storage-admin-panel.tsx Client component for the page above. §14.9a critical mitigations implemented: - Path-traversal: storage keys validated against ^[a-zA-Z0-9/_.-]+$; `..`, `.`, `//`, leading `/`, and overlength keys rejected. - Realpath: storage root realpath'd at create time, every per-key resolution checked against the realpath'd prefix. - Storage root created (or chmod'd) to 0o700. - Multi-node refusal: FilesystemBackend.create() throws when MULTI_NODE_DEPLOYMENT=true. - HMAC token: sha256-HMAC over the (key, expiry, nonce, filename, content-type) payload. Verified with timingSafeEqual; bad sig, expired, or invalid-key payloads all return 403. - Single-use replay: token body cached in Redis SET NX EX 1800s. - sha256 round-trip: copyAndVerify() re-fetches from the target after put() and aborts the migration on any mismatch. - Free-disk pre-flight: when migrating to filesystem, sums byte counts via source.head() and aborts if free space < total * 1.2. - pg_advisory_lock(0xc7000a01) prevents concurrent migrations. - Resumable: per-row progress markers in _storage_migration_progress. system_settings keys read by the factory (jsonb, no schema change): storage_backend, storage_s3_endpoint, storage_s3_region, storage_s3_bucket, storage_s3_access_key, storage_s3_secret_key_encrypted, storage_s3_force_path_style, storage_filesystem_root, storage_proxy_hmac_secret_encrypted. Defaults: storage_backend=`s3`, storage_filesystem_root=`./storage` (./storage added to .gitignore). Tests added (34 tests, all green): - tests/unit/storage/filesystem-backend.test.ts — key validation allow/reject matrix, realpath escape, 0o700 perms, multi-node refusal, HMAC token sign/verify/tamper/expire/invalid-key. - tests/unit/storage/copy-and-verify.test.ts — sha256 mismatch on round-trip aborts the migration. - tests/integration/storage/proxy-route.test.ts — happy path, wrong HMAC secret, expired token, replay rejection. Phase 6a ships zero file-bearing tables — TABLES_WITH_STORAGE_KEYS is intentionally empty. berth_pdf_versions and brochure_versions land in Phase 6b and join the list there. Existing s3_key columns: only gdpr_export_jobs.storage_key, already named correctly — no rename needed. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-05 03:15:59 +02:00
export function StorageAdminPanel() {
const queryClient = useQueryClient();
const [confirmOpen, setConfirmOpen] = useState(false);
const [dryRun, setDryRun] = useState<MigrationResult | null>(null);
feat: autonomous backlog push — admin UX overhaul + storage parity + residential parity + Documenso Phase 1 Massive multi-area push driven by docs/admin-ux-backlog.md. Every byte path now goes through getStorageBackend() so signed EOIs, contracts, brochures, berth PDFs, files, avatars, branding logos, and DB backups all work identically on S3 and filesystem backends. USER SETTINGS (rebuild) - Country + Timezone selectors with cross-defaulting - Browser-detected timezone banner ("Looks like you're in Europe/Paris…") - Email change with verification flow (user_email_changes table, OLD-address cancel link + NEW-address confirm link) + EMAIL_CHANGE_INSTANT=true dev shortcut - Password reset triggered via better-auth requestPasswordReset - Profile photo upload + crop (square 256×256) via shared <ImageCropperDialog> + /api/v1/me/avatar BRANDING - Shared <ImageCropperDialog> using react-easy-crop - Logo upload + crop in /admin/branding (writes via /api/v1/admin/settings/image -> storage backend) - Email header/footer HTML defaults injectable via "Insert default" - SettingsFormCard new field types: timezone (combobox), image-upload STORAGE ADMIN OVERHAUL - S3 config form FIRST, swap action SECOND - Test connection before any switch - Two-button switch: "Switch + migrate" vs "Switch only" with warning modals - runMigration() honours skipMigration flag - /api/ready + system-monitoring health check use the active storage backend instead of always probing MinIO - Filesystem backend already had full feature parity — verified BACKUP MANAGEMENT (real) - New backup_jobs table (id / status / trigger / size / storage_path) - runBackup() service spawns pg_dump --format=custom, streams to active storage backend via getStorageBackend().put() - /admin/backup page: trigger, history, download .dump for restore - Super-admin gated AI ADMIN PANEL - /admin/ai consolidates master switch + monthly token cap + provider credentials - Per-feature settings (OCR, berth-PDF parser, recommender) linked from the same page ONBOARDING WIZARD - /admin/onboarding now real with auto-checked steps - Reads each setting key + lists endpoint (roles/users/tags) to decide completion - Manual checkboxes for steps without an auto-detect signal - Progress bar + Mark done/Mark incomplete buttons - State persisted in system_settings.onboarding_manual_status RESIDENTIAL PARITY (full) - New residential_client_notes + residential_interest_notes tables (mirror marina-side shape) - Polymorphic notes.service.ts extended (verifyParent, listForEntity, create, update, delete) for residential_clients/_interests - <NotesList> component accepts the new entity types - 4 new note endpoints (GET/POST/PATCH/DELETE for clients + interests) - 2 new activity endpoints (residential clients + interests) - residential-client-tabs.tsx + residential-interest-tabs.tsx use DetailLayout (Overview / Interests / Notes / Activity) - residential-client-detail-header.tsx mirrors marina-side strip - useBreadcrumbHint wired into both detail components - Configurable Assigned-to dropdown (residential_interests.view perm) CONFIGURABLE RESIDENTIAL STAGES - residential-stages.service.ts with list / save / orphan-check - /api/v1/residential/stages GET/PUT - /admin/residential-stages admin UI with reassign-on-remove modal - Validators relaxed from z.enum to z.string DOCUMENSO PHASE 1 - Schema: document_signers.invited_at / opened_at / last_reminder_sent_at / signing_token (+ idx_ds_signing_token) - Schema: documents.completion_cc_emails (text[]) + auto_reminder_interval_days (int) - transformSigningUrl() now maps SignerRole -> URL segment via ROLE_TO_URL_SEGMENT (approver->cc, witness->witness) — fixes Risk #5 where approver invites landed on /sign/error - POST /api/v1/documents/[id]/send-invitation with auto-pick of next pending signer - Per-port settings: documenso_developer_label / _approver_label + documenso_developer_user_id / _approver_user_id (Phase 7 Project Director RBAC binding fields) ADMIN UX RAPID-FIRE - Sidebar collapse removed (always-expanded design) - Audit log: input sizes (h-9), date pickers w-44, action cell sub-label so single-row entries aren't blank - Sales email config: token list <details> + tooltips on threshold + body fields - Custom Settings card: long-form description - Reminder digest timezone uses TimezoneCombobox - Port form: currency dropdown (10 common currencies) + timezone combobox + brand color picker - Permissions count badge opens modal with granted/denied per resource - Role names display-normalized via prettifyRoleName - Tag form: native input type=color - Custom Fields page: amber heads-up about non-integration - Settings manager: select field type + fallthrough_policy as dropdown - Storage admin S3 fields ship as proper password + boolean LIST PAGES - Residential client list: clickable email/phone (mailto/tel/wa.me) - Residential interests + Documents Hub search inputs sized h-9 CURRENCY API - scripts/test-currency-api.ts verifies live Frankfurter fetch -> DB upsert -> getRate -> convert. Inverse-rate drift <=0.001 TESTS - 1185/1185 vitest passing - tsc clean - eslint 0 errors (16 pre-existing warnings) Note: WEBSITE_INTAKE_SECRET added to .env.example but committed separately due to pre-commit hook policy on .env* files. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-07 21:02:12 +02:00
const [confirmMode, setConfirmMode] = useState<'switch-only' | 'switch-and-migrate'>(
'switch-and-migrate',
);
feat(storage): pluggable s3-or-filesystem backend + migration CLI + admin UI Phase 6a from docs/berth-recommender-and-pdf-plan.md §4.7a + §14.9a. Lays the storage groundwork for Phase 6b/7 file-bearing schemas (per-berth PDFs, brochures) without touching those domains yet. New files: - src/lib/storage/index.ts StorageBackend interface + per-process factory keyed on system_settings. - src/lib/storage/s3.ts S3-compatible backend (MinIO/AWS/B2/R2/ Wasabi/Tigris) wrapping the existing minio JS client. Includes a healthCheck() used by the admin "Test connection" button. - src/lib/storage/filesystem.ts Local filesystem backend with all §14.9a mitigations baked in. - src/lib/storage/migrate.ts Shared migration core — pg_advisory_lock, per-row resumable progress markers, sha256 round-trip verification, atomic storage_backend flip on success. - scripts/migrate-storage.ts Thin CLI shim around runMigration(). - src/app/api/storage/[token]/route.ts Filesystem proxy GET. Verifies HMAC, enforces single-use replay protection via Redis SET NX, streams via NextResponse ReadableStream with explicit Content-Type + Content-Disposition. Node runtime only. - src/app/api/v1/admin/storage/route.ts GET status + POST connection test. - src/app/api/v1/admin/storage/migrate/route.ts Super-admin-only POST that runs the exact same runMigration() as the CLI. - src/app/(dashboard)/[portSlug]/admin/storage/page.tsx Super-admin admin UI (current backend, capacity stats, switch button with dry-run, test connection, backup hint). - src/components/admin/storage-admin-panel.tsx Client component for the page above. §14.9a critical mitigations implemented: - Path-traversal: storage keys validated against ^[a-zA-Z0-9/_.-]+$; `..`, `.`, `//`, leading `/`, and overlength keys rejected. - Realpath: storage root realpath'd at create time, every per-key resolution checked against the realpath'd prefix. - Storage root created (or chmod'd) to 0o700. - Multi-node refusal: FilesystemBackend.create() throws when MULTI_NODE_DEPLOYMENT=true. - HMAC token: sha256-HMAC over the (key, expiry, nonce, filename, content-type) payload. Verified with timingSafeEqual; bad sig, expired, or invalid-key payloads all return 403. - Single-use replay: token body cached in Redis SET NX EX 1800s. - sha256 round-trip: copyAndVerify() re-fetches from the target after put() and aborts the migration on any mismatch. - Free-disk pre-flight: when migrating to filesystem, sums byte counts via source.head() and aborts if free space < total * 1.2. - pg_advisory_lock(0xc7000a01) prevents concurrent migrations. - Resumable: per-row progress markers in _storage_migration_progress. system_settings keys read by the factory (jsonb, no schema change): storage_backend, storage_s3_endpoint, storage_s3_region, storage_s3_bucket, storage_s3_access_key, storage_s3_secret_key_encrypted, storage_s3_force_path_style, storage_filesystem_root, storage_proxy_hmac_secret_encrypted. Defaults: storage_backend=`s3`, storage_filesystem_root=`./storage` (./storage added to .gitignore). Tests added (34 tests, all green): - tests/unit/storage/filesystem-backend.test.ts — key validation allow/reject matrix, realpath escape, 0o700 perms, multi-node refusal, HMAC token sign/verify/tamper/expire/invalid-key. - tests/unit/storage/copy-and-verify.test.ts — sha256 mismatch on round-trip aborts the migration. - tests/integration/storage/proxy-route.test.ts — happy path, wrong HMAC secret, expired token, replay rejection. Phase 6a ships zero file-bearing tables — TABLES_WITH_STORAGE_KEYS is intentionally empty. berth_pdf_versions and brochure_versions land in Phase 6b and join the list there. Existing s3_key columns: only gdpr_export_jobs.storage_key, already named correctly — no rename needed. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-05 03:15:59 +02:00
const [testResult, setTestResult] = useState<{ ok: boolean; error?: string } | null>(null);
const status = useQuery({
queryKey: ['admin', 'storage', 'status'],
queryFn: () => apiFetch<{ data: StorageStatus }>('/api/v1/admin/storage'),
});
const dryRunMutation = useMutation({
mutationFn: async (opts: { from: BackendName; to: BackendName }) =>
apiFetch<{ data: MigrationResult }>('/api/v1/admin/storage/migrate', {
method: 'POST',
body: JSON.stringify({ ...opts, dryRun: true }),
}),
onSuccess: (result) => {
setDryRun(result.data);
setConfirmOpen(true);
},
onError: (e) => toastError(e),
feat(storage): pluggable s3-or-filesystem backend + migration CLI + admin UI Phase 6a from docs/berth-recommender-and-pdf-plan.md §4.7a + §14.9a. Lays the storage groundwork for Phase 6b/7 file-bearing schemas (per-berth PDFs, brochures) without touching those domains yet. New files: - src/lib/storage/index.ts StorageBackend interface + per-process factory keyed on system_settings. - src/lib/storage/s3.ts S3-compatible backend (MinIO/AWS/B2/R2/ Wasabi/Tigris) wrapping the existing minio JS client. Includes a healthCheck() used by the admin "Test connection" button. - src/lib/storage/filesystem.ts Local filesystem backend with all §14.9a mitigations baked in. - src/lib/storage/migrate.ts Shared migration core — pg_advisory_lock, per-row resumable progress markers, sha256 round-trip verification, atomic storage_backend flip on success. - scripts/migrate-storage.ts Thin CLI shim around runMigration(). - src/app/api/storage/[token]/route.ts Filesystem proxy GET. Verifies HMAC, enforces single-use replay protection via Redis SET NX, streams via NextResponse ReadableStream with explicit Content-Type + Content-Disposition. Node runtime only. - src/app/api/v1/admin/storage/route.ts GET status + POST connection test. - src/app/api/v1/admin/storage/migrate/route.ts Super-admin-only POST that runs the exact same runMigration() as the CLI. - src/app/(dashboard)/[portSlug]/admin/storage/page.tsx Super-admin admin UI (current backend, capacity stats, switch button with dry-run, test connection, backup hint). - src/components/admin/storage-admin-panel.tsx Client component for the page above. §14.9a critical mitigations implemented: - Path-traversal: storage keys validated against ^[a-zA-Z0-9/_.-]+$; `..`, `.`, `//`, leading `/`, and overlength keys rejected. - Realpath: storage root realpath'd at create time, every per-key resolution checked against the realpath'd prefix. - Storage root created (or chmod'd) to 0o700. - Multi-node refusal: FilesystemBackend.create() throws when MULTI_NODE_DEPLOYMENT=true. - HMAC token: sha256-HMAC over the (key, expiry, nonce, filename, content-type) payload. Verified with timingSafeEqual; bad sig, expired, or invalid-key payloads all return 403. - Single-use replay: token body cached in Redis SET NX EX 1800s. - sha256 round-trip: copyAndVerify() re-fetches from the target after put() and aborts the migration on any mismatch. - Free-disk pre-flight: when migrating to filesystem, sums byte counts via source.head() and aborts if free space < total * 1.2. - pg_advisory_lock(0xc7000a01) prevents concurrent migrations. - Resumable: per-row progress markers in _storage_migration_progress. system_settings keys read by the factory (jsonb, no schema change): storage_backend, storage_s3_endpoint, storage_s3_region, storage_s3_bucket, storage_s3_access_key, storage_s3_secret_key_encrypted, storage_s3_force_path_style, storage_filesystem_root, storage_proxy_hmac_secret_encrypted. Defaults: storage_backend=`s3`, storage_filesystem_root=`./storage` (./storage added to .gitignore). Tests added (34 tests, all green): - tests/unit/storage/filesystem-backend.test.ts — key validation allow/reject matrix, realpath escape, 0o700 perms, multi-node refusal, HMAC token sign/verify/tamper/expire/invalid-key. - tests/unit/storage/copy-and-verify.test.ts — sha256 mismatch on round-trip aborts the migration. - tests/integration/storage/proxy-route.test.ts — happy path, wrong HMAC secret, expired token, replay rejection. Phase 6a ships zero file-bearing tables — TABLES_WITH_STORAGE_KEYS is intentionally empty. berth_pdf_versions and brochure_versions land in Phase 6b and join the list there. Existing s3_key columns: only gdpr_export_jobs.storage_key, already named correctly — no rename needed. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-05 03:15:59 +02:00
});
const migrateMutation = useMutation({
feat: autonomous backlog push — admin UX overhaul + storage parity + residential parity + Documenso Phase 1 Massive multi-area push driven by docs/admin-ux-backlog.md. Every byte path now goes through getStorageBackend() so signed EOIs, contracts, brochures, berth PDFs, files, avatars, branding logos, and DB backups all work identically on S3 and filesystem backends. USER SETTINGS (rebuild) - Country + Timezone selectors with cross-defaulting - Browser-detected timezone banner ("Looks like you're in Europe/Paris…") - Email change with verification flow (user_email_changes table, OLD-address cancel link + NEW-address confirm link) + EMAIL_CHANGE_INSTANT=true dev shortcut - Password reset triggered via better-auth requestPasswordReset - Profile photo upload + crop (square 256×256) via shared <ImageCropperDialog> + /api/v1/me/avatar BRANDING - Shared <ImageCropperDialog> using react-easy-crop - Logo upload + crop in /admin/branding (writes via /api/v1/admin/settings/image -> storage backend) - Email header/footer HTML defaults injectable via "Insert default" - SettingsFormCard new field types: timezone (combobox), image-upload STORAGE ADMIN OVERHAUL - S3 config form FIRST, swap action SECOND - Test connection before any switch - Two-button switch: "Switch + migrate" vs "Switch only" with warning modals - runMigration() honours skipMigration flag - /api/ready + system-monitoring health check use the active storage backend instead of always probing MinIO - Filesystem backend already had full feature parity — verified BACKUP MANAGEMENT (real) - New backup_jobs table (id / status / trigger / size / storage_path) - runBackup() service spawns pg_dump --format=custom, streams to active storage backend via getStorageBackend().put() - /admin/backup page: trigger, history, download .dump for restore - Super-admin gated AI ADMIN PANEL - /admin/ai consolidates master switch + monthly token cap + provider credentials - Per-feature settings (OCR, berth-PDF parser, recommender) linked from the same page ONBOARDING WIZARD - /admin/onboarding now real with auto-checked steps - Reads each setting key + lists endpoint (roles/users/tags) to decide completion - Manual checkboxes for steps without an auto-detect signal - Progress bar + Mark done/Mark incomplete buttons - State persisted in system_settings.onboarding_manual_status RESIDENTIAL PARITY (full) - New residential_client_notes + residential_interest_notes tables (mirror marina-side shape) - Polymorphic notes.service.ts extended (verifyParent, listForEntity, create, update, delete) for residential_clients/_interests - <NotesList> component accepts the new entity types - 4 new note endpoints (GET/POST/PATCH/DELETE for clients + interests) - 2 new activity endpoints (residential clients + interests) - residential-client-tabs.tsx + residential-interest-tabs.tsx use DetailLayout (Overview / Interests / Notes / Activity) - residential-client-detail-header.tsx mirrors marina-side strip - useBreadcrumbHint wired into both detail components - Configurable Assigned-to dropdown (residential_interests.view perm) CONFIGURABLE RESIDENTIAL STAGES - residential-stages.service.ts with list / save / orphan-check - /api/v1/residential/stages GET/PUT - /admin/residential-stages admin UI with reassign-on-remove modal - Validators relaxed from z.enum to z.string DOCUMENSO PHASE 1 - Schema: document_signers.invited_at / opened_at / last_reminder_sent_at / signing_token (+ idx_ds_signing_token) - Schema: documents.completion_cc_emails (text[]) + auto_reminder_interval_days (int) - transformSigningUrl() now maps SignerRole -> URL segment via ROLE_TO_URL_SEGMENT (approver->cc, witness->witness) — fixes Risk #5 where approver invites landed on /sign/error - POST /api/v1/documents/[id]/send-invitation with auto-pick of next pending signer - Per-port settings: documenso_developer_label / _approver_label + documenso_developer_user_id / _approver_user_id (Phase 7 Project Director RBAC binding fields) ADMIN UX RAPID-FIRE - Sidebar collapse removed (always-expanded design) - Audit log: input sizes (h-9), date pickers w-44, action cell sub-label so single-row entries aren't blank - Sales email config: token list <details> + tooltips on threshold + body fields - Custom Settings card: long-form description - Reminder digest timezone uses TimezoneCombobox - Port form: currency dropdown (10 common currencies) + timezone combobox + brand color picker - Permissions count badge opens modal with granted/denied per resource - Role names display-normalized via prettifyRoleName - Tag form: native input type=color - Custom Fields page: amber heads-up about non-integration - Settings manager: select field type + fallthrough_policy as dropdown - Storage admin S3 fields ship as proper password + boolean LIST PAGES - Residential client list: clickable email/phone (mailto/tel/wa.me) - Residential interests + Documents Hub search inputs sized h-9 CURRENCY API - scripts/test-currency-api.ts verifies live Frankfurter fetch -> DB upsert -> getRate -> convert. Inverse-rate drift <=0.001 TESTS - 1185/1185 vitest passing - tsc clean - eslint 0 errors (16 pre-existing warnings) Note: WEBSITE_INTAKE_SECRET added to .env.example but committed separately due to pre-commit hook policy on .env* files. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-07 21:02:12 +02:00
mutationFn: async (opts: { from: BackendName; to: BackendName; skipMigration: boolean }) =>
feat(storage): pluggable s3-or-filesystem backend + migration CLI + admin UI Phase 6a from docs/berth-recommender-and-pdf-plan.md §4.7a + §14.9a. Lays the storage groundwork for Phase 6b/7 file-bearing schemas (per-berth PDFs, brochures) without touching those domains yet. New files: - src/lib/storage/index.ts StorageBackend interface + per-process factory keyed on system_settings. - src/lib/storage/s3.ts S3-compatible backend (MinIO/AWS/B2/R2/ Wasabi/Tigris) wrapping the existing minio JS client. Includes a healthCheck() used by the admin "Test connection" button. - src/lib/storage/filesystem.ts Local filesystem backend with all §14.9a mitigations baked in. - src/lib/storage/migrate.ts Shared migration core — pg_advisory_lock, per-row resumable progress markers, sha256 round-trip verification, atomic storage_backend flip on success. - scripts/migrate-storage.ts Thin CLI shim around runMigration(). - src/app/api/storage/[token]/route.ts Filesystem proxy GET. Verifies HMAC, enforces single-use replay protection via Redis SET NX, streams via NextResponse ReadableStream with explicit Content-Type + Content-Disposition. Node runtime only. - src/app/api/v1/admin/storage/route.ts GET status + POST connection test. - src/app/api/v1/admin/storage/migrate/route.ts Super-admin-only POST that runs the exact same runMigration() as the CLI. - src/app/(dashboard)/[portSlug]/admin/storage/page.tsx Super-admin admin UI (current backend, capacity stats, switch button with dry-run, test connection, backup hint). - src/components/admin/storage-admin-panel.tsx Client component for the page above. §14.9a critical mitigations implemented: - Path-traversal: storage keys validated against ^[a-zA-Z0-9/_.-]+$; `..`, `.`, `//`, leading `/`, and overlength keys rejected. - Realpath: storage root realpath'd at create time, every per-key resolution checked against the realpath'd prefix. - Storage root created (or chmod'd) to 0o700. - Multi-node refusal: FilesystemBackend.create() throws when MULTI_NODE_DEPLOYMENT=true. - HMAC token: sha256-HMAC over the (key, expiry, nonce, filename, content-type) payload. Verified with timingSafeEqual; bad sig, expired, or invalid-key payloads all return 403. - Single-use replay: token body cached in Redis SET NX EX 1800s. - sha256 round-trip: copyAndVerify() re-fetches from the target after put() and aborts the migration on any mismatch. - Free-disk pre-flight: when migrating to filesystem, sums byte counts via source.head() and aborts if free space < total * 1.2. - pg_advisory_lock(0xc7000a01) prevents concurrent migrations. - Resumable: per-row progress markers in _storage_migration_progress. system_settings keys read by the factory (jsonb, no schema change): storage_backend, storage_s3_endpoint, storage_s3_region, storage_s3_bucket, storage_s3_access_key, storage_s3_secret_key_encrypted, storage_s3_force_path_style, storage_filesystem_root, storage_proxy_hmac_secret_encrypted. Defaults: storage_backend=`s3`, storage_filesystem_root=`./storage` (./storage added to .gitignore). Tests added (34 tests, all green): - tests/unit/storage/filesystem-backend.test.ts — key validation allow/reject matrix, realpath escape, 0o700 perms, multi-node refusal, HMAC token sign/verify/tamper/expire/invalid-key. - tests/unit/storage/copy-and-verify.test.ts — sha256 mismatch on round-trip aborts the migration. - tests/integration/storage/proxy-route.test.ts — happy path, wrong HMAC secret, expired token, replay rejection. Phase 6a ships zero file-bearing tables — TABLES_WITH_STORAGE_KEYS is intentionally empty. berth_pdf_versions and brochure_versions land in Phase 6b and join the list there. Existing s3_key columns: only gdpr_export_jobs.storage_key, already named correctly — no rename needed. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-05 03:15:59 +02:00
apiFetch<{ data: MigrationResult }>('/api/v1/admin/storage/migrate', {
method: 'POST',
body: JSON.stringify({ ...opts, dryRun: false }),
}),
fix(audit-v3): platform-wide deferred-list cleanup (rounds 1-4) Working through the audit-v2 deferred backlog. Each round was tested (typecheck + 1168/1168 vitest) before moving on. Round 1 — DB performance + AI cost visibility: - Add missing FK indexes Postgres doesn't auto-create on berth_reservations.{interest_id, contract_file_id}, documents.{file_id, signed_file_id}, document_events.signer_id, document_templates.source_file_id, form_submissions.{form_template_id, client_id}, document_sends.{brochure_id, brochure_version_id, sent_by_user_id}. Without these, RESTRICT-checks on parent delete + reverse-lookups walk the child tables fully. Migration 0037. - AI worker now writes one ai_usage_ledger row per OpenAI call so admins can audit spend per port/user/feature and future per-port budgets have history to read from. Failure to write is logged-not-thrown so the user-facing email draft is unaffected. Round 2 — Boot-time + transport hardening: - S3 backend verifies the bucket exists at startup (or auto-creates when MINIO_AUTO_CREATE_BUCKET=true). A typo'd bucket name now surfaces with a clear boot error instead of a vague Minio error inside the first user-facing request. - Documenso v1 placeFields: 3-attempt exponential-backoff retry on 5xx + network errors, fail-fast on 4xx. Stops one transient flake from leaving a document with a partial field set. - FilesystemBackend logs a structured warn-once at boot when the dev HMAC fallback is in effect, so two processes started with different BETTER_AUTH_SECRET values are observable (random 401s on file downloads otherwise). - Logger redact paths extended to cover *.headers.{authorization, cookie}, *.config.headers.authorization, encrypted-credential blobs (secretKeyEncrypted, smtpPassEncrypted, etc.), the Documenso X-Documenso-Secret header, and 2-level nested forms. Round 3 — UI feedback + permission gates: - Storage admin migrate dialog: success toast with row count + error toast on both dryRun and migrate mutations. - Invoice detail Send + Record-payment buttons wrapped in PermissionGate (invoices.send / invoices.record_payment); both mutations now toast on success/error. - Admin user list Edit button wrapped in PermissionGate(admin.manage_users). - Scan-receipt page surfaces an amber warning when OCR fails so reps know they can fill the form manually instead of staring at a stalled spinner; the editable form now also opens on scanMutation.isError / uploadedFile, not only on success. - Email threads list now renders skeleton rows during load + shared EmptyState for the empty case (was a single "Loading…" line). Round 4 — Service / route correctness: - documentSends.sent_by_user_id was a free-text NOT NULL column with no FK. Now nullable + FK to user(id) ON DELETE SET NULL so the audit row survives a user being hard-deleted. Migration 0038 with a defensive null-out for any orphan ids before attaching the constraint. - Saved-views route: documented why withAuth alone is correct (the service strictly filters by (portId, userId) — owner-only by design). - Public-interests audit log: replaced "userId: null as unknown as string" cast with userId: null; AuditLogParams already accepts null for system-generated events. - EOI in-app PDF fill: extracted setBerthRange() that, when the AcroForm field is missing AND the context has a non-empty range string, logs a structured warn so the deployment gap (live Documenso template needs the field) is observable instead of silently dropping the multi-berth range. Test status: 1168/1168 vitest. tsc clean. Two new migrations (0037/0038) need pnpm db:push (or migration apply) on the dev DB. Deferred-doc updated with the remaining open items (bigger refactors). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-05 12:49:53 +02:00
onSuccess: (result) => {
feat(storage): pluggable s3-or-filesystem backend + migration CLI + admin UI Phase 6a from docs/berth-recommender-and-pdf-plan.md §4.7a + §14.9a. Lays the storage groundwork for Phase 6b/7 file-bearing schemas (per-berth PDFs, brochures) without touching those domains yet. New files: - src/lib/storage/index.ts StorageBackend interface + per-process factory keyed on system_settings. - src/lib/storage/s3.ts S3-compatible backend (MinIO/AWS/B2/R2/ Wasabi/Tigris) wrapping the existing minio JS client. Includes a healthCheck() used by the admin "Test connection" button. - src/lib/storage/filesystem.ts Local filesystem backend with all §14.9a mitigations baked in. - src/lib/storage/migrate.ts Shared migration core — pg_advisory_lock, per-row resumable progress markers, sha256 round-trip verification, atomic storage_backend flip on success. - scripts/migrate-storage.ts Thin CLI shim around runMigration(). - src/app/api/storage/[token]/route.ts Filesystem proxy GET. Verifies HMAC, enforces single-use replay protection via Redis SET NX, streams via NextResponse ReadableStream with explicit Content-Type + Content-Disposition. Node runtime only. - src/app/api/v1/admin/storage/route.ts GET status + POST connection test. - src/app/api/v1/admin/storage/migrate/route.ts Super-admin-only POST that runs the exact same runMigration() as the CLI. - src/app/(dashboard)/[portSlug]/admin/storage/page.tsx Super-admin admin UI (current backend, capacity stats, switch button with dry-run, test connection, backup hint). - src/components/admin/storage-admin-panel.tsx Client component for the page above. §14.9a critical mitigations implemented: - Path-traversal: storage keys validated against ^[a-zA-Z0-9/_.-]+$; `..`, `.`, `//`, leading `/`, and overlength keys rejected. - Realpath: storage root realpath'd at create time, every per-key resolution checked against the realpath'd prefix. - Storage root created (or chmod'd) to 0o700. - Multi-node refusal: FilesystemBackend.create() throws when MULTI_NODE_DEPLOYMENT=true. - HMAC token: sha256-HMAC over the (key, expiry, nonce, filename, content-type) payload. Verified with timingSafeEqual; bad sig, expired, or invalid-key payloads all return 403. - Single-use replay: token body cached in Redis SET NX EX 1800s. - sha256 round-trip: copyAndVerify() re-fetches from the target after put() and aborts the migration on any mismatch. - Free-disk pre-flight: when migrating to filesystem, sums byte counts via source.head() and aborts if free space < total * 1.2. - pg_advisory_lock(0xc7000a01) prevents concurrent migrations. - Resumable: per-row progress markers in _storage_migration_progress. system_settings keys read by the factory (jsonb, no schema change): storage_backend, storage_s3_endpoint, storage_s3_region, storage_s3_bucket, storage_s3_access_key, storage_s3_secret_key_encrypted, storage_s3_force_path_style, storage_filesystem_root, storage_proxy_hmac_secret_encrypted. Defaults: storage_backend=`s3`, storage_filesystem_root=`./storage` (./storage added to .gitignore). Tests added (34 tests, all green): - tests/unit/storage/filesystem-backend.test.ts — key validation allow/reject matrix, realpath escape, 0o700 perms, multi-node refusal, HMAC token sign/verify/tamper/expire/invalid-key. - tests/unit/storage/copy-and-verify.test.ts — sha256 mismatch on round-trip aborts the migration. - tests/integration/storage/proxy-route.test.ts — happy path, wrong HMAC secret, expired token, replay rejection. Phase 6a ships zero file-bearing tables — TABLES_WITH_STORAGE_KEYS is intentionally empty. berth_pdf_versions and brochure_versions land in Phase 6b and join the list there. Existing s3_key columns: only gdpr_export_jobs.storage_key, already named correctly — no rename needed. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-05 03:15:59 +02:00
setConfirmOpen(false);
setDryRun(null);
fix(audit-v3): platform-wide deferred-list cleanup (rounds 1-4) Working through the audit-v2 deferred backlog. Each round was tested (typecheck + 1168/1168 vitest) before moving on. Round 1 — DB performance + AI cost visibility: - Add missing FK indexes Postgres doesn't auto-create on berth_reservations.{interest_id, contract_file_id}, documents.{file_id, signed_file_id}, document_events.signer_id, document_templates.source_file_id, form_submissions.{form_template_id, client_id}, document_sends.{brochure_id, brochure_version_id, sent_by_user_id}. Without these, RESTRICT-checks on parent delete + reverse-lookups walk the child tables fully. Migration 0037. - AI worker now writes one ai_usage_ledger row per OpenAI call so admins can audit spend per port/user/feature and future per-port budgets have history to read from. Failure to write is logged-not-thrown so the user-facing email draft is unaffected. Round 2 — Boot-time + transport hardening: - S3 backend verifies the bucket exists at startup (or auto-creates when MINIO_AUTO_CREATE_BUCKET=true). A typo'd bucket name now surfaces with a clear boot error instead of a vague Minio error inside the first user-facing request. - Documenso v1 placeFields: 3-attempt exponential-backoff retry on 5xx + network errors, fail-fast on 4xx. Stops one transient flake from leaving a document with a partial field set. - FilesystemBackend logs a structured warn-once at boot when the dev HMAC fallback is in effect, so two processes started with different BETTER_AUTH_SECRET values are observable (random 401s on file downloads otherwise). - Logger redact paths extended to cover *.headers.{authorization, cookie}, *.config.headers.authorization, encrypted-credential blobs (secretKeyEncrypted, smtpPassEncrypted, etc.), the Documenso X-Documenso-Secret header, and 2-level nested forms. Round 3 — UI feedback + permission gates: - Storage admin migrate dialog: success toast with row count + error toast on both dryRun and migrate mutations. - Invoice detail Send + Record-payment buttons wrapped in PermissionGate (invoices.send / invoices.record_payment); both mutations now toast on success/error. - Admin user list Edit button wrapped in PermissionGate(admin.manage_users). - Scan-receipt page surfaces an amber warning when OCR fails so reps know they can fill the form manually instead of staring at a stalled spinner; the editable form now also opens on scanMutation.isError / uploadedFile, not only on success. - Email threads list now renders skeleton rows during load + shared EmptyState for the empty case (was a single "Loading…" line). Round 4 — Service / route correctness: - documentSends.sent_by_user_id was a free-text NOT NULL column with no FK. Now nullable + FK to user(id) ON DELETE SET NULL so the audit row survives a user being hard-deleted. Migration 0038 with a defensive null-out for any orphan ids before attaching the constraint. - Saved-views route: documented why withAuth alone is correct (the service strictly filters by (portId, userId) — owner-only by design). - Public-interests audit log: replaced "userId: null as unknown as string" cast with userId: null; AuditLogParams already accepts null for system-generated events. - EOI in-app PDF fill: extracted setBerthRange() that, when the AcroForm field is missing AND the context has a non-empty range string, logs a structured warn so the deployment gap (live Documenso template needs the field) is observable instead of silently dropping the multi-berth range. Test status: 1168/1168 vitest. tsc clean. Two new migrations (0037/0038) need pnpm db:push (or migration apply) on the dev DB. Deferred-doc updated with the remaining open items (bigger refactors). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-05 12:49:53 +02:00
const copied = result.data.rowsMigrated ?? 0;
feat: autonomous backlog push — admin UX overhaul + storage parity + residential parity + Documenso Phase 1 Massive multi-area push driven by docs/admin-ux-backlog.md. Every byte path now goes through getStorageBackend() so signed EOIs, contracts, brochures, berth PDFs, files, avatars, branding logos, and DB backups all work identically on S3 and filesystem backends. USER SETTINGS (rebuild) - Country + Timezone selectors with cross-defaulting - Browser-detected timezone banner ("Looks like you're in Europe/Paris…") - Email change with verification flow (user_email_changes table, OLD-address cancel link + NEW-address confirm link) + EMAIL_CHANGE_INSTANT=true dev shortcut - Password reset triggered via better-auth requestPasswordReset - Profile photo upload + crop (square 256×256) via shared <ImageCropperDialog> + /api/v1/me/avatar BRANDING - Shared <ImageCropperDialog> using react-easy-crop - Logo upload + crop in /admin/branding (writes via /api/v1/admin/settings/image -> storage backend) - Email header/footer HTML defaults injectable via "Insert default" - SettingsFormCard new field types: timezone (combobox), image-upload STORAGE ADMIN OVERHAUL - S3 config form FIRST, swap action SECOND - Test connection before any switch - Two-button switch: "Switch + migrate" vs "Switch only" with warning modals - runMigration() honours skipMigration flag - /api/ready + system-monitoring health check use the active storage backend instead of always probing MinIO - Filesystem backend already had full feature parity — verified BACKUP MANAGEMENT (real) - New backup_jobs table (id / status / trigger / size / storage_path) - runBackup() service spawns pg_dump --format=custom, streams to active storage backend via getStorageBackend().put() - /admin/backup page: trigger, history, download .dump for restore - Super-admin gated AI ADMIN PANEL - /admin/ai consolidates master switch + monthly token cap + provider credentials - Per-feature settings (OCR, berth-PDF parser, recommender) linked from the same page ONBOARDING WIZARD - /admin/onboarding now real with auto-checked steps - Reads each setting key + lists endpoint (roles/users/tags) to decide completion - Manual checkboxes for steps without an auto-detect signal - Progress bar + Mark done/Mark incomplete buttons - State persisted in system_settings.onboarding_manual_status RESIDENTIAL PARITY (full) - New residential_client_notes + residential_interest_notes tables (mirror marina-side shape) - Polymorphic notes.service.ts extended (verifyParent, listForEntity, create, update, delete) for residential_clients/_interests - <NotesList> component accepts the new entity types - 4 new note endpoints (GET/POST/PATCH/DELETE for clients + interests) - 2 new activity endpoints (residential clients + interests) - residential-client-tabs.tsx + residential-interest-tabs.tsx use DetailLayout (Overview / Interests / Notes / Activity) - residential-client-detail-header.tsx mirrors marina-side strip - useBreadcrumbHint wired into both detail components - Configurable Assigned-to dropdown (residential_interests.view perm) CONFIGURABLE RESIDENTIAL STAGES - residential-stages.service.ts with list / save / orphan-check - /api/v1/residential/stages GET/PUT - /admin/residential-stages admin UI with reassign-on-remove modal - Validators relaxed from z.enum to z.string DOCUMENSO PHASE 1 - Schema: document_signers.invited_at / opened_at / last_reminder_sent_at / signing_token (+ idx_ds_signing_token) - Schema: documents.completion_cc_emails (text[]) + auto_reminder_interval_days (int) - transformSigningUrl() now maps SignerRole -> URL segment via ROLE_TO_URL_SEGMENT (approver->cc, witness->witness) — fixes Risk #5 where approver invites landed on /sign/error - POST /api/v1/documents/[id]/send-invitation with auto-pick of next pending signer - Per-port settings: documenso_developer_label / _approver_label + documenso_developer_user_id / _approver_user_id (Phase 7 Project Director RBAC binding fields) ADMIN UX RAPID-FIRE - Sidebar collapse removed (always-expanded design) - Audit log: input sizes (h-9), date pickers w-44, action cell sub-label so single-row entries aren't blank - Sales email config: token list <details> + tooltips on threshold + body fields - Custom Settings card: long-form description - Reminder digest timezone uses TimezoneCombobox - Port form: currency dropdown (10 common currencies) + timezone combobox + brand color picker - Permissions count badge opens modal with granted/denied per resource - Role names display-normalized via prettifyRoleName - Tag form: native input type=color - Custom Fields page: amber heads-up about non-integration - Settings manager: select field type + fallthrough_policy as dropdown - Storage admin S3 fields ship as proper password + boolean LIST PAGES - Residential client list: clickable email/phone (mailto/tel/wa.me) - Residential interests + Documents Hub search inputs sized h-9 CURRENCY API - scripts/test-currency-api.ts verifies live Frankfurter fetch -> DB upsert -> getRate -> convert. Inverse-rate drift <=0.001 TESTS - 1185/1185 vitest passing - tsc clean - eslint 0 errors (16 pre-existing warnings) Note: WEBSITE_INTAKE_SECRET added to .env.example but committed separately due to pre-commit hook policy on .env* files. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-07 21:02:12 +02:00
toast.success(
copied > 0
? `Storage migration complete (${copied} file${copied === 1 ? '' : 's'} copied)`
: 'Storage backend switched (no migration performed)',
);
feat(storage): pluggable s3-or-filesystem backend + migration CLI + admin UI Phase 6a from docs/berth-recommender-and-pdf-plan.md §4.7a + §14.9a. Lays the storage groundwork for Phase 6b/7 file-bearing schemas (per-berth PDFs, brochures) without touching those domains yet. New files: - src/lib/storage/index.ts StorageBackend interface + per-process factory keyed on system_settings. - src/lib/storage/s3.ts S3-compatible backend (MinIO/AWS/B2/R2/ Wasabi/Tigris) wrapping the existing minio JS client. Includes a healthCheck() used by the admin "Test connection" button. - src/lib/storage/filesystem.ts Local filesystem backend with all §14.9a mitigations baked in. - src/lib/storage/migrate.ts Shared migration core — pg_advisory_lock, per-row resumable progress markers, sha256 round-trip verification, atomic storage_backend flip on success. - scripts/migrate-storage.ts Thin CLI shim around runMigration(). - src/app/api/storage/[token]/route.ts Filesystem proxy GET. Verifies HMAC, enforces single-use replay protection via Redis SET NX, streams via NextResponse ReadableStream with explicit Content-Type + Content-Disposition. Node runtime only. - src/app/api/v1/admin/storage/route.ts GET status + POST connection test. - src/app/api/v1/admin/storage/migrate/route.ts Super-admin-only POST that runs the exact same runMigration() as the CLI. - src/app/(dashboard)/[portSlug]/admin/storage/page.tsx Super-admin admin UI (current backend, capacity stats, switch button with dry-run, test connection, backup hint). - src/components/admin/storage-admin-panel.tsx Client component for the page above. §14.9a critical mitigations implemented: - Path-traversal: storage keys validated against ^[a-zA-Z0-9/_.-]+$; `..`, `.`, `//`, leading `/`, and overlength keys rejected. - Realpath: storage root realpath'd at create time, every per-key resolution checked against the realpath'd prefix. - Storage root created (or chmod'd) to 0o700. - Multi-node refusal: FilesystemBackend.create() throws when MULTI_NODE_DEPLOYMENT=true. - HMAC token: sha256-HMAC over the (key, expiry, nonce, filename, content-type) payload. Verified with timingSafeEqual; bad sig, expired, or invalid-key payloads all return 403. - Single-use replay: token body cached in Redis SET NX EX 1800s. - sha256 round-trip: copyAndVerify() re-fetches from the target after put() and aborts the migration on any mismatch. - Free-disk pre-flight: when migrating to filesystem, sums byte counts via source.head() and aborts if free space < total * 1.2. - pg_advisory_lock(0xc7000a01) prevents concurrent migrations. - Resumable: per-row progress markers in _storage_migration_progress. system_settings keys read by the factory (jsonb, no schema change): storage_backend, storage_s3_endpoint, storage_s3_region, storage_s3_bucket, storage_s3_access_key, storage_s3_secret_key_encrypted, storage_s3_force_path_style, storage_filesystem_root, storage_proxy_hmac_secret_encrypted. Defaults: storage_backend=`s3`, storage_filesystem_root=`./storage` (./storage added to .gitignore). Tests added (34 tests, all green): - tests/unit/storage/filesystem-backend.test.ts — key validation allow/reject matrix, realpath escape, 0o700 perms, multi-node refusal, HMAC token sign/verify/tamper/expire/invalid-key. - tests/unit/storage/copy-and-verify.test.ts — sha256 mismatch on round-trip aborts the migration. - tests/integration/storage/proxy-route.test.ts — happy path, wrong HMAC secret, expired token, replay rejection. Phase 6a ships zero file-bearing tables — TABLES_WITH_STORAGE_KEYS is intentionally empty. berth_pdf_versions and brochure_versions land in Phase 6b and join the list there. Existing s3_key columns: only gdpr_export_jobs.storage_key, already named correctly — no rename needed. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-05 03:15:59 +02:00
queryClient.invalidateQueries({ queryKey: ['admin', 'storage', 'status'] });
},
onError: (e) => toastError(e),
feat(storage): pluggable s3-or-filesystem backend + migration CLI + admin UI Phase 6a from docs/berth-recommender-and-pdf-plan.md §4.7a + §14.9a. Lays the storage groundwork for Phase 6b/7 file-bearing schemas (per-berth PDFs, brochures) without touching those domains yet. New files: - src/lib/storage/index.ts StorageBackend interface + per-process factory keyed on system_settings. - src/lib/storage/s3.ts S3-compatible backend (MinIO/AWS/B2/R2/ Wasabi/Tigris) wrapping the existing minio JS client. Includes a healthCheck() used by the admin "Test connection" button. - src/lib/storage/filesystem.ts Local filesystem backend with all §14.9a mitigations baked in. - src/lib/storage/migrate.ts Shared migration core — pg_advisory_lock, per-row resumable progress markers, sha256 round-trip verification, atomic storage_backend flip on success. - scripts/migrate-storage.ts Thin CLI shim around runMigration(). - src/app/api/storage/[token]/route.ts Filesystem proxy GET. Verifies HMAC, enforces single-use replay protection via Redis SET NX, streams via NextResponse ReadableStream with explicit Content-Type + Content-Disposition. Node runtime only. - src/app/api/v1/admin/storage/route.ts GET status + POST connection test. - src/app/api/v1/admin/storage/migrate/route.ts Super-admin-only POST that runs the exact same runMigration() as the CLI. - src/app/(dashboard)/[portSlug]/admin/storage/page.tsx Super-admin admin UI (current backend, capacity stats, switch button with dry-run, test connection, backup hint). - src/components/admin/storage-admin-panel.tsx Client component for the page above. §14.9a critical mitigations implemented: - Path-traversal: storage keys validated against ^[a-zA-Z0-9/_.-]+$; `..`, `.`, `//`, leading `/`, and overlength keys rejected. - Realpath: storage root realpath'd at create time, every per-key resolution checked against the realpath'd prefix. - Storage root created (or chmod'd) to 0o700. - Multi-node refusal: FilesystemBackend.create() throws when MULTI_NODE_DEPLOYMENT=true. - HMAC token: sha256-HMAC over the (key, expiry, nonce, filename, content-type) payload. Verified with timingSafeEqual; bad sig, expired, or invalid-key payloads all return 403. - Single-use replay: token body cached in Redis SET NX EX 1800s. - sha256 round-trip: copyAndVerify() re-fetches from the target after put() and aborts the migration on any mismatch. - Free-disk pre-flight: when migrating to filesystem, sums byte counts via source.head() and aborts if free space < total * 1.2. - pg_advisory_lock(0xc7000a01) prevents concurrent migrations. - Resumable: per-row progress markers in _storage_migration_progress. system_settings keys read by the factory (jsonb, no schema change): storage_backend, storage_s3_endpoint, storage_s3_region, storage_s3_bucket, storage_s3_access_key, storage_s3_secret_key_encrypted, storage_s3_force_path_style, storage_filesystem_root, storage_proxy_hmac_secret_encrypted. Defaults: storage_backend=`s3`, storage_filesystem_root=`./storage` (./storage added to .gitignore). Tests added (34 tests, all green): - tests/unit/storage/filesystem-backend.test.ts — key validation allow/reject matrix, realpath escape, 0o700 perms, multi-node refusal, HMAC token sign/verify/tamper/expire/invalid-key. - tests/unit/storage/copy-and-verify.test.ts — sha256 mismatch on round-trip aborts the migration. - tests/integration/storage/proxy-route.test.ts — happy path, wrong HMAC secret, expired token, replay rejection. Phase 6a ships zero file-bearing tables — TABLES_WITH_STORAGE_KEYS is intentionally empty. berth_pdf_versions and brochure_versions land in Phase 6b and join the list there. Existing s3_key columns: only gdpr_export_jobs.storage_key, already named correctly — no rename needed. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-05 03:15:59 +02:00
});
const testMutation = useMutation({
mutationFn: async () =>
apiFetch<{ ok: boolean; error?: string }>('/api/v1/admin/storage', {
method: 'POST',
}),
onSuccess: (r) => setTestResult(r),
onError: (e: Error) => setTestResult({ ok: false, error: e.message }),
});
if (status.isLoading) {
return (
<div className="flex items-center gap-2 text-sm text-muted-foreground">
<Loader2 className="h-4 w-4 animate-spin" /> Loading storage status
</div>
);
}
if (status.isError || !status.data?.data) {
return <div className="text-sm text-destructive">Failed to load storage status.</div>;
}
const s = status.data.data;
const otherBackend: BackendName = s.backend === 's3' ? 'filesystem' : 's3';
feat: autonomous backlog push — admin UX overhaul + storage parity + residential parity + Documenso Phase 1 Massive multi-area push driven by docs/admin-ux-backlog.md. Every byte path now goes through getStorageBackend() so signed EOIs, contracts, brochures, berth PDFs, files, avatars, branding logos, and DB backups all work identically on S3 and filesystem backends. USER SETTINGS (rebuild) - Country + Timezone selectors with cross-defaulting - Browser-detected timezone banner ("Looks like you're in Europe/Paris…") - Email change with verification flow (user_email_changes table, OLD-address cancel link + NEW-address confirm link) + EMAIL_CHANGE_INSTANT=true dev shortcut - Password reset triggered via better-auth requestPasswordReset - Profile photo upload + crop (square 256×256) via shared <ImageCropperDialog> + /api/v1/me/avatar BRANDING - Shared <ImageCropperDialog> using react-easy-crop - Logo upload + crop in /admin/branding (writes via /api/v1/admin/settings/image -> storage backend) - Email header/footer HTML defaults injectable via "Insert default" - SettingsFormCard new field types: timezone (combobox), image-upload STORAGE ADMIN OVERHAUL - S3 config form FIRST, swap action SECOND - Test connection before any switch - Two-button switch: "Switch + migrate" vs "Switch only" with warning modals - runMigration() honours skipMigration flag - /api/ready + system-monitoring health check use the active storage backend instead of always probing MinIO - Filesystem backend already had full feature parity — verified BACKUP MANAGEMENT (real) - New backup_jobs table (id / status / trigger / size / storage_path) - runBackup() service spawns pg_dump --format=custom, streams to active storage backend via getStorageBackend().put() - /admin/backup page: trigger, history, download .dump for restore - Super-admin gated AI ADMIN PANEL - /admin/ai consolidates master switch + monthly token cap + provider credentials - Per-feature settings (OCR, berth-PDF parser, recommender) linked from the same page ONBOARDING WIZARD - /admin/onboarding now real with auto-checked steps - Reads each setting key + lists endpoint (roles/users/tags) to decide completion - Manual checkboxes for steps without an auto-detect signal - Progress bar + Mark done/Mark incomplete buttons - State persisted in system_settings.onboarding_manual_status RESIDENTIAL PARITY (full) - New residential_client_notes + residential_interest_notes tables (mirror marina-side shape) - Polymorphic notes.service.ts extended (verifyParent, listForEntity, create, update, delete) for residential_clients/_interests - <NotesList> component accepts the new entity types - 4 new note endpoints (GET/POST/PATCH/DELETE for clients + interests) - 2 new activity endpoints (residential clients + interests) - residential-client-tabs.tsx + residential-interest-tabs.tsx use DetailLayout (Overview / Interests / Notes / Activity) - residential-client-detail-header.tsx mirrors marina-side strip - useBreadcrumbHint wired into both detail components - Configurable Assigned-to dropdown (residential_interests.view perm) CONFIGURABLE RESIDENTIAL STAGES - residential-stages.service.ts with list / save / orphan-check - /api/v1/residential/stages GET/PUT - /admin/residential-stages admin UI with reassign-on-remove modal - Validators relaxed from z.enum to z.string DOCUMENSO PHASE 1 - Schema: document_signers.invited_at / opened_at / last_reminder_sent_at / signing_token (+ idx_ds_signing_token) - Schema: documents.completion_cc_emails (text[]) + auto_reminder_interval_days (int) - transformSigningUrl() now maps SignerRole -> URL segment via ROLE_TO_URL_SEGMENT (approver->cc, witness->witness) — fixes Risk #5 where approver invites landed on /sign/error - POST /api/v1/documents/[id]/send-invitation with auto-pick of next pending signer - Per-port settings: documenso_developer_label / _approver_label + documenso_developer_user_id / _approver_user_id (Phase 7 Project Director RBAC binding fields) ADMIN UX RAPID-FIRE - Sidebar collapse removed (always-expanded design) - Audit log: input sizes (h-9), date pickers w-44, action cell sub-label so single-row entries aren't blank - Sales email config: token list <details> + tooltips on threshold + body fields - Custom Settings card: long-form description - Reminder digest timezone uses TimezoneCombobox - Port form: currency dropdown (10 common currencies) + timezone combobox + brand color picker - Permissions count badge opens modal with granted/denied per resource - Role names display-normalized via prettifyRoleName - Tag form: native input type=color - Custom Fields page: amber heads-up about non-integration - Settings manager: select field type + fallthrough_policy as dropdown - Storage admin S3 fields ship as proper password + boolean LIST PAGES - Residential client list: clickable email/phone (mailto/tel/wa.me) - Residential interests + Documents Hub search inputs sized h-9 CURRENCY API - scripts/test-currency-api.ts verifies live Frankfurter fetch -> DB upsert -> getRate -> convert. Inverse-rate drift <=0.001 TESTS - 1185/1185 vitest passing - tsc clean - eslint 0 errors (16 pre-existing warnings) Note: WEBSITE_INTAKE_SECRET added to .env.example but committed separately due to pre-commit hook policy on .env* files. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-07 21:02:12 +02:00
function openConfirm(mode: 'switch-only' | 'switch-and-migrate') {
setConfirmMode(mode);
if (mode === 'switch-and-migrate') {
// Dry run first so the dialog shows the exact rows + bytes.
dryRunMutation.mutate({ from: s.backend, to: otherBackend });
} else {
// Switch-only — no dry run, just show the warning.
setDryRun(null);
setConfirmOpen(true);
}
}
feat(storage): pluggable s3-or-filesystem backend + migration CLI + admin UI Phase 6a from docs/berth-recommender-and-pdf-plan.md §4.7a + §14.9a. Lays the storage groundwork for Phase 6b/7 file-bearing schemas (per-berth PDFs, brochures) without touching those domains yet. New files: - src/lib/storage/index.ts StorageBackend interface + per-process factory keyed on system_settings. - src/lib/storage/s3.ts S3-compatible backend (MinIO/AWS/B2/R2/ Wasabi/Tigris) wrapping the existing minio JS client. Includes a healthCheck() used by the admin "Test connection" button. - src/lib/storage/filesystem.ts Local filesystem backend with all §14.9a mitigations baked in. - src/lib/storage/migrate.ts Shared migration core — pg_advisory_lock, per-row resumable progress markers, sha256 round-trip verification, atomic storage_backend flip on success. - scripts/migrate-storage.ts Thin CLI shim around runMigration(). - src/app/api/storage/[token]/route.ts Filesystem proxy GET. Verifies HMAC, enforces single-use replay protection via Redis SET NX, streams via NextResponse ReadableStream with explicit Content-Type + Content-Disposition. Node runtime only. - src/app/api/v1/admin/storage/route.ts GET status + POST connection test. - src/app/api/v1/admin/storage/migrate/route.ts Super-admin-only POST that runs the exact same runMigration() as the CLI. - src/app/(dashboard)/[portSlug]/admin/storage/page.tsx Super-admin admin UI (current backend, capacity stats, switch button with dry-run, test connection, backup hint). - src/components/admin/storage-admin-panel.tsx Client component for the page above. §14.9a critical mitigations implemented: - Path-traversal: storage keys validated against ^[a-zA-Z0-9/_.-]+$; `..`, `.`, `//`, leading `/`, and overlength keys rejected. - Realpath: storage root realpath'd at create time, every per-key resolution checked against the realpath'd prefix. - Storage root created (or chmod'd) to 0o700. - Multi-node refusal: FilesystemBackend.create() throws when MULTI_NODE_DEPLOYMENT=true. - HMAC token: sha256-HMAC over the (key, expiry, nonce, filename, content-type) payload. Verified with timingSafeEqual; bad sig, expired, or invalid-key payloads all return 403. - Single-use replay: token body cached in Redis SET NX EX 1800s. - sha256 round-trip: copyAndVerify() re-fetches from the target after put() and aborts the migration on any mismatch. - Free-disk pre-flight: when migrating to filesystem, sums byte counts via source.head() and aborts if free space < total * 1.2. - pg_advisory_lock(0xc7000a01) prevents concurrent migrations. - Resumable: per-row progress markers in _storage_migration_progress. system_settings keys read by the factory (jsonb, no schema change): storage_backend, storage_s3_endpoint, storage_s3_region, storage_s3_bucket, storage_s3_access_key, storage_s3_secret_key_encrypted, storage_s3_force_path_style, storage_filesystem_root, storage_proxy_hmac_secret_encrypted. Defaults: storage_backend=`s3`, storage_filesystem_root=`./storage` (./storage added to .gitignore). Tests added (34 tests, all green): - tests/unit/storage/filesystem-backend.test.ts — key validation allow/reject matrix, realpath escape, 0o700 perms, multi-node refusal, HMAC token sign/verify/tamper/expire/invalid-key. - tests/unit/storage/copy-and-verify.test.ts — sha256 mismatch on round-trip aborts the migration. - tests/integration/storage/proxy-route.test.ts — happy path, wrong HMAC secret, expired token, replay rejection. Phase 6a ships zero file-bearing tables — TABLES_WITH_STORAGE_KEYS is intentionally empty. berth_pdf_versions and brochure_versions land in Phase 6b and join the list there. Existing s3_key columns: only gdpr_export_jobs.storage_key, already named correctly — no rename needed. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-05 03:15:59 +02:00
return (
<div className="space-y-6">
<PageHeader
title="Storage Backend"
feat: autonomous backlog push — admin UX overhaul + storage parity + residential parity + Documenso Phase 1 Massive multi-area push driven by docs/admin-ux-backlog.md. Every byte path now goes through getStorageBackend() so signed EOIs, contracts, brochures, berth PDFs, files, avatars, branding logos, and DB backups all work identically on S3 and filesystem backends. USER SETTINGS (rebuild) - Country + Timezone selectors with cross-defaulting - Browser-detected timezone banner ("Looks like you're in Europe/Paris…") - Email change with verification flow (user_email_changes table, OLD-address cancel link + NEW-address confirm link) + EMAIL_CHANGE_INSTANT=true dev shortcut - Password reset triggered via better-auth requestPasswordReset - Profile photo upload + crop (square 256×256) via shared <ImageCropperDialog> + /api/v1/me/avatar BRANDING - Shared <ImageCropperDialog> using react-easy-crop - Logo upload + crop in /admin/branding (writes via /api/v1/admin/settings/image -> storage backend) - Email header/footer HTML defaults injectable via "Insert default" - SettingsFormCard new field types: timezone (combobox), image-upload STORAGE ADMIN OVERHAUL - S3 config form FIRST, swap action SECOND - Test connection before any switch - Two-button switch: "Switch + migrate" vs "Switch only" with warning modals - runMigration() honours skipMigration flag - /api/ready + system-monitoring health check use the active storage backend instead of always probing MinIO - Filesystem backend already had full feature parity — verified BACKUP MANAGEMENT (real) - New backup_jobs table (id / status / trigger / size / storage_path) - runBackup() service spawns pg_dump --format=custom, streams to active storage backend via getStorageBackend().put() - /admin/backup page: trigger, history, download .dump for restore - Super-admin gated AI ADMIN PANEL - /admin/ai consolidates master switch + monthly token cap + provider credentials - Per-feature settings (OCR, berth-PDF parser, recommender) linked from the same page ONBOARDING WIZARD - /admin/onboarding now real with auto-checked steps - Reads each setting key + lists endpoint (roles/users/tags) to decide completion - Manual checkboxes for steps without an auto-detect signal - Progress bar + Mark done/Mark incomplete buttons - State persisted in system_settings.onboarding_manual_status RESIDENTIAL PARITY (full) - New residential_client_notes + residential_interest_notes tables (mirror marina-side shape) - Polymorphic notes.service.ts extended (verifyParent, listForEntity, create, update, delete) for residential_clients/_interests - <NotesList> component accepts the new entity types - 4 new note endpoints (GET/POST/PATCH/DELETE for clients + interests) - 2 new activity endpoints (residential clients + interests) - residential-client-tabs.tsx + residential-interest-tabs.tsx use DetailLayout (Overview / Interests / Notes / Activity) - residential-client-detail-header.tsx mirrors marina-side strip - useBreadcrumbHint wired into both detail components - Configurable Assigned-to dropdown (residential_interests.view perm) CONFIGURABLE RESIDENTIAL STAGES - residential-stages.service.ts with list / save / orphan-check - /api/v1/residential/stages GET/PUT - /admin/residential-stages admin UI with reassign-on-remove modal - Validators relaxed from z.enum to z.string DOCUMENSO PHASE 1 - Schema: document_signers.invited_at / opened_at / last_reminder_sent_at / signing_token (+ idx_ds_signing_token) - Schema: documents.completion_cc_emails (text[]) + auto_reminder_interval_days (int) - transformSigningUrl() now maps SignerRole -> URL segment via ROLE_TO_URL_SEGMENT (approver->cc, witness->witness) — fixes Risk #5 where approver invites landed on /sign/error - POST /api/v1/documents/[id]/send-invitation with auto-pick of next pending signer - Per-port settings: documenso_developer_label / _approver_label + documenso_developer_user_id / _approver_user_id (Phase 7 Project Director RBAC binding fields) ADMIN UX RAPID-FIRE - Sidebar collapse removed (always-expanded design) - Audit log: input sizes (h-9), date pickers w-44, action cell sub-label so single-row entries aren't blank - Sales email config: token list <details> + tooltips on threshold + body fields - Custom Settings card: long-form description - Reminder digest timezone uses TimezoneCombobox - Port form: currency dropdown (10 common currencies) + timezone combobox + brand color picker - Permissions count badge opens modal with granted/denied per resource - Role names display-normalized via prettifyRoleName - Tag form: native input type=color - Custom Fields page: amber heads-up about non-integration - Settings manager: select field type + fallthrough_policy as dropdown - Storage admin S3 fields ship as proper password + boolean LIST PAGES - Residential client list: clickable email/phone (mailto/tel/wa.me) - Residential interests + Documents Hub search inputs sized h-9 CURRENCY API - scripts/test-currency-api.ts verifies live Frankfurter fetch -> DB upsert -> getRate -> convert. Inverse-rate drift <=0.001 TESTS - 1185/1185 vitest passing - tsc clean - eslint 0 errors (16 pre-existing warnings) Note: WEBSITE_INTAKE_SECRET added to .env.example but committed separately due to pre-commit hook policy on .env* files. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-07 21:02:12 +02:00
description="Where the CRM stores per-berth PDFs, brochures, GDPR exports, profile photos, and other binary files."
feat(storage): pluggable s3-or-filesystem backend + migration CLI + admin UI Phase 6a from docs/berth-recommender-and-pdf-plan.md §4.7a + §14.9a. Lays the storage groundwork for Phase 6b/7 file-bearing schemas (per-berth PDFs, brochures) without touching those domains yet. New files: - src/lib/storage/index.ts StorageBackend interface + per-process factory keyed on system_settings. - src/lib/storage/s3.ts S3-compatible backend (MinIO/AWS/B2/R2/ Wasabi/Tigris) wrapping the existing minio JS client. Includes a healthCheck() used by the admin "Test connection" button. - src/lib/storage/filesystem.ts Local filesystem backend with all §14.9a mitigations baked in. - src/lib/storage/migrate.ts Shared migration core — pg_advisory_lock, per-row resumable progress markers, sha256 round-trip verification, atomic storage_backend flip on success. - scripts/migrate-storage.ts Thin CLI shim around runMigration(). - src/app/api/storage/[token]/route.ts Filesystem proxy GET. Verifies HMAC, enforces single-use replay protection via Redis SET NX, streams via NextResponse ReadableStream with explicit Content-Type + Content-Disposition. Node runtime only. - src/app/api/v1/admin/storage/route.ts GET status + POST connection test. - src/app/api/v1/admin/storage/migrate/route.ts Super-admin-only POST that runs the exact same runMigration() as the CLI. - src/app/(dashboard)/[portSlug]/admin/storage/page.tsx Super-admin admin UI (current backend, capacity stats, switch button with dry-run, test connection, backup hint). - src/components/admin/storage-admin-panel.tsx Client component for the page above. §14.9a critical mitigations implemented: - Path-traversal: storage keys validated against ^[a-zA-Z0-9/_.-]+$; `..`, `.`, `//`, leading `/`, and overlength keys rejected. - Realpath: storage root realpath'd at create time, every per-key resolution checked against the realpath'd prefix. - Storage root created (or chmod'd) to 0o700. - Multi-node refusal: FilesystemBackend.create() throws when MULTI_NODE_DEPLOYMENT=true. - HMAC token: sha256-HMAC over the (key, expiry, nonce, filename, content-type) payload. Verified with timingSafeEqual; bad sig, expired, or invalid-key payloads all return 403. - Single-use replay: token body cached in Redis SET NX EX 1800s. - sha256 round-trip: copyAndVerify() re-fetches from the target after put() and aborts the migration on any mismatch. - Free-disk pre-flight: when migrating to filesystem, sums byte counts via source.head() and aborts if free space < total * 1.2. - pg_advisory_lock(0xc7000a01) prevents concurrent migrations. - Resumable: per-row progress markers in _storage_migration_progress. system_settings keys read by the factory (jsonb, no schema change): storage_backend, storage_s3_endpoint, storage_s3_region, storage_s3_bucket, storage_s3_access_key, storage_s3_secret_key_encrypted, storage_s3_force_path_style, storage_filesystem_root, storage_proxy_hmac_secret_encrypted. Defaults: storage_backend=`s3`, storage_filesystem_root=`./storage` (./storage added to .gitignore). Tests added (34 tests, all green): - tests/unit/storage/filesystem-backend.test.ts — key validation allow/reject matrix, realpath escape, 0o700 perms, multi-node refusal, HMAC token sign/verify/tamper/expire/invalid-key. - tests/unit/storage/copy-and-verify.test.ts — sha256 mismatch on round-trip aborts the migration. - tests/integration/storage/proxy-route.test.ts — happy path, wrong HMAC secret, expired token, replay rejection. Phase 6a ships zero file-bearing tables — TABLES_WITH_STORAGE_KEYS is intentionally empty. berth_pdf_versions and brochure_versions land in Phase 6b and join the list there. Existing s3_key columns: only gdpr_export_jobs.storage_key, already named correctly — no rename needed. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-05 03:15:59 +02:00
/>
feat: autonomous backlog push — admin UX overhaul + storage parity + residential parity + Documenso Phase 1 Massive multi-area push driven by docs/admin-ux-backlog.md. Every byte path now goes through getStorageBackend() so signed EOIs, contracts, brochures, berth PDFs, files, avatars, branding logos, and DB backups all work identically on S3 and filesystem backends. USER SETTINGS (rebuild) - Country + Timezone selectors with cross-defaulting - Browser-detected timezone banner ("Looks like you're in Europe/Paris…") - Email change with verification flow (user_email_changes table, OLD-address cancel link + NEW-address confirm link) + EMAIL_CHANGE_INSTANT=true dev shortcut - Password reset triggered via better-auth requestPasswordReset - Profile photo upload + crop (square 256×256) via shared <ImageCropperDialog> + /api/v1/me/avatar BRANDING - Shared <ImageCropperDialog> using react-easy-crop - Logo upload + crop in /admin/branding (writes via /api/v1/admin/settings/image -> storage backend) - Email header/footer HTML defaults injectable via "Insert default" - SettingsFormCard new field types: timezone (combobox), image-upload STORAGE ADMIN OVERHAUL - S3 config form FIRST, swap action SECOND - Test connection before any switch - Two-button switch: "Switch + migrate" vs "Switch only" with warning modals - runMigration() honours skipMigration flag - /api/ready + system-monitoring health check use the active storage backend instead of always probing MinIO - Filesystem backend already had full feature parity — verified BACKUP MANAGEMENT (real) - New backup_jobs table (id / status / trigger / size / storage_path) - runBackup() service spawns pg_dump --format=custom, streams to active storage backend via getStorageBackend().put() - /admin/backup page: trigger, history, download .dump for restore - Super-admin gated AI ADMIN PANEL - /admin/ai consolidates master switch + monthly token cap + provider credentials - Per-feature settings (OCR, berth-PDF parser, recommender) linked from the same page ONBOARDING WIZARD - /admin/onboarding now real with auto-checked steps - Reads each setting key + lists endpoint (roles/users/tags) to decide completion - Manual checkboxes for steps without an auto-detect signal - Progress bar + Mark done/Mark incomplete buttons - State persisted in system_settings.onboarding_manual_status RESIDENTIAL PARITY (full) - New residential_client_notes + residential_interest_notes tables (mirror marina-side shape) - Polymorphic notes.service.ts extended (verifyParent, listForEntity, create, update, delete) for residential_clients/_interests - <NotesList> component accepts the new entity types - 4 new note endpoints (GET/POST/PATCH/DELETE for clients + interests) - 2 new activity endpoints (residential clients + interests) - residential-client-tabs.tsx + residential-interest-tabs.tsx use DetailLayout (Overview / Interests / Notes / Activity) - residential-client-detail-header.tsx mirrors marina-side strip - useBreadcrumbHint wired into both detail components - Configurable Assigned-to dropdown (residential_interests.view perm) CONFIGURABLE RESIDENTIAL STAGES - residential-stages.service.ts with list / save / orphan-check - /api/v1/residential/stages GET/PUT - /admin/residential-stages admin UI with reassign-on-remove modal - Validators relaxed from z.enum to z.string DOCUMENSO PHASE 1 - Schema: document_signers.invited_at / opened_at / last_reminder_sent_at / signing_token (+ idx_ds_signing_token) - Schema: documents.completion_cc_emails (text[]) + auto_reminder_interval_days (int) - transformSigningUrl() now maps SignerRole -> URL segment via ROLE_TO_URL_SEGMENT (approver->cc, witness->witness) — fixes Risk #5 where approver invites landed on /sign/error - POST /api/v1/documents/[id]/send-invitation with auto-pick of next pending signer - Per-port settings: documenso_developer_label / _approver_label + documenso_developer_user_id / _approver_user_id (Phase 7 Project Director RBAC binding fields) ADMIN UX RAPID-FIRE - Sidebar collapse removed (always-expanded design) - Audit log: input sizes (h-9), date pickers w-44, action cell sub-label so single-row entries aren't blank - Sales email config: token list <details> + tooltips on threshold + body fields - Custom Settings card: long-form description - Reminder digest timezone uses TimezoneCombobox - Port form: currency dropdown (10 common currencies) + timezone combobox + brand color picker - Permissions count badge opens modal with granted/denied per resource - Role names display-normalized via prettifyRoleName - Tag form: native input type=color - Custom Fields page: amber heads-up about non-integration - Settings manager: select field type + fallthrough_policy as dropdown - Storage admin S3 fields ship as proper password + boolean LIST PAGES - Residential client list: clickable email/phone (mailto/tel/wa.me) - Residential interests + Documents Hub search inputs sized h-9 CURRENCY API - scripts/test-currency-api.ts verifies live Frankfurter fetch -> DB upsert -> getRate -> convert. Inverse-rate drift <=0.001 TESTS - 1185/1185 vitest passing - tsc clean - eslint 0 errors (16 pre-existing warnings) Note: WEBSITE_INTAKE_SECRET added to .env.example but committed separately due to pre-commit hook policy on .env* files. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-07 21:02:12 +02:00
{/* STEP 1: configure connection details for the OTHER backend so the
admin can prep + test BEFORE attempting any switch. */}
<SettingsFormCard
title="S3 configuration"
description="Provider connection details. Save these first, run Test connection, and only then switch the active backend below. You can edit S3 config while filesystem is active without touching live storage."
fields={S3_FIELDS}
extra={
<div className="space-y-3 pt-2 border-t">
<Button
variant="outline"
onClick={() => testMutation.mutate()}
disabled={testMutation.isPending}
>
{testMutation.isPending && <Loader2 className="mr-2 h-4 w-4 animate-spin" />}
Test S3 connection
</Button>
{testResult && (
<div className="rounded-md border p-3 text-sm">
{testResult.ok ? (
<div className="flex items-center gap-2 text-emerald-600">
<CheckCircle2 className="h-4 w-4" /> Connection OK round-trip succeeded.
</div>
) : (
<div className="flex items-center gap-2 text-destructive">
<XCircle className="h-4 w-4" /> {testResult.error ?? 'Connection failed'}
</div>
)}
</div>
)}
</div>
}
/>
<SettingsFormCard
title="Filesystem configuration"
description="Used when the active backend is filesystem. Only single-node deployments — multi-node servers must use S3."
fields={FS_FIELDS}
/>
{/* STEP 2: visualize current state + switch options. Both options
are gated behind a confirmation dialog. */}
feat(storage): pluggable s3-or-filesystem backend + migration CLI + admin UI Phase 6a from docs/berth-recommender-and-pdf-plan.md §4.7a + §14.9a. Lays the storage groundwork for Phase 6b/7 file-bearing schemas (per-berth PDFs, brochures) without touching those domains yet. New files: - src/lib/storage/index.ts StorageBackend interface + per-process factory keyed on system_settings. - src/lib/storage/s3.ts S3-compatible backend (MinIO/AWS/B2/R2/ Wasabi/Tigris) wrapping the existing minio JS client. Includes a healthCheck() used by the admin "Test connection" button. - src/lib/storage/filesystem.ts Local filesystem backend with all §14.9a mitigations baked in. - src/lib/storage/migrate.ts Shared migration core — pg_advisory_lock, per-row resumable progress markers, sha256 round-trip verification, atomic storage_backend flip on success. - scripts/migrate-storage.ts Thin CLI shim around runMigration(). - src/app/api/storage/[token]/route.ts Filesystem proxy GET. Verifies HMAC, enforces single-use replay protection via Redis SET NX, streams via NextResponse ReadableStream with explicit Content-Type + Content-Disposition. Node runtime only. - src/app/api/v1/admin/storage/route.ts GET status + POST connection test. - src/app/api/v1/admin/storage/migrate/route.ts Super-admin-only POST that runs the exact same runMigration() as the CLI. - src/app/(dashboard)/[portSlug]/admin/storage/page.tsx Super-admin admin UI (current backend, capacity stats, switch button with dry-run, test connection, backup hint). - src/components/admin/storage-admin-panel.tsx Client component for the page above. §14.9a critical mitigations implemented: - Path-traversal: storage keys validated against ^[a-zA-Z0-9/_.-]+$; `..`, `.`, `//`, leading `/`, and overlength keys rejected. - Realpath: storage root realpath'd at create time, every per-key resolution checked against the realpath'd prefix. - Storage root created (or chmod'd) to 0o700. - Multi-node refusal: FilesystemBackend.create() throws when MULTI_NODE_DEPLOYMENT=true. - HMAC token: sha256-HMAC over the (key, expiry, nonce, filename, content-type) payload. Verified with timingSafeEqual; bad sig, expired, or invalid-key payloads all return 403. - Single-use replay: token body cached in Redis SET NX EX 1800s. - sha256 round-trip: copyAndVerify() re-fetches from the target after put() and aborts the migration on any mismatch. - Free-disk pre-flight: when migrating to filesystem, sums byte counts via source.head() and aborts if free space < total * 1.2. - pg_advisory_lock(0xc7000a01) prevents concurrent migrations. - Resumable: per-row progress markers in _storage_migration_progress. system_settings keys read by the factory (jsonb, no schema change): storage_backend, storage_s3_endpoint, storage_s3_region, storage_s3_bucket, storage_s3_access_key, storage_s3_secret_key_encrypted, storage_s3_force_path_style, storage_filesystem_root, storage_proxy_hmac_secret_encrypted. Defaults: storage_backend=`s3`, storage_filesystem_root=`./storage` (./storage added to .gitignore). Tests added (34 tests, all green): - tests/unit/storage/filesystem-backend.test.ts — key validation allow/reject matrix, realpath escape, 0o700 perms, multi-node refusal, HMAC token sign/verify/tamper/expire/invalid-key. - tests/unit/storage/copy-and-verify.test.ts — sha256 mismatch on round-trip aborts the migration. - tests/integration/storage/proxy-route.test.ts — happy path, wrong HMAC secret, expired token, replay rejection. Phase 6a ships zero file-bearing tables — TABLES_WITH_STORAGE_KEYS is intentionally empty. berth_pdf_versions and brochure_versions land in Phase 6b and join the list there. Existing s3_key columns: only gdpr_export_jobs.storage_key, already named correctly — no rename needed. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-05 03:15:59 +02:00
<div className="grid gap-4 lg:grid-cols-3">
<Card className="lg:col-span-2">
<CardHeader className="flex flex-row items-start gap-3 space-y-0 pb-2">
{s.backend === 's3' ? (
<ServerCog className="mt-0.5 h-5 w-5 text-muted-foreground" />
) : (
<HardDrive className="mt-0.5 h-5 w-5 text-muted-foreground" />
)}
<div>
<CardTitle className="text-base">Active backend: {s.backend}</CardTitle>
<CardDescription>
{s.backend === 's3'
feat: autonomous backlog push — admin UX overhaul + storage parity + residential parity + Documenso Phase 1 Massive multi-area push driven by docs/admin-ux-backlog.md. Every byte path now goes through getStorageBackend() so signed EOIs, contracts, brochures, berth PDFs, files, avatars, branding logos, and DB backups all work identically on S3 and filesystem backends. USER SETTINGS (rebuild) - Country + Timezone selectors with cross-defaulting - Browser-detected timezone banner ("Looks like you're in Europe/Paris…") - Email change with verification flow (user_email_changes table, OLD-address cancel link + NEW-address confirm link) + EMAIL_CHANGE_INSTANT=true dev shortcut - Password reset triggered via better-auth requestPasswordReset - Profile photo upload + crop (square 256×256) via shared <ImageCropperDialog> + /api/v1/me/avatar BRANDING - Shared <ImageCropperDialog> using react-easy-crop - Logo upload + crop in /admin/branding (writes via /api/v1/admin/settings/image -> storage backend) - Email header/footer HTML defaults injectable via "Insert default" - SettingsFormCard new field types: timezone (combobox), image-upload STORAGE ADMIN OVERHAUL - S3 config form FIRST, swap action SECOND - Test connection before any switch - Two-button switch: "Switch + migrate" vs "Switch only" with warning modals - runMigration() honours skipMigration flag - /api/ready + system-monitoring health check use the active storage backend instead of always probing MinIO - Filesystem backend already had full feature parity — verified BACKUP MANAGEMENT (real) - New backup_jobs table (id / status / trigger / size / storage_path) - runBackup() service spawns pg_dump --format=custom, streams to active storage backend via getStorageBackend().put() - /admin/backup page: trigger, history, download .dump for restore - Super-admin gated AI ADMIN PANEL - /admin/ai consolidates master switch + monthly token cap + provider credentials - Per-feature settings (OCR, berth-PDF parser, recommender) linked from the same page ONBOARDING WIZARD - /admin/onboarding now real with auto-checked steps - Reads each setting key + lists endpoint (roles/users/tags) to decide completion - Manual checkboxes for steps without an auto-detect signal - Progress bar + Mark done/Mark incomplete buttons - State persisted in system_settings.onboarding_manual_status RESIDENTIAL PARITY (full) - New residential_client_notes + residential_interest_notes tables (mirror marina-side shape) - Polymorphic notes.service.ts extended (verifyParent, listForEntity, create, update, delete) for residential_clients/_interests - <NotesList> component accepts the new entity types - 4 new note endpoints (GET/POST/PATCH/DELETE for clients + interests) - 2 new activity endpoints (residential clients + interests) - residential-client-tabs.tsx + residential-interest-tabs.tsx use DetailLayout (Overview / Interests / Notes / Activity) - residential-client-detail-header.tsx mirrors marina-side strip - useBreadcrumbHint wired into both detail components - Configurable Assigned-to dropdown (residential_interests.view perm) CONFIGURABLE RESIDENTIAL STAGES - residential-stages.service.ts with list / save / orphan-check - /api/v1/residential/stages GET/PUT - /admin/residential-stages admin UI with reassign-on-remove modal - Validators relaxed from z.enum to z.string DOCUMENSO PHASE 1 - Schema: document_signers.invited_at / opened_at / last_reminder_sent_at / signing_token (+ idx_ds_signing_token) - Schema: documents.completion_cc_emails (text[]) + auto_reminder_interval_days (int) - transformSigningUrl() now maps SignerRole -> URL segment via ROLE_TO_URL_SEGMENT (approver->cc, witness->witness) — fixes Risk #5 where approver invites landed on /sign/error - POST /api/v1/documents/[id]/send-invitation with auto-pick of next pending signer - Per-port settings: documenso_developer_label / _approver_label + documenso_developer_user_id / _approver_user_id (Phase 7 Project Director RBAC binding fields) ADMIN UX RAPID-FIRE - Sidebar collapse removed (always-expanded design) - Audit log: input sizes (h-9), date pickers w-44, action cell sub-label so single-row entries aren't blank - Sales email config: token list <details> + tooltips on threshold + body fields - Custom Settings card: long-form description - Reminder digest timezone uses TimezoneCombobox - Port form: currency dropdown (10 common currencies) + timezone combobox + brand color picker - Permissions count badge opens modal with granted/denied per resource - Role names display-normalized via prettifyRoleName - Tag form: native input type=color - Custom Fields page: amber heads-up about non-integration - Settings manager: select field type + fallthrough_policy as dropdown - Storage admin S3 fields ship as proper password + boolean LIST PAGES - Residential client list: clickable email/phone (mailto/tel/wa.me) - Residential interests + Documents Hub search inputs sized h-9 CURRENCY API - scripts/test-currency-api.ts verifies live Frankfurter fetch -> DB upsert -> getRate -> convert. Inverse-rate drift <=0.001 TESTS - 1185/1185 vitest passing - tsc clean - eslint 0 errors (16 pre-existing warnings) Note: WEBSITE_INTAKE_SECRET added to .env.example but committed separately due to pre-commit hook policy on .env* files. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-07 21:02:12 +02:00
? 'Files stored in an S3-compatible object store (AWS, MinIO, R2, B2, Wasabi, Tigris).'
: 'Files stored on the local filesystem under storage_filesystem_root. Single-node only.'}
feat(storage): pluggable s3-or-filesystem backend + migration CLI + admin UI Phase 6a from docs/berth-recommender-and-pdf-plan.md §4.7a + §14.9a. Lays the storage groundwork for Phase 6b/7 file-bearing schemas (per-berth PDFs, brochures) without touching those domains yet. New files: - src/lib/storage/index.ts StorageBackend interface + per-process factory keyed on system_settings. - src/lib/storage/s3.ts S3-compatible backend (MinIO/AWS/B2/R2/ Wasabi/Tigris) wrapping the existing minio JS client. Includes a healthCheck() used by the admin "Test connection" button. - src/lib/storage/filesystem.ts Local filesystem backend with all §14.9a mitigations baked in. - src/lib/storage/migrate.ts Shared migration core — pg_advisory_lock, per-row resumable progress markers, sha256 round-trip verification, atomic storage_backend flip on success. - scripts/migrate-storage.ts Thin CLI shim around runMigration(). - src/app/api/storage/[token]/route.ts Filesystem proxy GET. Verifies HMAC, enforces single-use replay protection via Redis SET NX, streams via NextResponse ReadableStream with explicit Content-Type + Content-Disposition. Node runtime only. - src/app/api/v1/admin/storage/route.ts GET status + POST connection test. - src/app/api/v1/admin/storage/migrate/route.ts Super-admin-only POST that runs the exact same runMigration() as the CLI. - src/app/(dashboard)/[portSlug]/admin/storage/page.tsx Super-admin admin UI (current backend, capacity stats, switch button with dry-run, test connection, backup hint). - src/components/admin/storage-admin-panel.tsx Client component for the page above. §14.9a critical mitigations implemented: - Path-traversal: storage keys validated against ^[a-zA-Z0-9/_.-]+$; `..`, `.`, `//`, leading `/`, and overlength keys rejected. - Realpath: storage root realpath'd at create time, every per-key resolution checked against the realpath'd prefix. - Storage root created (or chmod'd) to 0o700. - Multi-node refusal: FilesystemBackend.create() throws when MULTI_NODE_DEPLOYMENT=true. - HMAC token: sha256-HMAC over the (key, expiry, nonce, filename, content-type) payload. Verified with timingSafeEqual; bad sig, expired, or invalid-key payloads all return 403. - Single-use replay: token body cached in Redis SET NX EX 1800s. - sha256 round-trip: copyAndVerify() re-fetches from the target after put() and aborts the migration on any mismatch. - Free-disk pre-flight: when migrating to filesystem, sums byte counts via source.head() and aborts if free space < total * 1.2. - pg_advisory_lock(0xc7000a01) prevents concurrent migrations. - Resumable: per-row progress markers in _storage_migration_progress. system_settings keys read by the factory (jsonb, no schema change): storage_backend, storage_s3_endpoint, storage_s3_region, storage_s3_bucket, storage_s3_access_key, storage_s3_secret_key_encrypted, storage_s3_force_path_style, storage_filesystem_root, storage_proxy_hmac_secret_encrypted. Defaults: storage_backend=`s3`, storage_filesystem_root=`./storage` (./storage added to .gitignore). Tests added (34 tests, all green): - tests/unit/storage/filesystem-backend.test.ts — key validation allow/reject matrix, realpath escape, 0o700 perms, multi-node refusal, HMAC token sign/verify/tamper/expire/invalid-key. - tests/unit/storage/copy-and-verify.test.ts — sha256 mismatch on round-trip aborts the migration. - tests/integration/storage/proxy-route.test.ts — happy path, wrong HMAC secret, expired token, replay rejection. Phase 6a ships zero file-bearing tables — TABLES_WITH_STORAGE_KEYS is intentionally empty. berth_pdf_versions and brochure_versions land in Phase 6b and join the list there. Existing s3_key columns: only gdpr_export_jobs.storage_key, already named correctly — no rename needed. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-05 03:15:59 +02:00
</CardDescription>
</div>
</CardHeader>
<CardContent className="space-y-4">
<dl className="grid grid-cols-2 gap-3 text-sm">
<div>
<dt className="text-muted-foreground">Tracked tables</dt>
feat: autonomous backlog push — admin UX overhaul + storage parity + residential parity + Documenso Phase 1 Massive multi-area push driven by docs/admin-ux-backlog.md. Every byte path now goes through getStorageBackend() so signed EOIs, contracts, brochures, berth PDFs, files, avatars, branding logos, and DB backups all work identically on S3 and filesystem backends. USER SETTINGS (rebuild) - Country + Timezone selectors with cross-defaulting - Browser-detected timezone banner ("Looks like you're in Europe/Paris…") - Email change with verification flow (user_email_changes table, OLD-address cancel link + NEW-address confirm link) + EMAIL_CHANGE_INSTANT=true dev shortcut - Password reset triggered via better-auth requestPasswordReset - Profile photo upload + crop (square 256×256) via shared <ImageCropperDialog> + /api/v1/me/avatar BRANDING - Shared <ImageCropperDialog> using react-easy-crop - Logo upload + crop in /admin/branding (writes via /api/v1/admin/settings/image -> storage backend) - Email header/footer HTML defaults injectable via "Insert default" - SettingsFormCard new field types: timezone (combobox), image-upload STORAGE ADMIN OVERHAUL - S3 config form FIRST, swap action SECOND - Test connection before any switch - Two-button switch: "Switch + migrate" vs "Switch only" with warning modals - runMigration() honours skipMigration flag - /api/ready + system-monitoring health check use the active storage backend instead of always probing MinIO - Filesystem backend already had full feature parity — verified BACKUP MANAGEMENT (real) - New backup_jobs table (id / status / trigger / size / storage_path) - runBackup() service spawns pg_dump --format=custom, streams to active storage backend via getStorageBackend().put() - /admin/backup page: trigger, history, download .dump for restore - Super-admin gated AI ADMIN PANEL - /admin/ai consolidates master switch + monthly token cap + provider credentials - Per-feature settings (OCR, berth-PDF parser, recommender) linked from the same page ONBOARDING WIZARD - /admin/onboarding now real with auto-checked steps - Reads each setting key + lists endpoint (roles/users/tags) to decide completion - Manual checkboxes for steps without an auto-detect signal - Progress bar + Mark done/Mark incomplete buttons - State persisted in system_settings.onboarding_manual_status RESIDENTIAL PARITY (full) - New residential_client_notes + residential_interest_notes tables (mirror marina-side shape) - Polymorphic notes.service.ts extended (verifyParent, listForEntity, create, update, delete) for residential_clients/_interests - <NotesList> component accepts the new entity types - 4 new note endpoints (GET/POST/PATCH/DELETE for clients + interests) - 2 new activity endpoints (residential clients + interests) - residential-client-tabs.tsx + residential-interest-tabs.tsx use DetailLayout (Overview / Interests / Notes / Activity) - residential-client-detail-header.tsx mirrors marina-side strip - useBreadcrumbHint wired into both detail components - Configurable Assigned-to dropdown (residential_interests.view perm) CONFIGURABLE RESIDENTIAL STAGES - residential-stages.service.ts with list / save / orphan-check - /api/v1/residential/stages GET/PUT - /admin/residential-stages admin UI with reassign-on-remove modal - Validators relaxed from z.enum to z.string DOCUMENSO PHASE 1 - Schema: document_signers.invited_at / opened_at / last_reminder_sent_at / signing_token (+ idx_ds_signing_token) - Schema: documents.completion_cc_emails (text[]) + auto_reminder_interval_days (int) - transformSigningUrl() now maps SignerRole -> URL segment via ROLE_TO_URL_SEGMENT (approver->cc, witness->witness) — fixes Risk #5 where approver invites landed on /sign/error - POST /api/v1/documents/[id]/send-invitation with auto-pick of next pending signer - Per-port settings: documenso_developer_label / _approver_label + documenso_developer_user_id / _approver_user_id (Phase 7 Project Director RBAC binding fields) ADMIN UX RAPID-FIRE - Sidebar collapse removed (always-expanded design) - Audit log: input sizes (h-9), date pickers w-44, action cell sub-label so single-row entries aren't blank - Sales email config: token list <details> + tooltips on threshold + body fields - Custom Settings card: long-form description - Reminder digest timezone uses TimezoneCombobox - Port form: currency dropdown (10 common currencies) + timezone combobox + brand color picker - Permissions count badge opens modal with granted/denied per resource - Role names display-normalized via prettifyRoleName - Tag form: native input type=color - Custom Fields page: amber heads-up about non-integration - Settings manager: select field type + fallthrough_policy as dropdown - Storage admin S3 fields ship as proper password + boolean LIST PAGES - Residential client list: clickable email/phone (mailto/tel/wa.me) - Residential interests + Documents Hub search inputs sized h-9 CURRENCY API - scripts/test-currency-api.ts verifies live Frankfurter fetch -> DB upsert -> getRate -> convert. Inverse-rate drift <=0.001 TESTS - 1185/1185 vitest passing - tsc clean - eslint 0 errors (16 pre-existing warnings) Note: WEBSITE_INTAKE_SECRET added to .env.example but committed separately due to pre-commit hook policy on .env* files. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-07 21:02:12 +02:00
<dd>{s.tablesTracked.length === 0 ? '(none yet)' : s.tablesTracked.join(', ')}</dd>
feat(storage): pluggable s3-or-filesystem backend + migration CLI + admin UI Phase 6a from docs/berth-recommender-and-pdf-plan.md §4.7a + §14.9a. Lays the storage groundwork for Phase 6b/7 file-bearing schemas (per-berth PDFs, brochures) without touching those domains yet. New files: - src/lib/storage/index.ts StorageBackend interface + per-process factory keyed on system_settings. - src/lib/storage/s3.ts S3-compatible backend (MinIO/AWS/B2/R2/ Wasabi/Tigris) wrapping the existing minio JS client. Includes a healthCheck() used by the admin "Test connection" button. - src/lib/storage/filesystem.ts Local filesystem backend with all §14.9a mitigations baked in. - src/lib/storage/migrate.ts Shared migration core — pg_advisory_lock, per-row resumable progress markers, sha256 round-trip verification, atomic storage_backend flip on success. - scripts/migrate-storage.ts Thin CLI shim around runMigration(). - src/app/api/storage/[token]/route.ts Filesystem proxy GET. Verifies HMAC, enforces single-use replay protection via Redis SET NX, streams via NextResponse ReadableStream with explicit Content-Type + Content-Disposition. Node runtime only. - src/app/api/v1/admin/storage/route.ts GET status + POST connection test. - src/app/api/v1/admin/storage/migrate/route.ts Super-admin-only POST that runs the exact same runMigration() as the CLI. - src/app/(dashboard)/[portSlug]/admin/storage/page.tsx Super-admin admin UI (current backend, capacity stats, switch button with dry-run, test connection, backup hint). - src/components/admin/storage-admin-panel.tsx Client component for the page above. §14.9a critical mitigations implemented: - Path-traversal: storage keys validated against ^[a-zA-Z0-9/_.-]+$; `..`, `.`, `//`, leading `/`, and overlength keys rejected. - Realpath: storage root realpath'd at create time, every per-key resolution checked against the realpath'd prefix. - Storage root created (or chmod'd) to 0o700. - Multi-node refusal: FilesystemBackend.create() throws when MULTI_NODE_DEPLOYMENT=true. - HMAC token: sha256-HMAC over the (key, expiry, nonce, filename, content-type) payload. Verified with timingSafeEqual; bad sig, expired, or invalid-key payloads all return 403. - Single-use replay: token body cached in Redis SET NX EX 1800s. - sha256 round-trip: copyAndVerify() re-fetches from the target after put() and aborts the migration on any mismatch. - Free-disk pre-flight: when migrating to filesystem, sums byte counts via source.head() and aborts if free space < total * 1.2. - pg_advisory_lock(0xc7000a01) prevents concurrent migrations. - Resumable: per-row progress markers in _storage_migration_progress. system_settings keys read by the factory (jsonb, no schema change): storage_backend, storage_s3_endpoint, storage_s3_region, storage_s3_bucket, storage_s3_access_key, storage_s3_secret_key_encrypted, storage_s3_force_path_style, storage_filesystem_root, storage_proxy_hmac_secret_encrypted. Defaults: storage_backend=`s3`, storage_filesystem_root=`./storage` (./storage added to .gitignore). Tests added (34 tests, all green): - tests/unit/storage/filesystem-backend.test.ts — key validation allow/reject matrix, realpath escape, 0o700 perms, multi-node refusal, HMAC token sign/verify/tamper/expire/invalid-key. - tests/unit/storage/copy-and-verify.test.ts — sha256 mismatch on round-trip aborts the migration. - tests/integration/storage/proxy-route.test.ts — happy path, wrong HMAC secret, expired token, replay rejection. Phase 6a ships zero file-bearing tables — TABLES_WITH_STORAGE_KEYS is intentionally empty. berth_pdf_versions and brochure_versions land in Phase 6b and join the list there. Existing s3_key columns: only gdpr_export_jobs.storage_key, already named correctly — no rename needed. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-05 03:15:59 +02:00
</div>
<div>
<dt className="text-muted-foreground">File count</dt>
<dd>{s.fileCount}</dd>
</div>
</dl>
feat: autonomous backlog push — admin UX overhaul + storage parity + residential parity + Documenso Phase 1 Massive multi-area push driven by docs/admin-ux-backlog.md. Every byte path now goes through getStorageBackend() so signed EOIs, contracts, brochures, berth PDFs, files, avatars, branding logos, and DB backups all work identically on S3 and filesystem backends. USER SETTINGS (rebuild) - Country + Timezone selectors with cross-defaulting - Browser-detected timezone banner ("Looks like you're in Europe/Paris…") - Email change with verification flow (user_email_changes table, OLD-address cancel link + NEW-address confirm link) + EMAIL_CHANGE_INSTANT=true dev shortcut - Password reset triggered via better-auth requestPasswordReset - Profile photo upload + crop (square 256×256) via shared <ImageCropperDialog> + /api/v1/me/avatar BRANDING - Shared <ImageCropperDialog> using react-easy-crop - Logo upload + crop in /admin/branding (writes via /api/v1/admin/settings/image -> storage backend) - Email header/footer HTML defaults injectable via "Insert default" - SettingsFormCard new field types: timezone (combobox), image-upload STORAGE ADMIN OVERHAUL - S3 config form FIRST, swap action SECOND - Test connection before any switch - Two-button switch: "Switch + migrate" vs "Switch only" with warning modals - runMigration() honours skipMigration flag - /api/ready + system-monitoring health check use the active storage backend instead of always probing MinIO - Filesystem backend already had full feature parity — verified BACKUP MANAGEMENT (real) - New backup_jobs table (id / status / trigger / size / storage_path) - runBackup() service spawns pg_dump --format=custom, streams to active storage backend via getStorageBackend().put() - /admin/backup page: trigger, history, download .dump for restore - Super-admin gated AI ADMIN PANEL - /admin/ai consolidates master switch + monthly token cap + provider credentials - Per-feature settings (OCR, berth-PDF parser, recommender) linked from the same page ONBOARDING WIZARD - /admin/onboarding now real with auto-checked steps - Reads each setting key + lists endpoint (roles/users/tags) to decide completion - Manual checkboxes for steps without an auto-detect signal - Progress bar + Mark done/Mark incomplete buttons - State persisted in system_settings.onboarding_manual_status RESIDENTIAL PARITY (full) - New residential_client_notes + residential_interest_notes tables (mirror marina-side shape) - Polymorphic notes.service.ts extended (verifyParent, listForEntity, create, update, delete) for residential_clients/_interests - <NotesList> component accepts the new entity types - 4 new note endpoints (GET/POST/PATCH/DELETE for clients + interests) - 2 new activity endpoints (residential clients + interests) - residential-client-tabs.tsx + residential-interest-tabs.tsx use DetailLayout (Overview / Interests / Notes / Activity) - residential-client-detail-header.tsx mirrors marina-side strip - useBreadcrumbHint wired into both detail components - Configurable Assigned-to dropdown (residential_interests.view perm) CONFIGURABLE RESIDENTIAL STAGES - residential-stages.service.ts with list / save / orphan-check - /api/v1/residential/stages GET/PUT - /admin/residential-stages admin UI with reassign-on-remove modal - Validators relaxed from z.enum to z.string DOCUMENSO PHASE 1 - Schema: document_signers.invited_at / opened_at / last_reminder_sent_at / signing_token (+ idx_ds_signing_token) - Schema: documents.completion_cc_emails (text[]) + auto_reminder_interval_days (int) - transformSigningUrl() now maps SignerRole -> URL segment via ROLE_TO_URL_SEGMENT (approver->cc, witness->witness) — fixes Risk #5 where approver invites landed on /sign/error - POST /api/v1/documents/[id]/send-invitation with auto-pick of next pending signer - Per-port settings: documenso_developer_label / _approver_label + documenso_developer_user_id / _approver_user_id (Phase 7 Project Director RBAC binding fields) ADMIN UX RAPID-FIRE - Sidebar collapse removed (always-expanded design) - Audit log: input sizes (h-9), date pickers w-44, action cell sub-label so single-row entries aren't blank - Sales email config: token list <details> + tooltips on threshold + body fields - Custom Settings card: long-form description - Reminder digest timezone uses TimezoneCombobox - Port form: currency dropdown (10 common currencies) + timezone combobox + brand color picker - Permissions count badge opens modal with granted/denied per resource - Role names display-normalized via prettifyRoleName - Tag form: native input type=color - Custom Fields page: amber heads-up about non-integration - Settings manager: select field type + fallthrough_policy as dropdown - Storage admin S3 fields ship as proper password + boolean LIST PAGES - Residential client list: clickable email/phone (mailto/tel/wa.me) - Residential interests + Documents Hub search inputs sized h-9 CURRENCY API - scripts/test-currency-api.ts verifies live Frankfurter fetch -> DB upsert -> getRate -> convert. Inverse-rate drift <=0.001 TESTS - 1185/1185 vitest passing - tsc clean - eslint 0 errors (16 pre-existing warnings) Note: WEBSITE_INTAKE_SECRET added to .env.example but committed separately due to pre-commit hook policy on .env* files. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-07 21:02:12 +02:00
<div className="space-y-2">
<p className="text-xs font-medium uppercase tracking-wide text-muted-foreground">
Switch active backend
</p>
<div className="flex flex-wrap gap-2">
feat(storage): pluggable s3-or-filesystem backend + migration CLI + admin UI Phase 6a from docs/berth-recommender-and-pdf-plan.md §4.7a + §14.9a. Lays the storage groundwork for Phase 6b/7 file-bearing schemas (per-berth PDFs, brochures) without touching those domains yet. New files: - src/lib/storage/index.ts StorageBackend interface + per-process factory keyed on system_settings. - src/lib/storage/s3.ts S3-compatible backend (MinIO/AWS/B2/R2/ Wasabi/Tigris) wrapping the existing minio JS client. Includes a healthCheck() used by the admin "Test connection" button. - src/lib/storage/filesystem.ts Local filesystem backend with all §14.9a mitigations baked in. - src/lib/storage/migrate.ts Shared migration core — pg_advisory_lock, per-row resumable progress markers, sha256 round-trip verification, atomic storage_backend flip on success. - scripts/migrate-storage.ts Thin CLI shim around runMigration(). - src/app/api/storage/[token]/route.ts Filesystem proxy GET. Verifies HMAC, enforces single-use replay protection via Redis SET NX, streams via NextResponse ReadableStream with explicit Content-Type + Content-Disposition. Node runtime only. - src/app/api/v1/admin/storage/route.ts GET status + POST connection test. - src/app/api/v1/admin/storage/migrate/route.ts Super-admin-only POST that runs the exact same runMigration() as the CLI. - src/app/(dashboard)/[portSlug]/admin/storage/page.tsx Super-admin admin UI (current backend, capacity stats, switch button with dry-run, test connection, backup hint). - src/components/admin/storage-admin-panel.tsx Client component for the page above. §14.9a critical mitigations implemented: - Path-traversal: storage keys validated against ^[a-zA-Z0-9/_.-]+$; `..`, `.`, `//`, leading `/`, and overlength keys rejected. - Realpath: storage root realpath'd at create time, every per-key resolution checked against the realpath'd prefix. - Storage root created (or chmod'd) to 0o700. - Multi-node refusal: FilesystemBackend.create() throws when MULTI_NODE_DEPLOYMENT=true. - HMAC token: sha256-HMAC over the (key, expiry, nonce, filename, content-type) payload. Verified with timingSafeEqual; bad sig, expired, or invalid-key payloads all return 403. - Single-use replay: token body cached in Redis SET NX EX 1800s. - sha256 round-trip: copyAndVerify() re-fetches from the target after put() and aborts the migration on any mismatch. - Free-disk pre-flight: when migrating to filesystem, sums byte counts via source.head() and aborts if free space < total * 1.2. - pg_advisory_lock(0xc7000a01) prevents concurrent migrations. - Resumable: per-row progress markers in _storage_migration_progress. system_settings keys read by the factory (jsonb, no schema change): storage_backend, storage_s3_endpoint, storage_s3_region, storage_s3_bucket, storage_s3_access_key, storage_s3_secret_key_encrypted, storage_s3_force_path_style, storage_filesystem_root, storage_proxy_hmac_secret_encrypted. Defaults: storage_backend=`s3`, storage_filesystem_root=`./storage` (./storage added to .gitignore). Tests added (34 tests, all green): - tests/unit/storage/filesystem-backend.test.ts — key validation allow/reject matrix, realpath escape, 0o700 perms, multi-node refusal, HMAC token sign/verify/tamper/expire/invalid-key. - tests/unit/storage/copy-and-verify.test.ts — sha256 mismatch on round-trip aborts the migration. - tests/integration/storage/proxy-route.test.ts — happy path, wrong HMAC secret, expired token, replay rejection. Phase 6a ships zero file-bearing tables — TABLES_WITH_STORAGE_KEYS is intentionally empty. berth_pdf_versions and brochure_versions land in Phase 6b and join the list there. Existing s3_key columns: only gdpr_export_jobs.storage_key, already named correctly — no rename needed. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-05 03:15:59 +02:00
<Button
variant="outline"
feat: autonomous backlog push — admin UX overhaul + storage parity + residential parity + Documenso Phase 1 Massive multi-area push driven by docs/admin-ux-backlog.md. Every byte path now goes through getStorageBackend() so signed EOIs, contracts, brochures, berth PDFs, files, avatars, branding logos, and DB backups all work identically on S3 and filesystem backends. USER SETTINGS (rebuild) - Country + Timezone selectors with cross-defaulting - Browser-detected timezone banner ("Looks like you're in Europe/Paris…") - Email change with verification flow (user_email_changes table, OLD-address cancel link + NEW-address confirm link) + EMAIL_CHANGE_INSTANT=true dev shortcut - Password reset triggered via better-auth requestPasswordReset - Profile photo upload + crop (square 256×256) via shared <ImageCropperDialog> + /api/v1/me/avatar BRANDING - Shared <ImageCropperDialog> using react-easy-crop - Logo upload + crop in /admin/branding (writes via /api/v1/admin/settings/image -> storage backend) - Email header/footer HTML defaults injectable via "Insert default" - SettingsFormCard new field types: timezone (combobox), image-upload STORAGE ADMIN OVERHAUL - S3 config form FIRST, swap action SECOND - Test connection before any switch - Two-button switch: "Switch + migrate" vs "Switch only" with warning modals - runMigration() honours skipMigration flag - /api/ready + system-monitoring health check use the active storage backend instead of always probing MinIO - Filesystem backend already had full feature parity — verified BACKUP MANAGEMENT (real) - New backup_jobs table (id / status / trigger / size / storage_path) - runBackup() service spawns pg_dump --format=custom, streams to active storage backend via getStorageBackend().put() - /admin/backup page: trigger, history, download .dump for restore - Super-admin gated AI ADMIN PANEL - /admin/ai consolidates master switch + monthly token cap + provider credentials - Per-feature settings (OCR, berth-PDF parser, recommender) linked from the same page ONBOARDING WIZARD - /admin/onboarding now real with auto-checked steps - Reads each setting key + lists endpoint (roles/users/tags) to decide completion - Manual checkboxes for steps without an auto-detect signal - Progress bar + Mark done/Mark incomplete buttons - State persisted in system_settings.onboarding_manual_status RESIDENTIAL PARITY (full) - New residential_client_notes + residential_interest_notes tables (mirror marina-side shape) - Polymorphic notes.service.ts extended (verifyParent, listForEntity, create, update, delete) for residential_clients/_interests - <NotesList> component accepts the new entity types - 4 new note endpoints (GET/POST/PATCH/DELETE for clients + interests) - 2 new activity endpoints (residential clients + interests) - residential-client-tabs.tsx + residential-interest-tabs.tsx use DetailLayout (Overview / Interests / Notes / Activity) - residential-client-detail-header.tsx mirrors marina-side strip - useBreadcrumbHint wired into both detail components - Configurable Assigned-to dropdown (residential_interests.view perm) CONFIGURABLE RESIDENTIAL STAGES - residential-stages.service.ts with list / save / orphan-check - /api/v1/residential/stages GET/PUT - /admin/residential-stages admin UI with reassign-on-remove modal - Validators relaxed from z.enum to z.string DOCUMENSO PHASE 1 - Schema: document_signers.invited_at / opened_at / last_reminder_sent_at / signing_token (+ idx_ds_signing_token) - Schema: documents.completion_cc_emails (text[]) + auto_reminder_interval_days (int) - transformSigningUrl() now maps SignerRole -> URL segment via ROLE_TO_URL_SEGMENT (approver->cc, witness->witness) — fixes Risk #5 where approver invites landed on /sign/error - POST /api/v1/documents/[id]/send-invitation with auto-pick of next pending signer - Per-port settings: documenso_developer_label / _approver_label + documenso_developer_user_id / _approver_user_id (Phase 7 Project Director RBAC binding fields) ADMIN UX RAPID-FIRE - Sidebar collapse removed (always-expanded design) - Audit log: input sizes (h-9), date pickers w-44, action cell sub-label so single-row entries aren't blank - Sales email config: token list <details> + tooltips on threshold + body fields - Custom Settings card: long-form description - Reminder digest timezone uses TimezoneCombobox - Port form: currency dropdown (10 common currencies) + timezone combobox + brand color picker - Permissions count badge opens modal with granted/denied per resource - Role names display-normalized via prettifyRoleName - Tag form: native input type=color - Custom Fields page: amber heads-up about non-integration - Settings manager: select field type + fallthrough_policy as dropdown - Storage admin S3 fields ship as proper password + boolean LIST PAGES - Residential client list: clickable email/phone (mailto/tel/wa.me) - Residential interests + Documents Hub search inputs sized h-9 CURRENCY API - scripts/test-currency-api.ts verifies live Frankfurter fetch -> DB upsert -> getRate -> convert. Inverse-rate drift <=0.001 TESTS - 1185/1185 vitest passing - tsc clean - eslint 0 errors (16 pre-existing warnings) Note: WEBSITE_INTAKE_SECRET added to .env.example but committed separately due to pre-commit hook policy on .env* files. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-07 21:02:12 +02:00
disabled={dryRunMutation.isPending}
onClick={() => openConfirm('switch-and-migrate')}
feat(storage): pluggable s3-or-filesystem backend + migration CLI + admin UI Phase 6a from docs/berth-recommender-and-pdf-plan.md §4.7a + §14.9a. Lays the storage groundwork for Phase 6b/7 file-bearing schemas (per-berth PDFs, brochures) without touching those domains yet. New files: - src/lib/storage/index.ts StorageBackend interface + per-process factory keyed on system_settings. - src/lib/storage/s3.ts S3-compatible backend (MinIO/AWS/B2/R2/ Wasabi/Tigris) wrapping the existing minio JS client. Includes a healthCheck() used by the admin "Test connection" button. - src/lib/storage/filesystem.ts Local filesystem backend with all §14.9a mitigations baked in. - src/lib/storage/migrate.ts Shared migration core — pg_advisory_lock, per-row resumable progress markers, sha256 round-trip verification, atomic storage_backend flip on success. - scripts/migrate-storage.ts Thin CLI shim around runMigration(). - src/app/api/storage/[token]/route.ts Filesystem proxy GET. Verifies HMAC, enforces single-use replay protection via Redis SET NX, streams via NextResponse ReadableStream with explicit Content-Type + Content-Disposition. Node runtime only. - src/app/api/v1/admin/storage/route.ts GET status + POST connection test. - src/app/api/v1/admin/storage/migrate/route.ts Super-admin-only POST that runs the exact same runMigration() as the CLI. - src/app/(dashboard)/[portSlug]/admin/storage/page.tsx Super-admin admin UI (current backend, capacity stats, switch button with dry-run, test connection, backup hint). - src/components/admin/storage-admin-panel.tsx Client component for the page above. §14.9a critical mitigations implemented: - Path-traversal: storage keys validated against ^[a-zA-Z0-9/_.-]+$; `..`, `.`, `//`, leading `/`, and overlength keys rejected. - Realpath: storage root realpath'd at create time, every per-key resolution checked against the realpath'd prefix. - Storage root created (or chmod'd) to 0o700. - Multi-node refusal: FilesystemBackend.create() throws when MULTI_NODE_DEPLOYMENT=true. - HMAC token: sha256-HMAC over the (key, expiry, nonce, filename, content-type) payload. Verified with timingSafeEqual; bad sig, expired, or invalid-key payloads all return 403. - Single-use replay: token body cached in Redis SET NX EX 1800s. - sha256 round-trip: copyAndVerify() re-fetches from the target after put() and aborts the migration on any mismatch. - Free-disk pre-flight: when migrating to filesystem, sums byte counts via source.head() and aborts if free space < total * 1.2. - pg_advisory_lock(0xc7000a01) prevents concurrent migrations. - Resumable: per-row progress markers in _storage_migration_progress. system_settings keys read by the factory (jsonb, no schema change): storage_backend, storage_s3_endpoint, storage_s3_region, storage_s3_bucket, storage_s3_access_key, storage_s3_secret_key_encrypted, storage_s3_force_path_style, storage_filesystem_root, storage_proxy_hmac_secret_encrypted. Defaults: storage_backend=`s3`, storage_filesystem_root=`./storage` (./storage added to .gitignore). Tests added (34 tests, all green): - tests/unit/storage/filesystem-backend.test.ts — key validation allow/reject matrix, realpath escape, 0o700 perms, multi-node refusal, HMAC token sign/verify/tamper/expire/invalid-key. - tests/unit/storage/copy-and-verify.test.ts — sha256 mismatch on round-trip aborts the migration. - tests/integration/storage/proxy-route.test.ts — happy path, wrong HMAC secret, expired token, replay rejection. Phase 6a ships zero file-bearing tables — TABLES_WITH_STORAGE_KEYS is intentionally empty. berth_pdf_versions and brochure_versions land in Phase 6b and join the list there. Existing s3_key columns: only gdpr_export_jobs.storage_key, already named correctly — no rename needed. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-05 03:15:59 +02:00
>
feat: autonomous backlog push — admin UX overhaul + storage parity + residential parity + Documenso Phase 1 Massive multi-area push driven by docs/admin-ux-backlog.md. Every byte path now goes through getStorageBackend() so signed EOIs, contracts, brochures, berth PDFs, files, avatars, branding logos, and DB backups all work identically on S3 and filesystem backends. USER SETTINGS (rebuild) - Country + Timezone selectors with cross-defaulting - Browser-detected timezone banner ("Looks like you're in Europe/Paris…") - Email change with verification flow (user_email_changes table, OLD-address cancel link + NEW-address confirm link) + EMAIL_CHANGE_INSTANT=true dev shortcut - Password reset triggered via better-auth requestPasswordReset - Profile photo upload + crop (square 256×256) via shared <ImageCropperDialog> + /api/v1/me/avatar BRANDING - Shared <ImageCropperDialog> using react-easy-crop - Logo upload + crop in /admin/branding (writes via /api/v1/admin/settings/image -> storage backend) - Email header/footer HTML defaults injectable via "Insert default" - SettingsFormCard new field types: timezone (combobox), image-upload STORAGE ADMIN OVERHAUL - S3 config form FIRST, swap action SECOND - Test connection before any switch - Two-button switch: "Switch + migrate" vs "Switch only" with warning modals - runMigration() honours skipMigration flag - /api/ready + system-monitoring health check use the active storage backend instead of always probing MinIO - Filesystem backend already had full feature parity — verified BACKUP MANAGEMENT (real) - New backup_jobs table (id / status / trigger / size / storage_path) - runBackup() service spawns pg_dump --format=custom, streams to active storage backend via getStorageBackend().put() - /admin/backup page: trigger, history, download .dump for restore - Super-admin gated AI ADMIN PANEL - /admin/ai consolidates master switch + monthly token cap + provider credentials - Per-feature settings (OCR, berth-PDF parser, recommender) linked from the same page ONBOARDING WIZARD - /admin/onboarding now real with auto-checked steps - Reads each setting key + lists endpoint (roles/users/tags) to decide completion - Manual checkboxes for steps without an auto-detect signal - Progress bar + Mark done/Mark incomplete buttons - State persisted in system_settings.onboarding_manual_status RESIDENTIAL PARITY (full) - New residential_client_notes + residential_interest_notes tables (mirror marina-side shape) - Polymorphic notes.service.ts extended (verifyParent, listForEntity, create, update, delete) for residential_clients/_interests - <NotesList> component accepts the new entity types - 4 new note endpoints (GET/POST/PATCH/DELETE for clients + interests) - 2 new activity endpoints (residential clients + interests) - residential-client-tabs.tsx + residential-interest-tabs.tsx use DetailLayout (Overview / Interests / Notes / Activity) - residential-client-detail-header.tsx mirrors marina-side strip - useBreadcrumbHint wired into both detail components - Configurable Assigned-to dropdown (residential_interests.view perm) CONFIGURABLE RESIDENTIAL STAGES - residential-stages.service.ts with list / save / orphan-check - /api/v1/residential/stages GET/PUT - /admin/residential-stages admin UI with reassign-on-remove modal - Validators relaxed from z.enum to z.string DOCUMENSO PHASE 1 - Schema: document_signers.invited_at / opened_at / last_reminder_sent_at / signing_token (+ idx_ds_signing_token) - Schema: documents.completion_cc_emails (text[]) + auto_reminder_interval_days (int) - transformSigningUrl() now maps SignerRole -> URL segment via ROLE_TO_URL_SEGMENT (approver->cc, witness->witness) — fixes Risk #5 where approver invites landed on /sign/error - POST /api/v1/documents/[id]/send-invitation with auto-pick of next pending signer - Per-port settings: documenso_developer_label / _approver_label + documenso_developer_user_id / _approver_user_id (Phase 7 Project Director RBAC binding fields) ADMIN UX RAPID-FIRE - Sidebar collapse removed (always-expanded design) - Audit log: input sizes (h-9), date pickers w-44, action cell sub-label so single-row entries aren't blank - Sales email config: token list <details> + tooltips on threshold + body fields - Custom Settings card: long-form description - Reminder digest timezone uses TimezoneCombobox - Port form: currency dropdown (10 common currencies) + timezone combobox + brand color picker - Permissions count badge opens modal with granted/denied per resource - Role names display-normalized via prettifyRoleName - Tag form: native input type=color - Custom Fields page: amber heads-up about non-integration - Settings manager: select field type + fallthrough_policy as dropdown - Storage admin S3 fields ship as proper password + boolean LIST PAGES - Residential client list: clickable email/phone (mailto/tel/wa.me) - Residential interests + Documents Hub search inputs sized h-9 CURRENCY API - scripts/test-currency-api.ts verifies live Frankfurter fetch -> DB upsert -> getRate -> convert. Inverse-rate drift <=0.001 TESTS - 1185/1185 vitest passing - tsc clean - eslint 0 errors (16 pre-existing warnings) Note: WEBSITE_INTAKE_SECRET added to .env.example but committed separately due to pre-commit hook policy on .env* files. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-07 21:02:12 +02:00
{dryRunMutation.isPending && <Loader2 className="mr-2 h-4 w-4 animate-spin" />}
Switch + migrate existing files to {otherBackend}
</Button>
<Button variant="outline" onClick={() => openConfirm('switch-only')}>
Switch only (new files only) {otherBackend}
</Button>
<Button
variant="ghost"
onClick={() => status.refetch()}
disabled={status.isFetching}
>
<RefreshCw className="mr-2 h-4 w-4" /> Refresh
feat(storage): pluggable s3-or-filesystem backend + migration CLI + admin UI Phase 6a from docs/berth-recommender-and-pdf-plan.md §4.7a + §14.9a. Lays the storage groundwork for Phase 6b/7 file-bearing schemas (per-berth PDFs, brochures) without touching those domains yet. New files: - src/lib/storage/index.ts StorageBackend interface + per-process factory keyed on system_settings. - src/lib/storage/s3.ts S3-compatible backend (MinIO/AWS/B2/R2/ Wasabi/Tigris) wrapping the existing minio JS client. Includes a healthCheck() used by the admin "Test connection" button. - src/lib/storage/filesystem.ts Local filesystem backend with all §14.9a mitigations baked in. - src/lib/storage/migrate.ts Shared migration core — pg_advisory_lock, per-row resumable progress markers, sha256 round-trip verification, atomic storage_backend flip on success. - scripts/migrate-storage.ts Thin CLI shim around runMigration(). - src/app/api/storage/[token]/route.ts Filesystem proxy GET. Verifies HMAC, enforces single-use replay protection via Redis SET NX, streams via NextResponse ReadableStream with explicit Content-Type + Content-Disposition. Node runtime only. - src/app/api/v1/admin/storage/route.ts GET status + POST connection test. - src/app/api/v1/admin/storage/migrate/route.ts Super-admin-only POST that runs the exact same runMigration() as the CLI. - src/app/(dashboard)/[portSlug]/admin/storage/page.tsx Super-admin admin UI (current backend, capacity stats, switch button with dry-run, test connection, backup hint). - src/components/admin/storage-admin-panel.tsx Client component for the page above. §14.9a critical mitigations implemented: - Path-traversal: storage keys validated against ^[a-zA-Z0-9/_.-]+$; `..`, `.`, `//`, leading `/`, and overlength keys rejected. - Realpath: storage root realpath'd at create time, every per-key resolution checked against the realpath'd prefix. - Storage root created (or chmod'd) to 0o700. - Multi-node refusal: FilesystemBackend.create() throws when MULTI_NODE_DEPLOYMENT=true. - HMAC token: sha256-HMAC over the (key, expiry, nonce, filename, content-type) payload. Verified with timingSafeEqual; bad sig, expired, or invalid-key payloads all return 403. - Single-use replay: token body cached in Redis SET NX EX 1800s. - sha256 round-trip: copyAndVerify() re-fetches from the target after put() and aborts the migration on any mismatch. - Free-disk pre-flight: when migrating to filesystem, sums byte counts via source.head() and aborts if free space < total * 1.2. - pg_advisory_lock(0xc7000a01) prevents concurrent migrations. - Resumable: per-row progress markers in _storage_migration_progress. system_settings keys read by the factory (jsonb, no schema change): storage_backend, storage_s3_endpoint, storage_s3_region, storage_s3_bucket, storage_s3_access_key, storage_s3_secret_key_encrypted, storage_s3_force_path_style, storage_filesystem_root, storage_proxy_hmac_secret_encrypted. Defaults: storage_backend=`s3`, storage_filesystem_root=`./storage` (./storage added to .gitignore). Tests added (34 tests, all green): - tests/unit/storage/filesystem-backend.test.ts — key validation allow/reject matrix, realpath escape, 0o700 perms, multi-node refusal, HMAC token sign/verify/tamper/expire/invalid-key. - tests/unit/storage/copy-and-verify.test.ts — sha256 mismatch on round-trip aborts the migration. - tests/integration/storage/proxy-route.test.ts — happy path, wrong HMAC secret, expired token, replay rejection. Phase 6a ships zero file-bearing tables — TABLES_WITH_STORAGE_KEYS is intentionally empty. berth_pdf_versions and brochure_versions land in Phase 6b and join the list there. Existing s3_key columns: only gdpr_export_jobs.storage_key, already named correctly — no rename needed. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-05 03:15:59 +02:00
</Button>
</div>
feat: autonomous backlog push — admin UX overhaul + storage parity + residential parity + Documenso Phase 1 Massive multi-area push driven by docs/admin-ux-backlog.md. Every byte path now goes through getStorageBackend() so signed EOIs, contracts, brochures, berth PDFs, files, avatars, branding logos, and DB backups all work identically on S3 and filesystem backends. USER SETTINGS (rebuild) - Country + Timezone selectors with cross-defaulting - Browser-detected timezone banner ("Looks like you're in Europe/Paris…") - Email change with verification flow (user_email_changes table, OLD-address cancel link + NEW-address confirm link) + EMAIL_CHANGE_INSTANT=true dev shortcut - Password reset triggered via better-auth requestPasswordReset - Profile photo upload + crop (square 256×256) via shared <ImageCropperDialog> + /api/v1/me/avatar BRANDING - Shared <ImageCropperDialog> using react-easy-crop - Logo upload + crop in /admin/branding (writes via /api/v1/admin/settings/image -> storage backend) - Email header/footer HTML defaults injectable via "Insert default" - SettingsFormCard new field types: timezone (combobox), image-upload STORAGE ADMIN OVERHAUL - S3 config form FIRST, swap action SECOND - Test connection before any switch - Two-button switch: "Switch + migrate" vs "Switch only" with warning modals - runMigration() honours skipMigration flag - /api/ready + system-monitoring health check use the active storage backend instead of always probing MinIO - Filesystem backend already had full feature parity — verified BACKUP MANAGEMENT (real) - New backup_jobs table (id / status / trigger / size / storage_path) - runBackup() service spawns pg_dump --format=custom, streams to active storage backend via getStorageBackend().put() - /admin/backup page: trigger, history, download .dump for restore - Super-admin gated AI ADMIN PANEL - /admin/ai consolidates master switch + monthly token cap + provider credentials - Per-feature settings (OCR, berth-PDF parser, recommender) linked from the same page ONBOARDING WIZARD - /admin/onboarding now real with auto-checked steps - Reads each setting key + lists endpoint (roles/users/tags) to decide completion - Manual checkboxes for steps without an auto-detect signal - Progress bar + Mark done/Mark incomplete buttons - State persisted in system_settings.onboarding_manual_status RESIDENTIAL PARITY (full) - New residential_client_notes + residential_interest_notes tables (mirror marina-side shape) - Polymorphic notes.service.ts extended (verifyParent, listForEntity, create, update, delete) for residential_clients/_interests - <NotesList> component accepts the new entity types - 4 new note endpoints (GET/POST/PATCH/DELETE for clients + interests) - 2 new activity endpoints (residential clients + interests) - residential-client-tabs.tsx + residential-interest-tabs.tsx use DetailLayout (Overview / Interests / Notes / Activity) - residential-client-detail-header.tsx mirrors marina-side strip - useBreadcrumbHint wired into both detail components - Configurable Assigned-to dropdown (residential_interests.view perm) CONFIGURABLE RESIDENTIAL STAGES - residential-stages.service.ts with list / save / orphan-check - /api/v1/residential/stages GET/PUT - /admin/residential-stages admin UI with reassign-on-remove modal - Validators relaxed from z.enum to z.string DOCUMENSO PHASE 1 - Schema: document_signers.invited_at / opened_at / last_reminder_sent_at / signing_token (+ idx_ds_signing_token) - Schema: documents.completion_cc_emails (text[]) + auto_reminder_interval_days (int) - transformSigningUrl() now maps SignerRole -> URL segment via ROLE_TO_URL_SEGMENT (approver->cc, witness->witness) — fixes Risk #5 where approver invites landed on /sign/error - POST /api/v1/documents/[id]/send-invitation with auto-pick of next pending signer - Per-port settings: documenso_developer_label / _approver_label + documenso_developer_user_id / _approver_user_id (Phase 7 Project Director RBAC binding fields) ADMIN UX RAPID-FIRE - Sidebar collapse removed (always-expanded design) - Audit log: input sizes (h-9), date pickers w-44, action cell sub-label so single-row entries aren't blank - Sales email config: token list <details> + tooltips on threshold + body fields - Custom Settings card: long-form description - Reminder digest timezone uses TimezoneCombobox - Port form: currency dropdown (10 common currencies) + timezone combobox + brand color picker - Permissions count badge opens modal with granted/denied per resource - Role names display-normalized via prettifyRoleName - Tag form: native input type=color - Custom Fields page: amber heads-up about non-integration - Settings manager: select field type + fallthrough_policy as dropdown - Storage admin S3 fields ship as proper password + boolean LIST PAGES - Residential client list: clickable email/phone (mailto/tel/wa.me) - Residential interests + Documents Hub search inputs sized h-9 CURRENCY API - scripts/test-currency-api.ts verifies live Frankfurter fetch -> DB upsert -> getRate -> convert. Inverse-rate drift <=0.001 TESTS - 1185/1185 vitest passing - tsc clean - eslint 0 errors (16 pre-existing warnings) Note: WEBSITE_INTAKE_SECRET added to .env.example but committed separately due to pre-commit hook policy on .env* files. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-07 21:02:12 +02:00
<p className="text-xs text-muted-foreground">
<strong>Switch + migrate</strong> copies every existing file to the new backend then
flips the pointer atomically. Reversible with a follow-up reverse-migration.{' '}
<strong>Switch only</strong> flips the pointer immediately old files become
inaccessible until you migrate them or revert the backend.
</p>
</div>
feat(storage): pluggable s3-or-filesystem backend + migration CLI + admin UI Phase 6a from docs/berth-recommender-and-pdf-plan.md §4.7a + §14.9a. Lays the storage groundwork for Phase 6b/7 file-bearing schemas (per-berth PDFs, brochures) without touching those domains yet. New files: - src/lib/storage/index.ts StorageBackend interface + per-process factory keyed on system_settings. - src/lib/storage/s3.ts S3-compatible backend (MinIO/AWS/B2/R2/ Wasabi/Tigris) wrapping the existing minio JS client. Includes a healthCheck() used by the admin "Test connection" button. - src/lib/storage/filesystem.ts Local filesystem backend with all §14.9a mitigations baked in. - src/lib/storage/migrate.ts Shared migration core — pg_advisory_lock, per-row resumable progress markers, sha256 round-trip verification, atomic storage_backend flip on success. - scripts/migrate-storage.ts Thin CLI shim around runMigration(). - src/app/api/storage/[token]/route.ts Filesystem proxy GET. Verifies HMAC, enforces single-use replay protection via Redis SET NX, streams via NextResponse ReadableStream with explicit Content-Type + Content-Disposition. Node runtime only. - src/app/api/v1/admin/storage/route.ts GET status + POST connection test. - src/app/api/v1/admin/storage/migrate/route.ts Super-admin-only POST that runs the exact same runMigration() as the CLI. - src/app/(dashboard)/[portSlug]/admin/storage/page.tsx Super-admin admin UI (current backend, capacity stats, switch button with dry-run, test connection, backup hint). - src/components/admin/storage-admin-panel.tsx Client component for the page above. §14.9a critical mitigations implemented: - Path-traversal: storage keys validated against ^[a-zA-Z0-9/_.-]+$; `..`, `.`, `//`, leading `/`, and overlength keys rejected. - Realpath: storage root realpath'd at create time, every per-key resolution checked against the realpath'd prefix. - Storage root created (or chmod'd) to 0o700. - Multi-node refusal: FilesystemBackend.create() throws when MULTI_NODE_DEPLOYMENT=true. - HMAC token: sha256-HMAC over the (key, expiry, nonce, filename, content-type) payload. Verified with timingSafeEqual; bad sig, expired, or invalid-key payloads all return 403. - Single-use replay: token body cached in Redis SET NX EX 1800s. - sha256 round-trip: copyAndVerify() re-fetches from the target after put() and aborts the migration on any mismatch. - Free-disk pre-flight: when migrating to filesystem, sums byte counts via source.head() and aborts if free space < total * 1.2. - pg_advisory_lock(0xc7000a01) prevents concurrent migrations. - Resumable: per-row progress markers in _storage_migration_progress. system_settings keys read by the factory (jsonb, no schema change): storage_backend, storage_s3_endpoint, storage_s3_region, storage_s3_bucket, storage_s3_access_key, storage_s3_secret_key_encrypted, storage_s3_force_path_style, storage_filesystem_root, storage_proxy_hmac_secret_encrypted. Defaults: storage_backend=`s3`, storage_filesystem_root=`./storage` (./storage added to .gitignore). Tests added (34 tests, all green): - tests/unit/storage/filesystem-backend.test.ts — key validation allow/reject matrix, realpath escape, 0o700 perms, multi-node refusal, HMAC token sign/verify/tamper/expire/invalid-key. - tests/unit/storage/copy-and-verify.test.ts — sha256 mismatch on round-trip aborts the migration. - tests/integration/storage/proxy-route.test.ts — happy path, wrong HMAC secret, expired token, replay rejection. Phase 6a ships zero file-bearing tables — TABLES_WITH_STORAGE_KEYS is intentionally empty. berth_pdf_versions and brochure_versions land in Phase 6b and join the list there. Existing s3_key columns: only gdpr_export_jobs.storage_key, already named correctly — no rename needed. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-05 03:15:59 +02:00
</CardContent>
</Card>
<Card>
<CardHeader>
<CardTitle className="text-base">Backup notes</CardTitle>
</CardHeader>
<CardContent className="space-y-2 text-sm text-muted-foreground">
{s.backend === 's3' ? (
<p>
S3 mode: configure your provider&apos;s lifecycle / replication / versioning
policies as your primary backup. The CRM does not duplicate object storage in its
own backups.
</p>
) : (
<p>
Filesystem mode: include the storage root directory in your backup tool (restic,
borg, snapshots). It sits next to the database; the two should be backed up
together.
</p>
)}
<p className="pt-2 text-xs">
Filesystem mode refuses to start when MULTI_NODE_DEPLOYMENT=true. For multi-node
deployments, switch to an S3-compatible backend.
</p>
</CardContent>
</Card>
</div>
<Dialog open={confirmOpen} onOpenChange={setConfirmOpen}>
<DialogContent>
<DialogHeader>
feat: autonomous backlog push — admin UX overhaul + storage parity + residential parity + Documenso Phase 1 Massive multi-area push driven by docs/admin-ux-backlog.md. Every byte path now goes through getStorageBackend() so signed EOIs, contracts, brochures, berth PDFs, files, avatars, branding logos, and DB backups all work identically on S3 and filesystem backends. USER SETTINGS (rebuild) - Country + Timezone selectors with cross-defaulting - Browser-detected timezone banner ("Looks like you're in Europe/Paris…") - Email change with verification flow (user_email_changes table, OLD-address cancel link + NEW-address confirm link) + EMAIL_CHANGE_INSTANT=true dev shortcut - Password reset triggered via better-auth requestPasswordReset - Profile photo upload + crop (square 256×256) via shared <ImageCropperDialog> + /api/v1/me/avatar BRANDING - Shared <ImageCropperDialog> using react-easy-crop - Logo upload + crop in /admin/branding (writes via /api/v1/admin/settings/image -> storage backend) - Email header/footer HTML defaults injectable via "Insert default" - SettingsFormCard new field types: timezone (combobox), image-upload STORAGE ADMIN OVERHAUL - S3 config form FIRST, swap action SECOND - Test connection before any switch - Two-button switch: "Switch + migrate" vs "Switch only" with warning modals - runMigration() honours skipMigration flag - /api/ready + system-monitoring health check use the active storage backend instead of always probing MinIO - Filesystem backend already had full feature parity — verified BACKUP MANAGEMENT (real) - New backup_jobs table (id / status / trigger / size / storage_path) - runBackup() service spawns pg_dump --format=custom, streams to active storage backend via getStorageBackend().put() - /admin/backup page: trigger, history, download .dump for restore - Super-admin gated AI ADMIN PANEL - /admin/ai consolidates master switch + monthly token cap + provider credentials - Per-feature settings (OCR, berth-PDF parser, recommender) linked from the same page ONBOARDING WIZARD - /admin/onboarding now real with auto-checked steps - Reads each setting key + lists endpoint (roles/users/tags) to decide completion - Manual checkboxes for steps without an auto-detect signal - Progress bar + Mark done/Mark incomplete buttons - State persisted in system_settings.onboarding_manual_status RESIDENTIAL PARITY (full) - New residential_client_notes + residential_interest_notes tables (mirror marina-side shape) - Polymorphic notes.service.ts extended (verifyParent, listForEntity, create, update, delete) for residential_clients/_interests - <NotesList> component accepts the new entity types - 4 new note endpoints (GET/POST/PATCH/DELETE for clients + interests) - 2 new activity endpoints (residential clients + interests) - residential-client-tabs.tsx + residential-interest-tabs.tsx use DetailLayout (Overview / Interests / Notes / Activity) - residential-client-detail-header.tsx mirrors marina-side strip - useBreadcrumbHint wired into both detail components - Configurable Assigned-to dropdown (residential_interests.view perm) CONFIGURABLE RESIDENTIAL STAGES - residential-stages.service.ts with list / save / orphan-check - /api/v1/residential/stages GET/PUT - /admin/residential-stages admin UI with reassign-on-remove modal - Validators relaxed from z.enum to z.string DOCUMENSO PHASE 1 - Schema: document_signers.invited_at / opened_at / last_reminder_sent_at / signing_token (+ idx_ds_signing_token) - Schema: documents.completion_cc_emails (text[]) + auto_reminder_interval_days (int) - transformSigningUrl() now maps SignerRole -> URL segment via ROLE_TO_URL_SEGMENT (approver->cc, witness->witness) — fixes Risk #5 where approver invites landed on /sign/error - POST /api/v1/documents/[id]/send-invitation with auto-pick of next pending signer - Per-port settings: documenso_developer_label / _approver_label + documenso_developer_user_id / _approver_user_id (Phase 7 Project Director RBAC binding fields) ADMIN UX RAPID-FIRE - Sidebar collapse removed (always-expanded design) - Audit log: input sizes (h-9), date pickers w-44, action cell sub-label so single-row entries aren't blank - Sales email config: token list <details> + tooltips on threshold + body fields - Custom Settings card: long-form description - Reminder digest timezone uses TimezoneCombobox - Port form: currency dropdown (10 common currencies) + timezone combobox + brand color picker - Permissions count badge opens modal with granted/denied per resource - Role names display-normalized via prettifyRoleName - Tag form: native input type=color - Custom Fields page: amber heads-up about non-integration - Settings manager: select field type + fallthrough_policy as dropdown - Storage admin S3 fields ship as proper password + boolean LIST PAGES - Residential client list: clickable email/phone (mailto/tel/wa.me) - Residential interests + Documents Hub search inputs sized h-9 CURRENCY API - scripts/test-currency-api.ts verifies live Frankfurter fetch -> DB upsert -> getRate -> convert. Inverse-rate drift <=0.001 TESTS - 1185/1185 vitest passing - tsc clean - eslint 0 errors (16 pre-existing warnings) Note: WEBSITE_INTAKE_SECRET added to .env.example but committed separately due to pre-commit hook policy on .env* files. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-07 21:02:12 +02:00
<DialogTitle className="flex items-center gap-2">
<AlertTriangle className="h-5 w-5 text-amber-600" />
{confirmMode === 'switch-and-migrate'
? `Switch + migrate to ${otherBackend}?`
: `Switch active backend to ${otherBackend}?`}
</DialogTitle>
feat(storage): pluggable s3-or-filesystem backend + migration CLI + admin UI Phase 6a from docs/berth-recommender-and-pdf-plan.md §4.7a + §14.9a. Lays the storage groundwork for Phase 6b/7 file-bearing schemas (per-berth PDFs, brochures) without touching those domains yet. New files: - src/lib/storage/index.ts StorageBackend interface + per-process factory keyed on system_settings. - src/lib/storage/s3.ts S3-compatible backend (MinIO/AWS/B2/R2/ Wasabi/Tigris) wrapping the existing minio JS client. Includes a healthCheck() used by the admin "Test connection" button. - src/lib/storage/filesystem.ts Local filesystem backend with all §14.9a mitigations baked in. - src/lib/storage/migrate.ts Shared migration core — pg_advisory_lock, per-row resumable progress markers, sha256 round-trip verification, atomic storage_backend flip on success. - scripts/migrate-storage.ts Thin CLI shim around runMigration(). - src/app/api/storage/[token]/route.ts Filesystem proxy GET. Verifies HMAC, enforces single-use replay protection via Redis SET NX, streams via NextResponse ReadableStream with explicit Content-Type + Content-Disposition. Node runtime only. - src/app/api/v1/admin/storage/route.ts GET status + POST connection test. - src/app/api/v1/admin/storage/migrate/route.ts Super-admin-only POST that runs the exact same runMigration() as the CLI. - src/app/(dashboard)/[portSlug]/admin/storage/page.tsx Super-admin admin UI (current backend, capacity stats, switch button with dry-run, test connection, backup hint). - src/components/admin/storage-admin-panel.tsx Client component for the page above. §14.9a critical mitigations implemented: - Path-traversal: storage keys validated against ^[a-zA-Z0-9/_.-]+$; `..`, `.`, `//`, leading `/`, and overlength keys rejected. - Realpath: storage root realpath'd at create time, every per-key resolution checked against the realpath'd prefix. - Storage root created (or chmod'd) to 0o700. - Multi-node refusal: FilesystemBackend.create() throws when MULTI_NODE_DEPLOYMENT=true. - HMAC token: sha256-HMAC over the (key, expiry, nonce, filename, content-type) payload. Verified with timingSafeEqual; bad sig, expired, or invalid-key payloads all return 403. - Single-use replay: token body cached in Redis SET NX EX 1800s. - sha256 round-trip: copyAndVerify() re-fetches from the target after put() and aborts the migration on any mismatch. - Free-disk pre-flight: when migrating to filesystem, sums byte counts via source.head() and aborts if free space < total * 1.2. - pg_advisory_lock(0xc7000a01) prevents concurrent migrations. - Resumable: per-row progress markers in _storage_migration_progress. system_settings keys read by the factory (jsonb, no schema change): storage_backend, storage_s3_endpoint, storage_s3_region, storage_s3_bucket, storage_s3_access_key, storage_s3_secret_key_encrypted, storage_s3_force_path_style, storage_filesystem_root, storage_proxy_hmac_secret_encrypted. Defaults: storage_backend=`s3`, storage_filesystem_root=`./storage` (./storage added to .gitignore). Tests added (34 tests, all green): - tests/unit/storage/filesystem-backend.test.ts — key validation allow/reject matrix, realpath escape, 0o700 perms, multi-node refusal, HMAC token sign/verify/tamper/expire/invalid-key. - tests/unit/storage/copy-and-verify.test.ts — sha256 mismatch on round-trip aborts the migration. - tests/integration/storage/proxy-route.test.ts — happy path, wrong HMAC secret, expired token, replay rejection. Phase 6a ships zero file-bearing tables — TABLES_WITH_STORAGE_KEYS is intentionally empty. berth_pdf_versions and brochure_versions land in Phase 6b and join the list there. Existing s3_key columns: only gdpr_export_jobs.storage_key, already named correctly — no rename needed. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-05 03:15:59 +02:00
<DialogDescription>
feat: autonomous backlog push — admin UX overhaul + storage parity + residential parity + Documenso Phase 1 Massive multi-area push driven by docs/admin-ux-backlog.md. Every byte path now goes through getStorageBackend() so signed EOIs, contracts, brochures, berth PDFs, files, avatars, branding logos, and DB backups all work identically on S3 and filesystem backends. USER SETTINGS (rebuild) - Country + Timezone selectors with cross-defaulting - Browser-detected timezone banner ("Looks like you're in Europe/Paris…") - Email change with verification flow (user_email_changes table, OLD-address cancel link + NEW-address confirm link) + EMAIL_CHANGE_INSTANT=true dev shortcut - Password reset triggered via better-auth requestPasswordReset - Profile photo upload + crop (square 256×256) via shared <ImageCropperDialog> + /api/v1/me/avatar BRANDING - Shared <ImageCropperDialog> using react-easy-crop - Logo upload + crop in /admin/branding (writes via /api/v1/admin/settings/image -> storage backend) - Email header/footer HTML defaults injectable via "Insert default" - SettingsFormCard new field types: timezone (combobox), image-upload STORAGE ADMIN OVERHAUL - S3 config form FIRST, swap action SECOND - Test connection before any switch - Two-button switch: "Switch + migrate" vs "Switch only" with warning modals - runMigration() honours skipMigration flag - /api/ready + system-monitoring health check use the active storage backend instead of always probing MinIO - Filesystem backend already had full feature parity — verified BACKUP MANAGEMENT (real) - New backup_jobs table (id / status / trigger / size / storage_path) - runBackup() service spawns pg_dump --format=custom, streams to active storage backend via getStorageBackend().put() - /admin/backup page: trigger, history, download .dump for restore - Super-admin gated AI ADMIN PANEL - /admin/ai consolidates master switch + monthly token cap + provider credentials - Per-feature settings (OCR, berth-PDF parser, recommender) linked from the same page ONBOARDING WIZARD - /admin/onboarding now real with auto-checked steps - Reads each setting key + lists endpoint (roles/users/tags) to decide completion - Manual checkboxes for steps without an auto-detect signal - Progress bar + Mark done/Mark incomplete buttons - State persisted in system_settings.onboarding_manual_status RESIDENTIAL PARITY (full) - New residential_client_notes + residential_interest_notes tables (mirror marina-side shape) - Polymorphic notes.service.ts extended (verifyParent, listForEntity, create, update, delete) for residential_clients/_interests - <NotesList> component accepts the new entity types - 4 new note endpoints (GET/POST/PATCH/DELETE for clients + interests) - 2 new activity endpoints (residential clients + interests) - residential-client-tabs.tsx + residential-interest-tabs.tsx use DetailLayout (Overview / Interests / Notes / Activity) - residential-client-detail-header.tsx mirrors marina-side strip - useBreadcrumbHint wired into both detail components - Configurable Assigned-to dropdown (residential_interests.view perm) CONFIGURABLE RESIDENTIAL STAGES - residential-stages.service.ts with list / save / orphan-check - /api/v1/residential/stages GET/PUT - /admin/residential-stages admin UI with reassign-on-remove modal - Validators relaxed from z.enum to z.string DOCUMENSO PHASE 1 - Schema: document_signers.invited_at / opened_at / last_reminder_sent_at / signing_token (+ idx_ds_signing_token) - Schema: documents.completion_cc_emails (text[]) + auto_reminder_interval_days (int) - transformSigningUrl() now maps SignerRole -> URL segment via ROLE_TO_URL_SEGMENT (approver->cc, witness->witness) — fixes Risk #5 where approver invites landed on /sign/error - POST /api/v1/documents/[id]/send-invitation with auto-pick of next pending signer - Per-port settings: documenso_developer_label / _approver_label + documenso_developer_user_id / _approver_user_id (Phase 7 Project Director RBAC binding fields) ADMIN UX RAPID-FIRE - Sidebar collapse removed (always-expanded design) - Audit log: input sizes (h-9), date pickers w-44, action cell sub-label so single-row entries aren't blank - Sales email config: token list <details> + tooltips on threshold + body fields - Custom Settings card: long-form description - Reminder digest timezone uses TimezoneCombobox - Port form: currency dropdown (10 common currencies) + timezone combobox + brand color picker - Permissions count badge opens modal with granted/denied per resource - Role names display-normalized via prettifyRoleName - Tag form: native input type=color - Custom Fields page: amber heads-up about non-integration - Settings manager: select field type + fallthrough_policy as dropdown - Storage admin S3 fields ship as proper password + boolean LIST PAGES - Residential client list: clickable email/phone (mailto/tel/wa.me) - Residential interests + Documents Hub search inputs sized h-9 CURRENCY API - scripts/test-currency-api.ts verifies live Frankfurter fetch -> DB upsert -> getRate -> convert. Inverse-rate drift <=0.001 TESTS - 1185/1185 vitest passing - tsc clean - eslint 0 errors (16 pre-existing warnings) Note: WEBSITE_INTAKE_SECRET added to .env.example but committed separately due to pre-commit hook policy on .env* files. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-07 21:02:12 +02:00
{confirmMode === 'switch-and-migrate'
? 'Copy every existing file to the new backend, verify each via sha256, then atomically flip the active backend. The dry-run summary below shows what will move.'
: 'Flip the pointer immediately. Existing files stay on the old backend and become inaccessible until you migrate them or revert. Use this when you intentionally want a fresh storage tier (rare).'}
feat(storage): pluggable s3-or-filesystem backend + migration CLI + admin UI Phase 6a from docs/berth-recommender-and-pdf-plan.md §4.7a + §14.9a. Lays the storage groundwork for Phase 6b/7 file-bearing schemas (per-berth PDFs, brochures) without touching those domains yet. New files: - src/lib/storage/index.ts StorageBackend interface + per-process factory keyed on system_settings. - src/lib/storage/s3.ts S3-compatible backend (MinIO/AWS/B2/R2/ Wasabi/Tigris) wrapping the existing minio JS client. Includes a healthCheck() used by the admin "Test connection" button. - src/lib/storage/filesystem.ts Local filesystem backend with all §14.9a mitigations baked in. - src/lib/storage/migrate.ts Shared migration core — pg_advisory_lock, per-row resumable progress markers, sha256 round-trip verification, atomic storage_backend flip on success. - scripts/migrate-storage.ts Thin CLI shim around runMigration(). - src/app/api/storage/[token]/route.ts Filesystem proxy GET. Verifies HMAC, enforces single-use replay protection via Redis SET NX, streams via NextResponse ReadableStream with explicit Content-Type + Content-Disposition. Node runtime only. - src/app/api/v1/admin/storage/route.ts GET status + POST connection test. - src/app/api/v1/admin/storage/migrate/route.ts Super-admin-only POST that runs the exact same runMigration() as the CLI. - src/app/(dashboard)/[portSlug]/admin/storage/page.tsx Super-admin admin UI (current backend, capacity stats, switch button with dry-run, test connection, backup hint). - src/components/admin/storage-admin-panel.tsx Client component for the page above. §14.9a critical mitigations implemented: - Path-traversal: storage keys validated against ^[a-zA-Z0-9/_.-]+$; `..`, `.`, `//`, leading `/`, and overlength keys rejected. - Realpath: storage root realpath'd at create time, every per-key resolution checked against the realpath'd prefix. - Storage root created (or chmod'd) to 0o700. - Multi-node refusal: FilesystemBackend.create() throws when MULTI_NODE_DEPLOYMENT=true. - HMAC token: sha256-HMAC over the (key, expiry, nonce, filename, content-type) payload. Verified with timingSafeEqual; bad sig, expired, or invalid-key payloads all return 403. - Single-use replay: token body cached in Redis SET NX EX 1800s. - sha256 round-trip: copyAndVerify() re-fetches from the target after put() and aborts the migration on any mismatch. - Free-disk pre-flight: when migrating to filesystem, sums byte counts via source.head() and aborts if free space < total * 1.2. - pg_advisory_lock(0xc7000a01) prevents concurrent migrations. - Resumable: per-row progress markers in _storage_migration_progress. system_settings keys read by the factory (jsonb, no schema change): storage_backend, storage_s3_endpoint, storage_s3_region, storage_s3_bucket, storage_s3_access_key, storage_s3_secret_key_encrypted, storage_s3_force_path_style, storage_filesystem_root, storage_proxy_hmac_secret_encrypted. Defaults: storage_backend=`s3`, storage_filesystem_root=`./storage` (./storage added to .gitignore). Tests added (34 tests, all green): - tests/unit/storage/filesystem-backend.test.ts — key validation allow/reject matrix, realpath escape, 0o700 perms, multi-node refusal, HMAC token sign/verify/tamper/expire/invalid-key. - tests/unit/storage/copy-and-verify.test.ts — sha256 mismatch on round-trip aborts the migration. - tests/integration/storage/proxy-route.test.ts — happy path, wrong HMAC secret, expired token, replay rejection. Phase 6a ships zero file-bearing tables — TABLES_WITH_STORAGE_KEYS is intentionally empty. berth_pdf_versions and brochure_versions land in Phase 6b and join the list there. Existing s3_key columns: only gdpr_export_jobs.storage_key, already named correctly — no rename needed. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-05 03:15:59 +02:00
</DialogDescription>
</DialogHeader>
feat: autonomous backlog push — admin UX overhaul + storage parity + residential parity + Documenso Phase 1 Massive multi-area push driven by docs/admin-ux-backlog.md. Every byte path now goes through getStorageBackend() so signed EOIs, contracts, brochures, berth PDFs, files, avatars, branding logos, and DB backups all work identically on S3 and filesystem backends. USER SETTINGS (rebuild) - Country + Timezone selectors with cross-defaulting - Browser-detected timezone banner ("Looks like you're in Europe/Paris…") - Email change with verification flow (user_email_changes table, OLD-address cancel link + NEW-address confirm link) + EMAIL_CHANGE_INSTANT=true dev shortcut - Password reset triggered via better-auth requestPasswordReset - Profile photo upload + crop (square 256×256) via shared <ImageCropperDialog> + /api/v1/me/avatar BRANDING - Shared <ImageCropperDialog> using react-easy-crop - Logo upload + crop in /admin/branding (writes via /api/v1/admin/settings/image -> storage backend) - Email header/footer HTML defaults injectable via "Insert default" - SettingsFormCard new field types: timezone (combobox), image-upload STORAGE ADMIN OVERHAUL - S3 config form FIRST, swap action SECOND - Test connection before any switch - Two-button switch: "Switch + migrate" vs "Switch only" with warning modals - runMigration() honours skipMigration flag - /api/ready + system-monitoring health check use the active storage backend instead of always probing MinIO - Filesystem backend already had full feature parity — verified BACKUP MANAGEMENT (real) - New backup_jobs table (id / status / trigger / size / storage_path) - runBackup() service spawns pg_dump --format=custom, streams to active storage backend via getStorageBackend().put() - /admin/backup page: trigger, history, download .dump for restore - Super-admin gated AI ADMIN PANEL - /admin/ai consolidates master switch + monthly token cap + provider credentials - Per-feature settings (OCR, berth-PDF parser, recommender) linked from the same page ONBOARDING WIZARD - /admin/onboarding now real with auto-checked steps - Reads each setting key + lists endpoint (roles/users/tags) to decide completion - Manual checkboxes for steps without an auto-detect signal - Progress bar + Mark done/Mark incomplete buttons - State persisted in system_settings.onboarding_manual_status RESIDENTIAL PARITY (full) - New residential_client_notes + residential_interest_notes tables (mirror marina-side shape) - Polymorphic notes.service.ts extended (verifyParent, listForEntity, create, update, delete) for residential_clients/_interests - <NotesList> component accepts the new entity types - 4 new note endpoints (GET/POST/PATCH/DELETE for clients + interests) - 2 new activity endpoints (residential clients + interests) - residential-client-tabs.tsx + residential-interest-tabs.tsx use DetailLayout (Overview / Interests / Notes / Activity) - residential-client-detail-header.tsx mirrors marina-side strip - useBreadcrumbHint wired into both detail components - Configurable Assigned-to dropdown (residential_interests.view perm) CONFIGURABLE RESIDENTIAL STAGES - residential-stages.service.ts with list / save / orphan-check - /api/v1/residential/stages GET/PUT - /admin/residential-stages admin UI with reassign-on-remove modal - Validators relaxed from z.enum to z.string DOCUMENSO PHASE 1 - Schema: document_signers.invited_at / opened_at / last_reminder_sent_at / signing_token (+ idx_ds_signing_token) - Schema: documents.completion_cc_emails (text[]) + auto_reminder_interval_days (int) - transformSigningUrl() now maps SignerRole -> URL segment via ROLE_TO_URL_SEGMENT (approver->cc, witness->witness) — fixes Risk #5 where approver invites landed on /sign/error - POST /api/v1/documents/[id]/send-invitation with auto-pick of next pending signer - Per-port settings: documenso_developer_label / _approver_label + documenso_developer_user_id / _approver_user_id (Phase 7 Project Director RBAC binding fields) ADMIN UX RAPID-FIRE - Sidebar collapse removed (always-expanded design) - Audit log: input sizes (h-9), date pickers w-44, action cell sub-label so single-row entries aren't blank - Sales email config: token list <details> + tooltips on threshold + body fields - Custom Settings card: long-form description - Reminder digest timezone uses TimezoneCombobox - Port form: currency dropdown (10 common currencies) + timezone combobox + brand color picker - Permissions count badge opens modal with granted/denied per resource - Role names display-normalized via prettifyRoleName - Tag form: native input type=color - Custom Fields page: amber heads-up about non-integration - Settings manager: select field type + fallthrough_policy as dropdown - Storage admin S3 fields ship as proper password + boolean LIST PAGES - Residential client list: clickable email/phone (mailto/tel/wa.me) - Residential interests + Documents Hub search inputs sized h-9 CURRENCY API - scripts/test-currency-api.ts verifies live Frankfurter fetch -> DB upsert -> getRate -> convert. Inverse-rate drift <=0.001 TESTS - 1185/1185 vitest passing - tsc clean - eslint 0 errors (16 pre-existing warnings) Note: WEBSITE_INTAKE_SECRET added to .env.example but committed separately due to pre-commit hook policy on .env* files. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-07 21:02:12 +02:00
{dryRun && confirmMode === 'switch-and-migrate' && (
feat(storage): pluggable s3-or-filesystem backend + migration CLI + admin UI Phase 6a from docs/berth-recommender-and-pdf-plan.md §4.7a + §14.9a. Lays the storage groundwork for Phase 6b/7 file-bearing schemas (per-berth PDFs, brochures) without touching those domains yet. New files: - src/lib/storage/index.ts StorageBackend interface + per-process factory keyed on system_settings. - src/lib/storage/s3.ts S3-compatible backend (MinIO/AWS/B2/R2/ Wasabi/Tigris) wrapping the existing minio JS client. Includes a healthCheck() used by the admin "Test connection" button. - src/lib/storage/filesystem.ts Local filesystem backend with all §14.9a mitigations baked in. - src/lib/storage/migrate.ts Shared migration core — pg_advisory_lock, per-row resumable progress markers, sha256 round-trip verification, atomic storage_backend flip on success. - scripts/migrate-storage.ts Thin CLI shim around runMigration(). - src/app/api/storage/[token]/route.ts Filesystem proxy GET. Verifies HMAC, enforces single-use replay protection via Redis SET NX, streams via NextResponse ReadableStream with explicit Content-Type + Content-Disposition. Node runtime only. - src/app/api/v1/admin/storage/route.ts GET status + POST connection test. - src/app/api/v1/admin/storage/migrate/route.ts Super-admin-only POST that runs the exact same runMigration() as the CLI. - src/app/(dashboard)/[portSlug]/admin/storage/page.tsx Super-admin admin UI (current backend, capacity stats, switch button with dry-run, test connection, backup hint). - src/components/admin/storage-admin-panel.tsx Client component for the page above. §14.9a critical mitigations implemented: - Path-traversal: storage keys validated against ^[a-zA-Z0-9/_.-]+$; `..`, `.`, `//`, leading `/`, and overlength keys rejected. - Realpath: storage root realpath'd at create time, every per-key resolution checked against the realpath'd prefix. - Storage root created (or chmod'd) to 0o700. - Multi-node refusal: FilesystemBackend.create() throws when MULTI_NODE_DEPLOYMENT=true. - HMAC token: sha256-HMAC over the (key, expiry, nonce, filename, content-type) payload. Verified with timingSafeEqual; bad sig, expired, or invalid-key payloads all return 403. - Single-use replay: token body cached in Redis SET NX EX 1800s. - sha256 round-trip: copyAndVerify() re-fetches from the target after put() and aborts the migration on any mismatch. - Free-disk pre-flight: when migrating to filesystem, sums byte counts via source.head() and aborts if free space < total * 1.2. - pg_advisory_lock(0xc7000a01) prevents concurrent migrations. - Resumable: per-row progress markers in _storage_migration_progress. system_settings keys read by the factory (jsonb, no schema change): storage_backend, storage_s3_endpoint, storage_s3_region, storage_s3_bucket, storage_s3_access_key, storage_s3_secret_key_encrypted, storage_s3_force_path_style, storage_filesystem_root, storage_proxy_hmac_secret_encrypted. Defaults: storage_backend=`s3`, storage_filesystem_root=`./storage` (./storage added to .gitignore). Tests added (34 tests, all green): - tests/unit/storage/filesystem-backend.test.ts — key validation allow/reject matrix, realpath escape, 0o700 perms, multi-node refusal, HMAC token sign/verify/tamper/expire/invalid-key. - tests/unit/storage/copy-and-verify.test.ts — sha256 mismatch on round-trip aborts the migration. - tests/integration/storage/proxy-route.test.ts — happy path, wrong HMAC secret, expired token, replay rejection. Phase 6a ships zero file-bearing tables — TABLES_WITH_STORAGE_KEYS is intentionally empty. berth_pdf_versions and brochure_versions land in Phase 6b and join the list there. Existing s3_key columns: only gdpr_export_jobs.storage_key, already named correctly — no rename needed. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-05 03:15:59 +02:00
<div className="rounded-md border p-3 text-sm">
<dl className="grid grid-cols-2 gap-2">
<dt className="text-muted-foreground">Rows considered</dt>
<dd>{dryRun.rowsConsidered}</dd>
<dt className="text-muted-foreground">Already migrated (resumable)</dt>
<dd>{dryRun.rowsSkippedAlreadyDone}</dd>
<dt className="text-muted-foreground">Total bytes</dt>
<dd>{Math.round(dryRun.totalBytes / 1024)} KB</dd>
</dl>
</div>
)}
feat: autonomous backlog push — admin UX overhaul + storage parity + residential parity + Documenso Phase 1 Massive multi-area push driven by docs/admin-ux-backlog.md. Every byte path now goes through getStorageBackend() so signed EOIs, contracts, brochures, berth PDFs, files, avatars, branding logos, and DB backups all work identically on S3 and filesystem backends. USER SETTINGS (rebuild) - Country + Timezone selectors with cross-defaulting - Browser-detected timezone banner ("Looks like you're in Europe/Paris…") - Email change with verification flow (user_email_changes table, OLD-address cancel link + NEW-address confirm link) + EMAIL_CHANGE_INSTANT=true dev shortcut - Password reset triggered via better-auth requestPasswordReset - Profile photo upload + crop (square 256×256) via shared <ImageCropperDialog> + /api/v1/me/avatar BRANDING - Shared <ImageCropperDialog> using react-easy-crop - Logo upload + crop in /admin/branding (writes via /api/v1/admin/settings/image -> storage backend) - Email header/footer HTML defaults injectable via "Insert default" - SettingsFormCard new field types: timezone (combobox), image-upload STORAGE ADMIN OVERHAUL - S3 config form FIRST, swap action SECOND - Test connection before any switch - Two-button switch: "Switch + migrate" vs "Switch only" with warning modals - runMigration() honours skipMigration flag - /api/ready + system-monitoring health check use the active storage backend instead of always probing MinIO - Filesystem backend already had full feature parity — verified BACKUP MANAGEMENT (real) - New backup_jobs table (id / status / trigger / size / storage_path) - runBackup() service spawns pg_dump --format=custom, streams to active storage backend via getStorageBackend().put() - /admin/backup page: trigger, history, download .dump for restore - Super-admin gated AI ADMIN PANEL - /admin/ai consolidates master switch + monthly token cap + provider credentials - Per-feature settings (OCR, berth-PDF parser, recommender) linked from the same page ONBOARDING WIZARD - /admin/onboarding now real with auto-checked steps - Reads each setting key + lists endpoint (roles/users/tags) to decide completion - Manual checkboxes for steps without an auto-detect signal - Progress bar + Mark done/Mark incomplete buttons - State persisted in system_settings.onboarding_manual_status RESIDENTIAL PARITY (full) - New residential_client_notes + residential_interest_notes tables (mirror marina-side shape) - Polymorphic notes.service.ts extended (verifyParent, listForEntity, create, update, delete) for residential_clients/_interests - <NotesList> component accepts the new entity types - 4 new note endpoints (GET/POST/PATCH/DELETE for clients + interests) - 2 new activity endpoints (residential clients + interests) - residential-client-tabs.tsx + residential-interest-tabs.tsx use DetailLayout (Overview / Interests / Notes / Activity) - residential-client-detail-header.tsx mirrors marina-side strip - useBreadcrumbHint wired into both detail components - Configurable Assigned-to dropdown (residential_interests.view perm) CONFIGURABLE RESIDENTIAL STAGES - residential-stages.service.ts with list / save / orphan-check - /api/v1/residential/stages GET/PUT - /admin/residential-stages admin UI with reassign-on-remove modal - Validators relaxed from z.enum to z.string DOCUMENSO PHASE 1 - Schema: document_signers.invited_at / opened_at / last_reminder_sent_at / signing_token (+ idx_ds_signing_token) - Schema: documents.completion_cc_emails (text[]) + auto_reminder_interval_days (int) - transformSigningUrl() now maps SignerRole -> URL segment via ROLE_TO_URL_SEGMENT (approver->cc, witness->witness) — fixes Risk #5 where approver invites landed on /sign/error - POST /api/v1/documents/[id]/send-invitation with auto-pick of next pending signer - Per-port settings: documenso_developer_label / _approver_label + documenso_developer_user_id / _approver_user_id (Phase 7 Project Director RBAC binding fields) ADMIN UX RAPID-FIRE - Sidebar collapse removed (always-expanded design) - Audit log: input sizes (h-9), date pickers w-44, action cell sub-label so single-row entries aren't blank - Sales email config: token list <details> + tooltips on threshold + body fields - Custom Settings card: long-form description - Reminder digest timezone uses TimezoneCombobox - Port form: currency dropdown (10 common currencies) + timezone combobox + brand color picker - Permissions count badge opens modal with granted/denied per resource - Role names display-normalized via prettifyRoleName - Tag form: native input type=color - Custom Fields page: amber heads-up about non-integration - Settings manager: select field type + fallthrough_policy as dropdown - Storage admin S3 fields ship as proper password + boolean LIST PAGES - Residential client list: clickable email/phone (mailto/tel/wa.me) - Residential interests + Documents Hub search inputs sized h-9 CURRENCY API - scripts/test-currency-api.ts verifies live Frankfurter fetch -> DB upsert -> getRate -> convert. Inverse-rate drift <=0.001 TESTS - 1185/1185 vitest passing - tsc clean - eslint 0 errors (16 pre-existing warnings) Note: WEBSITE_INTAKE_SECRET added to .env.example but committed separately due to pre-commit hook policy on .env* files. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-07 21:02:12 +02:00
{confirmMode === 'switch-only' && (
<div className="rounded-md border border-amber-300 bg-amber-50 p-3 text-sm text-amber-900">
<strong>Warning:</strong> {s.fileCount} existing file
{s.fileCount === 1 ? '' : 's'} on <code className="text-xs">{s.backend}</code> will
not be reachable from the CRM after the switch unless you migrate them later. This is
rarely the right choice prefer Switch + migrate.
</div>
)}
feat(storage): pluggable s3-or-filesystem backend + migration CLI + admin UI Phase 6a from docs/berth-recommender-and-pdf-plan.md §4.7a + §14.9a. Lays the storage groundwork for Phase 6b/7 file-bearing schemas (per-berth PDFs, brochures) without touching those domains yet. New files: - src/lib/storage/index.ts StorageBackend interface + per-process factory keyed on system_settings. - src/lib/storage/s3.ts S3-compatible backend (MinIO/AWS/B2/R2/ Wasabi/Tigris) wrapping the existing minio JS client. Includes a healthCheck() used by the admin "Test connection" button. - src/lib/storage/filesystem.ts Local filesystem backend with all §14.9a mitigations baked in. - src/lib/storage/migrate.ts Shared migration core — pg_advisory_lock, per-row resumable progress markers, sha256 round-trip verification, atomic storage_backend flip on success. - scripts/migrate-storage.ts Thin CLI shim around runMigration(). - src/app/api/storage/[token]/route.ts Filesystem proxy GET. Verifies HMAC, enforces single-use replay protection via Redis SET NX, streams via NextResponse ReadableStream with explicit Content-Type + Content-Disposition. Node runtime only. - src/app/api/v1/admin/storage/route.ts GET status + POST connection test. - src/app/api/v1/admin/storage/migrate/route.ts Super-admin-only POST that runs the exact same runMigration() as the CLI. - src/app/(dashboard)/[portSlug]/admin/storage/page.tsx Super-admin admin UI (current backend, capacity stats, switch button with dry-run, test connection, backup hint). - src/components/admin/storage-admin-panel.tsx Client component for the page above. §14.9a critical mitigations implemented: - Path-traversal: storage keys validated against ^[a-zA-Z0-9/_.-]+$; `..`, `.`, `//`, leading `/`, and overlength keys rejected. - Realpath: storage root realpath'd at create time, every per-key resolution checked against the realpath'd prefix. - Storage root created (or chmod'd) to 0o700. - Multi-node refusal: FilesystemBackend.create() throws when MULTI_NODE_DEPLOYMENT=true. - HMAC token: sha256-HMAC over the (key, expiry, nonce, filename, content-type) payload. Verified with timingSafeEqual; bad sig, expired, or invalid-key payloads all return 403. - Single-use replay: token body cached in Redis SET NX EX 1800s. - sha256 round-trip: copyAndVerify() re-fetches from the target after put() and aborts the migration on any mismatch. - Free-disk pre-flight: when migrating to filesystem, sums byte counts via source.head() and aborts if free space < total * 1.2. - pg_advisory_lock(0xc7000a01) prevents concurrent migrations. - Resumable: per-row progress markers in _storage_migration_progress. system_settings keys read by the factory (jsonb, no schema change): storage_backend, storage_s3_endpoint, storage_s3_region, storage_s3_bucket, storage_s3_access_key, storage_s3_secret_key_encrypted, storage_s3_force_path_style, storage_filesystem_root, storage_proxy_hmac_secret_encrypted. Defaults: storage_backend=`s3`, storage_filesystem_root=`./storage` (./storage added to .gitignore). Tests added (34 tests, all green): - tests/unit/storage/filesystem-backend.test.ts — key validation allow/reject matrix, realpath escape, 0o700 perms, multi-node refusal, HMAC token sign/verify/tamper/expire/invalid-key. - tests/unit/storage/copy-and-verify.test.ts — sha256 mismatch on round-trip aborts the migration. - tests/integration/storage/proxy-route.test.ts — happy path, wrong HMAC secret, expired token, replay rejection. Phase 6a ships zero file-bearing tables — TABLES_WITH_STORAGE_KEYS is intentionally empty. berth_pdf_versions and brochure_versions land in Phase 6b and join the list there. Existing s3_key columns: only gdpr_export_jobs.storage_key, already named correctly — no rename needed. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-05 03:15:59 +02:00
<DialogFooter>
<Button variant="outline" onClick={() => setConfirmOpen(false)}>
Cancel
</Button>
<Button
disabled={migrateMutation.isPending}
feat: autonomous backlog push — admin UX overhaul + storage parity + residential parity + Documenso Phase 1 Massive multi-area push driven by docs/admin-ux-backlog.md. Every byte path now goes through getStorageBackend() so signed EOIs, contracts, brochures, berth PDFs, files, avatars, branding logos, and DB backups all work identically on S3 and filesystem backends. USER SETTINGS (rebuild) - Country + Timezone selectors with cross-defaulting - Browser-detected timezone banner ("Looks like you're in Europe/Paris…") - Email change with verification flow (user_email_changes table, OLD-address cancel link + NEW-address confirm link) + EMAIL_CHANGE_INSTANT=true dev shortcut - Password reset triggered via better-auth requestPasswordReset - Profile photo upload + crop (square 256×256) via shared <ImageCropperDialog> + /api/v1/me/avatar BRANDING - Shared <ImageCropperDialog> using react-easy-crop - Logo upload + crop in /admin/branding (writes via /api/v1/admin/settings/image -> storage backend) - Email header/footer HTML defaults injectable via "Insert default" - SettingsFormCard new field types: timezone (combobox), image-upload STORAGE ADMIN OVERHAUL - S3 config form FIRST, swap action SECOND - Test connection before any switch - Two-button switch: "Switch + migrate" vs "Switch only" with warning modals - runMigration() honours skipMigration flag - /api/ready + system-monitoring health check use the active storage backend instead of always probing MinIO - Filesystem backend already had full feature parity — verified BACKUP MANAGEMENT (real) - New backup_jobs table (id / status / trigger / size / storage_path) - runBackup() service spawns pg_dump --format=custom, streams to active storage backend via getStorageBackend().put() - /admin/backup page: trigger, history, download .dump for restore - Super-admin gated AI ADMIN PANEL - /admin/ai consolidates master switch + monthly token cap + provider credentials - Per-feature settings (OCR, berth-PDF parser, recommender) linked from the same page ONBOARDING WIZARD - /admin/onboarding now real with auto-checked steps - Reads each setting key + lists endpoint (roles/users/tags) to decide completion - Manual checkboxes for steps without an auto-detect signal - Progress bar + Mark done/Mark incomplete buttons - State persisted in system_settings.onboarding_manual_status RESIDENTIAL PARITY (full) - New residential_client_notes + residential_interest_notes tables (mirror marina-side shape) - Polymorphic notes.service.ts extended (verifyParent, listForEntity, create, update, delete) for residential_clients/_interests - <NotesList> component accepts the new entity types - 4 new note endpoints (GET/POST/PATCH/DELETE for clients + interests) - 2 new activity endpoints (residential clients + interests) - residential-client-tabs.tsx + residential-interest-tabs.tsx use DetailLayout (Overview / Interests / Notes / Activity) - residential-client-detail-header.tsx mirrors marina-side strip - useBreadcrumbHint wired into both detail components - Configurable Assigned-to dropdown (residential_interests.view perm) CONFIGURABLE RESIDENTIAL STAGES - residential-stages.service.ts with list / save / orphan-check - /api/v1/residential/stages GET/PUT - /admin/residential-stages admin UI with reassign-on-remove modal - Validators relaxed from z.enum to z.string DOCUMENSO PHASE 1 - Schema: document_signers.invited_at / opened_at / last_reminder_sent_at / signing_token (+ idx_ds_signing_token) - Schema: documents.completion_cc_emails (text[]) + auto_reminder_interval_days (int) - transformSigningUrl() now maps SignerRole -> URL segment via ROLE_TO_URL_SEGMENT (approver->cc, witness->witness) — fixes Risk #5 where approver invites landed on /sign/error - POST /api/v1/documents/[id]/send-invitation with auto-pick of next pending signer - Per-port settings: documenso_developer_label / _approver_label + documenso_developer_user_id / _approver_user_id (Phase 7 Project Director RBAC binding fields) ADMIN UX RAPID-FIRE - Sidebar collapse removed (always-expanded design) - Audit log: input sizes (h-9), date pickers w-44, action cell sub-label so single-row entries aren't blank - Sales email config: token list <details> + tooltips on threshold + body fields - Custom Settings card: long-form description - Reminder digest timezone uses TimezoneCombobox - Port form: currency dropdown (10 common currencies) + timezone combobox + brand color picker - Permissions count badge opens modal with granted/denied per resource - Role names display-normalized via prettifyRoleName - Tag form: native input type=color - Custom Fields page: amber heads-up about non-integration - Settings manager: select field type + fallthrough_policy as dropdown - Storage admin S3 fields ship as proper password + boolean LIST PAGES - Residential client list: clickable email/phone (mailto/tel/wa.me) - Residential interests + Documents Hub search inputs sized h-9 CURRENCY API - scripts/test-currency-api.ts verifies live Frankfurter fetch -> DB upsert -> getRate -> convert. Inverse-rate drift <=0.001 TESTS - 1185/1185 vitest passing - tsc clean - eslint 0 errors (16 pre-existing warnings) Note: WEBSITE_INTAKE_SECRET added to .env.example but committed separately due to pre-commit hook policy on .env* files. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-07 21:02:12 +02:00
onClick={() =>
migrateMutation.mutate({
from: s.backend,
to: otherBackend,
skipMigration: confirmMode === 'switch-only',
})
}
feat(storage): pluggable s3-or-filesystem backend + migration CLI + admin UI Phase 6a from docs/berth-recommender-and-pdf-plan.md §4.7a + §14.9a. Lays the storage groundwork for Phase 6b/7 file-bearing schemas (per-berth PDFs, brochures) without touching those domains yet. New files: - src/lib/storage/index.ts StorageBackend interface + per-process factory keyed on system_settings. - src/lib/storage/s3.ts S3-compatible backend (MinIO/AWS/B2/R2/ Wasabi/Tigris) wrapping the existing minio JS client. Includes a healthCheck() used by the admin "Test connection" button. - src/lib/storage/filesystem.ts Local filesystem backend with all §14.9a mitigations baked in. - src/lib/storage/migrate.ts Shared migration core — pg_advisory_lock, per-row resumable progress markers, sha256 round-trip verification, atomic storage_backend flip on success. - scripts/migrate-storage.ts Thin CLI shim around runMigration(). - src/app/api/storage/[token]/route.ts Filesystem proxy GET. Verifies HMAC, enforces single-use replay protection via Redis SET NX, streams via NextResponse ReadableStream with explicit Content-Type + Content-Disposition. Node runtime only. - src/app/api/v1/admin/storage/route.ts GET status + POST connection test. - src/app/api/v1/admin/storage/migrate/route.ts Super-admin-only POST that runs the exact same runMigration() as the CLI. - src/app/(dashboard)/[portSlug]/admin/storage/page.tsx Super-admin admin UI (current backend, capacity stats, switch button with dry-run, test connection, backup hint). - src/components/admin/storage-admin-panel.tsx Client component for the page above. §14.9a critical mitigations implemented: - Path-traversal: storage keys validated against ^[a-zA-Z0-9/_.-]+$; `..`, `.`, `//`, leading `/`, and overlength keys rejected. - Realpath: storage root realpath'd at create time, every per-key resolution checked against the realpath'd prefix. - Storage root created (or chmod'd) to 0o700. - Multi-node refusal: FilesystemBackend.create() throws when MULTI_NODE_DEPLOYMENT=true. - HMAC token: sha256-HMAC over the (key, expiry, nonce, filename, content-type) payload. Verified with timingSafeEqual; bad sig, expired, or invalid-key payloads all return 403. - Single-use replay: token body cached in Redis SET NX EX 1800s. - sha256 round-trip: copyAndVerify() re-fetches from the target after put() and aborts the migration on any mismatch. - Free-disk pre-flight: when migrating to filesystem, sums byte counts via source.head() and aborts if free space < total * 1.2. - pg_advisory_lock(0xc7000a01) prevents concurrent migrations. - Resumable: per-row progress markers in _storage_migration_progress. system_settings keys read by the factory (jsonb, no schema change): storage_backend, storage_s3_endpoint, storage_s3_region, storage_s3_bucket, storage_s3_access_key, storage_s3_secret_key_encrypted, storage_s3_force_path_style, storage_filesystem_root, storage_proxy_hmac_secret_encrypted. Defaults: storage_backend=`s3`, storage_filesystem_root=`./storage` (./storage added to .gitignore). Tests added (34 tests, all green): - tests/unit/storage/filesystem-backend.test.ts — key validation allow/reject matrix, realpath escape, 0o700 perms, multi-node refusal, HMAC token sign/verify/tamper/expire/invalid-key. - tests/unit/storage/copy-and-verify.test.ts — sha256 mismatch on round-trip aborts the migration. - tests/integration/storage/proxy-route.test.ts — happy path, wrong HMAC secret, expired token, replay rejection. Phase 6a ships zero file-bearing tables — TABLES_WITH_STORAGE_KEYS is intentionally empty. berth_pdf_versions and brochure_versions land in Phase 6b and join the list there. Existing s3_key columns: only gdpr_export_jobs.storage_key, already named correctly — no rename needed. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-05 03:15:59 +02:00
>
{migrateMutation.isPending && <Loader2 className="mr-2 h-4 w-4 animate-spin" />}
feat: autonomous backlog push — admin UX overhaul + storage parity + residential parity + Documenso Phase 1 Massive multi-area push driven by docs/admin-ux-backlog.md. Every byte path now goes through getStorageBackend() so signed EOIs, contracts, brochures, berth PDFs, files, avatars, branding logos, and DB backups all work identically on S3 and filesystem backends. USER SETTINGS (rebuild) - Country + Timezone selectors with cross-defaulting - Browser-detected timezone banner ("Looks like you're in Europe/Paris…") - Email change with verification flow (user_email_changes table, OLD-address cancel link + NEW-address confirm link) + EMAIL_CHANGE_INSTANT=true dev shortcut - Password reset triggered via better-auth requestPasswordReset - Profile photo upload + crop (square 256×256) via shared <ImageCropperDialog> + /api/v1/me/avatar BRANDING - Shared <ImageCropperDialog> using react-easy-crop - Logo upload + crop in /admin/branding (writes via /api/v1/admin/settings/image -> storage backend) - Email header/footer HTML defaults injectable via "Insert default" - SettingsFormCard new field types: timezone (combobox), image-upload STORAGE ADMIN OVERHAUL - S3 config form FIRST, swap action SECOND - Test connection before any switch - Two-button switch: "Switch + migrate" vs "Switch only" with warning modals - runMigration() honours skipMigration flag - /api/ready + system-monitoring health check use the active storage backend instead of always probing MinIO - Filesystem backend already had full feature parity — verified BACKUP MANAGEMENT (real) - New backup_jobs table (id / status / trigger / size / storage_path) - runBackup() service spawns pg_dump --format=custom, streams to active storage backend via getStorageBackend().put() - /admin/backup page: trigger, history, download .dump for restore - Super-admin gated AI ADMIN PANEL - /admin/ai consolidates master switch + monthly token cap + provider credentials - Per-feature settings (OCR, berth-PDF parser, recommender) linked from the same page ONBOARDING WIZARD - /admin/onboarding now real with auto-checked steps - Reads each setting key + lists endpoint (roles/users/tags) to decide completion - Manual checkboxes for steps without an auto-detect signal - Progress bar + Mark done/Mark incomplete buttons - State persisted in system_settings.onboarding_manual_status RESIDENTIAL PARITY (full) - New residential_client_notes + residential_interest_notes tables (mirror marina-side shape) - Polymorphic notes.service.ts extended (verifyParent, listForEntity, create, update, delete) for residential_clients/_interests - <NotesList> component accepts the new entity types - 4 new note endpoints (GET/POST/PATCH/DELETE for clients + interests) - 2 new activity endpoints (residential clients + interests) - residential-client-tabs.tsx + residential-interest-tabs.tsx use DetailLayout (Overview / Interests / Notes / Activity) - residential-client-detail-header.tsx mirrors marina-side strip - useBreadcrumbHint wired into both detail components - Configurable Assigned-to dropdown (residential_interests.view perm) CONFIGURABLE RESIDENTIAL STAGES - residential-stages.service.ts with list / save / orphan-check - /api/v1/residential/stages GET/PUT - /admin/residential-stages admin UI with reassign-on-remove modal - Validators relaxed from z.enum to z.string DOCUMENSO PHASE 1 - Schema: document_signers.invited_at / opened_at / last_reminder_sent_at / signing_token (+ idx_ds_signing_token) - Schema: documents.completion_cc_emails (text[]) + auto_reminder_interval_days (int) - transformSigningUrl() now maps SignerRole -> URL segment via ROLE_TO_URL_SEGMENT (approver->cc, witness->witness) — fixes Risk #5 where approver invites landed on /sign/error - POST /api/v1/documents/[id]/send-invitation with auto-pick of next pending signer - Per-port settings: documenso_developer_label / _approver_label + documenso_developer_user_id / _approver_user_id (Phase 7 Project Director RBAC binding fields) ADMIN UX RAPID-FIRE - Sidebar collapse removed (always-expanded design) - Audit log: input sizes (h-9), date pickers w-44, action cell sub-label so single-row entries aren't blank - Sales email config: token list <details> + tooltips on threshold + body fields - Custom Settings card: long-form description - Reminder digest timezone uses TimezoneCombobox - Port form: currency dropdown (10 common currencies) + timezone combobox + brand color picker - Permissions count badge opens modal with granted/denied per resource - Role names display-normalized via prettifyRoleName - Tag form: native input type=color - Custom Fields page: amber heads-up about non-integration - Settings manager: select field type + fallthrough_policy as dropdown - Storage admin S3 fields ship as proper password + boolean LIST PAGES - Residential client list: clickable email/phone (mailto/tel/wa.me) - Residential interests + Documents Hub search inputs sized h-9 CURRENCY API - scripts/test-currency-api.ts verifies live Frankfurter fetch -> DB upsert -> getRate -> convert. Inverse-rate drift <=0.001 TESTS - 1185/1185 vitest passing - tsc clean - eslint 0 errors (16 pre-existing warnings) Note: WEBSITE_INTAKE_SECRET added to .env.example but committed separately due to pre-commit hook policy on .env* files. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-07 21:02:12 +02:00
{confirmMode === 'switch-and-migrate' ? 'Migrate now' : 'Switch now'}
feat(storage): pluggable s3-or-filesystem backend + migration CLI + admin UI Phase 6a from docs/berth-recommender-and-pdf-plan.md §4.7a + §14.9a. Lays the storage groundwork for Phase 6b/7 file-bearing schemas (per-berth PDFs, brochures) without touching those domains yet. New files: - src/lib/storage/index.ts StorageBackend interface + per-process factory keyed on system_settings. - src/lib/storage/s3.ts S3-compatible backend (MinIO/AWS/B2/R2/ Wasabi/Tigris) wrapping the existing minio JS client. Includes a healthCheck() used by the admin "Test connection" button. - src/lib/storage/filesystem.ts Local filesystem backend with all §14.9a mitigations baked in. - src/lib/storage/migrate.ts Shared migration core — pg_advisory_lock, per-row resumable progress markers, sha256 round-trip verification, atomic storage_backend flip on success. - scripts/migrate-storage.ts Thin CLI shim around runMigration(). - src/app/api/storage/[token]/route.ts Filesystem proxy GET. Verifies HMAC, enforces single-use replay protection via Redis SET NX, streams via NextResponse ReadableStream with explicit Content-Type + Content-Disposition. Node runtime only. - src/app/api/v1/admin/storage/route.ts GET status + POST connection test. - src/app/api/v1/admin/storage/migrate/route.ts Super-admin-only POST that runs the exact same runMigration() as the CLI. - src/app/(dashboard)/[portSlug]/admin/storage/page.tsx Super-admin admin UI (current backend, capacity stats, switch button with dry-run, test connection, backup hint). - src/components/admin/storage-admin-panel.tsx Client component for the page above. §14.9a critical mitigations implemented: - Path-traversal: storage keys validated against ^[a-zA-Z0-9/_.-]+$; `..`, `.`, `//`, leading `/`, and overlength keys rejected. - Realpath: storage root realpath'd at create time, every per-key resolution checked against the realpath'd prefix. - Storage root created (or chmod'd) to 0o700. - Multi-node refusal: FilesystemBackend.create() throws when MULTI_NODE_DEPLOYMENT=true. - HMAC token: sha256-HMAC over the (key, expiry, nonce, filename, content-type) payload. Verified with timingSafeEqual; bad sig, expired, or invalid-key payloads all return 403. - Single-use replay: token body cached in Redis SET NX EX 1800s. - sha256 round-trip: copyAndVerify() re-fetches from the target after put() and aborts the migration on any mismatch. - Free-disk pre-flight: when migrating to filesystem, sums byte counts via source.head() and aborts if free space < total * 1.2. - pg_advisory_lock(0xc7000a01) prevents concurrent migrations. - Resumable: per-row progress markers in _storage_migration_progress. system_settings keys read by the factory (jsonb, no schema change): storage_backend, storage_s3_endpoint, storage_s3_region, storage_s3_bucket, storage_s3_access_key, storage_s3_secret_key_encrypted, storage_s3_force_path_style, storage_filesystem_root, storage_proxy_hmac_secret_encrypted. Defaults: storage_backend=`s3`, storage_filesystem_root=`./storage` (./storage added to .gitignore). Tests added (34 tests, all green): - tests/unit/storage/filesystem-backend.test.ts — key validation allow/reject matrix, realpath escape, 0o700 perms, multi-node refusal, HMAC token sign/verify/tamper/expire/invalid-key. - tests/unit/storage/copy-and-verify.test.ts — sha256 mismatch on round-trip aborts the migration. - tests/integration/storage/proxy-route.test.ts — happy path, wrong HMAC secret, expired token, replay rejection. Phase 6a ships zero file-bearing tables — TABLES_WITH_STORAGE_KEYS is intentionally empty. berth_pdf_versions and brochure_versions land in Phase 6b and join the list there. Existing s3_key columns: only gdpr_export_jobs.storage_key, already named correctly — no rename needed. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-05 03:15:59 +02:00
</Button>
</DialogFooter>
</DialogContent>
</Dialog>
</div>
);
}