fe863a588ec728637a0bfb57ee45af619f53ecb1
6 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
| fe863a588e |
feat(backup): full DR bundle export + admin-configurable offsite destinations
Backend-agnostic disaster-recovery backup engine that runs on the current storage backend (no storage cutover required): - Full-bundle export: db.dump (pg_dump custom) + every storage blob + manifest.json with per-object SHA-256, streamed as a tar. Entry points: admin UI download, GET /api/v1/admin/backup/export, scripts/create-full-backup.ts. - Admin-configurable push destinations (backup_destinations table, migration 0091): SFTP/SSH, S3-compatible (reuses the minio client), and mounted path/NAS behind one transport interface (test/push/prune). Secrets AES-GCM at rest; API returns only *IsSet markers. - Opt-in per-destination AES-256 bundle encryption (scrypt KDF, streamed) + scripts/decrypt-backup.ts for restore. - Wired the previously-dead database-backup cron to runScheduledBackupPush (push to enabled destinations, prune to retention, alert super-admins on failure). Tests: 1608 unit/integration pass; tsc + lint clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
|||
| 221ae5784e |
chore(autonomous-session): consolidate uncommitted work from prior session
Bundles the prior autonomous-session output that was sitting unstaged: - Em-dash sweep across src/ + tests/ (en-dash/em-dash to hyphen, ~2280 instances) - country-flag-icons rollout (CountryFlag component, replaces emoji glyphs that never rendered on Windows; lazy-loads the 3x2 SVG index as a single chunk after the per-subpath dynamic-import approach silently failed in webpack) - Admin IA Phase 1+2: 7-domain regroup, 41 to 38 pages, /admin/berths index, redirects (ocr to ai, reports to dashboard, invitations to users), docs/admin-ia-proposal.md - Per-template email tester (registry + endpoint + UI on Email admin page) - Cancel-document mode picker (delete-from-Documenso vs keep-for-audit) - Dashboard PDF report: 25 widgets, SVG charts, date-range picker, 11 resolvers - Customize-widgets per-region sortables at xl+ (charts/rails/feed); single flat sortable below xl when the layout stacks; per-viewport saved orders - Audit doc updates capturing each shipped item - Lint fixes: react-compiler immutability in DonutChart (reduce instead of let-reassign), set-state-in-effect disables in CountryFlag and UploadForSigning preview-bytes effect, unused 'confirm' destructures in interest contract + reservation tabs, unescaped apostrophe in test-template card copy |
|||
| 0baca41693 |
audit: Tier 0 quick wins — EMAIL_REDIRECT_TO prod guard + storage routing + metadata masking
Tier 0.2: src/lib/env.ts now refuses boot when NODE_ENV=production AND EMAIL_REDIRECT_TO is set. Sendmail logs the rewrite at warn (was debug) so dev/staging windows where someone forgets to unset are immediately visible. Tier 0.6: backup_jobs.storage_path added to TABLES_WITH_STORAGE_KEYS in src/lib/storage/migrate.ts. Flipping the storage backend used to silently orphan every pg_dump artefact — last-resort recovery path is now actually portable. Tier 1.7: createAuditLog now runs metadata through maskSensitiveFields (was only applied to old/new value diffs). Portal-auth, crm-invite, hard-delete and email-accounts services were writing raw emails into this column unbounded. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
|||
| 9a5ba87d6c |
fix(integration): webhook v2 events, storage migrate, test theatre
- F1: DOCUMENT_DECLINED handler (v2 Decline vs Reject) — routes to same handler as DOCUMENT_REJECTED until product refines downstream UX - Add RECIPIENT_VIEWED / RECIPIENT_SIGNED v2-alias cases with telemetry logging so we see when v2 deployments emit them - D1: populate TABLES_WITH_STORAGE_KEYS (files, berth_pdf_versions, brochure_versions, gdpr_exports) — was an empty list, migrated 0 files - MinIO putObject/getObject/statObject/removeObject socket timeout wrapper to prevent worker hangs on TCP blackhole (30s deadline) - E1: convert test.skip on smoke-setup infra failure to throw new Error so green-skipped silence becomes a real test failure (Playwright doesn't expose vitest's expect.fail) - Regression tests: folderId='' → null transform, applyEntityRestoredSuffix no-op (never-archived), syncEntityFolderName collision loop past (2) Note: matching .env.example documentation (D2 — bare DOCUMENSO_API_URL, DOCUMENSO_API_VERSION, MINIO_AUTO_CREATE_BUCKET, DOCUMENSO_TEMPLATE_ID_EOI, recipient role id vars) prepared but not committed — pre-commit hook blocks .env*. Apply manually via the separate .env workflow. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
|||
| 5c8c12ba1f |
feat: autonomous backlog push — admin UX overhaul + storage parity + residential parity + Documenso Phase 1
Massive multi-area push driven by docs/admin-ux-backlog.md. Every byte
path now goes through getStorageBackend() so signed EOIs, contracts,
brochures, berth PDFs, files, avatars, branding logos, and DB backups
all work identically on S3 and filesystem backends.
USER SETTINGS (rebuild)
- Country + Timezone selectors with cross-defaulting
- Browser-detected timezone banner ("Looks like you're in Europe/Paris…")
- Email change with verification flow (user_email_changes table,
OLD-address cancel link + NEW-address confirm link)
+ EMAIL_CHANGE_INSTANT=true dev shortcut
- Password reset triggered via better-auth requestPasswordReset
- Profile photo upload + crop (square 256×256) via shared
<ImageCropperDialog> + /api/v1/me/avatar
BRANDING
- Shared <ImageCropperDialog> using react-easy-crop
- Logo upload + crop in /admin/branding (writes via
/api/v1/admin/settings/image -> storage backend)
- Email header/footer HTML defaults injectable via "Insert default"
- SettingsFormCard new field types: timezone (combobox), image-upload
STORAGE ADMIN OVERHAUL
- S3 config form FIRST, swap action SECOND
- Test connection before any switch
- Two-button switch: "Switch + migrate" vs "Switch only" with
warning modals
- runMigration() honours skipMigration flag
- /api/ready + system-monitoring health check use the active
storage backend instead of always probing MinIO
- Filesystem backend already had full feature parity — verified
BACKUP MANAGEMENT (real)
- New backup_jobs table (id / status / trigger / size / storage_path)
- runBackup() service spawns pg_dump --format=custom, streams to
active storage backend via getStorageBackend().put()
- /admin/backup page: trigger, history, download .dump for restore
- Super-admin gated
AI ADMIN PANEL
- /admin/ai consolidates master switch + monthly token cap +
provider credentials
- Per-feature settings (OCR, berth-PDF parser, recommender)
linked from the same page
ONBOARDING WIZARD
- /admin/onboarding now real with auto-checked steps
- Reads each setting key + lists endpoint (roles/users/tags) to
decide completion
- Manual checkboxes for steps without an auto-detect signal
- Progress bar + Mark done/Mark incomplete buttons
- State persisted in system_settings.onboarding_manual_status
RESIDENTIAL PARITY (full)
- New residential_client_notes + residential_interest_notes tables
(mirror marina-side shape)
- Polymorphic notes.service.ts extended (verifyParent, listForEntity,
create, update, delete) for residential_clients/_interests
- <NotesList> component accepts the new entity types
- 4 new note endpoints (GET/POST/PATCH/DELETE for clients + interests)
- 2 new activity endpoints (residential clients + interests)
- residential-client-tabs.tsx + residential-interest-tabs.tsx use
DetailLayout (Overview / Interests / Notes / Activity)
- residential-client-detail-header.tsx mirrors marina-side strip
- useBreadcrumbHint wired into both detail components
- Configurable Assigned-to dropdown (residential_interests.view perm)
CONFIGURABLE RESIDENTIAL STAGES
- residential-stages.service.ts with list / save / orphan-check
- /api/v1/residential/stages GET/PUT
- /admin/residential-stages admin UI with reassign-on-remove modal
- Validators relaxed from z.enum to z.string
DOCUMENSO PHASE 1
- Schema: document_signers.invited_at / opened_at /
last_reminder_sent_at / signing_token (+ idx_ds_signing_token)
- Schema: documents.completion_cc_emails (text[]) +
auto_reminder_interval_days (int)
- transformSigningUrl() now maps SignerRole -> URL segment via
ROLE_TO_URL_SEGMENT (approver->cc, witness->witness) — fixes
Risk #5 where approver invites landed on /sign/error
- POST /api/v1/documents/[id]/send-invitation with auto-pick of
next pending signer
- Per-port settings: documenso_developer_label / _approver_label
+ documenso_developer_user_id / _approver_user_id (Phase 7
Project Director RBAC binding fields)
ADMIN UX RAPID-FIRE
- Sidebar collapse removed (always-expanded design)
- Audit log: input sizes (h-9), date pickers w-44, action cell
sub-label so single-row entries aren't blank
- Sales email config: token list <details> + tooltips on
threshold + body fields
- Custom Settings card: long-form description
- Reminder digest timezone uses TimezoneCombobox
- Port form: currency dropdown (10 common currencies) + timezone
combobox + brand color picker
- Permissions count badge opens modal with granted/denied per
resource
- Role names display-normalized via prettifyRoleName
- Tag form: native input type=color
- Custom Fields page: amber heads-up about non-integration
- Settings manager: select field type + fallthrough_policy as dropdown
- Storage admin S3 fields ship as proper password + boolean
LIST PAGES
- Residential client list: clickable email/phone (mailto/tel/wa.me)
- Residential interests + Documents Hub search inputs sized h-9
CURRENCY API
- scripts/test-currency-api.ts verifies live Frankfurter fetch
-> DB upsert -> getRate -> convert. Inverse-rate drift <=0.001
TESTS
- 1185/1185 vitest passing
- tsc clean
- eslint 0 errors (16 pre-existing warnings)
Note: WEBSITE_INTAKE_SECRET added to .env.example but committed
separately due to pre-commit hook policy on .env* files.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
|||
Matt Ciaccio
|
83693dd993 |
feat(storage): pluggable s3-or-filesystem backend + migration CLI + admin UI
Phase 6a from docs/berth-recommender-and-pdf-plan.md §4.7a + §14.9a. Lays
the storage groundwork for Phase 6b/7 file-bearing schemas (per-berth PDFs,
brochures) without touching those domains yet.
New files:
- src/lib/storage/index.ts StorageBackend interface + per-process
factory keyed on system_settings.
- src/lib/storage/s3.ts S3-compatible backend (MinIO/AWS/B2/R2/
Wasabi/Tigris) wrapping the existing minio
JS client. Includes a healthCheck() used
by the admin "Test connection" button.
- src/lib/storage/filesystem.ts Local filesystem backend with all §14.9a
mitigations baked in.
- src/lib/storage/migrate.ts Shared migration core — pg_advisory_lock,
per-row resumable progress markers,
sha256 round-trip verification, atomic
storage_backend flip on success.
- scripts/migrate-storage.ts Thin CLI shim around runMigration().
- src/app/api/storage/[token]/route.ts
Filesystem proxy GET. Verifies HMAC,
enforces single-use replay protection
via Redis SET NX, streams via NextResponse
ReadableStream with explicit Content-Type
+ Content-Disposition. Node runtime only.
- src/app/api/v1/admin/storage/route.ts
GET status + POST connection test.
- src/app/api/v1/admin/storage/migrate/route.ts
Super-admin-only POST that runs the
exact same runMigration() as the CLI.
- src/app/(dashboard)/[portSlug]/admin/storage/page.tsx
Super-admin admin UI (current backend,
capacity stats, switch button with
dry-run, test connection, backup hint).
- src/components/admin/storage-admin-panel.tsx
Client component for the page above.
§14.9a critical mitigations implemented:
- Path-traversal: storage keys validated against ^[a-zA-Z0-9/_.-]+$;
`..`, `.`, `//`, leading `/`, and overlength keys rejected.
- Realpath: storage root realpath'd at create time, every per-key
resolution checked against the realpath'd prefix.
- Storage root created (or chmod'd) to 0o700.
- Multi-node refusal: FilesystemBackend.create() throws when
MULTI_NODE_DEPLOYMENT=true.
- HMAC token: sha256-HMAC over the (key, expiry, nonce, filename,
content-type) payload. Verified with timingSafeEqual; bad sig,
expired, or invalid-key payloads all return 403.
- Single-use replay: token body cached in Redis SET NX EX 1800s.
- sha256 round-trip: copyAndVerify() re-fetches from the target after
put() and aborts the migration on any mismatch.
- Free-disk pre-flight: when migrating to filesystem, sums byte counts
via source.head() and aborts if free space < total * 1.2.
- pg_advisory_lock(0xc7000a01) prevents concurrent migrations.
- Resumable: per-row progress markers in _storage_migration_progress.
system_settings keys read by the factory (jsonb, no schema change):
storage_backend, storage_s3_endpoint, storage_s3_region,
storage_s3_bucket, storage_s3_access_key,
storage_s3_secret_key_encrypted, storage_s3_force_path_style,
storage_filesystem_root, storage_proxy_hmac_secret_encrypted.
Defaults: storage_backend=`s3`, storage_filesystem_root=`./storage`
(./storage added to .gitignore).
Tests added (34 tests, all green):
- tests/unit/storage/filesystem-backend.test.ts — key validation
allow/reject matrix, realpath escape, 0o700 perms, multi-node
refusal, HMAC token sign/verify/tamper/expire/invalid-key.
- tests/unit/storage/copy-and-verify.test.ts — sha256 mismatch on
round-trip aborts the migration.
- tests/integration/storage/proxy-route.test.ts — happy path, wrong
HMAC secret, expired token, replay rejection.
Phase 6a ships zero file-bearing tables — TABLES_WITH_STORAGE_KEYS is
intentionally empty. berth_pdf_versions and brochure_versions land in
Phase 6b and join the list there. Existing s3_key columns: only
gdpr_export_jobs.storage_key, already named correctly — no rename needed.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
