letsbe/pn-new-crm
1
0
Fork 0
Code Issues Pull Requests 1 Actions Packages Projects Releases Wiki Activity
394 Commits 8 Branches 0 Tags
4a50bab389f00687ef673a77bdecbc8a1d7aad6f
Commit Graph

4 Commits

Author SHA1 Message Date
Matt
5c8c12ba1f feat: autonomous backlog push — admin UX overhaul + storage parity + residential parity + Documenso Phase 1
Some checks failed
Build & Push Docker Images / lint (push) Successful in 1m32s
Details
Build & Push Docker Images / build-and-push (push) Failing after 32s
Details 
Massive multi-area push driven by docs/admin-ux-backlog.md. Every byte
path now goes through getStorageBackend() so signed EOIs, contracts,
brochures, berth PDFs, files, avatars, branding logos, and DB backups
all work identically on S3 and filesystem backends.

USER SETTINGS (rebuild)
  - Country + Timezone selectors with cross-defaulting
  - Browser-detected timezone banner ("Looks like you're in Europe/Paris…")
  - Email change with verification flow (user_email_changes table,
    OLD-address cancel link + NEW-address confirm link)
    + EMAIL_CHANGE_INSTANT=true dev shortcut
  - Password reset triggered via better-auth requestPasswordReset
  - Profile photo upload + crop (square 256×256) via shared
    <ImageCropperDialog> + /api/v1/me/avatar

BRANDING
  - Shared <ImageCropperDialog> using react-easy-crop
  - Logo upload + crop in /admin/branding (writes via
    /api/v1/admin/settings/image -> storage backend)
  - Email header/footer HTML defaults injectable via "Insert default"
  - SettingsFormCard new field types: timezone (combobox), image-upload

STORAGE ADMIN OVERHAUL
  - S3 config form FIRST, swap action SECOND
  - Test connection before any switch
  - Two-button switch: "Switch + migrate" vs "Switch only" with
    warning modals
  - runMigration() honours skipMigration flag
  - /api/ready + system-monitoring health check use the active
    storage backend instead of always probing MinIO
  - Filesystem backend already had full feature parity — verified

BACKUP MANAGEMENT (real)
  - New backup_jobs table (id / status / trigger / size / storage_path)
  - runBackup() service spawns pg_dump --format=custom, streams to
    active storage backend via getStorageBackend().put()
  - /admin/backup page: trigger, history, download .dump for restore
  - Super-admin gated

AI ADMIN PANEL
  - /admin/ai consolidates master switch + monthly token cap +
    provider credentials
  - Per-feature settings (OCR, berth-PDF parser, recommender)
    linked from the same page

ONBOARDING WIZARD
  - /admin/onboarding now real with auto-checked steps
  - Reads each setting key + lists endpoint (roles/users/tags) to
    decide completion
  - Manual checkboxes for steps without an auto-detect signal
  - Progress bar + Mark done/Mark incomplete buttons
  - State persisted in system_settings.onboarding_manual_status

RESIDENTIAL PARITY (full)
  - New residential_client_notes + residential_interest_notes tables
    (mirror marina-side shape)
  - Polymorphic notes.service.ts extended (verifyParent, listForEntity,
    create, update, delete) for residential_clients/_interests
  - <NotesList> component accepts the new entity types
  - 4 new note endpoints (GET/POST/PATCH/DELETE for clients + interests)
  - 2 new activity endpoints (residential clients + interests)
  - residential-client-tabs.tsx + residential-interest-tabs.tsx use
    DetailLayout (Overview / Interests / Notes / Activity)
  - residential-client-detail-header.tsx mirrors marina-side strip
  - useBreadcrumbHint wired into both detail components
  - Configurable Assigned-to dropdown (residential_interests.view perm)

CONFIGURABLE RESIDENTIAL STAGES
  - residential-stages.service.ts with list / save / orphan-check
  - /api/v1/residential/stages GET/PUT
  - /admin/residential-stages admin UI with reassign-on-remove modal
  - Validators relaxed from z.enum to z.string

DOCUMENSO PHASE 1
  - Schema: document_signers.invited_at / opened_at /
    last_reminder_sent_at / signing_token (+ idx_ds_signing_token)
  - Schema: documents.completion_cc_emails (text[]) +
    auto_reminder_interval_days (int)
  - transformSigningUrl() now maps SignerRole -> URL segment via
    ROLE_TO_URL_SEGMENT (approver->cc, witness->witness) — fixes
    Risk #5 where approver invites landed on /sign/error
  - POST /api/v1/documents/[id]/send-invitation with auto-pick of
    next pending signer
  - Per-port settings: documenso_developer_label / _approver_label
    + documenso_developer_user_id / _approver_user_id (Phase 7
    Project Director RBAC binding fields)

ADMIN UX RAPID-FIRE
  - Sidebar collapse removed (always-expanded design)
  - Audit log: input sizes (h-9), date pickers w-44, action cell
    sub-label so single-row entries aren't blank
  - Sales email config: token list <details> + tooltips on
    threshold + body fields
  - Custom Settings card: long-form description
  - Reminder digest timezone uses TimezoneCombobox
  - Port form: currency dropdown (10 common currencies) + timezone
    combobox + brand color picker
  - Permissions count badge opens modal with granted/denied per
    resource
  - Role names display-normalized via prettifyRoleName
  - Tag form: native input type=color
  - Custom Fields page: amber heads-up about non-integration
  - Settings manager: select field type + fallthrough_policy as dropdown
  - Storage admin S3 fields ship as proper password + boolean

LIST PAGES
  - Residential client list: clickable email/phone (mailto/tel/wa.me)
  - Residential interests + Documents Hub search inputs sized h-9

CURRENCY API
  - scripts/test-currency-api.ts verifies live Frankfurter fetch
    -> DB upsert -> getRate -> convert. Inverse-rate drift <=0.001

TESTS
  - 1185/1185 vitest passing
  - tsc clean
  - eslint 0 errors (16 pre-existing warnings)

Note: WEBSITE_INTAKE_SECRET added to .env.example but committed
separately due to pre-commit hook policy on .env* files.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-07 21:02:12 +02:00
Matt Ciaccio
fc7595faf8 fix(audit-tier-2): error-surface hygiene — toastError + CodedError sweep 
Two mechanical sweeps closing the audit's HIGH §16 + MED §11 findings:

* 38 client components / 56 toast.error sites converted to
  toastError(err) so the new admin error inspector becomes usable from
  user-reported issues — every failed inline-edit, save, send, archive,
  upload, etc. now carries the request-id + error-code (Copy ID action).
* 26 service files / 62 bare-Error throws converted to CodedError or
  the existing AppError subclasses.  Adds new error codes:
  DOCUMENSO_UPSTREAM_ERROR (502), DOCUMENSO_AUTH_FAILURE (502),
  DOCUMENSO_TIMEOUT (504), OCR_UPSTREAM_ERROR (502),
  IMAP_UPSTREAM_ERROR (502), UMAMI_UPSTREAM_ERROR (502),
  UMAMI_NOT_CONFIGURED (409), and INSERT_RETURNING_EMPTY (500) for
  post-insert returning-empty guards.
* Five vitest assertions updated to match the new user-facing wording
  (client-merge "already been merged", expense/interest "couldn't find
  that …", documenso "signing service didn't respond").

Test status: 1168/1168 vitest, tsc clean.

Refs: docs/audit-comprehensive-2026-05-05.md HIGH §16 (auditor-H Issue 1)
+ MED §11 (auditor-G Issue 1).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-05 20:18:05 +02:00
Matt Ciaccio
687a1f1c2f fix(audit-v3): platform-wide deferred-list cleanup (rounds 1-4) 
Working through the audit-v2 deferred backlog. Each round was tested
(typecheck + 1168/1168 vitest) before moving on.

Round 1 — DB performance + AI cost visibility:
- Add missing FK indexes Postgres doesn't auto-create on
  berth_reservations.{interest_id, contract_file_id},
  documents.{file_id, signed_file_id}, document_events.signer_id,
  document_templates.source_file_id, form_submissions.{form_template_id,
  client_id}, document_sends.{brochure_id, brochure_version_id,
  sent_by_user_id}. Without these, RESTRICT-checks on parent delete +
  reverse-lookups walk the child tables fully. Migration 0037.
- AI worker now writes one ai_usage_ledger row per OpenAI call so admins
  can audit spend per port/user/feature and future per-port budgets have
  history to read from. Failure to write is logged-not-thrown so the
  user-facing email draft is unaffected.

Round 2 — Boot-time + transport hardening:
- S3 backend verifies the bucket exists at startup (or auto-creates
  when MINIO_AUTO_CREATE_BUCKET=true). A typo'd bucket name now
  surfaces with a clear boot error instead of a vague Minio error
  inside the first user-facing request.
- Documenso v1 placeFields: 3-attempt exponential-backoff retry on 5xx
  + network errors, fail-fast on 4xx. Stops one transient flake from
  leaving a document with a partial field set.
- FilesystemBackend logs a structured warn-once at boot when the dev
  HMAC fallback is in effect, so two processes started with different
  BETTER_AUTH_SECRET values are observable (random 401s on file
  downloads otherwise).
- Logger redact paths extended to cover *.headers.{authorization,
  cookie}, *.config.headers.authorization, encrypted-credential blobs
  (secretKeyEncrypted, smtpPassEncrypted, etc.), the Documenso
  X-Documenso-Secret header, and 2-level nested forms.

Round 3 — UI feedback + permission gates:
- Storage admin migrate dialog: success toast with row count + error
  toast on both dryRun and migrate mutations.
- Invoice detail Send + Record-payment buttons wrapped in
  PermissionGate (invoices.send / invoices.record_payment); both
  mutations now toast on success/error.
- Admin user list Edit button wrapped in PermissionGate(admin.manage_users).
- Scan-receipt page surfaces an amber warning when OCR fails so reps
  know they can fill the form manually instead of staring at a stalled
  spinner; the editable form now also opens on scanMutation.isError
  / uploadedFile, not only on success.
- Email threads list now renders skeleton rows during load + shared
  EmptyState for the empty case (was a single "Loading…" line).

Round 4 — Service / route correctness:
- documentSends.sent_by_user_id was a free-text NOT NULL column with no
  FK. Now nullable + FK to user(id) ON DELETE SET NULL so the audit row
  survives a user being hard-deleted. Migration 0038 with a defensive
  null-out for any orphan ids before attaching the constraint.
- Saved-views route: documented why withAuth alone is correct (the
  service strictly filters by (portId, userId) — owner-only by design).
- Public-interests audit log: replaced "userId: null as unknown as
  string" cast with userId: null; AuditLogParams already accepts null
  for system-generated events.
- EOI in-app PDF fill: extracted setBerthRange() that, when the
  AcroForm field is missing AND the context has a non-empty range
  string, logs a structured warn so the deployment gap (live Documenso
  template needs the field) is observable instead of silently dropping
  the multi-berth range.

Test status: 1168/1168 vitest. tsc clean. Two new migrations
(0037/0038) need pnpm db:push (or migration apply) on the dev DB.
Deferred-doc updated with the remaining open items (bigger refactors).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-05 12:49:53 +02:00
Matt Ciaccio
83693dd993 feat(storage): pluggable s3-or-filesystem backend + migration CLI + admin UI 
Phase 6a from docs/berth-recommender-and-pdf-plan.md §4.7a + §14.9a. Lays
the storage groundwork for Phase 6b/7 file-bearing schemas (per-berth PDFs,
brochures) without touching those domains yet.

New files:
- src/lib/storage/index.ts        StorageBackend interface + per-process
                                  factory keyed on system_settings.
- src/lib/storage/s3.ts           S3-compatible backend (MinIO/AWS/B2/R2/
                                  Wasabi/Tigris) wrapping the existing minio
                                  JS client. Includes a healthCheck() used
                                  by the admin "Test connection" button.
- src/lib/storage/filesystem.ts   Local filesystem backend with all §14.9a
                                  mitigations baked in.
- src/lib/storage/migrate.ts      Shared migration core — pg_advisory_lock,
                                  per-row resumable progress markers,
                                  sha256 round-trip verification, atomic
                                  storage_backend flip on success.
- scripts/migrate-storage.ts      Thin CLI shim around runMigration().
- src/app/api/storage/[token]/route.ts
                                  Filesystem proxy GET. Verifies HMAC,
                                  enforces single-use replay protection
                                  via Redis SET NX, streams via NextResponse
                                  ReadableStream with explicit Content-Type
                                  + Content-Disposition. Node runtime only.
- src/app/api/v1/admin/storage/route.ts
                                  GET status + POST connection test.
- src/app/api/v1/admin/storage/migrate/route.ts
                                  Super-admin-only POST that runs the
                                  exact same runMigration() as the CLI.
- src/app/(dashboard)/[portSlug]/admin/storage/page.tsx
                                  Super-admin admin UI (current backend,
                                  capacity stats, switch button with
                                  dry-run, test connection, backup hint).
- src/components/admin/storage-admin-panel.tsx
                                  Client component for the page above.

§14.9a critical mitigations implemented:
- Path-traversal: storage keys validated against ^[a-zA-Z0-9/_.-]+$;
  `..`, `.`, `//`, leading `/`, and overlength keys rejected.
- Realpath: storage root realpath'd at create time, every per-key
  resolution checked against the realpath'd prefix.
- Storage root created (or chmod'd) to 0o700.
- Multi-node refusal: FilesystemBackend.create() throws when
  MULTI_NODE_DEPLOYMENT=true.
- HMAC token: sha256-HMAC over the (key, expiry, nonce, filename,
  content-type) payload. Verified with timingSafeEqual; bad sig,
  expired, or invalid-key payloads all return 403.
- Single-use replay: token body cached in Redis SET NX EX 1800s.
- sha256 round-trip: copyAndVerify() re-fetches from the target after
  put() and aborts the migration on any mismatch.
- Free-disk pre-flight: when migrating to filesystem, sums byte counts
  via source.head() and aborts if free space < total * 1.2.
- pg_advisory_lock(0xc7000a01) prevents concurrent migrations.
- Resumable: per-row progress markers in _storage_migration_progress.

system_settings keys read by the factory (jsonb, no schema change):
storage_backend, storage_s3_endpoint, storage_s3_region,
storage_s3_bucket, storage_s3_access_key,
storage_s3_secret_key_encrypted, storage_s3_force_path_style,
storage_filesystem_root, storage_proxy_hmac_secret_encrypted.

Defaults: storage_backend=`s3`, storage_filesystem_root=`./storage`
(./storage added to .gitignore).

Tests added (34 tests, all green):
- tests/unit/storage/filesystem-backend.test.ts — key validation
  allow/reject matrix, realpath escape, 0o700 perms, multi-node
  refusal, HMAC token sign/verify/tamper/expire/invalid-key.
- tests/unit/storage/copy-and-verify.test.ts — sha256 mismatch on
  round-trip aborts the migration.
- tests/integration/storage/proxy-route.test.ts — happy path, wrong
  HMAC secret, expired token, replay rejection.

Phase 6a ships zero file-bearing tables — TABLES_WITH_STORAGE_KEYS is
intentionally empty. berth_pdf_versions and brochure_versions land in
Phase 6b and join the list there. Existing s3_key columns: only
gdpr_export_jobs.storage_key, already named correctly — no rename needed.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-05 03:15:59 +02:00