9240cf18084b5a260dbd8d42e0e9e0863798b16b
12 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Matt Ciaccio
|
180912ba9f |
fix(audit-final): pre-merge hardening + expense receipt UI
Final audit pass on feat/berth-recommender (3 parallel Opus agents) caught 5 critical and ~12 high-severity findings. All addressed in-branch; medium/low items deferred to docs/audit-final-deferred.md. Critical: - Add filesystem-backend PUT handler at /api/storage/[token] so presigned uploads stop 405-ing in filesystem mode (every browser-driven berth-PDF + brochure upload was broken). Same token-verify + replay protection as GET, plus magic-byte gate when c=application/pdf. - Forward req.signal into streamExpensePdf so an aborted 1000-receipt export no longer keeps grinding for minutes. - Strengthen Content-Disposition filename sanitization: \s matches CR/LF which would let documentName forge headers; restrict to [\w. -]+ and add filename* RFC 5987 fallback. - Lock public berths feed behind an explicit slug allowlist instead of ?portSlug= enumeration. - Reject cross-port interest_berths upserts (defense-in-depth on top of the recommender SQL port filter). High: - Recommender: width-only feasibility now caps length via L/W ratio so a 200ft berth doesn't surface for a 30ft beam request; total_interest_count filters out junction rows whose interest is in another port. - Mooring normalization follow-up migration (0034) catches un-hyphenated padded forms (A01) the original 0024 WHERE missed. - Send-out rate limit moved AFTER validation and scoped per-(port, user) so typos don't burn a slot and a multi-port rep can't be DoS'd by another tenant. - Default-brochure path now blocks an archived row from sneaking through the partial unique index. - NocoDB import --update-snapshot honoured under --dry-run so reps can refresh the seed JSON without committing DB writes. - PDF export: orderBy desc(expenseDate); apply isNull(archivedAt) when expenseIds are passed (was bypassed); flag rate-unavailable rows with an amber footer instead of silently treating them as 1:1; skip the USD->EUR chain when source already matches target. - expense-form-dialog: revokeObjectURL captures the URL in the closure instead of revoking the still-displayed one; reset upload state on close. - scan/page: handleClearReceipt resets in-flight scan/upload mutations; Save disabled while upload pending. - updateExpense re-asserts receipt-or-acknowledgement at the merged row so PATCH can't slip past the create-time refine. Plus the in-progress receipt upload UI for the expense form dialog (receipt picker + "I have no receipt" checkbox + warning banner) and a noReceiptAcknowledged flag on ExpenseRow for edit-mode hydration. Includes the canonical plan doc (referenced in CLAUDE.md), the handoff prompt, and a deferred-findings index for follow-up issues. 1163/1163 vitest passing. Typecheck clean. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Matt Ciaccio
|
83693dd993 |
feat(storage): pluggable s3-or-filesystem backend + migration CLI + admin UI
Phase 6a from docs/berth-recommender-and-pdf-plan.md §4.7a + §14.9a. Lays
the storage groundwork for Phase 6b/7 file-bearing schemas (per-berth PDFs,
brochures) without touching those domains yet.
New files:
- src/lib/storage/index.ts StorageBackend interface + per-process
factory keyed on system_settings.
- src/lib/storage/s3.ts S3-compatible backend (MinIO/AWS/B2/R2/
Wasabi/Tigris) wrapping the existing minio
JS client. Includes a healthCheck() used
by the admin "Test connection" button.
- src/lib/storage/filesystem.ts Local filesystem backend with all §14.9a
mitigations baked in.
- src/lib/storage/migrate.ts Shared migration core — pg_advisory_lock,
per-row resumable progress markers,
sha256 round-trip verification, atomic
storage_backend flip on success.
- scripts/migrate-storage.ts Thin CLI shim around runMigration().
- src/app/api/storage/[token]/route.ts
Filesystem proxy GET. Verifies HMAC,
enforces single-use replay protection
via Redis SET NX, streams via NextResponse
ReadableStream with explicit Content-Type
+ Content-Disposition. Node runtime only.
- src/app/api/v1/admin/storage/route.ts
GET status + POST connection test.
- src/app/api/v1/admin/storage/migrate/route.ts
Super-admin-only POST that runs the
exact same runMigration() as the CLI.
- src/app/(dashboard)/[portSlug]/admin/storage/page.tsx
Super-admin admin UI (current backend,
capacity stats, switch button with
dry-run, test connection, backup hint).
- src/components/admin/storage-admin-panel.tsx
Client component for the page above.
§14.9a critical mitigations implemented:
- Path-traversal: storage keys validated against ^[a-zA-Z0-9/_.-]+$;
`..`, `.`, `//`, leading `/`, and overlength keys rejected.
- Realpath: storage root realpath'd at create time, every per-key
resolution checked against the realpath'd prefix.
- Storage root created (or chmod'd) to 0o700.
- Multi-node refusal: FilesystemBackend.create() throws when
MULTI_NODE_DEPLOYMENT=true.
- HMAC token: sha256-HMAC over the (key, expiry, nonce, filename,
content-type) payload. Verified with timingSafeEqual; bad sig,
expired, or invalid-key payloads all return 403.
- Single-use replay: token body cached in Redis SET NX EX 1800s.
- sha256 round-trip: copyAndVerify() re-fetches from the target after
put() and aborts the migration on any mismatch.
- Free-disk pre-flight: when migrating to filesystem, sums byte counts
via source.head() and aborts if free space < total * 1.2.
- pg_advisory_lock(0xc7000a01) prevents concurrent migrations.
- Resumable: per-row progress markers in _storage_migration_progress.
system_settings keys read by the factory (jsonb, no schema change):
storage_backend, storage_s3_endpoint, storage_s3_region,
storage_s3_bucket, storage_s3_access_key,
storage_s3_secret_key_encrypted, storage_s3_force_path_style,
storage_filesystem_root, storage_proxy_hmac_secret_encrypted.
Defaults: storage_backend=`s3`, storage_filesystem_root=`./storage`
(./storage added to .gitignore).
Tests added (34 tests, all green):
- tests/unit/storage/filesystem-backend.test.ts — key validation
allow/reject matrix, realpath escape, 0o700 perms, multi-node
refusal, HMAC token sign/verify/tamper/expire/invalid-key.
- tests/unit/storage/copy-and-verify.test.ts — sha256 mismatch on
round-trip aborts the migration.
- tests/integration/storage/proxy-route.test.ts — happy path, wrong
HMAC secret, expired token, replay rejection.
Phase 6a ships zero file-bearing tables — TABLES_WITH_STORAGE_KEYS is
intentionally empty. berth_pdf_versions and brochure_versions land in
Phase 6b and join the list there. Existing s3_key columns: only
gdpr_export_jobs.storage_key, already named correctly — no rename needed.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
Matt Ciaccio
|
cb83b09b2d |
Merge feat/dedup-migration: client dedup library + NocoDB migration script + admin queue
# Conflicts: # .gitignore # src/lib/db/migrations/meta/_journal.json |
||
|
Matt Ciaccio
|
80fc5932be |
chore: ignore tooling caches, scratch screenshots, sister website project
Three categories added to .gitignore:
Tool caches / runtime state:
.claude/ — Claude Code session state + lock files
(scheduled_tasks.lock untracked here)
.serena/ — Serena MCP project cache
ruvector.db — RuVector AgentDB binary
Scratch screenshots:
/*.jpg — debug captures dropped at repo root
/.audit-screenshots/ — UX audit run output (regenerable)
Sister project:
/website/ — separate Nuxt marketing site, kept on disk
for reference but not tracked in this repo
The single tracked file in .claude/ (scheduled_tasks.lock) is removed
from the index here; future dev sessions won't bring it back.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
Matt Ciaccio
|
18e5c124b0 |
feat(dedup): NocoDB migration script + tables (P3 dry-run)
Lands the one-shot migration pipeline from the legacy NocoDB Interests base into the new client/interest schema. Dry-run mode is fully operational: pulls the live snapshot, runs the dedup library, and writes a CSV + Markdown report under .migration/<timestamp>/. The --apply phase is stubbed for a follow-up PR per the design's P3 implementation sequence. Schema additions ================ - `client_merge_candidates` — pairs flagged by the background scoring job for the /admin/duplicates review queue. Status enum: pending / dismissed / merged. Unique-(portId, clientAId, clientBId) so the same pair can't surface twice. Empty until P2 lands the cron. - `migration_source_links` — idempotency ledger. Maps source-system rows (NocoDB Interest #624 → new client UUID) so re-running --apply against the same dry-run report skips already-imported entities. Both tables ship with the migration `0020_unusual_azazel.sql` — already applied to the local dev DB during this commit's preparation. Library ======= src/lib/dedup/nocodb-source.ts Read-only adapter for the legacy NocoDB v2 API. xc-token auth, auto-paginates until isLastPage, captures the table IDs from the 2026-05-03 audit. `fetchSnapshot()` pulls every relevant table in parallel into one in-memory object the transform layer consumes. src/lib/dedup/migration-transform.ts Pure function: NocoDB snapshot in, MigrationPlan out. Per row: - normalizes name / email / phone / country via the dedup library - parses the legacy DD-MM-YYYY / DD/MM/YYYY / ISO date formats - maps the 8-stage `Sales Process Level` enum to the new 9-stage pipelineStage - filters yacht-name placeholders ('TBC', 'Na', etc.) - merges Internal Notes + Extra Comments + Berth Size Desired into a single notes blob Then runs `findClientMatches` pairwise (with blocking) and union-finds clusters of rows whose score crosses the auto-link threshold (90). Lower-scoring pairs (50–89) become 'needs review'. Each cluster's "lead" row is picked by completeness score with recency tie-break. src/lib/dedup/migration-report.ts Writes three artifacts to .migration/<timestamp>/: - report.csv — one row per planned op, RFC-4180 escaped - summary.md — human-skimmable overview - plan.json — full structured plan for the --apply phase CSV cells with comma / quote / newline are quoted; internal quotes are doubled. No external CSV dep. src/lib/dedup/phone-parse.ts Script-safe wrapper around libphonenumber-js's `core` entry that loads `metadata.min.json` directly. The default `index.cjs.js` bundled by libphonenumber hits a metadata-shape interop bug under Node 25 + tsx (`{ default }` wrapping); core+JSON sidesteps it. The dedup `normalizePhone` and `find-matches` both use this wrapper now so the same code path runs in vitest, Next.js, and the migration CLI without surprises. src/lib/dedup/normalize.ts Tightened country resolution: added Caribbean short-form aliases ('antigua' → AG, 'st kitts' → KN, etc.) and a city map covering the US locations seen in the NocoDB dump (Boston, Tampa, Fort Lauderdale, Port Jefferson, Nantucket). Also relaxed phone parsing to drop the `isValid()` strict check — the libphonenumber min build rejects many real NANP-territory numbers, and dedup only needs a canonical E.164 to compare. CLI === scripts/migrate-from-nocodb.ts pnpm tsx scripts/migrate-from-nocodb.ts --dry-run → Pulls the live NocoDB base (NOCODB_URL + NOCODB_TOKEN env vars), runs the transform, writes report. No DB writes. pnpm tsx scripts/migrate-from-nocodb.ts --apply --report .migration/<dir>/ → Stubbed; exits with `not yet implemented` and a pointer to the design doc. Apply phase ships in a follow-up. Tests ===== tests/unit/dedup/migration-transform.test.ts (7 cases) Fixture-based regression. A frozen 12-row NocoDB snapshot covers every duplicate pattern in the design (§1.2). The test asserts: - 12 input rows → 7 unique clients (cluster math is right) - Patterns A / B / C / E auto-link - Pattern F (Etiennette Clamouze) does NOT auto-link - Every interest preserved as its own row even when clients merge - 8-stage → 9-stage enum mapping is correct per spec - Multi-yacht merge (Constanzo CALYPSO + Costanzo GEMINI under one client) — the design's signature win - Output is deterministic (run twice, identical) Validation against real data ============================ Ran `pnpm tsx scripts/migrate-from-nocodb.ts --dry-run` against the live NocoDB. Result on 252 Interests rows: - 237 clients (15 merged into 13 clusters) - 252 interests (one per source row) - 406 contacts, 52 addresses - 13 auto-linked clusters (every confirmed cluster from §1.2 audit) - 3 pairs flagged for review (Camazou, Zasso, one new) - 1 phone placeholder flagged Total dedup test count: 57 (50 from P1 + 7 fixture tests). Lint: clean. Tsc: clean for new files. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Matt Ciaccio
|
ba5fb6db5e |
feat(sales): EOI queue route + invoice→deposit auto-advance + won/lost outcomes
Three independent strengthenings of the sales spine that the prior coherence
sweep made it possible to do cleanly.
1. EOI queue page
- Sidebar entry under Documents → "EOI queue".
- Route /[port]/documents/eoi renders DocumentsHub with the existing
eoi_queue tab pre-selected (filters in-flight EOIs only).
- .gitignore: tightened root-only `eoi/` ignore so the documents/eoi
route is no longer silently excluded.
2. Invoice ↔ deposit link
- invoices.interestId (FK, ON DELETE SET NULL) + invoices.kind
('general' | 'deposit'). Indexed on (port_id, interest_id).
- createInvoiceSchema requires interestId when kind === 'deposit';
the service validates the linked interest belongs to the same port
before insert.
- recordPayment auto-advances pipelineStage to deposit_10pct (via
advanceStageIfBehind) when a paid invoice is kind=deposit and has
an interestId. No-op if the interest is already further along.
- "Create deposit invoice" link added to the Deposit milestone on the
interest detail. Links to /invoices/new?interestId=…&kind=deposit;
the form prefills the billing entity from the linked interest's
client and shows a context banner.
3. Won / lost terminal outcomes
- interests.outcome ('won' | 'lost_other_marina' | 'lost_unqualified'
| 'lost_no_response' | 'cancelled') + outcomeReason text +
outcomeAt timestamp. Indexed on (port_id, outcome).
- setInterestOutcome / clearInterestOutcome services + POST/DELETE
/api/v1/interests/:id/outcome endpoints (gated by change_stage
permission). Setting an outcome moves the interest to `completed`
in the same write; clearing reopens to `in_communication` (or a
caller-specified stage).
- Mark Won / Mark Lost icon buttons on the interest detail header,
plus an outcome badge that replaces the stage pill once a terminal
outcome is set, plus a Reopen button.
- Funnel + dashboard math updated to exclude lost/cancelled outcomes
from active calculations (KPIs.activeInterests, pipelineValueUsd,
getPipelineCounts, computePipelineFunnel, getRevenueForecast).
The funnel now also returns a `lost` summary so callers can
surface leakage without polluting conversion percentages.
Schema changes shipped via 0019_lazy_vampiro.sql; applied to dev DB
manually via psql because drizzle-kit push hits a pre-existing zod
parsing issue on the companies index. Dev server may need a restart
to flush prepared-statement caches.
tsc clean. vitest 832/832 pass. ESLint clean on every file touched.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
Matt Ciaccio
|
fbb1f1f366 |
scaffold(mobile): branch setup — audit harness, spec, plan, gitignore + client-portal cleanup
Pre-execution baseline for the mobile foundation PR: - Mobile audit harness (tests/e2e/audit/mobile.spec.ts + mobile-audit Playwright project) — visits every page at four anchor iPhone viewports (375/393/402/440), screenshots full-page to .audit/mobile/, generates index.md - Design spec (docs/superpowers/specs/2026-04-29-mobile-optimization-design.md) — adaptive shell + responsive content; full active-iPhone-range coverage; foundation + per-page migration phases - Implementation plan (docs/superpowers/plans/2026-04-29-mobile-foundation.md) — 24 TDD tasks for the foundation PR - .gitignore: ignore /client-portal/ (legacy nested Nuxt repo) and /.audit/ (regenerable screenshots) - Remove phantom client-portal gitlink (mode 160000 with no .gitmodules) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Matt Ciaccio
|
e8d61c91c4 |
feat(platform): residential module + admin UI + reliability fixes
Residential platform - New schema: residentialClients, residentialInterests (separate from marina/yacht clients) with migration 0010 - Service layer with CRUD + audit + sockets + per-port portal toggle - v1 + public API routes (/api/v1/residential/*, /api/public/residential-inquiries) - List + detail pages with inline editing for clients and interests - Per-user residentialAccess toggle on userPortRoles (migration 0011) - Permission keys: residential_clients, residential_interests - Sidebar nav + role form integration - Smoke spec covering page loads, UI create flow, public endpoint Admin & shared UI - Admin → Forms (form templates CRUD) with validators + service - Notification preferences page (in-app + email per type) - Email composition + accounts list + threads view - Branded auth shell shared across CRM + portal auth surfaces - Inline editing extended to yacht/company/interest detail pages - InlineTagEditor + per-entity tags endpoints (yachts, companies) - Notes service polymorphic across clients/interests/yachts/companies - Client list columns: yachtCount + companyCount badges - Reservation file-download via presigned URL (replaces stale <a href>) Route handler refactor - Extracted yachts/companies/berths reservation handlers to sibling handlers.ts files (Next.js 15 route.ts only allows specific exports) Reliability fixes - apiFetch double-stringify bug fixed across 13 components (apiFetch already JSON.stringifies its body; passing a stringified body produced double-encoded JSON which failed zod validation) - SocketProvider gated behind useSyncExternalStore-based mount check to avoid useSession() SSR crashes under React 19 + Next 15 - apiFetch falls back to URL-pathname → port-id resolution when the Zustand store hasn't hydrated yet (fresh contexts, e2e tests) - CRM invite flow (schema, service, route, email, dev script) - Dashboard route → [portSlug]/dashboard/page.tsx + redirect - Document the dev-server restart-after-migration gotcha in CLAUDE.md Tests - 5-case residential smoke spec - Integration test updates for new service signatures Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Matt Ciaccio
|
475b051e29 |
feat(portal): replace magic-link with email/password + admin-initiated activation
The client portal no longer uses passwordless / magic-link sign-in. Each
client now has a `portal_users` row with a scrypt-hashed password,
created by an admin from the client detail page; the admin's invite
mails an activation link that the client uses to set their own password.
Forgot-password is wired through the same token mechanism.
Schema (migration `0009_outgoing_rumiko_fujikawa.sql`):
- `portal_users` — one per client account, separate from the CRM
`users` table (better-auth) so the auth realms stay isolated. Email
is globally unique, password is null until activation.
- `portal_auth_tokens` — single-use activation / reset tokens. Stores
only the SHA-256 hash so a DB compromise never leaks live tokens.
Services:
- `src/lib/portal/passwords.ts` — scrypt hash/verify (no new deps;
uses node:crypto), token mint+hash helpers.
- `src/lib/services/portal-auth.service.ts` — createPortalUser,
resendActivation, activateAccount, signIn (timing-safe),
requestPasswordReset, resetPassword. Auth failures throw the new
UnauthorizedError (401); enumeration-safe behaviour everywhere.
Routes:
- POST /api/portal/auth/sign-in — sets the existing portal JWT cookie.
- POST /api/portal/auth/forgot-password — always 200.
- POST /api/portal/auth/reset-password — token + new password.
- POST /api/portal/auth/activate — token + initial password.
- POST /api/v1/clients/:id/portal-user — admin invite (and `?action=resend`).
- Removed: /api/portal/auth/request, /api/portal/auth/verify (magic link).
UI:
- /portal/login — replaced email-only magic-link form with email +
password + "forgot password" link.
- /portal/forgot-password, /portal/reset-password, /portal/activate — new.
- New shared `PasswordSetForm` component used by activate + reset.
- New `PortalInviteButton` rendered on the client detail header.
Email send:
- `createTransporter` now wires SMTP auth when SMTP_USER+SMTP_PASS are
set (gmail app-password or marina-server creds, configured via env).
- `SMTP_FROM` env var lets the sender address be overridden without
pinning it to `noreply@${SMTP_HOST}`.
Tests:
- Smoke spec 17 (client-portal) updated to the new flow: 7/7 green.
- Smoke specs 02-crud-spine, 05-invoices, 20-critical-path updated to
match the post-refactor client + invoice forms (drop companyName,
use OwnerPicker + billingEmail).
- Vitest 652/652 still green; type-check clean.
Drops the dead `requestMagicLink` from portal.service.ts.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
Matt Ciaccio
|
ed40662b99 |
chore: gitignore docker-compose.override.yml and .remember/
The override file is a local-only port remap for when the default dev postgres port is already bound by another project. .remember/ is skill-maintained session-state storage. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
| a13d7503cc |
Fix Docker build and production server infrastructure
- Add SKIP_ENV_VALIDATION to bypass Zod env check during next build - Bundle custom server.ts with esbuild so production uses Socket.io - Create worker entry point (src/worker.ts) with all BullMQ workers - Add esbuild build scripts for server and worker bundles - Fix Dockerfile.worker to include its own build stage - Fix pre-commit hook to work without global pnpm - Add CLAUDE.md with project conventions and quick reference Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
|||
| 67d7e6e3d5 |
Initial commit: Port Nimara CRM (Layers 0-4)
Full CRM rebuild with Next.js 15, TypeScript, Tailwind, Drizzle ORM, PostgreSQL, Redis, BullMQ, MinIO, and Socket.io. Includes 461 source files covering clients, berths, interests/pipeline, documents/EOI, expenses/invoices, email, notifications, dashboard, admin, and client portal. CI/CD via Gitea Actions with Docker builds. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
