98211066a585234158ed4a0e0c28a3e943a47033
8 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
| 98211066a5 |
fix(legacy-stage): purge 9-stage enum keys from rank tables and stale copy
L-001 hunt landed these:
- src/lib/services/clients.service.ts — stageRank used pre-refactor
9-stage names exclusively (`contract_signed`, `deposit_10pct`, …).
Every modern 7-stage interest fell to rank 0, making client-list
"most-progressed deal" sort effectively random. Modern values now
own the canonical ranks; legacy aliases map to their 7-stage
equivalents so historical audit data still sorts.
- src/lib/services/berth-recommender.service.ts — STAGE_ORDER had
the same 9-stage shape. LATE_STAGE_THRESHOLD pointed at the (now
nonexistent) `deposit_10pct` slot. Reworked to the 7-stage scale;
threshold now at `deposit_paid` (5).
- Stale comments referencing `deposit_10pct` in schema (clients,
financial) and client-archive services updated to current copy.
- Smart-archive dialog rendered `i.pipelineStage` as raw enum; now
routes through `stageLabelFor` (the new helper added with A2).
Test fixture updates: berth-recommender.test.ts numeric inputs
re-mapped to the new 7-stage scale (eoi_signed=5 → eoi=3, etc.).
1373/1373 vitest pass.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
|||
| 465650957b |
fix(pipeline-refactor): purge stale 9-stage name references
Audit of every '*_sent' / '*_signed' / 'in_communication' / 'details_sent' / 'deposit_10pct' / 'completed' literal under src/ caught four genuinely broken sites that migration 0062 collapsed away but the runtime code never followed through on: 1. alert-rules.ts: `interest.stale` matched 'details_sent' / 'in_communication' / 'eoi_sent' — none of which exist post-migration. The alert never fired. Updated to the new mid-funnel canon (enquiry / qualified / nurturing). 2. berth-recommender.service.ts: TWO copies of the same stage-rank CASE (one for active history, one for fallthrough scoring) referenced the full legacy 8-stage ladder. Every WHEN missed → MAX(...) returned 0 → tier-ladder + heat-score logic collapsed silently. Rebuilt both against the 7-stage canon mirroring getHotDeals. 3. interests.service.ts: clearInterestOutcome reopen default was the dead 'in_communication'. Switched to 'qualified' (closest analog; rep can still override via data.reopenStage). Pre-fix, any reopened deal fell through safeStage() to 'enquiry'. 4. report-generators.ts: revenue-PDF "total completed" filter intersected pipeline_stage='completed' AND outcome='won'. The stage filter is redundant today (setInterestOutcome always writes 'completed' for terminal outcomes) and is brittle to the upcoming sentinel-stage cleanup. Dropped the stage filter — outcome='won' is the canonical money-changed-hands signal. Follow-up flagged: setInterestOutcome still writes pipeline_stage = 'completed' as a sentinel, which is non-canonical under the new 7-stage type (PIPELINE_STAGES doesn't include 'completed'). Migration 0062's intent is `outcome` carries terminal state forward; pipeline_stage stays in-canon. Cleaning up requires sweeping every consumer of pipeline_stage='completed' as a terminal marker — separate commit. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
|||
| 60365dc3de |
fix(audit): backlog sweep — partial archived indexes, custom-fields per-entity gate, polish
Wave through the 2026-05-07 backlog of small/concrete audit-final-deferred
items (deferring the Documenso Phases 2-7 build and items needing design
decisions or live external instances).
DB schema:
- Migration 0046 converts 5 composite (port_id, archived_at) indexes to
partial WHERE archived_at IS NULL — clients, interests, yachts, and
both residential tables. Smaller, faster planner choice for the
dominant list-query shape.
Multi-tenant isolation:
- document_sends now verifies recipient.interestId belongs to the port
before landing on the audit row (the surrounding clientId check was
already port-scoped; interestId pollution was the gap).
Routes / API:
- /api/v1/custom-fields/[entityId] requires entityType query param and
gates on the matching resource permission (clients/interests/berths/
yachts/companies). Fixes the cross-resource gap where a user with
clients.view could read company custom-field values.
- Admin user list trash button wrapped in PermissionGate (edit was
already gated; remove was not).
Service polish:
- berth-recommender accepts string-shaped JSONB booleans
('true'/'false') so admin UIs that wrap values as strings don't
silently fall through to defaults.
- expense-pdf renderReceiptHeader anchors all text positions to a
captured baseY rather than reading mutating doc.y after rect+stroke.
Headers no longer drift on the first receipt page after a soft page
break.
- berth-pdf apply: collect non-finite numeric coercion drops + warn-log
them so partial silent drops are observable (was invisible because
the no-fields-supplied check only fires when ALL drop).
- Storage cache fingerprint comment documenting the encrypted-secret
invariant + the explicit invalidation hook.
UI polish:
- invoice-detail typed: replaced two `any` casts with a proper
InvoiceDetailData / LineItem / LinkedExpense interface set.
- YachtForm now accepts initialOwner prop. Wired through:
- client-yachts-tab passes { type: 'client', id: clientId }
- interest-form passes { type: 'client', id: selectedClientId }
- Interest-form yacht picker now includes company-owned yachts where
the selected client is a member (fetches client.companies and feeds
YachtPicker an array filter). Plus an inline "Add new" button that
opens YachtForm pre-bound to the client.
- YachtPicker accepts ownerFilter as single OR array for "match any"
semantics.
BACKLOG.md updated with what landed vs what's still deferred (and why
each deferred item is genuinely larger than this push warrants).
Tests: 1185/1185 vitest, tsc clean.
|
|||
Matt Ciaccio
|
4723994bdc |
feat(errors): platform-wide request ids + error codes + admin inspector
End-to-end error-handling overhaul. A user hitting any failure now sees
a plain-text message + stable error code + reference id. A super admin
can paste the id into /admin/errors/<id> for the full request shape,
sanitized body, error stack, and a heuristic likely-cause hint.
REQUEST CONTEXT (AsyncLocalStorage)
- src/lib/request-context.ts mints a per-request frame carrying
requestId + portId + userId + method + path + start timestamp.
- withAuth wraps every authenticated handler in runWithRequestContext
and accepts an upstream X-Request-Id header (validated shape) or
generates a fresh UUID. The id ALWAYS leaves on the X-Request-Id
response header, including early-return 401/403/4xx paths.
- Pino logger reads from the same context via mixin — every log
line emitted during the request automatically carries the ids
with no per-call threading.
ERROR CODE REGISTRY
- src/lib/error-codes.ts defines stable DOMAIN_REASON codes with
HTTP status + plain-text user-facing message (no jargon, written
for the rep on the phone with a customer).
- New CodedError class wraps a registered code + optional
internalMessage (admin-only — never sent to client).
- Existing AppError subclasses got plain-text default rewrites so
legacy throw sites improve immediately without migration.
- High-impact services migrated to specific codes:
expenses (RECEIPT_REQUIRED, INVOICE_LINKED), interest-berths
(CROSS_PORT_LINK_REJECTED), berth-pdf (PDF_MAGIC_BYTE / PDF_EMPTY /
PDF_TOO_LARGE / VERSION_ALREADY_CURRENT), recommender
(INTEREST_PORT_MISMATCH).
ERROR ENVELOPE
- errorResponse always sets X-Request-Id header + requestId field.
- 5xx responses include a "Quote error ID …" friendly line.
- 4xx kept clean (validation, permission, not-found don't pollute
the inspector — they're already in audit log).
PERSISTENCE (error_events table, migration 0040)
- One row per 5xx, keyed on requestId, with method/path/status/error
name+message/stack head (4KB cap)/sanitized body excerpt (1KB cap;
password/token/secret/etc keys redacted)/duration/IP/UA/metadata.
- captureErrorEvent extracts Postgres SQLSTATE/severity/cause.code
so the classifier can recognize FK / unique / NOT NULL / schema-
drift violations.
- Failure to persist is logged-not-thrown.
LIKELY-CULPRIT CLASSIFIER (src/lib/error-classifier.ts)
- 4-pass heuristic (first match wins):
1. Postgres SQLSTATE → human reason (23503 FK, 23505 unique,
42703 schema drift, 53300 connection limit, …)
2. Error class name (AbortError, TimeoutError, FetchError,
ZodError)
3. Stack-path patterns (/lib/storage/, /lib/email/, documenso,
openai|claude, /queue/workers/)
4. Free-text message keywords (econnrefused, rate limit, timeout,
unauthorized|invalid api key)
- Returns { label, hint, subsystem } for the inspector badge.
CLIENT SIDE
- apiFetch throws structured ApiError with message + code + requestId
+ details + retryAfter.
- toastError() helper renders the standard 3-line toast:
plain message / Error code: X / Reference ID: Y [Copy ID].
ADMIN INSPECTOR
- /<port>/admin/errors lists captured 5xx with status badge + path +
likely-culprit badge + truncated message + reference id. Filter by
status code; auto-refresh via TanStack Query.
- /<port>/admin/errors/<requestId> deep-dive: request shape, full
error name+message+stack, sanitized body excerpt, raw metadata,
registered-code lookup (so admin can compare to what user saw),
likely-culprit hint with subsystem tag.
- /<port>/admin/errors/codes is the in-app code reference page —
every registered code grouped by domain prefix, searchable, with
HTTP status + user message inline. Linked from inspector header
so admins can flip to it while triaging.
- Permission: admin.view_audit_log. Super admins see all ports;
regular admins port-scoped.
- system-monitoring dashboard now surfaces error_events alongside
permission_denied audit + queue failed jobs (RecentError gains
source: 'request' variant).
DOCS
- docs/error-handling.md walks through coded errors, plain-text
message guidelines, client toasting, admin inspector usage,
persistence rules, classifier internals, pruning, and the
legacy → CodedError migration path.
MIGRATION SAFETY
- Audit confirmed all 41 migrations (0000-0040) apply cleanly in
journal order against an empty DB. 0040 references ports(id)
which exists from 0000. 0035/0038 don't deadlock under sequential
psql -f. Removed redundant idx_ds_sent_by from 0038 (created in
0037).
Tests: 1168/1168 vitest passing. tsc clean.
- security-error-responses tests updated for plain-text messages
+ new optional response keys (code/requestId/message).
- berth-pdf-versions tests assert stable error codes via
toMatchObject({ code }) rather than message regex.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
Matt Ciaccio
|
180912ba9f |
fix(audit-final): pre-merge hardening + expense receipt UI
Final audit pass on feat/berth-recommender (3 parallel Opus agents) caught 5 critical and ~12 high-severity findings. All addressed in-branch; medium/low items deferred to docs/audit-final-deferred.md. Critical: - Add filesystem-backend PUT handler at /api/storage/[token] so presigned uploads stop 405-ing in filesystem mode (every browser-driven berth-PDF + brochure upload was broken). Same token-verify + replay protection as GET, plus magic-byte gate when c=application/pdf. - Forward req.signal into streamExpensePdf so an aborted 1000-receipt export no longer keeps grinding for minutes. - Strengthen Content-Disposition filename sanitization: \s matches CR/LF which would let documentName forge headers; restrict to [\w. -]+ and add filename* RFC 5987 fallback. - Lock public berths feed behind an explicit slug allowlist instead of ?portSlug= enumeration. - Reject cross-port interest_berths upserts (defense-in-depth on top of the recommender SQL port filter). High: - Recommender: width-only feasibility now caps length via L/W ratio so a 200ft berth doesn't surface for a 30ft beam request; total_interest_count filters out junction rows whose interest is in another port. - Mooring normalization follow-up migration (0034) catches un-hyphenated padded forms (A01) the original 0024 WHERE missed. - Send-out rate limit moved AFTER validation and scoped per-(port, user) so typos don't burn a slot and a multi-port rep can't be DoS'd by another tenant. - Default-brochure path now blocks an archived row from sneaking through the partial unique index. - NocoDB import --update-snapshot honoured under --dry-run so reps can refresh the seed JSON without committing DB writes. - PDF export: orderBy desc(expenseDate); apply isNull(archivedAt) when expenseIds are passed (was bypassed); flag rate-unavailable rows with an amber footer instead of silently treating them as 1:1; skip the USD->EUR chain when source already matches target. - expense-form-dialog: revokeObjectURL captures the URL in the closure instead of revoking the still-displayed one; reset upload state on close. - scan/page: handleClearReceipt resets in-flight scan/upload mutations; Save disabled while upload pending. - updateExpense re-asserts receipt-or-acknowledgement at the merged row so PATCH can't slip past the create-time refine. Plus the in-progress receipt upload UI for the expense form dialog (receipt picker + "I have no receipt" checkbox + warning banner) and a noReceiptAcknowledged flag on ExpenseRow for edit-mode hydration. Includes the canonical plan doc (referenced in CLAUDE.md), the handoff prompt, and a deferred-findings index for follow-up issues. 1163/1163 vitest passing. Typecheck clean. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Matt Ciaccio
|
a3e002852b |
fix(audit-2): integration regressions + data-integrity from second-pass review
Two reviewer agents did a second-pass deep audit of the 21-commit refactor. Eight findings; four fixed here (one was deferred with a schema comment, three were 🟡 nice-to-haves left for follow-up). Integration regressions (🟠 high): - Outbound webhook `interest.berth_linked` now fires from the new junction-add handler. Was emitting a socket-only event, leaving external integrations silent post-refactor. - Two new webhook events `interest.berth_unlinked` and `interest.berth_link_updated` added to WEBHOOK_EVENTS + INTERNAL_TO_WEBHOOK_MAP. PATCH and DELETE handlers now dispatch them alongside the existing socket emits — lifecycle parity restored. - BerthInterestPulse adds useRealtimeInvalidation for berth-link events. The query key was berth-scoped while the linked-berths dialog invalidates interest-scoped keys (no prefix match), so the pulse went stale. Bridges via the realtime hook now. Recommender semantic fix (🟠 medium-high): - aggregates CTE: active_interest_count now filters on `ib.is_specific_interest = true`, matching the public-map "Under Offer" derivation. EOI-bundle-only links no longer demote a berth to Tier C for other reps. Smoke test confirms previously-all-Tier-C results now correctly classify as Tier A. - Same CTE: `total_interest_count` uses COUNT(ib.berth_id) instead of COUNT(*) so a berth with no junction rows reports 0 (not 1 from the LEFT JOIN's NULL-right-side row). Prevents heat over-counting. Data integrity (🟠): - AcroForm tier rejects negative numerics in coerceFieldValue (was letting through `length_ft="-50"` which would poison the recommender feasibility filter on apply). - FilesystemBackend.resolveHmacSecret throws in production when storage_proxy_hmac_secret_encrypted is null. Dev still derives from BETTER_AUTH_SECRET for ergonomics; prod must explicitly configure. - Documented the circular FK between berths.current_pdf_version_id and berth_pdf_versions.id. Drizzle's `.references()` can't express the cycle so the schema column is plain text + a comment; the FK is authoritatively maintained by migration 0030. Tests still 1163/1163. tsc clean. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Matt Ciaccio
|
86372a857f |
fix(audit): post-review hardening across phases 0-7
15 of 17 findings from the consolidated audit (3 reviewer agents on the previously-shipped phase commits). Remaining two are nice-to-have follow-ups deferred. Critical (data integrity / security): - Public berths API: closed-deal junction rows no longer flip a berth to "Under Offer" - filter on `interests.outcome IS NULL` so won/ lost/cancelled don't pollute public-map status. Both list + single-mooring routes. - Recommender heat: cancelled outcomes now count as fall-throughs (SQL was `LIKE 'lost%'` which silently dropped them, leaving cancelled-only berths stuck in tier A). - Filesystem presignDownload returns an absolute URL (origin from APP_URL) so emailed download links resolve from external mail clients. - Magic-byte verification on the presigned-PUT path: both per-berth PDFs and brochures stream the first 5 bytes via the storage backend and reject + delete on `%PDF-` mismatch (was only enforced when the server saw the buffer; presign-PUT was wide open). - Replay-protection TTL aligned to the token's own expiry (was a fixed 30 min, but send-out tokens live 24 h). Floor 60 s, ceiling 25 days. - Brochures unique partial index on (port_id) WHERE is_default=true + 0032 migration. Closes the read-then-write race in the create/ update transactions. Important: - Recommender SQL: defense-in-depth `i.port_id = $portId` filter on the aggregates CTE. - berth-pdf service: per-berth pg_advisory_xact_lock around the version-number SELECT + insert. Storage key is now UUID-based so concurrent uploads can't collide on blob paths. Replaces `nextVersionNumber` with the tx-bound variant. - berth-pdf apply: rejects with ConflictError when parse_results contain a mooring-mismatch warning unless the caller passes `confirmMooringMismatch: true` (force-reconfirm gate was UI-only). - Send-out body: HTML-escape brochure filename in the download-link fallback (XSS guard). - parseDecimalWithUnit rejects negative numbers. - listClients DISTINCT ON for primary contact resolution: bounds contact-row count to ~2 per client. Defensive: - verifyProxyToken rejects NaN/Infinity expiries via Number.isFinite. - Replaced sql ANY() with inArray() in interest-berths. Tests: 1145 -> 1163 passing. Deferred: bulk-send rate limit (no bulk endpoint today), markdown italic regex breaking links with asterisks (cosmetic). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Matt Ciaccio
|
b1e787e55c |
feat(recommender): SQL ranking + tier ladder + heat scoring
Plan §4.4 + §13: pure SQL recommender, no AI. Single CTE chain (feasible -> aggregates) + JS-side tier classification, fall-through cooldown filter, heat scoring, and fit ranking. Per-port settings via system_settings layered over global + DEFAULT_RECOMMENDER_SETTINGS. Tier ladder (default): A : no interest history B : lost-only history (still recommendable + boosted by heat) C : active interest in early stage (open..eoi_signed) D : active interest at deposit_10pct or beyond (hidden by default) Heat (only for tier B): recency weight 30 full @ <=30 days, decays to 0 @ 365 days furthest stage weight 40 full when prior reached deposit interest count weight 15 saturates at 5+ EOI count weight 15 saturates at 3+ Multi-port isolation enforced (§14.10 critical): the SQL filters by port_id AND the entry-point function rejects cross-port interest lookups with an explicit error. Fall-through policy supports immediate_with_heat (default), cooldown, and never_auto_recommend. 15 unit tests covering tier classification, heat saturation, weight tuning, zero-weight guard. Smoke-tested end-to-end via scripts/dev-recommender-smoke.ts. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
