Commit Graph

3 Commits

Author SHA1 Message Date
25988dbfad fix(audit): import cluster — M27 (commit idempotency), M25 (in-file dedup preview), M26 (undo destructive-update reporting), L33 (mapping/mooring), L35 (port-auth doc)
M25 DB unique-index backstop deferred: needs a migration (column + backfill +
insert-stamp trigger + dedup) — tracked as a follow-up. The classify in-file
dedup (preview accuracy) ships now.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-02 12:41:00 +02:00
c7325010e6 feat(import): commit runner + undo + wired BullMQ worker
Third importer increment — the write path, fully testable without UI.

- commit.ts: commitBatch streams classified rows, applies insert/update per
  the conflict policy via the adapter (each row in its own try/catch so valid
  rows still land), records every action in import_batch_rows, and keeps live
  counts on the batch header. undoBatch hard-deletes a batch's inserted rows
  (port-scoped); a delete blocked by a dependent FK is reported, not forced,
  and the batch flips to `undone` only when every inserted row was removed.
- import worker: replaced the no-op placeholder with the real processor —
  loads the batch, re-reads the uploaded file from storage, parses, and runs
  commitBatch under the batch's mapping + policy. Marks the batch failed on
  error. Concurrency 1 so imports don't race each other's dedup lookups.

Tests: commit (skip/insert/error counts + per-row ledger + real inserted
entity), undo (removes exactly the inserted row, flips status), and
update-matches overwrite. 2 passing.

Engine is now functional end-to-end at the service layer: parse → map →
dry-run → commit → undo. Remaining: 4 FK adapters, API routes + permission,
wizard UI + history.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-01 22:36:42 +02:00
3cf12b3015 feat(import): engine core + companies/clients/berths adapters
Second importer increment: the generic engine + the three no-FK adapters,
fully unit + integration tested.

- types: ImportAdapter contract (targetFields, matchKey, findExisting,
  resolveForeignKeys, insert/update) + engine types.
- mapping: fuzzy header → target-field auto-suggest (exact / alias / edit
  distance, one header per field) + applyMapping (drops empty cells).
- classify: per-field zod + cross-field extraValidate, FK resolution hook,
  natural-key dedup, and the conflict-policy matrix
  (skip-matches / update-matches / error-on-match) → row outcomes + summary.
- engine: CSV (papaparse) + XLSX (ExcelJS) parse into a uniform
  {headers, rows} of trimmed strings.
- adapters (delegating to existing create/update services for audit +
  validation): companies (name dedup, update), clients (flat email/phone →
  contacts[], email-or-phone dedup, insert-only), berths (canonical mooring
  dedup, numeric coercion, update).
- registry: implemented adapters in dependency order.

Tests: 11 unit (mapping/validation/matchKey/parse) + 3 integration
(dedup + all three conflict policies on a seeded DB). 14 passing.

Next increments: FK adapters (yachts/interests/tenancies/expenses),
commit runner + worker, API routes + permission, wizard UI + undo.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-01 22:32:19 +02:00