# Legacy → New CRM Data Migration — Design Spec > **Status:** DRAFT (2026-06-01) · scope locked · awaiting stage-map sign-off > **Goal:** Translate all live legacy data + reconnect documents/EOIs so the > new CRM "picks up exactly where we left off." > **Companion:** `docs/launch-readiness.md` Initiative 5 · `docs/deployment-plan.md` > **Source snapshot:** read-only `pg_dump` of prod NocoDB at > `private/nocodb-snapshot/` (gitignored), restored locally as `nocodb_legacy`. ## 1. Source landscape (verified 2026-06-01) Legacy data is spread across these systems (portal has **no DB of its own**): | System | What | Migrate? | | ------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------- | | **NocoDB "Port Nimara"** base `plplouets5zw1um` | Interests (255), Berths (117), Residences (45), multi-berth junction `_nc_m2m_Berths_Interests` (83), Website subs (Interest 64 / Contact 50 / BerthEOI 1), Newsletter (69), reminder/alert settings | ✅ | | **NocoDB "Expenses"** base `p3hq2fxdevqcaq8` | Expenses (165); `invoices` empty | ✅ | | **MinIO bucket `client-portal`** | EOIs, berth PDFs, receipts, business cards, general files | ✅ (Phase 2) | | **MinIO bucket `signatures`** | Documenso signed PDFs | ✅ (Phase 2) | | **Documenso v1.13.1** | Signing envelopes, linked per-deal by `documensoID` | ✅ (Phase 2) | | 9 other NocoDB bases (Customer_List, Registered Interest, Form Submissions, 2nd Residential, Image Uploads, EOI Queue, …) | Old imports/experiments/backups | ❌ **excluded** — zero code refs; stale 7–14 months | | Gmail (IMAP), Keycloak | Email archive, portal auth | ❌ out of scope (per Matt) | **Authority for scope:** the live portal + website code reference table IDs in **only** the two active bases above; the recency check confirms `Interests` is the only actively-written table (last write 2026-05-21). **Legacy has no Company entity** (everything is attributed to a person), so the migration creates **clients + yachts (client-owned) + deals** — no companies. ## 2. Key linking facts - **Client + yacht are inline on each Interests row** → extract + dedup. - **`documensoID`** (e.g. `"82"`) on each deal → resolves to Documenso `Envelope.secondaryId = 'document_' || documensoID` (verified: deal `doc=114` → envelope `document_114`). The envelope's completed PDF = the signed EOI. (Prod Documenso = v1.13.1, 140 migrations — confirmed.) - **`Berth Number`** (mooring, e.g. `D31`) + the `_nc_m2m_Berths_Interests` junction → multi-berth links. - **Notes** = inline `Internal Notes` + `Extra Comments` (+ 5 rows in `nc_comments`). - Dedup key for people: **lowercased email → fallback canonical phone**. ## 3. Phase 1 — NocoDB → new CRM (data) Build against the local `nocodb_legacy` snapshot; idempotent; every new row stamped with its `legacy_nocodb_id` (add a nullable column or a side mapping table `migration_id_map(entity, legacy_id, new_id)`). **Import order (FK-safe):** clients → yachts → interests → interest_berths → notes → residential → expenses → website_submissions → settings. ### 3.1 Clients (from Interests, deduped) Source fields → `clients`: `Full Name`→fullName (title-cased via the legacy `normalizePersonName` rule), `Email Address`→primary email, `Phone Number`→ canonical phone, `Address`+`Place of Residence`→address/locality, `Contact Method Preferred`→preferredContactMethod, `Source`→source, `Lead Category`→(deal-level, see below). **Dedup:** group all 255 interests by lowercased email (fallback canonical phone); one client per unique person, N deals. ### 3.2 Yachts (from Interests) `Yacht Name`→name (skip `TBC`/blank), `Length`/`Width`/`Depth`→dims. **Unit note:** legacy stores strings like `"50ft"` — parse number + unit, convert ft→m to match the berth/yacht numeric schema (store original string in a note if ambiguous). Owner = the deduped client (polymorphic `client`). ### 3.3 Interests / deals - **Stage:** map `Sales Process Level` (8) → new 7-stage pipeline — **see §4 (needs sign-off).** - `Lead Category` (General / Friends and Family)→leadCategory, `Source`→source. - Statuses: `EOI Status`, `Deposit 10% Status`, `Contract Status`, `Contract Sent Status`, `Berth Info Sent Status` → drive stage + the new EOI/contract/deposit fields; `Deposit 10% Status='Received'` → a `payments` row (deposit) + auto-advance. - Dates: `Date Added`/`Created At`→createdAt (DD-MM-YYYY → ISO; many are null — fall back to Documenso/earliest signal), `EOI Time Sent`, `Time LOI Sent`. - `documensoID` → stored for Phase 2 EOI relink. - **Outcome:** `Sales Process Level='Contract Signed'` + deposit/contract complete → won; otherwise open. (No explicit "lost" in legacy.) ### 3.4 interest_berths (multi-berth) From `_nc_m2m_Berths_Interests` (83 links) → `interest_berths` via `interestBerthsService`. `is_primary` = the `Berth Number` plain-text mooring (or first link); `is_in_eoi_bundle` = true for signed/sent EOIs. Resolve berth by mooring against the migrated 117 berths. ### 3.5 Notes `Internal Notes` + `Extra Comments` (and `nc_comments`) → `interestNotes` via `notes.service`, preserving original timestamps where present. ### 3.6 Residential `Interests (Residences)` (45) → `residential_clients` + `residential_interests` (dedup by email). The 2nd residential base (16 rows) is **excluded** (stale). ### 3.7 Expenses `Expenses` base (165) → the expenses module. Map Time→date, Payer→payer, Category→category, Price (string `"€1,234"`)→numeric+currency. Receipts linked in Phase 2 (the `Receipts` images live in MinIO). ### 3.8 Website submissions + settings Website Interest/Contact/BerthEOI subs → `website_submissions`. `reminder_settings` /`alert_settings` → best-effort into `system_settings`. ## 4. Stage mapping (8 → 7) — NEEDS SIGN-OFF Legacy `Sales Process Level` → new pipeline stage (proposed): | Legacy | New stage | | ------------------------------- | --------------------------- | | General Qualified Interest | `qualified` | | Specific Qualified Interest | `nurturing` | | EOI and NDA Sent | `eoi` | | Signed EOI and NDA | `eoi` (EOI signed) | | Made Reservation | `reservation` | | Contract Negotiation | `reservation` → `contract`? | | Contract Negotiations Finalized | `contract` | | Contract Signed | `contract` (won) | Open questions for Matt: (a) is "General Qualified Interest" really `qualified` or should some map to `enquiry`? (b) does "Contract Negotiation" belong in `reservation` or `contract`? (c) treat `Contract Signed` as a closed-won outcome? ## 5. Phase 2 — documents & EOIs (MinIO inventoried 2026-06-01) Documents live in **three** MinIO buckets (verified): - **`client-portal`** (248 objects, 240 MB) — cleanly foldered: `Berth-PDFs/` (114, mooring in filename), `EOIs/` (95 signed EOIs foldered by client name), `Client Documents/` (6), `Legal/` (14), `expense-sheets/` (2), `client-emails/` (3 sent-email JSONs keyed `interest-`). - **`signatures`** (323) — Documenso's raw per-envelope store (many test dupes — secondary source). - **`database`** — NocoDB's own attachment store at `database/nc/uploads/noco/plplouets5zw1um/mbs9hjauug4eseo/cjzx7y2h9sxwd0n/…` (field `cjzx7y2h9sxwd0n` = `EOI_Document`). **This is where the pre-Documenso ("before/aside") signed EOIs live**, as NocoDB attachments. **EOI coverage — verified, no missing signed EOI.** Of 255 interests, 48 are EOI-signed; every one resolves to a recoverable PDF: 1. **~38 via `documensoID`** → `Envelope.secondaryId='document_'||id` → completed PDF (+ curated copy in `client-portal/EOIs//`). 2. **~10 old LOI-process deals** (no documensoID, `LOI=Signing Complete`) → `EOI_Document` attachment in the **`database`** bucket. 3. **3 via explicit `S3_Documenso_Path`** → `client-portal/EOIs/`. Backfill order per deal: prefer the curated `client-portal/EOIs/` copy → fall back to Documenso (by secondaryId) → then the NocoDB `database` attachment. Each → store via `getStorageBackend()` → `files`+`documents` rows → `ensureEntityFolder`. Still run a file↔deal reconciliation to flag orphan EOI files + confirm each envelope PDF actually downloads. 4. **Berth PDFs:** `client-portal/Berth-PDFs/` (114) → `berth_pdf_versions` (mooring parsed from filename). 5. **Receipts / business cards:** NOT in `client-portal` — likely in `forms`/ `images`/`directus` buckets (OpnForm uploads). Hunt only if wanted. 6. Unresolved → manual-review CSV. ### ⚠ Crossover gate — in-flight Documenso signings Documenso currently holds **6 PENDING** (sent, awaiting signature) + **6 DRAFT** envelopes (of 58 total; 46 COMPLETED). PENDING: Thomas Nemic (2026-02-04), Davy Morée (2025-11-28), Matthew Ciaccio (2025-11-24), Ben Sturge (2025-10-11), Van der Merwe (2025-10-02), Charles Davis (2025-08-22) — most stale/likely abandoned, only one from 2026. **Before the Documenso upgrade/crossover, review these:** void the dead ones, let any genuine one finish — don't strand an active signature. ## 6. Verification & reconcile **Validated run (2026-06-01, `extract-nocodb.ts`):** 255 interests → **232 unique clients** (1.10×; 21 with >1 deal roll up correctly), 39 yachts, 84 deal↔berth links (12 multi-berth), 63 notes. Stages 8→7: qualified 171 · eoi 51 · nurturing 30 · reservation 2 · contract 1. **EOI coverage 48/48 resolvable.** Signing state (Documenso-authoritative): signed 48 · **awaiting_signature 3** (interests 581/633/639 → migrate as "awaiting" + keep envelope link + display pending) · none 204. Duplicate review: 1 exact-name (Etiennette Clamouze ×2), 0 fuzzy. Residential 45→35. Expenses 165 (0 parse fails). Output → `private/migration-output/` (gitignored). **In-flight signing display:** the 3 `awaiting_signature` deals load with the interest's EOI state = sent/awaiting + the Documenso envelope linked, so the new CRM's webhook/poll completes them and the UI shows "Waiting for signatures." Reconcile the 6 Documenso PENDING: 3 link to deals (in-flight above); 3 are abandoned re-sends of already-signed deals → void-review before crossover. Remaining: spot-check 5 deals end-to-end after load. ## 7. Deliverables (scripts/migration/) - `probe-minio.ts` — bucket inventory (Phase 2 sizing; answers "are the business cards there?"). - `extract-nocodb.ts` — read the snapshot, emit normalized JSON per entity. - `transform-load.ts` — dedup + map + load via service helpers, idempotent. - `backfill-documents.ts` — Phase 2 EOI/PDF/receipt backfill. - `reconcile.ts` — final report. ## 8. Decisions locked (2026-06-01) - Scope = the 2 active bases only; 9 others excluded; email/Keycloak out. - Extract via read-only pg_dump snapshot (done). - No company entities (legacy has none). - Idempotent, keyed on `legacy_nocodb_id`.