# NocoDB → PostgreSQL Migration Mapping

**Purpose:** Field-by-field mapping from the current NocoDB tables to the new PostgreSQL schema, including transformation rules, data quality fixes, deduplication logic, and migration order.
**References:** `07-DATABASE-SCHEMA.md`, `IMPL-L6-MIGRATION.md`, annotated source `utils/types.ts`, `server/utils/nocodb.ts`, `server/utils/nocodb-invoice.ts`

---

## Source System Summary

The current NocoDB system has four primary tables:

| NocoDB Table | Table ID          | Est. Records | Destination Tables                                                                           |
| ------------ | ----------------- | ------------ | -------------------------------------------------------------------------------------------- |
| Interests    | `mbs9hjauug4eseo` | ~200–500     | `clients`, `client_contacts`, `interests`, `interest_notes`, `documents`, `document_signers` |
| Berths       | `mczgos9hr3oa9qc` | ~100–200     | `berths`, `berth_map_data`                                                                   |
| Expenses     | `mxfcefkk4dqs6uq` | ~500–2000    | `expenses`, `files` (receipts)                                                               |
| Invoices     | `mvyvz0lpc30p01s` | ~200–500     | `invoices`, `invoice_line_items`, `invoice_expenses`                                         |

**Key structural change:** NocoDB's Interests table is a "mega-table" that combines client identity data, contact details, vessel specifications, pipeline tracking, milestone dates, and document references into a single 60+ field row. The new schema normalizes this into separate `clients`, `client_contacts`, `interests`, `interest_notes`, and `documents` tables.

---

## Migration Order (Respects Foreign Keys)

```
1. ports              ← Seed from L0 (already exists)
2. clients            ← Derived from NocoDB Interests (dedup)
3. client_contacts    ← Derived from NocoDB Interests (dedup)
4. berths             ← From NocoDB Berths
5. berth_map_data     ← From NocoDB Berths (if SVG data exists)
6. interests          ← From NocoDB Interests (references clients + berths)
7. interest_notes     ← From NocoDB Interests "Extra Comments" field
8. documents          ← From NocoDB Interests (EOI/contract references)
9. document_signers   ← From NocoDB Interests (signing link fields)
10. expenses          ← From NocoDB Expenses
11. files             ← From MinIO scan (receipts, EOI PDFs, contracts)
12. invoices          ← From NocoDB Invoices
13. invoice_line_items ← Extracted from NocoDB Invoice data
14. invoice_expenses  ← Cross-reference invoices ↔ expenses
15. tags              ← Derived from any categorical data worth tagging
16. audit_logs        ← Synthetic historical entries (created_at events)
```

---

## Table 1: NocoDB Interests → `clients`

### Deduplication Strategy

The Interests table has one row per interest, but multiple interests can belong to the same client. Client identity must be deduplicated before insertion.

**Primary match key:** `Email Address` (case-insensitive, trimmed)
**Secondary match key:** `Full Name` (normalized) + `Place of Residence` (when email is null)
**On conflict:** Merge contacts, keep earliest `Created At`, concatenate notes with separator

```
normalizeClientName(name: string): string
  → trim whitespace
  → collapse multiple spaces
  → title case
  → strip honorifics (Mr., Mrs., Dr., Capt., etc.)
  → return normalized name
```

### Field Mapping

| NocoDB Interests Field     | Type (NocoDB) | →   | New Table.Column                   | Type (PG)   | Transformation                                                                                                                     |
| -------------------------- | ------------- | --- | ---------------------------------- | ----------- | ---------------------------------------------------------------------------------------------------------------------------------- |
| `Full Name`                | string        | →   | `clients.full_name`                | TEXT        | `normalizeClientName()` — trim, title-case, strip honorifics                                                                       |
| — (no company field)       | —             | →   | `clients.company_name`             | TEXT        | NULL — NocoDB has no company field; leave empty                                                                                    |
| — (no nationality)         | —             | →   | `clients.nationality`              | TEXT        | NULL                                                                                                                               |
| `Yacht Name`               | string        | →   | `clients.yacht_name`               | TEXT        | Trim, title-case                                                                                                                   |
| `Length`                   | string        | →   | `clients.yacht_length_ft`          | NUMERIC     | `parseFloat()` — NocoDB stores as string. If contains "m" suffix, convert m→ft (×3.28084). If contains "ft", strip suffix.         |
| `Width`                    | string        | →   | `clients.yacht_width_ft`           | NUMERIC     | Same string→number parsing as Length                                                                                               |
| `Depth`                    | string        | →   | `clients.yacht_draft_ft`           | NUMERIC     | Same string→number parsing as Length                                                                                               |
| `Length`                   | string        | →   | `clients.yacht_length_m`           | NUMERIC     | If originally in ft, convert ft→m (÷3.28084). If in m, use directly.                                                               |
| `Width`                    | string        | →   | `clients.yacht_width_m`            | NUMERIC     | Same dual-unit logic                                                                                                               |
| `Depth`                    | string        | →   | `clients.yacht_draft_m`            | NUMERIC     | Same dual-unit logic                                                                                                               |
| `Berth Size Desired`       | string        | →   | `clients.berth_size_desired`       | TEXT        | Direct copy, trim                                                                                                                  |
| `Contact Method Preferred` | string        | →   | `clients.preferred_contact_method` | TEXT        | Map: `"Email"` → `"email"`, `"Phone"` → `"phone"`, `"WhatsApp"` → `"whatsapp"`. Default `"email"`.                                 |
| `Source`                   | string        | →   | `clients.source`                   | TEXT        | Map: `"Website"` → `"website"`, `"Referral"` → `"referral"`, `"Direct"` → `"manual"`, `"Broker"` → `"broker"`. Default `"manual"`. |
| `Place of Residence`       | string        | →   | `clients.source_details`           | TEXT        | Store as additional context (may indicate referral origin)                                                                         |
| `Created At`               | datetime      | →   | `clients.created_at`               | TIMESTAMPTZ | Use earliest `Created At` from all interests belonging to this client                                                              |
| `Created At`               | datetime      | →   | `clients.updated_at`               | TIMESTAMPTZ | Use latest `Updated At` from all interests belonging to this client                                                                |
| —                          | —             | →   | `clients.port_id`                  | UUID        | Constant: Port Nimara UUID (from seed data)                                                                                        |
| —                          | —             | →   | `clients.is_proxy`                 | BOOLEAN     | `false` (no proxy data in NocoDB)                                                                                                  |
| —                          | —             | →   | `clients.archived_at`              | TIMESTAMPTZ | NULL                                                                                                                               |

**Unit parsing helper:**

```typescript
function parseDimension(value: string | null): { ft: number | null; m: number | null } {
  if (!value || value.trim() === '') return { ft: null, m: null };
  const cleaned = value.replace(/['"]/g, '').trim();
  const num = parseFloat(cleaned);
  if (isNaN(num)) return { ft: null, m: null };

  if (cleaned.toLowerCase().endsWith('m')) {
    return { ft: Math.round(num * 3.28084 * 100) / 100, m: num };
  }
  // Default: assume feet (most common in NocoDB data)
  return { ft: num, m: Math.round((num / 3.28084) * 100) / 100 };
}
```

---

## Table 2: NocoDB Interests → `client_contacts`

Each unique client may have up to 3 contacts extracted from the Interests table. Deduplicate across all interests belonging to the same client.

| NocoDB Interests Field | →   | New Table.Column             | Type (PG) | Transformation                                                                  |
| ---------------------- | --- | ---------------------------- | --------- | ------------------------------------------------------------------------------- |
| `Email Address`        | →   | `client_contacts.value`      | TEXT      | Trim, lowercase. Channel = `"email"`. Label = `"primary"`.                      |
| `Phone Number`         | →   | `client_contacts.value`      | TEXT      | Trim, normalize to E.164 if possible. Channel = `"phone"`. Label = `"primary"`. |
| `Address`              | →   | `client_contacts.value`      | TEXT      | Trim. Channel = `"other"`. Label = `"address"`.                                 |
| —                      | →   | `client_contacts.client_id`  | UUID      | From dedup map: NocoDB Interest → client UUID                                   |
| —                      | →   | `client_contacts.is_primary` | BOOLEAN   | `true` for first email, first phone. `false` for subsequent.                    |
| —                      | →   | `client_contacts.channel`    | TEXT      | Determined by source field (email/phone/other)                                  |

**Deduplication rules:**

- If the same email appears across multiple interests for the same client → create one contact record
- If different emails appear across interests for the same client → create multiple records, first is `is_primary=true`
- Skip empty/null values

---

## Table 3: NocoDB Interests → `interests`

One NocoDB Interest row = one new `interests` row (after client dedup, the interest still exists as a pipeline record).

| NocoDB Interests Field     | Type (NocoDB) | →   | New Table.Column               | Type (PG)   | Transformation                                                                                                                                           |
| -------------------------- | ------------- | --- | ------------------------------ | ----------- | -------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `Id`                       | number        | →   | `interests.metadata`           | JSONB       | Store as `{ "nocodb_id": value }` for reference                                                                                                          |
| —                          | —             | →   | `interests.client_id`          | UUID        | From client dedup map                                                                                                                                    |
| `Berths_id`                | number        | →   | `interests.berth_id`           | UUID        | From berth ID map (NocoDB numeric ID → new UUID). Nullable if `Berths_id` is null.                                                                       |
| `Sales Process Level`      | string        | →   | `interests.pipeline_stage`     | TEXT        | **See Pipeline Stage Mapping below**                                                                                                                     |
| `Lead Category`            | string        | →   | `interests.lead_category`      | TEXT        | Map: `"General"` → `"general_interest"`, `"Specific"` → `"specific_qualified"`, `"Hot"` → `"hot_lead"`.                                                  |
| `Source`                   | string        | →   | `interests.source`             | TEXT        | Same mapping as clients.source                                                                                                                           |
| `EOI Status`               | string        | →   | `interests.eoi_status`         | TEXT        | Map: `"Pending"` → `"waiting_for_signatures"`, `"Signed"` → `"signed"`, `"Expired"` → `"expired"`, `"Sent"` → `"waiting_for_signatures"`. NULL if empty. |
| `documensoID`              | string        | →   | `interests.documenso_id`       | TEXT        | Direct copy. Trim.                                                                                                                                       |
| `Contract Status`          | string        | →   | `interests.contract_status`    | TEXT        | Lowercase, snake_case conversion                                                                                                                         |
| `Deposit 10% Status`       | string        | →   | `interests.deposit_status`     | TEXT        | Lowercase, snake_case conversion                                                                                                                         |
| `Date Added`               | datetime      | →   | `interests.date_first_contact` | TIMESTAMPTZ | Parse date string → ISO 8601                                                                                                                             |
| `EOI Time Sent`            | datetime      | →   | `interests.date_eoi_sent`      | TIMESTAMPTZ | Parse date string → ISO 8601                                                                                                                             |
| `Time LOI Sent`            | datetime      | →   | `interests.date_eoi_signed`    | TIMESTAMPTZ | NocoDB uses "LOI" but this is actually EOI signed date. Parse date → ISO 8601.                                                                           |
| `Contract Sent Status`     | string        | →   | `interests.date_contract_sent` | TIMESTAMPTZ | If status indicates "Sent", use `Updated At` as approximate date. Otherwise NULL.                                                                        |
| `Extra Comments`           | string        | →   | `interests.notes`              | TEXT        | Direct copy, trim                                                                                                                                        |
| `Berth Number`             | string        | →   | —                              | —           | Cross-reference field; use for berth_id resolution if `Berths_id` is null                                                                                |
| `Request More Information` | boolean       | →   | `interests.metadata`           | JSONB       | Store in metadata: `{ "request_more_info": true/false }`                                                                                                 |
| `Request Form Sent`        | string        | →   | `interests.metadata`           | JSONB       | Store in metadata                                                                                                                                        |
| `Borth Info Sent Status`   | string        | →   | `interests.metadata`           | JSONB       | Store in metadata (note: field name has typo "Borth")                                                                                                    |
| `Created At`               | datetime      | →   | `interests.created_at`         | TIMESTAMPTZ | Direct copy                                                                                                                                              |
| —                          | —             | →   | `interests.port_id`            | UUID        | Constant: Port Nimara UUID                                                                                                                               |
| —                          | —             | →   | `interests.archived_at`        | TIMESTAMPTZ | NULL                                                                                                                                                     |
| —                          | —             | →   | `interests.reminder_enabled`   | BOOLEAN     | `false` (configure post-migration)                                                                                                                       |

### Pipeline Stage Mapping

| NocoDB `Sales Process Level`      | →   | New `pipeline_stage` | Notes                                  |
| --------------------------------- | --- | -------------------- | -------------------------------------- |
| `General Qualified Interest`      | →   | `open`               | Broadest inquiry stage                 |
| `Specific Qualified Interest`     | →   | `details_sent`       | Client interested in specific berth(s) |
| `EOI and NDA Sent`                | →   | `in_communication`   | Active document exchange               |
| `Signed EOI and NDA`              | →   | `signed_eoi_nda`     | EOI/NDA completed                      |
| `Made Reservation`                | →   | `deposit_10pct`      | Reservation ≈ deposit stage            |
| `Contract Negotiation`            | →   | `contract`           | Active contract discussion             |
| `Contract Negotiations Finalized` | →   | `contract`           | Same stage, later substatus            |
| `Contract Signed`                 | →   | `completed`          | Deal closed                            |
| NULL or empty                     | →   | `open`               | Default for missing data               |

**Important:** The NocoDB schema uses 8 stage names while the new schema uses 8 stage codes. "Contract Negotiation" and "Contract Negotiations Finalized" both map to `contract` — distinguish them via `date_contract_sent` (null = negotiation, not null = finalized).

---

## Table 4: NocoDB Interests → `interest_notes`

The `Extra Comments` field on each Interest becomes an `interest_notes` record if it contains content.

| Source                          | →   | New Table.Column             | Transformation                                |
| ------------------------------- | --- | ---------------------------- | --------------------------------------------- |
| `Extra Comments` (if non-empty) | →   | `interest_notes.content`     | Trim. Prefix with `"[Migrated from NocoDB] "` |
| —                               | →   | `interest_notes.author_id`   | System migration user ID                      |
| —                               | →   | `interest_notes.interest_id` | From interest UUID map                        |
| —                               | →   | `interest_notes.is_locked`   | `true` (historical, non-editable)             |
| `Created At`                    | →   | `interest_notes.created_at`  | From parent interest `Created At`             |

---

## Table 5: NocoDB Interests → `documents` + `document_signers`

Interests with EOI/contract document references create `documents` and `document_signers` records.

**Trigger:** Any interest where `EOI Document` is not null, or `LOI-NDA Document` is not null, or `documensoID` is not null.

| NocoDB Interests Field | →   | New Table.Column          | Transformation                                                                     |
| ---------------------- | --- | ------------------------- | ---------------------------------------------------------------------------------- |
| `EOI Document`         | →   | `documents.file_id`       | Match to MinIO file → `files` table entry                                          |
| `LOI-NDA Document`     | →   | `documents.file_id`       | Match to MinIO file → `files` table entry                                          |
| `documensoID`          | →   | `documents.documenso_id`  | Direct copy                                                                        |
| `EOI Status`           | →   | `documents.status`        | Map: `"Signed"` → `"completed"`, `"Pending"` → `"sent"`, `"Expired"` → `"expired"` |
| —                      | →   | `documents.document_type` | `"eoi"` for EOI documents, `"nda"` for LOI-NDA documents                           |
| —                      | →   | `documents.port_id`       | Port Nimara UUID                                                                   |
| —                      | →   | `documents.interest_id`   | From interest UUID map                                                             |
| —                      | →   | `documents.client_id`     | From client UUID map                                                               |
| —                      | →   | `documents.title`         | Generate: `"EOI - {client_name} - {berth_number}"`                                 |
| —                      | →   | `documents.created_by`    | System migration user ID                                                           |

**Signing links → `document_signers`:**

NocoDB stores up to 9 signing link fields per interest. These map to `document_signers` entries.

| NocoDB Field Pattern      | →   | `document_signers` Column | Transformation                                          |
| ------------------------- | --- | ------------------------- | ------------------------------------------------------- |
| Signing link fields (1-9) | →   | `signer_email`            | Extract email from Documenso signing URL if available   |
| —                         | →   | `signer_role`             | `"client"` (default for most), infer from context       |
| —                         | →   | `signing_order`           | 1-based index of the link field                         |
| —                         | →   | `status`                  | If EOI Status = "Signed" → `"signed"`, else `"pending"` |
| —                         | →   | `signing_url`             | Direct copy of Documenso signing link URL               |

---

## Table 6: NocoDB Berths → `berths`

One NocoDB Berth row = one new `berths` row. Berths have cleaner data than Interests.

| NocoDB Berths Field        | Type (NocoDB) | →   | New Table.Column                           | Type (PG)   | Transformation                                                                                                                                                                                   |
| -------------------------- | ------------- | --- | ------------------------------------------ | ----------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| `Id`                       | number        | →   | —                                          | —           | Store in berth ID map: NocoDB ID → new UUID                                                                                                                                                      |
| `Mooring Number`           | string        | →   | `berths.mooring_number`                    | TEXT        | Direct copy, trim                                                                                                                                                                                |
| `Area`                     | string (enum) | →   | `berths.area`                              | TEXT        | Direct copy. NocoDB enum A-E; new system supports A-G. Validate.                                                                                                                                 |
| `Status`                   | string (enum) | →   | `berths.status`                            | TEXT        | Map: `"Available"` → `"available"`, `"UnderOffer"` → `"under_offer"`, `"Sold"` → `"sold"`. Add missing: `"Reserved"` → `"under_offer"`, `"Not Available"` → `"sold"`, `"Draft"` → `"available"`. |
| `status_override_mode`     | boolean       | →   | —                                          | —           | Not migrated (new system handles status rules differently)                                                                                                                                       |
| `Nominal Boat Size`        | string        | →   | `berths.nominal_boat_size`                 | TEXT        | Direct copy (already in ft format)                                                                                                                                                               |
| `Nominal Boat Size_metric` | string        | →   | `berths.nominal_boat_size_m`               | TEXT        | Direct copy (m format)                                                                                                                                                                           |
| `Length`                   | number        | →   | `berths.length_ft`                         | NUMERIC     | Direct copy (NocoDB berths store as numbers, not strings)                                                                                                                                        |
| `Length_metric`            | number        | →   | `berths.length_m`                          | NUMERIC     | Direct copy. If missing, calculate: `length_ft / 3.28084`                                                                                                                                        |
| `Width`                    | number        | →   | `berths.width_ft`                          | NUMERIC     | Direct copy                                                                                                                                                                                      |
| `Width_metric`             | number        | →   | `berths.width_m`                           | NUMERIC     | Direct copy. If missing, calculate.                                                                                                                                                              |
| `Draft`                    | number        | →   | `berths.draft_ft`                          | NUMERIC     | Direct copy                                                                                                                                                                                      |
| `Draft_metric`             | number        | →   | `berths.draft_m`                           | NUMERIC     | Direct copy. If missing, calculate.                                                                                                                                                              |
| `Water Depth`              | number        | →   | `berths.water_depth`                       | NUMERIC     | Direct copy (ft)                                                                                                                                                                                 |
| `Water Depth_metric`       | number        | →   | `berths.water_depth_m`                     | NUMERIC     | Direct copy. If missing, calculate.                                                                                                                                                              |
| `Side Pontoon`             | string (enum) | →   | `berths.side_pontoon`                      | TEXT        | Lowercase: `"Port"` → `"port"`, `"Starboard"` → `"starboard"`, `"Both"` → `"both"`                                                                                                               |
| `Power Capacity`           | string        | →   | `berths.power_capacity`                    | TEXT        | Direct copy                                                                                                                                                                                      |
| `Voltage`                  | string        | →   | `berths.voltage`                           | TEXT        | Direct copy                                                                                                                                                                                      |
| `Mooring Type`             | string (enum) | →   | `berths.mooring_type`                      | TEXT        | Lowercase snake_case: `"Med Mooring"` → `"med_mooring"`, `"Alongside"` → `"alongside"`                                                                                                           |
| `Cleat Type/Capacity`      | string        | →   | `berths.cleat_type` + `cleat_capacity`     | TEXT        | Split on delimiter if combined. NocoDB may store as single field or separate enum.                                                                                                               |
| `Bollard Type/Capacity`    | string        | →   | `berths.bollard_type` + `bollard_capacity` | TEXT        | Same split logic                                                                                                                                                                                 |
| `Bow Facing`               | string        | →   | `berths.bow_facing`                        | TEXT        | Direct copy, lowercase                                                                                                                                                                           |
| `Access`                   | string (enum) | →   | `berths.access`                            | TEXT        | Lowercase: `"Walk On"` → `"walk_on"`, `"Dinghy"` → `"dinghy"`                                                                                                                                    |
| `Price`                    | number        | →   | `berths.price`                             | NUMERIC     | Direct copy                                                                                                                                                                                      |
| —                          | —             | →   | `berths.price_currency`                    | TEXT        | Default `"USD"`                                                                                                                                                                                  |
| —                          | —             | →   | `berths.tenure_type`                       | TEXT        | Default `"permanent"` (most Port Nimara berths are permanent tenure)                                                                                                                             |
| `Created At`               | datetime      | →   | `berths.created_at`                        | TIMESTAMPTZ | Direct copy                                                                                                                                                                                      |
| `Updated At`               | datetime      | →   | `berths.updated_at`                        | TIMESTAMPTZ | Direct copy                                                                                                                                                                                      |
| —                          | —             | →   | `berths.port_id`                           | UUID        | Constant: Port Nimara UUID                                                                                                                                                                       |
| —                          | —             | →   | `berths.berth_approved`                    | BOOLEAN     | `true` (all existing berths are approved)                                                                                                                                                        |
| `Interested Parties`       | link field    | →   | —                                          | —           | Not migrated directly; relationship is rebuilt via `interests.berth_id`                                                                                                                          |

### Berth Area Expansion

NocoDB BerthArea enum: `A | B | C | D | E`
New system supports: `A | B | C | D | E | F | G`

Berths in areas F and G exist in production but the NocoDB enum doesn't define them. The extraction script should treat area as a free-text field and not validate against the old enum.

### Berth Status Mapping

| NocoDB Status   | →   | New Status    | Notes                                        |
| --------------- | --- | ------------- | -------------------------------------------- |
| `Available`     | →   | `available`   |                                              |
| `UnderOffer`    | →   | `under_offer` |                                              |
| `Sold`          | →   | `sold`        |                                              |
| `Reserved`      | →   | `under_offer` | NocoDB may have this in data but not in enum |
| `Not Available` | →   | `sold`        | Treat as sold/occupied                       |
| `Draft`         | →   | `available`   | Draft berths become available                |
| NULL or empty   | →   | `available`   | Default                                      |

---

## Table 7: NocoDB Berths → `berth_map_data`

If the NocoDB berths table or a related data source contains SVG positioning data for the marina map, extract it.

| Source                    | →   | New Table.Column           | Transformation      |
| ------------------------- | --- | -------------------------- | ------------------- |
| SVG path data (if exists) | →   | `berth_map_data.svg_path`  | Direct copy         |
| X coordinate              | →   | `berth_map_data.x`         | Direct copy         |
| Y coordinate              | →   | `berth_map_data.y`         | Direct copy         |
| Transform data            | →   | `berth_map_data.transform` | Direct copy         |
| —                         | →   | `berth_map_data.berth_id`  | From berth UUID map |

**Note:** The current berth map may be rendered from a static SVG with hardcoded positions. If no per-berth coordinate data exists in NocoDB, the `berth_map_data` table will be populated manually post-migration or via the L4 berth spec import tool.

---

## Table 8: NocoDB Expenses → `expenses`

One NocoDB Expense row = one new `expenses` row.

| NocoDB Expenses Field | Type (NocoDB)      | →   | New Table.Column              | Type (PG)   | Transformation                                                                                                                |
| --------------------- | ------------------ | --- | ----------------------------- | ----------- | ----------------------------------------------------------------------------------------------------------------------------- |
| `Id`                  | number             | →   | —                             | —           | Store in expense ID map: NocoDB ID → new UUID                                                                                 |
| `Establishment Name`  | string             | →   | `expenses.establishment_name` | TEXT        | Trim                                                                                                                          |
| `Price`               | string             | →   | `expenses.amount`             | NUMERIC     | `parseFloat()` — NocoDB stores price as STRING. Strip currency symbols ($, €, £). Strip commas. Parse to number.              |
| `currency`            | string             | →   | `expenses.currency`           | TEXT        | Uppercase: `"usd"` → `"USD"`, `"eur"` → `"EUR"`. Default `"USD"` if missing.                                                  |
| —                     | —                  | →   | `expenses.amount_usd`         | NUMERIC     | If currency ≠ USD, convert using historical exchange rate for the expense date. Otherwise same as amount.                     |
| `Payment Method`      | string (enum)      | →   | `expenses.payment_method`     | TEXT        | Lowercase snake_case: `"Credit Card"` → `"credit_card"`, `"Cash"` → `"cash"`, `"Bank Transfer"` → `"bank_transfer"`           |
| `Category`            | string (enum)      | →   | `expenses.category`           | TEXT        | Lowercase snake_case: `"Fuel"` → `"fuel"`, `"Maintenance"` → `"maintenance"`, `"Office Supplies"` → `"office_supplies"`, etc. |
| `Payer`               | string             | →   | `expenses.payer`              | TEXT        | Trim                                                                                                                          |
| `Time`                | datetime           | →   | `expenses.expense_date`       | TIMESTAMPTZ | Parse date string → ISO 8601                                                                                                  |
| `Contents`            | string             | →   | `expenses.description`        | TEXT        | Trim                                                                                                                          |
| `Receipt`             | file attachment(s) | →   | `expenses.receipt_file_ids`   | UUID[]      | For each receipt file: create `files` table entry, store MinIO path. Collect UUIDs into array.                                |
| `Paid`                | boolean            | →   | `expenses.payment_status`     | TEXT        | `true` → `"paid"`, `false` → `"unpaid"`, NULL → `"unpaid"`                                                                    |
| `payment_date`        | date (optional)    | →   | `expenses.payment_date`       | DATE        | Direct copy if present                                                                                                        |
| `payment_reference`   | string (optional)  | →   | `expenses.payment_reference`  | TEXT        | Direct copy if present                                                                                                        |
| `payment_notes`       | string (optional)  | →   | `expenses.payment_notes`      | TEXT        | Direct copy if present                                                                                                        |
| `CreatedAt`           | datetime           | →   | `expenses.created_at`         | TIMESTAMPTZ | Direct copy. Note: NocoDB uses `CreatedAt` (camelCase), not `Created At` (spaced).                                            |
| `UpdatedAt`           | datetime           | →   | `expenses.updated_at`         | TIMESTAMPTZ | Direct copy                                                                                                                   |
| —                     | —                  | →   | `expenses.port_id`            | UUID        | Constant: Port Nimara UUID                                                                                                    |
| —                     | —                  | →   | `expenses.created_by`         | TEXT        | System migration user ID                                                                                                      |
| —                     | —                  | →   | `expenses.archived_at`        | TIMESTAMPTZ | NULL                                                                                                                          |

**Price parsing helper:**

```typescript
function parsePrice(value: string | null): number | null {
  if (!value || value.trim() === '') return null;
  // Strip currency symbols and thousand separators
  const cleaned = value.replace(/[$€£¥,\s]/g, '').trim();
  const num = parseFloat(cleaned);
  return isNaN(num) ? null : Math.round(num * 100) / 100;
}
```

**Expense category normalization:**

```typescript
const CATEGORY_MAP: Record<string, string> = {
  Fuel: 'fuel',
  Maintenance: 'maintenance',
  'Office Supplies': 'office_supplies',
  Utilities: 'utilities',
  'Marina Operations': 'marina_operations',
  Travel: 'travel',
  Equipment: 'equipment',
  Insurance: 'insurance',
  Legal: 'legal',
  Other: 'other',
};

function normalizeCategory(value: string | null): string {
  if (!value) return 'other';
  return CATEGORY_MAP[value.trim()] ?? value.toLowerCase().replace(/\s+/g, '_');
}
```

---

## Table 9: NocoDB Invoices → `invoices`

| NocoDB Invoices Field | Type (NocoDB) | →   | New Table.Column                                                                 | Type (PG)   | Transformation                                                                                                                 |
| --------------------- | ------------- | --- | -------------------------------------------------------------------------------- | ----------- | ------------------------------------------------------------------------------------------------------------------------------ |
| `Id`                  | number        | →   | —                                                                                | —           | Store in invoice ID map                                                                                                        |
| Invoice number        | string        | →   | `invoices.invoice_number`                                                        | TEXT        | Preserve existing numbering (format: `INV-YYYYMM-XYZ`). Verify uniqueness.                                                     |
| Client reference      | string/link   | →   | `invoices.client_name`                                                           | TEXT        | Resolve client name from interest/client data                                                                                  |
| Client email          | string        | →   | `invoices.billing_email`                                                         | TEXT        | From client contacts                                                                                                           |
| Client address        | string        | →   | `invoices.billing_address`                                                       | TEXT        | From client contacts                                                                                                           |
| Due date              | date          | →   | `invoices.due_date`                                                              | DATE        | Direct copy                                                                                                                    |
| Currency              | string        | →   | `invoices.currency`                                                              | TEXT        | Uppercase. Default `"USD"`                                                                                                     |
| Subtotal              | number        | →   | `invoices.subtotal`                                                              | NUMERIC     | Direct copy or calculate from line items                                                                                       |
| Discount              | number        | →   | `invoices.discount_pct` + `discount_amount`                                      | NUMERIC     | Parse from invoice data                                                                                                        |
| Fee                   | number        | →   | `invoices.fee_pct` + `fee_amount`                                                | NUMERIC     | Parse from invoice data                                                                                                        |
| Total                 | number        | →   | `invoices.total`                                                                 | NUMERIC     | Direct copy or calculate: `subtotal - discount + fee`                                                                          |
| Status                | string        | →   | `invoices.status`                                                                | TEXT        | Map: `"Draft"` → `"draft"`, `"Sent"` → `"sent"`, `"Paid"` → `"paid"`, `"Overdue"` → `"overdue"`, `"Cancelled"` → `"cancelled"` |
| Payment info          | object        | →   | `invoices.payment_status`, `payment_date`, `payment_method`, `payment_reference` | mixed       | Extract from payment fields                                                                                                    |
| PDF reference         | file          | →   | `invoices.pdf_file_id`                                                           | UUID        | Create `files` table entry for stored PDF                                                                                      |
| Notes                 | string        | →   | `invoices.notes`                                                                 | TEXT        | Trim                                                                                                                           |
| Created at            | datetime      | →   | `invoices.created_at`                                                            | TIMESTAMPTZ | Direct copy                                                                                                                    |
| —                     | —             | →   | `invoices.port_id`                                                               | UUID        | Constant: Port Nimara UUID                                                                                                     |
| —                     | —             | →   | `invoices.created_by`                                                            | TEXT        | System migration user ID                                                                                                       |
| —                     | —             | →   | `invoices.payment_terms`                                                         | TEXT        | Default `"net30"` (can be refined post-migration)                                                                              |

### Invoice → Line Items

NocoDB invoices may store line item data inline or as linked records. Each expense linked to an invoice becomes an `invoice_line_items` entry.

| Source              | →   | New Table.Column                 | Transformation                                         |
| ------------------- | --- | -------------------------------- | ------------------------------------------------------ |
| Expense description | →   | `invoice_line_items.description` | From linked expense `Contents` / `Establishment Name`  |
| Quantity            | →   | `invoice_line_items.quantity`    | Default `1` (NocoDB doesn't track quantity separately) |
| Amount              | →   | `invoice_line_items.unit_price`  | From linked expense `Price`                            |
| —                   | →   | `invoice_line_items.total`       | `quantity × unit_price`                                |
| —                   | →   | `invoice_line_items.sort_order`  | Auto-increment from 0                                  |

### Invoice ↔ Expense Junction

For each expense linked to an invoice in NocoDB, create an `invoice_expenses` junction record:

| Source            | →   | New Table                     | Transformation        |
| ----------------- | --- | ----------------------------- | --------------------- |
| Invoice NocoDB ID | →   | `invoice_expenses.invoice_id` | From invoice UUID map |
| Expense NocoDB ID | →   | `invoice_expenses.expense_id` | From expense UUID map |

---

## Table 10: MinIO Files → `files`

Scan MinIO storage and create `files` table entries. Map files to clients where possible.

| MinIO Metadata     | →   | New Table.Column       | Transformation                                                                                                                        |
| ------------------ | --- | ---------------------- | ------------------------------------------------------------------------------------------------------------------------------------- |
| Object key (path)  | →   | `files.storage_path`   | Direct copy (current path). Will be reorganized post-load.                                                                            |
| —                  | →   | `files.storage_bucket` | `"crm-files"`                                                                                                                         |
| Filename from path | →   | `files.filename`       | Extract from object key                                                                                                               |
| Original filename  | →   | `files.original_name`  | Same as filename (original names may not be preserved in MinIO)                                                                       |
| Content-Type       | →   | `files.mime_type`      | From MinIO object metadata                                                                                                            |
| Content-Length     | →   | `files.size_bytes`     | From MinIO object metadata                                                                                                            |
| Client association | →   | `files.client_id`      | AI-assisted matching: filename patterns, folder structure, PDF text extraction. Log unmatched to CSV.                                 |
| File category      | →   | `files.category`       | Infer from filename/path: `*eoi*` → `"eoi"`, `*contract*` → `"contract"`, `*receipt*` → `"receipt"`, `*nda*` → `"eoi"`, else `"misc"` |
| —                  | →   | `files.port_id`        | Port Nimara UUID                                                                                                                      |
| —                  | →   | `files.uploaded_by`    | System migration user ID                                                                                                              |

**New MinIO path structure (post-migration reorganization):**

```
{portSlug}/clients/{clientId}/eoi/filename.pdf
{portSlug}/clients/{clientId}/contracts/filename.pdf
{portSlug}/clients/{clientId}/receipts/filename.jpg
{portSlug}/clients/{clientId}/correspondence/filename.pdf
{portSlug}/clients/{clientId}/misc/filename.pdf
{portSlug}/invoices/{invoiceId}/invoice.pdf
{portSlug}/expenses/{expenseId}/receipt.jpg
```

---

## Data Quality Issues & Fixes

### Issue 1: String Dimensions on Interests

**Problem:** Interest table stores vessel dimensions (Length, Width, Depth) as strings. Values may include unit suffixes ("45ft", "12m"), be empty strings, or contain non-numeric characters.

**Fix:** `parseDimension()` helper (defined above) handles all cases. Log any unparseable values to `migration-warnings.csv`.

### Issue 2: Mixed Naming Conventions

**Problem:** NocoDB fields use inconsistent naming: `"Full Name"` (Title Spaced), `"documensoID"` (camelCase), `"status_override_mode"` (snake_case), `CreatedAt` (PascalCase on Expenses vs `Created At` on Interests).

**Fix:** Access each field by its exact NocoDB name. The extraction layer handles the naming; transformation maps to consistent snake_case.

### Issue 3: LOI → EOI Terminology

**Problem:** The codebase has remnants of "LOI" (Letter of Intent) terminology that was renamed to "EOI" (Expression of Interest). Fields like `LOI-NDA Document` and `Time LOI Sent` actually refer to EOI documents.

**Fix:** Map all LOI references to EOI in the new schema. `LOI-NDA Document` → `documents` with `document_type = "eoi"`. `Time LOI Sent` → `interests.date_eoi_signed`.

### Issue 4: Incomplete Enums

**Problem:** BerthStatus enum (`Available | UnderOffer | Sold`) is missing `Reserved`, `Not Available`, `Draft` which may exist in production data. BerthArea enum covers A-E but production has F and G.

**Fix:** Status mapping table handles all known values (see Table 6). Area is treated as free text. Any unrecognized values are logged and defaulted.

### Issue 5: Price as String on Expenses

**Problem:** Expense `Price` field is typed as `string` in the NocoDB interface. Values may contain currency symbols, commas, or be formatted inconsistently.

**Fix:** `parsePrice()` helper strips non-numeric characters and parses to NUMERIC. NULL values logged.

### Issue 6: Berth References

**Problem:** Interests reference berths via both `Berths_id` (numeric link field) and `Berth Number` (string display field). These may be inconsistent.

**Fix:** Primary resolution via `Berths_id` → berth UUID map. Fallback: match `Berth Number` string against `berths.mooring_number`. If both fail, set `berth_id = NULL` and log.

### Issue 7: Invoice Number Collision Risk

**Problem:** The current `generateInvoiceNumber()` function uses format `INV-YYYYMM-XYZ` with a random 3-char suffix, which has collision risk.

**Fix:** During migration, preserve all existing invoice numbers exactly. Post-migration, the new system uses a sequential counter per port per month, eliminating collision risk.

### Issue 8: Duplicate Clients

**Problem:** The same person may appear as multiple NocoDB Interest rows with slightly different name spellings or different email addresses.

**Fix:** Three-pass dedup:

1. Exact email match (case-insensitive) → same client
2. Fuzzy name match (Levenshtein distance ≤ 2 on normalized name) + same residence → candidate merge (log for manual review)
3. Remaining → create as separate clients (can be merged manually in new system)

All fuzzy-match candidates logged to `dedup-candidates.csv` for Matt's review before committing.

---

## Synthetic Records (Created During Migration)

These records don't exist in NocoDB but are generated to provide a complete dataset in the new system:

| Record Type               | Source                   | Logic                                                                                                                                                                                                                            |
| ------------------------- | ------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `audit_logs` (historical) | All migrated records     | For each migrated entity, create a synthetic `action: "create"` audit log entry with `created_at` matching the original record's creation date, `user_id` = system migration user, `metadata = { "source": "nocodb_migration" }` |
| `tags`                    | Lead categories, sources | Create tags for common categories: "Hot Lead", "Website Lead", "Referral", "Broker", etc.                                                                                                                                        |
| `client_tags`             | Interest `Lead Category` | Auto-tag clients based on their highest lead category across all interests                                                                                                                                                       |
| `interest_tags`           | Interest `Source`        | Auto-tag interests by source                                                                                                                                                                                                     |
| `client_notes`            | Dedup merge info         | When two interests are merged into one client, create a note documenting the merge                                                                                                                                               |

---

## ID Mapping Tables (In-Memory During Migration)

The migration script maintains these maps for cross-referencing:

```typescript
interface MigrationMaps {
  // NocoDB numeric ID → new PostgreSQL UUID
  clientMap: Map<string, string>; // emailOrNameKey → client UUID
  berthMap: Map<number, string>; // NocoDB Berth Id → berth UUID
  interestMap: Map<number, string>; // NocoDB Interest Id → interest UUID
  expenseMap: Map<number, string>; // NocoDB Expense Id → expense UUID
  invoiceMap: Map<number, string>; // NocoDB Invoice Id → invoice UUID

  // Reverse lookups
  interestToClient: Map<number, string>; // NocoDB Interest Id → client UUID
  berthNumberToId: Map<string, string>; // Mooring number string → berth UUID
}
```

---

## Validation Checklist

After migration completes, run these validation checks:

| Check                 | Query                                                                                                   | Expected                                      |
| --------------------- | ------------------------------------------------------------------------------------------------------- | --------------------------------------------- |
| Client count          | `SELECT COUNT(*) FROM clients`                                                                          | ≤ NocoDB interest count (dedup reduces)       |
| Contact count         | `SELECT COUNT(*) FROM client_contacts`                                                                  | ≥ client count (most clients have ≥1 contact) |
| Interest count        | `SELECT COUNT(*) FROM interests`                                                                        | = NocoDB interest count (1:1)                 |
| Berth count           | `SELECT COUNT(*) FROM berths`                                                                           | = NocoDB berth count (1:1)                    |
| Expense count         | `SELECT COUNT(*) FROM expenses`                                                                         | = NocoDB expense count (1:1)                  |
| Invoice count         | `SELECT COUNT(*) FROM invoices`                                                                         | = NocoDB invoice count (1:1)                  |
| Orphan interests      | `SELECT COUNT(*) FROM interests WHERE client_id NOT IN (SELECT id FROM clients)`                        | 0                                             |
| Orphan contacts       | `SELECT COUNT(*) FROM client_contacts WHERE client_id NOT IN (SELECT id FROM clients)`                  | 0                                             |
| NULL berth refs       | `SELECT COUNT(*) FROM interests WHERE berth_id IS NOT NULL AND berth_id NOT IN (SELECT id FROM berths)` | 0                                             |
| Invoice-expense links | `SELECT COUNT(*) FROM invoice_expenses WHERE expense_id NOT IN (SELECT id FROM expenses)`               | 0                                             |
| File records          | `SELECT COUNT(*) FROM files`                                                                            | ≥ (receipt files + EOI docs + contract docs)  |
| Pipeline distribution | `SELECT pipeline_stage, COUNT(*) FROM interests GROUP BY 1`                                             | Reasonable distribution, no unexpected NULLs  |
| Duplicate emails      | `SELECT value, COUNT(*) FROM client_contacts WHERE channel='email' GROUP BY 1 HAVING COUNT(*) > 1`      | Review each — may be valid (shared email)     |
| Amount sanity         | `SELECT COUNT(*) FROM expenses WHERE amount <= 0`                                                       | 0 (or review outliers)                        |

---

## Migration Script Configuration

```typescript
// scripts/migrate-nocodb/config.ts
export const MIGRATION_CONFIG = {
  // NocoDB connection
  nocodb: {
    apiUrl: process.env.NOCODB_API_URL, // e.g., 'https://nocodb.portnimara.com'
    token: process.env.NOCODB_AUTH_TOKEN,
    projectId: process.env.NOCODB_PROJECT_ID,
    tables: {
      interests: 'mbs9hjauug4eseo',
      berths: 'mczgos9hr3oa9qc',
      expenses: 'mxfcefkk4dqs6uq',
      invoices: 'mvyvz0lpc30p01s',
    },
  },

  // Target PostgreSQL
  postgres: {
    connectionString: process.env.DATABASE_URL,
  },

  // MinIO
  minio: {
    endpoint: process.env.MINIO_ENDPOINT,
    accessKey: process.env.MINIO_ACCESS_KEY,
    secretKey: process.env.MINIO_SECRET_KEY,
    bucket: 'crm-files',
  },

  // Migration defaults
  defaults: {
    portId: process.env.PORT_NIMARA_UUID, // seeded in L0
    portSlug: 'port-nimara',
    migrationUserId: 'system-migration',
    defaultCurrency: 'USD',
    defaultTimezone: 'America/Anguilla',
  },

  // Deduplication thresholds
  dedup: {
    nameLevenshteinThreshold: 2, // max edit distance for fuzzy name match
    requireResidenceMatch: true, // secondary match requires same residence
    logCandidates: true, // write fuzzy candidates to CSV
  },

  // Output files
  output: {
    warningsFile: 'migration-warnings.csv', // unparseable values, unexpected data
    dedupCandidatesFile: 'dedup-candidates.csv', // fuzzy match candidates for review
    unmatchedFilesFile: 'unmatched-files.csv', // MinIO files that couldn't be linked to a client
    validationReport: 'validation-report.json', // final counts and check results
  },
};
```

---

## Post-Migration Manual Tasks

| Task                                                                            | Owner | When                      |
| ------------------------------------------------------------------------------- | ----- | ------------------------- |
| Review `dedup-candidates.csv` — approve or reject client merges                 | Matt  | Saturday morning          |
| Review `unmatched-files.csv` — manually assign files to clients or mark as misc | Matt  | Saturday morning          |
| Review `migration-warnings.csv` — fix any data issues                           | Matt  | Saturday morning          |
| Spot-check 5 high-profile clients — verify all data migrated correctly          | Matt  | Saturday after validation |
| Spot-check 5 berths — verify dimensions, status, map data                       | Matt  | Saturday after validation |
| Spot-check 3 invoices — verify amounts, line items, linked expenses             | Matt  | Saturday after validation |
| Create user accounts and assign roles in new system                             | Matt  | Saturday afternoon        |
| Configure SMTP (Poste.io) connection                                            | Matt  | Saturday afternoon        |
| Test Documenso API connection                                                   | Matt  | Saturday afternoon        |
| Populate `berth_map_data` if not extracted from NocoDB                          | Matt  | Post-migration week       |
