# Launch Readiness — Pre-Prod Initiative

> **Scope:** the user enumerated five launch-blocking initiatives on
> 2026-05-27. This doc is the single home for all of them so we can
> track progress without losing items between sessions. Companion to
> `docs/superpowers/audits/active-uat.md` (which keeps the live UAT
> findings) and `docs/BACKLOG.md` (master backlog index).
>
> Status tags per item: `OPEN | IN PROGRESS | SHIPPED in <hash> | BLOCKED | DEFERRED`.

## Initiative 1 — Reports overhaul

**Status:** IN PROGRESS · Active phase

Goals (per user, 2026-05-27):

- Cover all four report categories: **Sales performance**, **Financial**,
  **Marketing / funnel**, **Operational**.
- Template system: load template → modify → re-save OR save as new.
- Rich data density: more charts, more graphs, more KPIs.
- Output formats: **PDF + CSV + Excel** for each report.
- Scheduled reports: cron-driven; auto-email is **optional** (so the
  admin can schedule a run without forcing an email blast).
- Custom builder: full ad-hoc (pick entity, columns, filters, group-by),
  save as template — but quality-first; we don't ship a janky composer.
- UI/UX: stunning, fluid, beautiful. Within the existing white/navy
  brand language — no off-brand experimental themes.

Decisions locked (2026-05-27):

- **Currency**: port branding default
- **Rep visibility**: port-scoped admin setting (default depends on
  team size; PN is single-rep so default = full team)
- **AR aging buckets**: standard 30-day (current / 1-30 / 31-60 /
  61-90 / 90+)
- **Custom builder entity scope**: all 10 entities
- **Pulse data**: fold into Sales report
- **Inquiry-link audit**: yes, audit + fix; no website-repo edits
  required for the audit itself (link logic is server-side)
- **Scope cut for launch**: Sales + Operational ship first as
  fully-functional reports; Marketing + Financial ship in tandem with
  their data sources being wired (see Initiatives 2c + 2d below).

Phases (status snapshot 2026-05-27):

1. ✅ Foundation + UX overhaul — landing page (within existing
   design system); charts library audit done; ExcelJS installed.
2. ✅ Sales Performance + Operational builders — full report pages
   with KPIs / charts / tables; client-side Export to CSV + Excel +
   PDF; server-side PDF endpoint for branded output. _See gaps
   below._
3. ❌ Marketing report — NOT BUILT. Pending Init 1b cutover.
   **Beta gate (2026-06-02):** the `marketing` kind in
   `reports/[kind]/page.tsx` now returns `notFound()` (via
   `UNAVAILABLE_NEW_KINDS`) instead of the "in development" placeholder,
   so the beta reports surface reads as complete — the landing page only
   advertises Sales / Operational / Financial / Custom, and the
   hand-typed `/reports/marketing` URL 404s. **Remove the
   `UNAVAILABLE_NEW_KINDS` entry when this report ships.** Decision: keep
   the reports page live for beta rather than hiding it behind a module
   toggle — 3 of 4 reports are fully built + verified (export, templates,
   scheduling) and strictly beat the dashboard-only fallback.
4. ✅ Financial report — **SHIPPED in b690fb8d.** Built on the canonical
   payments + expenses tables (invoices module stays OFF); the
   invoice-centric spec was reframed onto the payments model
   ("outstanding AR" → expected-deposit shortfall; "AR aging" →
   outstanding deposits by deal age). 7 KPIs, 6 charts, 4 tables, port-
   currency normalised, 1y default range, templates + export. Marketing
   is the only remaining unbuilt report.
5. ⚠️ Custom (ad-hoc) report builder — partial ship.
6. ✅ Scheduled reports with optional emailing — BullMQ poll +
   render path live; recipients optional; PDF-only output.
7. ✅ Templates — load / modify / save / save-as / URL deep-link.

Open considerations carried forward:

- **Chart library mix.** Project already has `recharts` (simple bar/line/pie)
  and `echarts` (heatmaps, funnels, complex). Lean on each where it fits;
  don't add a third unless something specific is missing.
- **PDF cover-page treatment.** Each report PDF should open with a
  branded cover (port logo, title, date range, generated-on stamp). Reuse
  the existing `branded-document.tsx` shell.

Working spec: `docs/reports-content-spec.md` (per-category KPIs +
charts + tables proposed; updated as we walk through each).

### Reports — what's left (gap audit 2026-05-27)

Comparing the working spec against shipped code, here's the bucketed
backlog. **Items marked LAUNCH-BLOCK** are needed for the beta cutover;
everything else is post-launch polish unless promoted.

#### Cross-cutting capabilities (apply to every report)

- ⚠️ **Period comparison toggle** — "this period vs prior period" delta
  arrows on KPI cards. **Sales: SHIPPED locally (2026-05-31)** — a
  "Compare to prior period" toggle in the header computes an
  equal-length preceding window (`previousPeriodBounds`), the API
  recomputes KPIs for that window behind `?compare=1`, and the five
  window-derived tiles (Won, Lost, Win rate, Avg time-to-close, New
  leads) render colour-correct "vs prior" deltas. Point-in-time tiles
  (Active interests, Pipeline value) intentionally have no delta.
  Persisted in the saved-template config. TDD'd:
  `previousPeriodBounds` + `computeSalesKpiComparison` unit tests.
  Operational already rendered period-start deltas. **Still open:** the
  spec's "on every report" — Operational uses a different
  "vs period start" baseline; reconcile the two semantics if a single
  consistent comparison is wanted.
- ✅ **Rep multi-select filter** — **SHIPPED in b97f6e94** (Sales).
  Dynamic "Assigned to" multi-select populated from a window-independent
  `getRepFilterOptions` (distinct assigned reps port-wide); hidden when
  the port has no assigned interests.
- ✅ **Source multi-select filter** — **SHIPPED in b97f6e94** (Sales).
  Static Source multi-select (website / manual / referral / broker /
  other) allowlisted against `SOURCES`. Both filters thread through the 5
  filtered Sales queries via a pure, unit-tested `parseSalesFilters`.
  _Still open: replicate both on Operational + the other report pages._
- ✅ **Empty-state copy per report** — **SHIPPED (2026-06-02).** A
  window-independent `hasData` flag on the Sales / Operational /
  Financial routes drives a shared `<ReportEmptyState>` hero (named icon
  - one-line body + onboarding action button) when the port has no
    underlying data at all — distinct from the per-chart "no data in this
    window" states, which already degraded gracefully. Targets: Sales →
    Interests, Operational → Berths, Financial → Expenses. Spec:
    `docs/superpowers/specs/2026-06-02-reports-polish-design.md`.

#### Phase 2 — Sales report gaps

- ✅ **Operational-style filter set on Sales** — stage / lead-cat /
  outcome + period comparison + rep multi-select + source multi-select
  all shipped (rep/source in b97f6e94). Sales filter set is complete.

#### Phase 2 — Operational report gaps

- ⚠️ **Operational-specific filters**: **Area SHIPPED (2026-06-02)** —
  a berth-area scope (`parseOperationalFilters` +
  `getOperationalAreaOptions`, threaded through the 5 berth-derived
  service fns) re-queries the berth-count KPIs, occupancy-by-area,
  utilisation heatmap, and vacant lists for the selected areas; trend +
  tenancy/signing/docs panels stay port-wide with a "scoped to {areas}"
  caption. Browser-verified (area A: total berths 117→11). **Status /
  tenure type / document type deferred** — Status proved a light filter
  here (can't retro-apply to historical trend charts; the vacant lists
  are available-by-definition); see
  `docs/superpowers/specs/2026-06-02-reports-polish-design.md`.

#### Phase 3 — Marketing report (LAUNCH-BLOCK if Marketing is in beta scope)

Not built. Spec at `docs/reports-content-spec.md` § Report 03 calls for:

- 6 KPIs (inquiries, inquiry→interest %, inquiry→EOI %, inquiry→won %,
  top source, avg time-to-respond)
- 6 charts (inquiries by source donut, source ROI stacked bar, full
  funnel, conversion trend, country geo map via `react-simple-maps`,
  time-to-respond histogram)
- 3 tables (top-converting sources, recent inquiries, stuck inquiries)
- Filters: specific source, mooring, UTM campaign

**Blocker:** depends on the website actually sending UTM params (Init
1b step 4 — CRM-side shipped, website-side pending) AND on inquiry
data flowing from the new intake endpoint (Init 1b step 1 — pending
website env flip).

#### Phase 4 — Financial report ✅ SHIPPED in b690fb8d

**Decision taken (2026-06-02):** ship on the canonical `payments` +
`expenses` tables; invoices module stays OFF. The invoice-centric spec
(§ Report 02) was reframed onto the payments model so the report is
populated rather than 90% empty:

- 7 KPIs: revenue collected (net of refunds), deposits, balance,
  pipeline (expected deposits), outstanding deposits (expected−collected
  on open deals = the AR analogue), expenses, net contribution.
- 6 charts: revenue by month (deposit/balance, with month/quarter/year
  toggle), collection funnel (EOI → deposit → contract → won),
  outstanding deposits by deal age (AR-aging analogue, no invoice due
  dates exist), cash flow (inflow vs outflow), expense breakdown donut.
- 4 tables: outstanding deposits, recent payments, refund/write-off log,
  expense ledger.
- All money normalised to port currency; 1y default range; templates +
  CSV/XLSX/PDF export.

**Follow-up (deferred, not launch-blocking):** if the user later flips
the invoices module ON, add invoice-sourced AR (due dates → true aging)

- the invoice/payment-status/billing-entity filters from the original
  spec. Browser-verified against live data (0 payment rows in dev → revenue
  $0 correct; 165 expenses populate the expense surfaces).

#### Phase 5 — Custom builder gaps

v1 ships 4 entities; full spec wants 10 + advanced composition.

- ❌ **Missing entities**: yachts, companies, invoices, expenses,
  documents, websiteSubmissions, payments. Each is a registry-only
  extension — add a `CustomEntityDefinition` to
  `src/lib/reports/custom/registry.ts`. ~30 min per entity.
- ❌ **Filters beyond date range** — spec wants per-column filter rows
  (column → operator → value, AND/OR between rows). Today only the
  date range filter exists.
- ❌ **Group by + aggregate** — single group-by dimension + per-column
  aggregate (count / sum / avg / min / max). Today only a flat list.
- ❌ **Column sort** — pick a column + direction. Today rows return
  with the registry's hardcoded `orderBy`.
- ❌ **Live preview as you build** — spec wants debounced re-render on
  filter / column change. Today the rep clicks "Run query" to fetch.
- ❌ **Column whitelist per role** — PII columns (`email`, `phone`)
  should be gated by `clients.view_pii`. Today all listed columns are
  available to anyone with `reports.export`.
- ❌ **Run-once vs Save-as-template** — the spec asks for three buttons
  on save (Run once / Save as template / Update existing). Today only
  the template-save path exists.

#### Phase 6 — Scheduled runs gaps

- ❌ **Custom cron strings** — three hardcoded cadences (weekly Mon 9 ·
  monthly 1st 9 · quarterly 1st 9). Spec implies arbitrary cron.
  `nextRunFor` in `report-schedules.service.ts` switches on the enum;
  extend to support a `cron_expression` mode.
- ❌ **Scheduled CSV / XLSX** — only PDF is wired through the worker
  (`renderStandaloneReportRun` in `report-render.service.ts`). For
  CSV/XLSX, the worker would need to either run the existing client-side
  exporter server-side (drop ExcelJS into the worker bundle) or build
  format-specific server renderers.

#### Phase 7 — Templates gaps

- ❌ **"Modified ●" indicator** — when the rep changes view state after
  loading a template, the active-template badge currently just clears.
  Spec wants a visible "modified" marker so they know they've drifted.
- ❌ **Personal vs port-wide scope** — schema has the `visibility`
  column with `'private' | 'team'` but the UI always saves as port-wide.
  The Save dialog needs a scope picker.
- ❌ **"Owned by" attribution** — templates with `visibility='team'`
  should show creator name. Schema captures `createdBy`; UI doesn't
  surface it.
- ❌ **Promote-to-port-wide affordance** — once shipped, a "Share with
  team" action on personal templates that flips visibility.

#### Net launch-readiness for reports

If the launch scope is **Sales + Operational only**, reports are
launch-ready with the polish items above as post-launch follow-ups.

If the launch scope includes **Marketing + Financial**, both reports
need to be built AND their data plumbing finished (Init 1b website
flip + UTM forwarding for Marketing; invoices module + rep training
for Financial).

The cross-cutting filter set (period comparison, rep / source
multi-select, empty-state copy) is the highest-value polish that's
visible on every report — call it ~6-8 hours of work spread across
both shipped report pages + the shared FilterBar component.

---

## Initiative 1b — Marketing data pipeline cutover

**Status:** OPEN · Blocks the Marketing report

The CRM has the **full infrastructure** for marketing intake +
attribution; it's just not connected end-to-end.

What's built:

- **Email-open pixel tracking**: `src/app/api/public/email-pixel/[sendId]/route.ts`
  - `src/lib/email/tracking-pixel.ts`. Sales sends with
    `trackOpens=true` get a 1×1 pixel; opens record to
    `email_send_opens` and cross-post to Umami.
- **Umami integration**: `@umami/node` installed; `src/lib/services/umami.service.ts`
  is the wrapper. Outcome events (EOI sent, deposit received, etc.)
  already cross-post into Umami.
- **Website inquiry intake endpoint**: `/api/public/website-inquiries`
  in the CRM, paired with `/api/public/residential-inquiries`. Both
  validate + dual-write into `website_submissions`.
- **Website posting code**: `Port Nimara/Website/server/utils/crmIntake.ts:72`
  has the matching POST. Just needs the env var to point at the new
  CRM.

What's NOT connected yet:

1. **Website env `CRM_INTAKE_URL`** still points at the old portal (or
   isn't set). Flipping this is a ~5-min config change inside the
   website Nuxt deploy. After flip, every website inquiry lands in
   `website_submissions` + auto-routes to the inquiry-triage queue.
2. **Backfill of historical inquiries** from the old portal so the
   Marketing report has launch-day history rather than starting from
   zero. Reads from `client_portal_v2`'s inquiry table, inserts into
   `website_submissions` with original `receivedAt` timestamps,
   re-links to existing CRM clients via dedup (email/phone).
3. **Umami funnel events on the marketing site itself**. The Umami
   project exists; what's unclear is whether the marketing site is
   firing `event:` calls on key actions (form submitted, brochure
   downloaded, virtual-tour started). Audit needed.
4. **UTM column wiring**. ✅ CRM-side SHIPPED — migration `0089_website_submissions_utm.sql`
   adds `utm_source / utm_medium / utm_campaign / utm_term / utm_content`
   to `website_submissions` plus a `(port_id, utm_source, received_at)`
   composite index for per-campaign rollups. `/api/public/website-inquiries`
   accepts the five fields in the request body and persists them on
   insert. **Pending website-side change**: the marketing site's
   `crmIntake.ts` POST must forward UTM params from the form's query
   string / cookies. **Pending residential parity**: residential
   inquiries (`/api/public/residential-inquiries`) don't go through
   `website_submissions`; if Marketing report needs UTM attribution on
   residential leads too, add the same columns to `residential_clients`
   in a follow-up.

Sequencing:

- Step 1 is the cutover unblock (do during launch window itself).
- Step 2 is part of Initiative 5 (data migration).
- Step 3 is a website-side audit (Initiative 3).
- Step 4 is a small CRM-side schema add (one migration + 4 column
  reads). Decision pending: ship at launch or defer to Phase 2.

---

## Initiative 1c — Invoicing audit-and-finish

**Status:** SPIKE COMPLETE · Module-toggle shipped · Financial report deferred

### Audit findings (2026-05-27 spike)

The CRM has two parallel money-receiving flows in active code:

1. **`payments` table — canonical, in active use.** Schema comment at
   `src/lib/db/schema/pipeline.ts:75` is unambiguous: "The CRM does
   NOT generate invoices — clients pay banks directly. We record that
   money was received." Linked to `interests`. `recordPayment`
   auto-advances pipeline to `deposit_paid` when the cumulative
   deposit total hits `depositExpectedAmount`. This is the surface
   reps actually use; payments are recorded from the per-interest
   **Payments** tab.
2. **`invoices` + `invoice_line_items` table — orphaned in the UI.**
   Full builder (line items, PDF, send, mark-paid) exists at
   `/[portSlug]/invoices/new`. The sidebar nav entry was removed
   earlier; only the page itself can link to `invoices/new`. Dev DB
   has zero rows. The standalone surface is parallel infrastructure
   for the rare case where an operator wants to invoice a client
   directly from the CRM, plus the employee-expense-report flow
   (`expenses → invoices` PDF).

### Decision (per the existing "intentionally manual elsewhere" branch)

Ship a port-level module toggle, default OFF, identical pattern to
the Tenancies and Expenses toggles. The Financial report stays
deferred from launch since the canonical Payments tab feeds the
Sales report (which is shipping) — separate Financial dashboard adds
no value when there's no second money-receiving flow.

**What shipped (2026-05-27):**

- `system_settings` registry entry `invoices_module_enabled` (boolean,
  port-scoped, default `false`) — added to
  `src/lib/settings/registry.ts`.
- New module-gate service `src/lib/services/invoices-module.service.ts`
  with `isInvoicesModuleEnabled(portId)` (same shape as
  `isExpensesModuleEnabled`).
- Layout-level guard at `src/app/(dashboard)/[portSlug]/invoices/layout.tsx`
  — every `/invoices/*` route renders `<ModuleDisabledPage>` when the
  port hasn't opted in. Admins can flip on from Admin → Settings;
  historical rows preserved.

**What's NOT changed:**

- API endpoints (`/api/v1/invoices/*`) still respond — historical PDF
  links + send-flow webhooks keep resolving regardless of the toggle.
- The `payments` flow is untouched and continues to be the canonical
  money-received path.
- The expense → invoice flow (employee expense reports) is
  unaffected since employee-expense PDFs flow through a different
  surface (`/expenses`) that lives behind its own module gate.

**Follow-up:** if the user later wants per-port branded
client-facing invoicing from inside the CRM, the surface is ready to
turn on with no schema work — just flip `invoices_module_enabled = true`.

---

## Initiative 2 — Multi-agent codebase audit

**Status:** OPEN · Awaiting kickoff

User ask: "deep, multi-agent audit of all routes, naming, text, UX, and
… dig through the entire code of everything in the system (especially
related to the sales process) and find any issues in the logic or how
the functionality interacts with each other, how data is shared and
persists where needed. Also a deep security audit."

Audit dimensions (use one specialised agent per dimension, in parallel):

| #   | Dimension                     | Specialised agent                        | Output                                                                                                                                                                               |
| --- | ----------------------------- | ---------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| 1   | **Sales pipeline logic**      | `feature-dev:code-explorer`              | Trace every stage transition; verify auto-advance rules, EOI gating, deposit handling, contract signing. Look for stale enum references (the 9→7 stage migration left some bugs).    |
| 2   | **Cross-entity data flow**    | `feature-dev:code-explorer`              | Map polymorphic ownership (yacht/company), interest_berths (multi-berth), document folders (aggregated projection), notes (4-table dispatch). Find divergence between docs and code. |
| 3   | **Security**                  | `security-review` (existing skill)       | OWASP API Top 10, auth bypass, IDOR, injection, secret leakage, GDPR exposure. Multi-tenant boundary checks (port_id at every join).                                                 |
| 4   | **API surface consistency**   | `code-review:code-review`                | `{ data: T }` envelope adherence, `errorResponse(error)` usage, `parseBody(req, schema)` usage, 204 vs JSON, withAuth+withPermission composition.                                    |
| 5   | **UI/UX consistency**         | `frontend-design:frontend-design` review | Visual inconsistencies, copy/text issues, accessibility, mobile parity, brand drift, em-dashes, generic SaaS slop.                                                                   |
| 6   | **Schema vs code divergence** | `feature-dev:code-explorer`              | Migrations vs Drizzle schema files vs service helpers — find any column the DB has that no service touches, or any service field with no migration.                                  |
| 7   | **Documenso integration**     | `feature-dev:code-explorer`              | Full v1↔v2 path coverage, webhook idempotency, template field mapping, EOI generation (both pathways), error recovery.                                                               |
| 8   | **Storage & file lifecycle**  | `feature-dev:code-explorer`              | S3↔filesystem switching, file orphans, signed-URL expiry, GDPR export coverage, magic-byte validation everywhere.                                                                    |

Coordination:

- Use a **single coordinator session** that fans out via `Agent` /
  `TaskCreate` with `subagent_type` set per dimension. Each agent writes
  findings to a per-dimension scratch file under
  `docs/audits/2026-05-27/<dimension>.md`, then the coordinator
  consolidates into a single triage doc with severity tags.
- Pass `model: "opus"` on every agent spawn — Sonnet/Haiku context
  windows compact too fast under MCP baseline (per memory
  `feedback_subagent_context_bloat`).

Output: `docs/audits/2026-05-27/findings-master.md` with per-finding
severity (`CRITICAL | HIGH | MED | LOW`), file:line refs, and
recommended fix. Critical + High get fixed before launch.

---

## Initiative 3 — Marketing website integration

**Status:** OPEN · Needs scope clarification

User ask: "make our relevant edits to the marketing website to prepare
for the deployment and integration of our new system."

The marketing site lives in `/Users/matt/Repos/Port Nimara/Website`
(separate Nuxt repo). Integration touch points the CRM exposes today:

- **`/api/public/berths`** + **`/api/public/berths/[mooringNumber]`** —
  feeds the marketing site's berth list / detail. Status precedence
  Sold > Under Offer > Available is already wired.
- **`/api/public/health`** — dual-mode health check; the website should
  call the authenticated variant (with `WEBSITE_INTAKE_SECRET`) on
  startup so it refuses to start when pointed at the wrong CRM env.
- **`/api/public/website-inquiries`** — intake endpoint for the contact
  form; dual-writes inquiry into the CRM.
- **Inquiry email ownership** — at cutover, inquiry emails move from
  the website to the CRM (per memory
  `project_email_ownership_at_cutover`). Templates + settings keys
  already exist; berth public endpoint + admin recipient UI still
  needed (per existing memory).
- **Cover photography + branding assets** — the new system uses
  `branding_email_background_url` etc.; ensure the website assets
  match.

Open work (needs user input on priority):

- Wire the website's contact form to `/api/public/website-inquiries`
  with the new payload shape.
- Add the `WEBSITE_INTAKE_SECRET` to the website's env, point at the
  authenticated `/api/public/health`.
- Update berth-detail page to consume the new `/api/public/berths/...`
  shape (the JSON mirrors the legacy NocoDB shape so this should be
  a no-op — VERIFY).
- Replace any hard-coded "noreply@portnimara.com" sender on the
  website side with the CRM-controlled From address (so per-port
  branding wins).
- Confirm the website's caching headers don't fight ours
  (`s-maxage=300, stale-while-revalidate=60` on berth endpoints).

---

## Initiative 4 — End-to-end testing

**Status:** OPEN · Needs scope clarification

User ask: "end to end testing of all sales functions, generating
EOIs/documents (especially), ensuring all UX/UI is fluid, beautiful,
relevant and helps the user go through the sales process effortlessly."

Existing infrastructure (per `CLAUDE.md`):

- `tests/e2e/smoke` — fast click-through (~10 min, ~125 specs)
- `tests/e2e/exhaustive` — deeper UI coverage
- `tests/e2e/destructive` — archive/delete/cancel paths
- `tests/e2e/realapi` — opt-in real Documenso + IMAP round-trip
- `tests/e2e/visual` — pixel-diff baselines

Pre-launch test gaps to fill (proposed):

1. **End-to-end sales journey** (single Playwright spec, real-API): new
   inquiry → qualified → EOI generated (Documenso) → client signs →
   developer countersigns → reservation → deposit recorded → contract
   generated → contract signed → tenancy auto-created → berth marked
   sold. Assert every stage transition + every email fires.
2. **EOI generation parity** between both pathways (in-app
   `fill-eoi-form` vs Documenso template). Same `EoiContext` should
   produce equivalent PDFs.
3. **Multi-berth EOI rendering** — berth range formatter assertion
   (`A1-A3, B5-B7` from `interest_berths`).
4. **Documenso webhook idempotency** — replay the same `DOCUMENT_COMPLETED`
   webhook three times; assert single `files.folder_id` write + no
   duplicate audit-log rows.
5. **Storage backend swap** — switch port to filesystem, generate EOI,
   verify file lands; switch back to S3, confirm migrate script moves
   the blob correctly.
6. **Visual snapshot refresh** for the new Reports UI + back-button
   smart-back changes (this conversation).
7. **Mobile parity** for the entire sales journey (different Playwright
   project or `--config` variant).

Each gap above becomes one or two new spec files. Coordinate with
Initiative 2's audit so we don't double-test.

---

## Initiative 5 — Data migration (legacy → new)

**Status:** OPEN · High effort · Likely blocker for cutover

> **Infra cutover plan:** `docs/deployment-plan.md` — prod deploy of the CRM
> to `crm.portnimara.com` (nginx + certbot + registry-image compose),
> Gitea/CI access, and the Documenso backup + safe-upgrade procedure. Access
> (SSH + Gitea API) established 2026-05-31; no prod changes without explicit
> approval. Deployment creds in `private/deployment-creds.md` (gitignored).

User ask: "start pulling all existing prod data from the old system and
connected systems (we'll have to backfill the EOIs by pulling them
through MinIO — it's a fucking mess so I'll really need your help
automating/speeding up that process) and initiate a preliminary switch
over."

Sources to drain:

| Source                          | Storage            | Entities                                                       | Notes                                                                                                            |
| ------------------------------- | ------------------ | -------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------- |
| Old NocoDB tables               | Postgres / NocoDB  | Clients, yachts, companies, interests, berths, EOIs (metadata) | Already imported in earlier migration; verify currency vs prod NocoDB.                                           |
| Old portal (`client_portal_v2`) | Nuxt + Postgres    | Portal users, signing history, sent invitations                | Need to confirm what hasn't been migrated yet.                                                                   |
| MinIO (legacy bucket)           | Object storage     | EOI PDFs (signed + unsigned), receipts, contracts              | The "fucking mess" — naming is inconsistent, organisation unclear, need to map each blob back to its CRM entity. |
| Documenso v1 (live)             | Documenso server   | In-flight signing envelopes + signed PDFs                      | Migration question: do we cut new EOIs to v2 and let v1 envelopes finish, or migrate the in-flight?              |
| Email archives                  | IMAP / mail server | Inquiry replies, signing reminders, deposit confirmations      | Probably out of scope for cutover (read-only history).                                                           |

Migration script plan (write under `scripts/migration/`):

1. **`probe-minio.ts`** — scan the legacy MinIO bucket, list every blob,
   try to extract a client / interest / berth identifier from filename
   patterns. Produce `docs/migration/minio-blob-inventory.csv` with
   `key, size_bytes, mime, probable_entity_type, probable_entity_id, confidence`.
2. **`backfill-eoi-pdfs.ts`** — for each inventoried blob with confidence
   ≥ HIGH, copy from legacy MinIO into the new storage backend, create a
   matching `files` row + `documents` row, deposit into the right
   entity folder via the existing `ensureEntityFolder` helper. Idempotent
   via `legacy_minio_key` column (add via migration if missing).
3. **`reconcile-nocodb.ts`** — diff the live NocoDB tables against our
   imported state; report rows added/changed/deleted since last import.
4. **`preflight-cutover.sh`** — orchestrator script that runs the three
   above in order, writes a final report.

Cutover plan:

1. Freeze writes on the old system (NocoDB read-only, portal
   maintenance page).
2. Run `preflight-cutover.sh` against frozen sources.
3. Manual reconciliation of probe-minio rows where confidence < HIGH
   (likely a few hundred blobs — the user explicitly flagged this is
   manual labour, automation helps but doesn't replace it).
4. DNS / website pointer flip.
5. Watch error_events for 24h; rollback plan = re-enable old system
   writes and stop the cutover commit.

---

## Cross-initiative open questions

- **When to wrap the launch audit doc.** I'd suggest: after Initiative
  2's findings are triaged AND Initiatives 3-5 reach IN PROGRESS. At
  that point this file becomes the launch-day-runbook.
- **Who's the launch sponsor / decision-maker?** Different from
  "user / matt"? Affects who signs off on cutover.
- **Soft launch vs hard cutover?** Hard cutover is simpler operationally
  but risky; soft launch (parallel writes for a week) is safer but
  requires the old system to keep accepting writes for longer.

---

## 2026-06-01 — Feature-completeness sweep & launch-prep decisions

A read-only sweep (ahead of the ~same-day launch) checked the whole
platform for half-built / stubbed surfaces beyond the known Reports
gaps. It resolved two stale-doc contradictions: **Documenso signing
phases 2–7 are fully built and wired** (`BACKLOG.md` §A is stale on
this), and the **interest Contract/Reservation tabs are fully built**
(not "coming soon" cards). Findings + decisions below.

**Decision (per Matt, 2026-06-01):** launch is ~today, so **ship what's
done, hide what's not, defer the big builds** — do NOT revert to the old
desktop-spreadsheet reports (a downgrade), and do NOT rush the
unproven full builds onto a same-day prod launch.

### Shipped today (launch-prep, low-risk; SHIPPED)

- **Hid Financial + Marketing report cards** from the reports landing
  (`reports/page.tsx`) — both were "Builder in development" placeholders
  gated on unbuilt data sources (Init 1b/1c). The reports section ships
  with the working **Sales + Operational + Custom** reports + templates +
  scheduling + PDF/CSV/Excel exports. The basic Custom builder already
  covers the old desktop-report use case (entity + columns + date range +
  export) — parity-plus, not a regression.
- **Trimmed the Custom-report card copy** so it stops promising
  group-by/filters/dimensions it doesn't yet have (the builder page
  header was already honest).
- **Hid the Bulk Import mockup** from nav + search
  (`admin-sections-browser.tsx`, `search-nav-catalog.ts`). The static
  `/admin/import` mockup is now unreachable from the UI (route still
  resolves by direct URL).
- **Corrected client-facing doc over-claims** in `features-list.md` +
  `new-system-feature-summary.md` (removed the waiting-list
  "next-in-line notification" claim — built but hidden; removed Import
  from the admin-pages list, 43→42).

### Deferred to post-launch (tracked here; none launch-blocking)

- **Full Bulk CSV/XLSX importer** — design APPROVED + spec written:
  `docs/superpowers/specs/2026-06-01-bulk-import-design.md` (generic
  engine + per-entity adapter registry; 7 entities; column-mapping,
  dry-run, dedup, per-batch undo). Cutover data migration runs through
  the existing CLI scripts (`import-berths-from-nocodb.ts` + the
  Initiative 5 migration scripts), so the UI importer is **not needed
  for launch**.
- **Full Custom-report builder** — group-by + aggregates, sort,
  per-column filter rows (AND/OR), debounced live preview, the remaining
  6 of 10 entities, per-role PII column whitelist. Architecture decided
  (per-column expression map + generic Drizzle query composer); spec
  deferred. Basic builder ships as-is.
- **Berth Waiting List** — ✅ **SHIPPED in 8be7a6e2.** `WaitingListManager`
  tab un-hidden + wired. _Still deferred: the availability-triggered
  next-in-line notification (today only a `notifyPref` column is stored;
  no sender exists)._
- **Berth Maintenance Log** — ✅ **SHIPPED in 8be7a6e2.** UI tab mirroring
  the waiting-list manager, on the existing API + service.
- **Contract/Reservation paper-upload misroute (BUG)** — ✅ **SHIPPED in
  d98aa5cc.** Added contract/reservation paper-upload endpoints +
  pointed `ExternalEoiUploadDialog` at the right one per docType, so a
  paper-signed contract/reservation no longer files as an EOI.
- **Marketing + Financial reports** — remain unbuilt + now hidden; gated
  on Init 1b (website UTM/inquiry cutover) and Init 1c (invoices-module
  decision) respectively.
