letsbe/pn-new-crm
1
0
Fork 0
Code Issues Pull Requests 1 Actions Packages Projects Releases Wiki Activity
Files
14ab8a8161fccc658e1d565771fe58b79f5d414a
pn-new-crm/docs/website-analytics-flesh-out-plan.md
Matt bac253b360 feat(analytics): Umami website-analytics suite — world map, realtime, sessions, heatmap, pixel tracking, tracked links 
Adds the read-side Umami integration queued in last week's
website-analytics plan (Phases 1–6 of `docs/website-analytics-flesh-out-plan.md`):

- Realtime panel polls Umami at 5s intervals; world map renders visitor
  origins via echarts + `public/world-map/echarts-world.json` topo.
- Sessions list + session-detail-sheet drill-down (per-session event
  timeline pulled from `/api/v1/website-analytics`).
- Weekly heatmap (day-of-week × hour-of-day) for engagement timing.
- Metric-detail pages under `/[portSlug]/website-analytics/[metric]`
  for pageviews / referrers / events deep-dives.
- Email-pixel write path: `/api/public/email-pixel/[sendId]` 1×1 GIF
  beacon backed by `email_open_tracking` (migration 0076); resolves
  inline on render in inbox.
- Tracked-link redirect: `/q/[slug]` routes through `tracked_links`
  (migration 0077) and forwards to the canonical destination after
  logging the click.
- Dashboard `website-glance-tile` now reads from the live Umami service
  instead of placeholder data.

Deps: `@umami/node`, `echarts`, `echarts-for-react`, `@types/geojson`,
`@types/topojson-client`.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-20 15:53:41 +02:00

429 lines
41 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# Website Analytics — flesh-out plan
**Goal:** rebuild `/{portSlug}/website-analytics` so it feels like a polished native CRM panel that _mirrors_ Umami's idiom rather than reading as a stripped-down embed. Keep a "View in Umami →" deep-link in the header for power users; render most data in-app via the API. Also extend usage into adjacent CRM surfaces (dashboard tiles, inquiry detail, email open-tracking) so Umami stops being "the analytics page" and becomes a cross-cutting data layer.
**Inputs to this plan:**
1. Live API capabilities reference — `docs/umami-api-capabilities.md` (verified empirically against v3.1.0 on analytics.portnimara.com).
2. Live UI tour via Playwright — screenshots `umami-tour-1-overview.png` through `umami-tour-9-compare.png` (10 surfaces captured).
3. Pixel-tracking probe — confirmed the `/p/<slug>` and `/q/<slug>` endpoints + their UI creation forms.
---
## 1. What Umami's UI actually does — design patterns to mirror
Tour findings (from 17 sub-pages + 4 team pages):
| Surface | Visual idiom | Adopt for CRM? |
| ------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ----------------------------------------------------------------------------------------------------------------------------- |
| **Overview** | 5-tile KPI row (Visitors / Visits / Views / Bounce rate / Visit duration) — each tile shows headline number + colored arrow chip (green ↑ 58% / red ↓ 39%) + percentage delta. Single stacked bar chart below for traffic time-series (visitors stacked over visits, dual-shade blue). Filter pill + date-range nav top-right. | **Yes** — already mostly there, missing the bounce-rate + visit-duration tiles. |
| **Events** | List of custom event names with per-event count + time-series spark. | **Yes** — needs marketing-site event firing first (Phase 4). |
| **Sessions** | Dense table: avatar + per-session row showing Visits / Views / Events / Location (flag + city, country) / Browser icon / OS icon / Device icon / Last seen. Tabs for Activity vs Properties (custom session props). | **Yes** — high-leverage; lets reps see _who_ is browsing right now. |
| **Realtime** | 4 stat tiles (Views/Visitors/Events/Countries) + auto-refreshing line chart of last 30 min. | **Yes** — already partial via the glance tile. |
| **Performance** | Likely page-speed / Core Web Vitals. | Skip — not relevant to marina sales. |
| **Compare** | Pick two date ranges side-by-side. | **Partial** — single `compare=prev` overlay on the existing trend chart suffices. |
| **Breakdown** | Pivot table view across dimensions. | Skip in v1; expose via Reports later. |
| **Goals** | Define event/page-view goals, see completion rate over time. | **Yes** — defer to Phase 5. |
| **Funnels** | Multi-step conversion funnel (e.g. /berths → /berths/A12 → /inquire → submit). | **Yes** — Phase 5; high-value for inquiry conversion. |
| **Journeys** | Most common navigation paths (Sankey-like). | **Maybe** — defer; nice-to-have. |
| **Retention** | Cohort retention grid. | Skip — wrong fit for one-and-done marina inquiry traffic. |
| **Replays** | Session replay (likely paid). | Skip — unavailable on our tier. |
| **Segments / Cohorts** | Saved filters / user groups. | Skip in v1. |
| **UTM** | Campaign attribution by UTM params. | **Yes** — Phase 5 for paid-campaign tracking. |
| **Revenue** | Revenue attribution. | Skip — would require firing `purchase` events from CRM after EOI close (consider Phase 6 if leadership wants funnel→revenue). |
| **Attribution** | First/last-click attribution model. | **Maybe** — defer. |
| **Team-Boards / Websites / Links / Pixels** | Account admin surfaces. | **Pixels + Links: YES — see Phase 4.** Boards/Websites stay in Umami. |
### Visual specifics worth copying
- **KPI tile design**: large bold number, label above in muted-grey, arrow + percentage delta below in a colored chip (green-bg for positive, red-bg for negative, fixed-width for alignment). Our `KPITile` already does the right shape — we just need to add the missing two metrics.
- **Stacked bar chart for traffic**: dual-shade single bar (visitors as light-blue base, views stacked dark-blue on top). Reads cleaner than two overlapping lines.
- **Location rendering**: flag emoji + "City, Country" inline. Use `getCountryName()` + a flag library (twemoji or unicode regional indicators).
- **Browser/OS/Device icons**: small colored brand glyphs inline. Use `simple-icons` or `lucide` equivalents.
- **Filter chip + date nav**: `<` `>` arrows step through the date range; dropdown opens to preset list. Adopt the same pattern on our shell — currently we only have presets, no step-arrows.
---
## 2. Phased build plan
### Phase 1 — Fill out the Overview tiles & chart (~3-4h)
Quick wins that close visual parity with Umami's Overview:
| Task | File | Effort |
| -------------------------------------------------------------- | ------------------------------------------------------------------------ | ----------------------------------------------------------------------------- |
| Add **Bounce rate** KPI tile | `website-analytics-shell.tsx` | derive `bounces / visits * 100`; service field already there |
| Add **Avg visit duration** KPI tile | `website-analytics-shell.tsx` | derive `totaltime / visits` formatted as `Xm Ys`; service field already there |
| Add **`<` `>` date-step arrows** on the date-range chip | `date-range-picker.tsx` | step the current preset by one window (today→yesterday, 7d→prior-7d, etc.) |
| Convert pageviews trend to **stacked bar** (visitors vs views) | `pageviews-chart.tsx` | recharts `BarChart` stacked, light/dark blue |
| Add **`compare=prev` overlay toggle** on the trend chart | `pageviews-chart.tsx` + service `getPageviewsSeries` | optional "vs prior period" series rendered as dashed line |
| Add **Top browsers / OS / devices** ranked-list cards | new `<TopList>` consumers; service already exposes via `getMetric(type)` | mirror Top Pages/Referrers/Countries layout |
| **World choropleth heatmap** card (already queued separately) | new `visitor-world-map.tsx` (Natural Earth topojson + react-simple-maps) | ~4-6h on its own |
**Cumulative result:** Overview surface reads at ~80% parity with Umami's Overview.
---
### Phase 2 — Sessions surface (~4-5h)
New `/website-analytics/sessions` tab + supporting service wrappers:
| Task | File | Effort |
| ------------------------------------------------------------------------------------------------------------------------ | ------------------ | ------- |
| Service: `getSessions(portId, range, opts)` → `/api/websites/:id/sessions` (paged) | `umami.service.ts` | ~30 min |
| Service: `getSession(portId, sessionId)` → single-session detail | `umami.service.ts` | ~15 min |
| Service: `getSessionActivity(portId, sessionId, range)` → event timeline | `umami.service.ts` | ~15 min |
| Service: `getSessionsWeekly(portId, range)` → hour-of-week heatmap | `umami.service.ts` | ~15 min |
| API route: `/api/v1/website-analytics?metric=sessions[&sessionId=...]` | route.ts | ~30 min |
| UI: `sessions-table.tsx` — dense rows mirroring Umami (avatar + location flag + browser/OS/device icons + Last seen) | new component | ~2h |
| UI: `session-detail-sheet.tsx` — right-side Sheet drawer showing the session's full event timeline when a row is clicked | new component | ~1h |
| UI: `weekly-heatmap-card.tsx` — 7×24 grid colour-scaled by session count, hover for tooltip | new component | ~1h |
**Unlock:** rep can see "who is currently browsing right now, where from, on what device, what they're looking at" — directly actionable for sales follow-up.
---
### Phase 3 — Events surface (~3-4h, BLOCKED on Phase 4a)
| Task | File | Effort |
| -------------------------------------------------------------------------------------------- | ------------------ | ------- |
| Service: `getEvents(portId, range, opts)` → `/events` paged list | `umami.service.ts` | ~30 min |
| Service: `getEventsStats(portId, range)` → totals | `umami.service.ts` | ~15 min |
| Service: `getEventsSeries(portId, range, eventName, unit)` → per-event time-series | `umami.service.ts` | ~15 min |
| API route addition | route.ts | ~30 min |
| UI: `events-tab.tsx` — list of event names with per-event count + spark + drill-in | new component | ~1.5h |
| UI: `event-detail-sheet.tsx` — single event's time-series chart + filter by payload property | new component | ~1h |
**Dependency:** the marketing site must fire `umami.track(name, payload)` calls (Phase 4a). Without this, Events tab is empty.
---
### Phase 4 — Pixel tracking + link tracking + marketing-site event push
**Phase 4a — Marketing-site event tracking (~2-3h on marketing repo)**
Add `umami.track()` calls in the marketing site:
- `inquiry-submitted` with `{berth, source}` payload — fires on EOI form submit
- `brochure-download` with `{brochureId}` — fires on brochure download
- `berth-detail-viewed` with `{berthId, mooring}` — fires on `/berths/[mooring]` page view
- `phone-revealed` / `email-revealed` — fires when contact details are exposed
These light up the Events tab + enable funnel analysis in Phase 5.
**Phase 4b — Pixel-based email open tracking (~3-4h CRM-side)**
Probe finding: Umami exposes pixel URLs at `https://analytics.portnimara.com/p/<slug>` — fetching the URL records an event. Use case: embed in HTML emails as a 1x1 image.
**Two architecture options:**
**Option A — One Umami pixel per email type** (simple, low fidelity)
- Create a pixel manually in Umami for each templated email type (`portal-invite`, `eoi-sent`, `reservation-reminder`, etc.)
- Embed the static pixel URL in each template
- Pro: zero CRM-side code beyond template HTML. Open rates roll up in Umami by pixel.
- Con: can't tell _which recipient_ opened — only aggregate counts per template.
**Option B — One Umami pixel + CRM-side per-send tracking endpoint** (richer, recommended)
- Build `GET /api/public/email-pixel/:sendId.gif` in our CRM that:
1. Returns a 1×1 transparent GIF
2. Records the open in `document_sends.opened_at` (already a table; per CLAUDE.md "send-from accounts" section)
3. Optionally proxies the hit to Umami via `POST /api/send` with the email type + send id as event properties for cross-correlation
- Embed `<img src="https://crm.portnimara.com/api/public/email-pixel/{sendId}.gif" width="1" height="1" />` in every templated email
- Pro: per-recipient open tracking + open-time + CRM-attached. Funnels by email type via Umami too.
- Con: needs the public endpoint + a schema column (or reuse `document_sends.opened_at`).
**Recommendation: ship Option B.** The CRM-side hook gives us per-deal attribution ("client X opened the EOI reminder twice but hasn't signed"), and Umami still gets the aggregate.
| Task | File | Effort |
| ------------------------------------------------------------------------------------------------------------- | ---------------------------------------------- | ------- |
| New endpoint `/api/public/email-pixel/[sendId]/route.ts` returning a 1×1 GIF + recording open | new route | ~1h |
| Migration: add `opened_at`, `open_count`, `last_opened_user_agent` to `document_sends` if not present | drizzle migration | ~30 min |
| Email template helper: inject the pixel HTML into every transactional template | `src/lib/email/render.ts` | ~30 min |
| UI surface: on each `document_sends` row in the activity feed, show "Opened N times, last at X" badge | `email-activity-row.tsx` | ~1h |
| Cross-post to Umami via `trackEvent('email-opened', {emailType, sendId})` so Umami funnel data includes opens | new `trackEvent` wrapper in `umami.service.ts` | ~30 min |
| Privacy: respect `EMAIL_REDIRECT_TO` dev gate; don't fire pixels for redirected dev emails | ditto | ~15 min |
**Phase 4c — Tracked redirect links (~1.5h)**
Umami's `/q/<slug>` endpoint is a tracked redirect — records a click then 302s to the destination URL. Use for outbound CTAs:
- "View brochure" links in emails → wrap via Umami link → records click → opens brochure
- "Schedule a viewing" buttons → wrap via Umami link → click attribution
- Marketing-site CTAs → wrap → measure engagement
| Task | File | Effort |
| ------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------- | ------- |
| Service: `createTrackedLink(name, destinationUrl)` → POST to Umami's links endpoint via authenticated API | `umami.service.ts` | ~45 min |
| Email template helper: `<trackedLink href="..." name="...">` JSX wrapper that auto-creates the Umami link on first render + caches the slug | `src/lib/email/components/` | ~45 min |
---
### Phase 5 — Reports surfaces (Funnels, UTM, Journeys) (~6-8h)
| Task | File | Effort |
| ---------------------------------------------------------------------------------------------------------------------------------------------- | ------------------ | ------ |
| Service: `getReport(reportType, body)` POST wrapper covering `/funnel`, `/journey`, `/utm`, `/goals`, `/retention`, `/revenue`, `/attribution` | `umami.service.ts` | ~1h |
| UI: `/website-analytics/funnels` page — admin-configurable funnel definitions (steps as event names or URL paths), per-step drop-off chart | new page | ~3h |
| UI: `/website-analytics/utm` page — UTM source/medium/campaign breakdown with click-through to attributed sessions | new page | ~2h |
| UI: `/website-analytics/journeys` page — top navigation paths rendered as ranked list (skip Sankey for v1) | new page | ~1.5h |
| Defer: Goals / Retention / Revenue / Attribution to v2 (low signal for marina sales) | | |
**High-leverage funnels to wire as defaults:**
- **Inquiry funnel**: `/` → `/berths` → `/berths/[mooring]` → `inquiry-submitted` event → CRM `eoi-signed` (cross-system!) → CRM `reservation-paid` (cross-system!)
- **Email funnel**: `email-sent` → `email-opened` (pixel) → tracked-link click → CRM action
The cross-system funnels require Phase 4 to be live first.
---
### Phase 6 — CRM → Umami event push for outcome attribution (~2-3h)
Close the funnel from "marketing site click" → "CRM closed deal" by firing CRM-side events back into Umami via `POST /api/send`:
| Event | Fired by | Payload |
| ---------------------- | -------------------------------------------- | --------------------------------------- |
| `crm-inquiry-created` | `createInterest()` in `interests.service.ts` | `{interestId, source, leadCategory}` |
| `crm-eoi-sent` | `generateAndSign()` after EOI dispatch | `{interestId, berth, pathway}` |
| `crm-eoi-signed` | Documenso `DOCUMENT_COMPLETED` webhook | `{interestId, berth}` |
| `crm-reservation-paid` | manual stage advance to `deposit_paid` | `{interestId, berth, amount, currency}` |
| `crm-contract-signed` | manual stage advance to `contract` | `{interestId, berth, amount, currency}` |
| Task | File | Effort |
| ----------------------------------------------------------------------------------------- | ------------------- | -------- |
| Service: `trackEvent(name, payload, sessionId?)` → `POST /api/send` on the Umami instance | `umami.service.ts` | ~45 min |
| Hook into the 5 service entry points above (one event per outcome milestone) | each service file | ~1.5h |
| Audit log entry per event sent so we can verify Umami received it | `audit_logs` insert | included |
**Unlock:** Umami's Revenue + Attribution reports start showing CRM outcomes attributed to marketing-site channels — closes the leadership question "which traffic sources actually generate signed deals, not just leads?"
---
### Phase 7 — Cross-cutting CRM placements (~3-4h)
Beyond the dedicated `/website-analytics` page, surface Umami data inside CRM context:
| Placement | What | Effort |
| ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------- | ------------------------------------ |
| **Dashboard rail tile** (already shipped) — Pageviews + active now | already done in this session | — |
| **Inquiry detail page** — "Source attribution" card showing the inquiry's UTM params, landing page, time-on-site, pages-viewed-before-submit. Pulls from `getSession(sessionId)` if the inquiry's create payload includes a session ID (requires marketing-site change to pass it). | new `inquiry-attribution-card.tsx` | ~1.5h + marketing-site change |
| **Client detail page** — "Website activity" card: total sessions, pageviews, last-seen, top pages visited. Requires `umami.identify({email})` on marketing site to link sessions back to clients. | new `client-web-activity-card.tsx` | ~1.5h + marketing-site identify call |
| **Berth detail page** — "Marketing demand" card: pageviews to `/berths/{mooring}` over time + referrer breakdown. Drives "this berth is being viewed but not inquired-about — flag for outreach." | new `berth-demand-card.tsx` | ~1h |
| **Document send activity** — pixel opens per recipient (from Phase 4b) | inline on existing `document_sends` rows | included in 4b |
---
## 2b. Library adoptions (changes the plan materially)
Context7 lookup surfaced three official libraries that reshape the plan. **Adopt all three.**
### `@umami/api-client` — official read-side client
Covers every read endpoint we need including all the report types. Built-in filter support, login/JWT auth handled internally, `{ok, data}` discriminated union for clean error handling.
**Replaces:** ~60-70% of our current `umami.service.ts` (drop `umamiFetch`, JWT cache, decrypt boilerplate; keep thin wrappers with existing signatures so consumers don't change).
**One-time refactor (~2h):**
```ts
const clientByPort = new Map<string, UmamiApiClient>();
async function getClient(portId: string): Promise<UmamiApiClient | null> {
if (clientByPort.has(portId)) return clientByPort.get(portId)!;
const cfg = await loadUmamiConfig(portId);
if (!cfg) return null;
const client = new UmamiApiClient({
apiEndpoint: `${cfg.apiUrl}/api`,
apiKey: cfg.apiToken ?? undefined,
});
if (!cfg.apiToken && cfg.username && cfg.password) await client.login(cfg.username, cfg.password);
clientByPort.set(portId, client);
return client;
}
export async function getStats(portId: string, range: DateRange) {
const client = await getClient(portId);
if (!client) return null;
const { from, to } = rangeToBounds(range);
const result = await client.getWebsiteStats(WEBSITE_ID, {
startAt: from.getTime(),
endAt: to.getTime(),
});
return result.ok ? result.data : null;
}
```
Same pattern for `getPageviewsSeries`, `getMetric`, `getActiveVisitors`, plus new ones from the SDK: `getRealtime`, `getWebsiteSessionStats`, `runFunnelReport`, `runJourneyReport`, etc.
### `@umami/node` — official write-side SDK
For Phase 6 (CRM → Umami push) and Phase 4b cross-post:
```ts
const umami = new Umami({ websiteId, hostUrl });
await umami.track({
url: '/crm/eoi-signed',
name: 'crm-eoi-signed',
data: { interestId, berth, dealValue },
});
await umami.identify({ sessionId, email, interestId });
```
**Replaces:** the planned hand-rolled `trackEvent()` wrapper. Single line per outcome milestone.
### `react-simple-maps` — for the world heatmap (Phase 1b)
Declarative SVG choropleth on d3-geo + topojson-client. SSR-safe. Use `topojson/world-atlas` (110m resolution ~30KB) cached in `public/`. Bundle ~30-50KB + topojson 30-100KB.
```jsx
<ComposableMap projection="geoMercator">
<Geographies geography="/world-110m.json">
{({ geographies }) =>
geographies.map((geo) => (
<Geography
key={geo.rsmKey}
geography={geo}
fill={scaleByVisitorCount(visitorsByCountry[geo.properties.iso_a2] ?? 0)}
onClick={() => onCountryClick(geo.properties.iso_a2)}
/>
))
}
</Geographies>
</ComposableMap>
```
**Chose this over visx/Nivo/Chart.js Geo:** visx is overkill for one map; Nivo + Chart.js force a different charting idiom (we use recharts everywhere); react-simple-maps' compose-primitives shape matches our recharts pattern.
### Net effect on phase efforts
| Phase | Original estimate | Revised after library adoption |
| ---------------------------------------- | ----------------- | --------------------------------------------------------------- |
| Service refactor (one-time) | — | **+2h** (one-time foundation; pays back across all phases) |
| Phase 1 — Overview parity | 3-4h | 3-4h (unchanged; api-client makes the filter additions trivial) |
| Phase 1b — World heatmap | 4-6h | 3-4h (library choice locked in) |
| Phase 2 — Sessions | 4-5h | 3-4h (api-client has session methods built-in) |
| Phase 3 — Events | 3-4h | 2-3h (api-client provides) |
| Phase 4b — Pixel hybrid | 3-4h | 2.5-3h (cross-post is one line) |
| Phase 5 — Reports (funnels/UTM/journeys) | 6-8h | 3-4h (every report method pre-wrapped) |
| Phase 6 — CRM → Umami push | 2-3h | 1.5h (`@umami/node` handles transport) |
**Total scope drops from ~30-40h to ~22-28h** with these adoptions.
---
## 3. Service-layer additions consolidated
Add to `src/lib/services/umami.service.ts` (each is a thin wrapper around existing `umamiFetch` / new `umamiPost`):
```ts
// Sessions (Phase 2)
getSessions(portId, range, { page?, pageSize?, query? }) → /sessions
getSession(portId, sessionId) → /sessions/:id
getSessionActivity(portId, sessionId, range) → /sessions/:id/activity
getSessionProperties(portId, sessionId) → /sessions/:id/properties
getSessionsWeekly(portId, range, timezone) → /sessions/weekly
// Events (Phase 3)
getEvents(portId, range, opts) → /events
getEventsStats(portId, range) → /events/stats
getEventsSeries(portId, range, eventName, unit) → /events/series
getEventDataProperties(portId, range) → /event-data/properties
// Realtime (Phase 1)
getRealtime(portId, range) → /api/realtime/:id (richer than /active)
// Reports (Phase 5)
getReport(portId, reportType, body) → POST /api/reports/:type (funnel/journey/utm/goals/retention/revenue/attribution)
// CRM → Umami (Phase 6)
trackEvent(portId, name, payload, sessionId?) → POST /api/send
// Links + Pixels admin (Phase 4)
createTrackedLink(portId, name, destinationUrl) → POST team-level /links
createTrackingPixel(portId, name) → POST team-level /pixels
```
Plus a new `umamiPost(config, path, body)` helper alongside the existing `umamiFetch` since GET-only doesn't cover reports + send.
---
## 4. Pixel-tracking answer (the user's specific question)
**Q: Can we use Umami's pixel tracking for email opens?**
**A: Yes — and recommended in hybrid form.** Direct verification on the live instance:
- Pixel UI at `/teams/[teamId]/pixels` lets an admin create named pixels. Each gets an auto-generated slug.
- The pixel URL is `https://analytics.portnimara.com/p/<slug>` — fetching it records an event (no auth required from the email client side; the slug is the credential).
- Embedded as `<img src="..." width="1" height="1" />` in HTML emails, it fires when the email is rendered (Outlook/Apple Mail/etc.).
- Standard caveats: Apple Mail privacy protection pre-fetches images server-side → opens may be over-counted for iOS users. Some recipients block images entirely → opens under-counted. Same caveats as every email tracking pixel ever.
**Recommended hybrid (Phase 4b above):** build a CRM-side pixel endpoint `/api/public/email-pixel/[sendId].gif` that:
- Returns the 1×1 GIF
- Records `opened_at` in `document_sends`
- Cross-posts the hit to Umami via `POST /api/send` so the Umami Events tab + funnels include opens
This way: per-recipient attribution in the CRM, aggregate roll-ups in Umami, single source of truth for both.
---
## 5. Effort summary + prioritization
| Phase | Scope | Effort | Priority |
| ----- | ------------------------------------------------------------------------------------ | ---------------------- | ------------------------------------------------------------ |
| 1 | Overview parity (KPI tiles, stacked-bar chart, date arrows, browser/OS/device cards) | ~3-4h | **High** — most visible polish, no dependencies |
| 1b | World choropleth heatmap (already queued separately) | ~4-6h | **High** if leadership wants the visual |
| 2 | Sessions surface (table + detail sheet + weekly heatmap) | ~4-5h | **High** — biggest "wow" + actionable |
| 3 | Events surface | ~3-4h | **Medium** — blocked on 4a |
| 4a | Marketing-site event tracking | ~2-3h (marketing repo) | **High** — unblocks 3 + 5 |
| 4b | Pixel-based email open tracking (hybrid) | ~3-4h | **High** — direct ask + immediate value |
| 4c | Tracked redirect links | ~1.5h | **Medium** |
| 5 | Reports (Funnels, UTM, Journeys) | ~6-8h | **Medium** — depends on 4a being live |
| 6 | CRM → Umami event push for outcome attribution | ~2-3h | **Medium-high** — needed to close marketing→outcome loop |
| 7 | Cross-cutting placements (inquiry / client / berth detail cards) | ~3-4h | **Medium** — depends on `umami.identify()` on marketing site |
**Recommended build order (updated 2026-05-19 per user):**
1. ✅ **Service refactor** — Kept hand-rolled `umamiFetch` (the official `@umami/api-client` transitively pulls `next-basics` which requires React at module-import time, breaking SSR + tsx scripts). Adopted `@umami/node` for the write side.
2. ✅ **Phase 1** — Overview parity (KPI tiles + browser/OS/device cards + date arrows + stacked-bar chart + `compare=prev` overlay)
3. ✅ **Phase 1b** — World heatmap. **Switched from `react-simple-maps` to ECharts + `public/world-map/echarts-world.json`** — the `world-atlas/110m` topojson has antimeridian-crossing polygons (Russia/Fiji/Antarctica) that render a horizontal line through the equator regardless of projection. ECharts' world.json is pre-cleaned.
4. ✅ **Phase 4b** — Pixel-based email open tracking. `document_send_opens` table + `/api/public/email-pixel/[sendId]` endpoint + `injectTrackingPixel` helper wired into `performSend`. Per-port kill switch `email_open_tracking_enabled` (admin UI on `/admin/website-analytics`). Cross-posts to Umami as `email-opened`.
5. ✅ **Phase 2** — Sessions surface. `SessionsList` (paginated, click-through to detail), `SessionDetailSheet` (full activity stream), `WeeklyHeatmap` (7×24 grid). API endpoints `sessions`, `session`, `session-activity`, `sessions-weekly`.
6. ✅ **Phase 6** — CRM → Umami event push. `trackEvent` calls wired into `createInterest` (`interest-created`), `updateInterestStage` (`interest-stage-changed`), `setInterestOutcome` (`interest-outcome-set`).
7. ✅ **Phase 7** — Cross-cutting placements. `email-sent` (in `performSend`), `eoi-signed` (in `handleDocumentCompleted`). Remaining placements (inquiry / berth detail attribution cards) defer until UI surfaces them.
8. ✅ **Phase 4c** — Tracked redirect links. `tracked_links` + `tracked_link_clicks` tables + `/q/[slug]` redirect endpoint + `createTrackedLink` / `buildTrackedUrl` service helpers. Email-composer integration deferred to UI follow-up.
9. **Phase 3 + Phase 5 — DEFERRED to the end.** Events tab is empty until marketing-site `umami.track()` calls land (Phase 4a, deferred). Funnels save for the end per user direction — pageview-only marketing funnel is the v1; richer event-based funnels come later.
10. **Phase 4a + cross-system funnels** — when there's appetite for marketing-site repo changes, unlock Events tab + cross-system funnels.
**Total scope: ~22-28h** with library adoptions, of which ~13-15h is the high-priority Phases 1 + 1b + 4b + 2 + 6 that ship first.
Total scope: ~30-40h end-to-end for the full flesh-out.
---
## 6. What stays in Umami vs. mirrored in CRM
| In CRM (mirror) | In Umami only (deep-link) |
| ----------------------------------------------------------- | ----------------------------------------------------------------- |
| Overview / KPI tiles / trend chart | Replays (paid) |
| Sessions list + detail | Retention (low signal) |
| Top pages / referrers / countries / browsers / OS / devices | Saved Boards (admin power-user) |
| Events + per-event drill | Pixels/Links admin CRUD (use Umami for setup; render data in CRM) |
| Funnels + UTM + Journeys | Performance / Web Vitals |
| World heatmap | Cohorts / Segments (defer until use case emerges) |
| Email open tracking | Multi-website CRUD |
Every page header in the CRM analytics surface gets a small "View in Umami →" outbound link in the corner for power users who want the full feature surface.
---
## 7. Open questions for the user before implementation
1. **Marketing site repo access**: Phase 4a (umami.track calls), Phase 4 (umami.identify for client linkage), and Phase 7 (passing sessionId to inquiry intake) all require changes there. Confirm whoever owns the marketing site is in the loop.
2. **Pixel hybrid vs Umami-only**: do you want per-recipient open tracking (hybrid) or just aggregate (Umami-only)? Recommended hybrid above; switch to Umami-only if the engineering cost isn't worth it.
3. **Funnel definitions**: who defines the canonical funnels? Suggest admins set them up via a CRM-side admin page that POSTs to Umami's `/api/reports/funnel`, with the most important funnels (inquiry, email-conversion) seeded as defaults at install time.
4. **Privacy / GDPR**: email pixel tracking + `umami.identify({email})` linkage both touch PII. Confirm consent model — likely already handled by the marketing-site cookie banner, but the email pixel needs explicit opt-out handling (e.g. don't fire pixel if the recipient is in a do-not-track list).
Reference in New Issue View Git Blame Copy Permalink