feat(analytics): Umami website-analytics suite — world map, realtime, sessions, heatmap, pixel tracking, tracked links
Adds the read-side Umami integration queued in last week's website-analytics plan (Phases 1–6 of `docs/website-analytics-flesh-out-plan.md`): - Realtime panel polls Umami at 5s intervals; world map renders visitor origins via echarts + `public/world-map/echarts-world.json` topo. - Sessions list + session-detail-sheet drill-down (per-session event timeline pulled from `/api/v1/website-analytics`). - Weekly heatmap (day-of-week × hour-of-day) for engagement timing. - Metric-detail pages under `/[portSlug]/website-analytics/[metric]` for pageviews / referrers / events deep-dives. - Email-pixel write path: `/api/public/email-pixel/[sendId]` 1×1 GIF beacon backed by `email_open_tracking` (migration 0076); resolves inline on render in inbox. - Tracked-link redirect: `/q/[slug]` routes through `tracked_links` (migration 0077) and forwards to the canonical destination after logging the click. - Dashboard `website-glance-tile` now reads from the live Umami service instead of placeholder data. Deps: `@umami/node`, `echarts`, `echarts-for-react`, `@types/geojson`, `@types/topojson-client`. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
189
docs/umami-api-capabilities.md
Normal file
189
docs/umami-api-capabilities.md
Normal file
@@ -0,0 +1,189 @@
|
||||
# Umami v2 / v3 API capabilities — reference for flesh-out planning
|
||||
|
||||
**Verified against:** analytics.portnimara.com (Umami v3.1.0), 2026-05-19.
|
||||
**Auth:** username/password → JWT via `POST /api/auth/login`, Bearer on every request, 1h TTL (we cache 55min).
|
||||
**Companion code:** `src/lib/services/umami.service.ts` (currently wraps stats/pageviews/metrics/active).
|
||||
|
||||
Endpoints below are listed by topic area, with what we currently use, what's available but unused, and where it could plug into the CRM.
|
||||
|
||||
---
|
||||
|
||||
## 1. Stats & traffic snapshots — `/api/websites/:id/stats`
|
||||
|
||||
**Currently used.** Returns the flat aggregate over the requested window plus a `comparison` block for the prior window of equal length.
|
||||
|
||||
```json
|
||||
{
|
||||
"pageviews": 2081, "visitors": 726, "visits": 872,
|
||||
"bounces": 457, "totaltime": 109519,
|
||||
"comparison": { "pageviews": 1935, "visitors": 642, ... }
|
||||
}
|
||||
```
|
||||
|
||||
**Unused fields we could surface:**
|
||||
|
||||
- `totaltime` — total seconds on site → derive avg session time (`totaltime / visits`).
|
||||
- `bounces / visits` → bounce-rate KPI.
|
||||
- Period-over-period deltas (already wired for trend arrows, but the _full_ comparison object has more we could use for a "what changed since last period" panel).
|
||||
|
||||
**Filters supported** (per Umami docs, mostly untested by us): `url`, `referrer`, `title`, `query`, `event`, `host`, `os`, `browser`, `device`, `country`, `region`, `city` — meaning every stats call can be sliced. **Big unlock:** show stats for a specific landing-page URL on the berth detail (e.g. `/berths/A12` stats), or filter by referrer to see which channels drove signed EOIs.
|
||||
|
||||
---
|
||||
|
||||
## 2. Time-series — `/api/websites/:id/pageviews`
|
||||
|
||||
**Currently used** for the trend chart. Returns `{pageviews: [{x, y}], sessions?: [{x, y}]}` (sessions only when `compare` is requested).
|
||||
|
||||
**Parameters:** `startAt`, `endAt`, `unit` (`year|month|day|hour`), `timezone`, `compare` (untapped), `filters` (untapped).
|
||||
|
||||
**Unused:** `compare=prev` gives the same series for the previous period — could power a dual-line "vs last period" overlay on the chart.
|
||||
|
||||
---
|
||||
|
||||
## 3. Top-N metrics — `/api/websites/:id/metrics`
|
||||
|
||||
**Currently used** for Top Pages / Referrers / Countries (limit 10). Returns `[{x, y}]`.
|
||||
|
||||
**Available `type` values** (we surface 4, Umami offers 17):
|
||||
|
||||
| Type | What it returns | CRM use case |
|
||||
| --------------------------- | -------------------------- | --------------------------------------------------------- |
|
||||
| `path` | Top URLs | ✅ Already shown (we mis-typed as `url`, now fixed) |
|
||||
| `referrer` | Top referring sites | ✅ Already shown |
|
||||
| `country` | Visitors by country | ✅ Already shown |
|
||||
| `browser` / `os` / `device` | Tech breakdown | Not surfaced — useful for "is mobile traffic converting?" |
|
||||
| `region` / `city` | Geographic drill-down | Strong fit for marina marketing |
|
||||
| `language` | Visitor browser language | Could feed i18n decisions |
|
||||
| `screen` | Resolution | Low value |
|
||||
| `event` | Top custom events | Big unlock — see §6 below |
|
||||
| `tag` | Event tags | Same |
|
||||
| `query` | Top URL query strings | UTM-debug surface |
|
||||
| `entry` / `exit` | First/last page in session | Funnel analysis |
|
||||
| `title` | Top page titles (vs paths) | Better labels for non-slug URLs |
|
||||
| `hostname` | Multi-domain sites | Probably N/A |
|
||||
| `distinctId` | Custom user identifiers | If we ever pipe CRM user IDs into Umami |
|
||||
|
||||
---
|
||||
|
||||
## 4. Live visitors — `/api/websites/:id/active`
|
||||
|
||||
**Currently used** for the green-dot "N active right now" indicator. Returns `{visitors: number}` (last-5-min count).
|
||||
|
||||
**Alternative for richer realtime:** `/api/realtime/:websiteId` (live realtime feed) returns far more — current top URLs being viewed, current top countries, recent event stream, a 30-minute time-series, totals, plus a `timestamp` you can poll against. We could surface a "live" panel on the dashboard showing the most-viewed pages right now.
|
||||
|
||||
---
|
||||
|
||||
## 5. Sessions API — `/api/websites/:id/sessions/*`
|
||||
|
||||
**Not currently used.** Multiple endpoints worth integrating:
|
||||
|
||||
- `GET /sessions` — list every session in a range with full device/geo/visits/views columns. Pageable. Could power a "recent visitors" surface — see who's browsing the berth detail pages right now.
|
||||
- `GET /sessions/stats` — summary aggregate (pageviews, visitors, visits, countries, events) keyed by session.
|
||||
- `GET /sessions/:sessionId` — drill into a single session: device, OS, browser, country, subdivision, city, screen, language, firstAt, lastAt, visits, views, events, totaltime.
|
||||
- `GET /sessions/:sessionId/activity` — full event timeline for one session (urlPath, eventName, referrerDomain, timestamps).
|
||||
- `GET /sessions/:sessionId/properties` — custom session properties (email, name, etc. — if Umami's `identify()` is called from the marketing site).
|
||||
- `GET /session-data/properties` + `/session-data/values` — aggregate custom session properties.
|
||||
- `GET /sessions/weekly` — heatmap of session count by hour-of-week. Direct fit for an "engagement heatmap" widget.
|
||||
|
||||
**Big unlock:** if marketing site calls `umami.identify({email})` after EOI form submit, sessions can be linked back to a specific client. We could then show "this client's website journey" on their CRM detail page.
|
||||
|
||||
---
|
||||
|
||||
## 6. Events API — `/api/websites/:id/events/*`
|
||||
|
||||
**Not currently used.** Umami auto-tracks pageviews; custom events are fired explicitly (e.g. button clicks, form submits, video plays). Endpoints:
|
||||
|
||||
- `GET /events` — list custom events in a range.
|
||||
- `GET /events/stats` — totals.
|
||||
- `GET /events/series` — time-series per event.
|
||||
- `GET /event-data/*` — aggregate over event payload properties.
|
||||
|
||||
**High-leverage CRM use cases:**
|
||||
|
||||
- Fire an event on the marketing site when someone clicks "Inquire about berth A12" → CRM Activity feed shows it in real-time on the inquiry record.
|
||||
- Fire an event when someone downloads a brochure → see which brochures convert.
|
||||
- Fire an event on EOI form-step completions → drop-off funnel analysis.
|
||||
|
||||
We'd need to add `umami.track('event-name', {payload})` calls on the marketing site (~1-2h work there) and a new admin surface to define/view these events.
|
||||
|
||||
---
|
||||
|
||||
## 7. Reports API — `/api/reports/*`
|
||||
|
||||
**Not currently used.** Umami's "saved reports" system. Endpoints:
|
||||
|
||||
- `GET /reports` + `GET /reports/:id` — list / retrieve saved reports.
|
||||
- `POST /reports/insights` — slice-and-dice with arbitrary filters/dimensions.
|
||||
- `POST /reports/funnel` — multi-step conversion analysis.
|
||||
- `POST /reports/retention` — cohort retention over time.
|
||||
- `POST /reports/utm` — UTM-tagged campaign performance.
|
||||
- `POST /reports/journey` — most common navigation paths.
|
||||
- `POST /reports/goals` — pageview/event-goal completion tracking.
|
||||
- `POST /reports/revenue` — revenue attribution (if we fire `purchase` events with amount).
|
||||
- `POST /reports/attribution` — first/last-click attribution modelling.
|
||||
|
||||
**Best fits for the CRM:**
|
||||
|
||||
- **Funnel report** for the EOI flow: `/berths → /berths/A12 → /inquire?berth=A12 → form submit → CRM EOI signed`. Surface drop-off percentages on the Pulse-style dashboard.
|
||||
- **Journey report** to see "what paths do visitors take before signing an EOI?" — informs marketing-site IA.
|
||||
- **UTM report** to plumb campaign attribution into the lead-source breakdown (currently CRM-side; could be cross-validated against marketing's UTM-tagged traffic).
|
||||
- **Attribution report** to give Pipeline-by-Source a "first-click vs last-click" toggle.
|
||||
|
||||
---
|
||||
|
||||
## 8. Send events from CRM → Umami — `/api/send`
|
||||
|
||||
**Not currently used.** The collect endpoint accepts page hits + custom events from any client. CRM doesn't currently push events, but we could:
|
||||
|
||||
- Fire `umami.track('signed-eoi', {berth: 'A12', deal_value: 50000})` from the CRM after EOI completion — closes the loop between marketing-site funnel and CRM outcome.
|
||||
- Fire `umami.track('contract-signed')`, `umami.track('deposit-received')` — full funnel visible in Umami without leaving it.
|
||||
|
||||
---
|
||||
|
||||
## 9. Multi-website + team admin — `/api/websites`, `/api/teams`, `/api/users`
|
||||
|
||||
**Not currently used.** We hard-code a single `umami_website_id` per port. Useful if a port runs multiple sites (e.g. main marina + residential subdomain): admin UI could list-and-pick from the configured Umami instance's websites instead of requiring manual ID copy-paste. Same for team membership.
|
||||
|
||||
---
|
||||
|
||||
## Prioritized opportunity list
|
||||
|
||||
Ranked by leverage-vs-effort, assuming the v3.1.0 fix in this commit is the baseline:
|
||||
|
||||
1. **Avg session time + bounce rate KPI tiles** (~20 min) — already in the `/stats` response, just need new tiles.
|
||||
2. **`compare=prev` overlay on the pageviews trend chart** (~30 min) — dual-line "vs last period" surface.
|
||||
3. **Country choropleth heatmap** (~4-6h) — already queued in Bucket 3 of the UAT findings doc as "World-map heatmap of Umami visitor origins."
|
||||
4. **Surface top browsers / OS / devices** (~30 min) — additional `TopList` columns; pure UI work.
|
||||
5. **Fire CRM-side events back into Umami** (~2-3h marketing-site + CRM hook) — closes the funnel between marketing and outcomes.
|
||||
6. **EOI funnel via `/api/reports/funnel`** (~3-4h) — drop-off analysis from berth view → inquiry → signed EOI.
|
||||
7. **Identify visitors → link sessions to clients** (~4-6h spread across marketing site + CRM detail surfaces) — biggest unlock but needs marketing-site changes.
|
||||
8. **Sessions-list "recent visitors" panel** (~2-3h) — see who's browsing right now, drill into individual sessions.
|
||||
9. **Saved-reports admin surface** (~6-10h) — let admins create + share Umami reports without leaving the CRM. Bigger product surface; defer until #1-#5 land.
|
||||
|
||||
---
|
||||
|
||||
## Service-layer additions needed to support the above
|
||||
|
||||
`src/lib/services/umami.service.ts` currently exports: `getStats`, `getPageviewsSeries`, `getMetric`, `getActiveVisitors`, `testConnection`. To unlock the opportunities above, add:
|
||||
|
||||
- `getSessions(portId, range, opts)` → `/sessions` (paged)
|
||||
- `getSession(portId, sessionId)` → single-session drill-in
|
||||
- `getSessionActivity(portId, sessionId, range)` → event timeline
|
||||
- `getSessionsWeekly(portId, range)` → heatmap source
|
||||
- `getEvents(portId, range)` + `getEventsStats(portId, range)` + `getEventsSeries(portId, range, eventName, unit)` → custom events
|
||||
- `getRealtime(portId, range)` → `/api/realtime/:id` for the live panel
|
||||
- `getReport(portId, reportType, body)` → POST wrappers for funnel/retention/journey/utm/goals/revenue/attribution
|
||||
- `trackEvent(portId, name, payload)` → POST to `/api/send` for CRM → Umami event emission
|
||||
|
||||
Each is a thin wrapper around the existing `umamiFetch` (or a new `umamiPost` variant for the reports endpoints). The auth + JWT cache + retry logic already in place handles them all.
|
||||
|
||||
---
|
||||
|
||||
## Known gotchas (verified against v3.1.0)
|
||||
|
||||
- Metric `type=url` returns 400 — use `type=path` (handled in our code via back-compat alias).
|
||||
- `/api/websites/:id/pageviews` returns `sessions` only when `compare` is in the query string — keep `.sessions` optional in TS types.
|
||||
- Stats response is **flat** (`pageviews: number`), not nested (`pageviews: {value, prev}`). The v1 nested shape isn't in v2/v3.
|
||||
- `/api/auth/login` returns a JWT with no `expires_in` field — we assume 1h and refresh proactively at 55min.
|
||||
- Visiting `/api` in a browser returns nothing — base path has no GET handler. Use `/api/heartbeat` to check liveness.
|
||||
- Filters are passed as query params (e.g. `&country=DE`), NOT as a JSON `filters` body, per actual API behaviour (docs occasionally show JSON which doesn't work for GET endpoints).
|
||||
428
docs/website-analytics-flesh-out-plan.md
Normal file
428
docs/website-analytics-flesh-out-plan.md
Normal file
@@ -0,0 +1,428 @@
|
||||
# Website Analytics — flesh-out plan
|
||||
|
||||
**Goal:** rebuild `/{portSlug}/website-analytics` so it feels like a polished native CRM panel that _mirrors_ Umami's idiom rather than reading as a stripped-down embed. Keep a "View in Umami →" deep-link in the header for power users; render most data in-app via the API. Also extend usage into adjacent CRM surfaces (dashboard tiles, inquiry detail, email open-tracking) so Umami stops being "the analytics page" and becomes a cross-cutting data layer.
|
||||
|
||||
**Inputs to this plan:**
|
||||
|
||||
1. Live API capabilities reference — `docs/umami-api-capabilities.md` (verified empirically against v3.1.0 on analytics.portnimara.com).
|
||||
2. Live UI tour via Playwright — screenshots `umami-tour-1-overview.png` through `umami-tour-9-compare.png` (10 surfaces captured).
|
||||
3. Pixel-tracking probe — confirmed the `/p/<slug>` and `/q/<slug>` endpoints + their UI creation forms.
|
||||
|
||||
---
|
||||
|
||||
## 1. What Umami's UI actually does — design patterns to mirror
|
||||
|
||||
Tour findings (from 17 sub-pages + 4 team pages):
|
||||
|
||||
| Surface | Visual idiom | Adopt for CRM? |
|
||||
| ------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ----------------------------------------------------------------------------------------------------------------------------- |
|
||||
| **Overview** | 5-tile KPI row (Visitors / Visits / Views / Bounce rate / Visit duration) — each tile shows headline number + colored arrow chip (green ↑ 58% / red ↓ 39%) + percentage delta. Single stacked bar chart below for traffic time-series (visitors stacked over visits, dual-shade blue). Filter pill + date-range nav top-right. | **Yes** — already mostly there, missing the bounce-rate + visit-duration tiles. |
|
||||
| **Events** | List of custom event names with per-event count + time-series spark. | **Yes** — needs marketing-site event firing first (Phase 4). |
|
||||
| **Sessions** | Dense table: avatar + per-session row showing Visits / Views / Events / Location (flag + city, country) / Browser icon / OS icon / Device icon / Last seen. Tabs for Activity vs Properties (custom session props). | **Yes** — high-leverage; lets reps see _who_ is browsing right now. |
|
||||
| **Realtime** | 4 stat tiles (Views/Visitors/Events/Countries) + auto-refreshing line chart of last 30 min. | **Yes** — already partial via the glance tile. |
|
||||
| **Performance** | Likely page-speed / Core Web Vitals. | Skip — not relevant to marina sales. |
|
||||
| **Compare** | Pick two date ranges side-by-side. | **Partial** — single `compare=prev` overlay on the existing trend chart suffices. |
|
||||
| **Breakdown** | Pivot table view across dimensions. | Skip in v1; expose via Reports later. |
|
||||
| **Goals** | Define event/page-view goals, see completion rate over time. | **Yes** — defer to Phase 5. |
|
||||
| **Funnels** | Multi-step conversion funnel (e.g. /berths → /berths/A12 → /inquire → submit). | **Yes** — Phase 5; high-value for inquiry conversion. |
|
||||
| **Journeys** | Most common navigation paths (Sankey-like). | **Maybe** — defer; nice-to-have. |
|
||||
| **Retention** | Cohort retention grid. | Skip — wrong fit for one-and-done marina inquiry traffic. |
|
||||
| **Replays** | Session replay (likely paid). | Skip — unavailable on our tier. |
|
||||
| **Segments / Cohorts** | Saved filters / user groups. | Skip in v1. |
|
||||
| **UTM** | Campaign attribution by UTM params. | **Yes** — Phase 5 for paid-campaign tracking. |
|
||||
| **Revenue** | Revenue attribution. | Skip — would require firing `purchase` events from CRM after EOI close (consider Phase 6 if leadership wants funnel→revenue). |
|
||||
| **Attribution** | First/last-click attribution model. | **Maybe** — defer. |
|
||||
| **Team-Boards / Websites / Links / Pixels** | Account admin surfaces. | **Pixels + Links: YES — see Phase 4.** Boards/Websites stay in Umami. |
|
||||
|
||||
### Visual specifics worth copying
|
||||
|
||||
- **KPI tile design**: large bold number, label above in muted-grey, arrow + percentage delta below in a colored chip (green-bg for positive, red-bg for negative, fixed-width for alignment). Our `KPITile` already does the right shape — we just need to add the missing two metrics.
|
||||
- **Stacked bar chart for traffic**: dual-shade single bar (visitors as light-blue base, views stacked dark-blue on top). Reads cleaner than two overlapping lines.
|
||||
- **Location rendering**: flag emoji + "City, Country" inline. Use `getCountryName()` + a flag library (twemoji or unicode regional indicators).
|
||||
- **Browser/OS/Device icons**: small colored brand glyphs inline. Use `simple-icons` or `lucide` equivalents.
|
||||
- **Filter chip + date nav**: `<` `>` arrows step through the date range; dropdown opens to preset list. Adopt the same pattern on our shell — currently we only have presets, no step-arrows.
|
||||
|
||||
---
|
||||
|
||||
## 2. Phased build plan
|
||||
|
||||
### Phase 1 — Fill out the Overview tiles & chart (~3-4h)
|
||||
|
||||
Quick wins that close visual parity with Umami's Overview:
|
||||
|
||||
| Task | File | Effort |
|
||||
| -------------------------------------------------------------- | ------------------------------------------------------------------------ | ----------------------------------------------------------------------------- |
|
||||
| Add **Bounce rate** KPI tile | `website-analytics-shell.tsx` | derive `bounces / visits * 100`; service field already there |
|
||||
| Add **Avg visit duration** KPI tile | `website-analytics-shell.tsx` | derive `totaltime / visits` formatted as `Xm Ys`; service field already there |
|
||||
| Add **`<` `>` date-step arrows** on the date-range chip | `date-range-picker.tsx` | step the current preset by one window (today→yesterday, 7d→prior-7d, etc.) |
|
||||
| Convert pageviews trend to **stacked bar** (visitors vs views) | `pageviews-chart.tsx` | recharts `BarChart` stacked, light/dark blue |
|
||||
| Add **`compare=prev` overlay toggle** on the trend chart | `pageviews-chart.tsx` + service `getPageviewsSeries` | optional "vs prior period" series rendered as dashed line |
|
||||
| Add **Top browsers / OS / devices** ranked-list cards | new `<TopList>` consumers; service already exposes via `getMetric(type)` | mirror Top Pages/Referrers/Countries layout |
|
||||
| **World choropleth heatmap** card (already queued separately) | new `visitor-world-map.tsx` (Natural Earth topojson + react-simple-maps) | ~4-6h on its own |
|
||||
|
||||
**Cumulative result:** Overview surface reads at ~80% parity with Umami's Overview.
|
||||
|
||||
---
|
||||
|
||||
### Phase 2 — Sessions surface (~4-5h)
|
||||
|
||||
New `/website-analytics/sessions` tab + supporting service wrappers:
|
||||
|
||||
| Task | File | Effort |
|
||||
| ------------------------------------------------------------------------------------------------------------------------ | ------------------ | ------- |
|
||||
| Service: `getSessions(portId, range, opts)` → `/api/websites/:id/sessions` (paged) | `umami.service.ts` | ~30 min |
|
||||
| Service: `getSession(portId, sessionId)` → single-session detail | `umami.service.ts` | ~15 min |
|
||||
| Service: `getSessionActivity(portId, sessionId, range)` → event timeline | `umami.service.ts` | ~15 min |
|
||||
| Service: `getSessionsWeekly(portId, range)` → hour-of-week heatmap | `umami.service.ts` | ~15 min |
|
||||
| API route: `/api/v1/website-analytics?metric=sessions[&sessionId=...]` | route.ts | ~30 min |
|
||||
| UI: `sessions-table.tsx` — dense rows mirroring Umami (avatar + location flag + browser/OS/device icons + Last seen) | new component | ~2h |
|
||||
| UI: `session-detail-sheet.tsx` — right-side Sheet drawer showing the session's full event timeline when a row is clicked | new component | ~1h |
|
||||
| UI: `weekly-heatmap-card.tsx` — 7×24 grid colour-scaled by session count, hover for tooltip | new component | ~1h |
|
||||
|
||||
**Unlock:** rep can see "who is currently browsing right now, where from, on what device, what they're looking at" — directly actionable for sales follow-up.
|
||||
|
||||
---
|
||||
|
||||
### Phase 3 — Events surface (~3-4h, BLOCKED on Phase 4a)
|
||||
|
||||
| Task | File | Effort |
|
||||
| -------------------------------------------------------------------------------------------- | ------------------ | ------- |
|
||||
| Service: `getEvents(portId, range, opts)` → `/events` paged list | `umami.service.ts` | ~30 min |
|
||||
| Service: `getEventsStats(portId, range)` → totals | `umami.service.ts` | ~15 min |
|
||||
| Service: `getEventsSeries(portId, range, eventName, unit)` → per-event time-series | `umami.service.ts` | ~15 min |
|
||||
| API route addition | route.ts | ~30 min |
|
||||
| UI: `events-tab.tsx` — list of event names with per-event count + spark + drill-in | new component | ~1.5h |
|
||||
| UI: `event-detail-sheet.tsx` — single event's time-series chart + filter by payload property | new component | ~1h |
|
||||
|
||||
**Dependency:** the marketing site must fire `umami.track(name, payload)` calls (Phase 4a). Without this, Events tab is empty.
|
||||
|
||||
---
|
||||
|
||||
### Phase 4 — Pixel tracking + link tracking + marketing-site event push
|
||||
|
||||
**Phase 4a — Marketing-site event tracking (~2-3h on marketing repo)**
|
||||
|
||||
Add `umami.track()` calls in the marketing site:
|
||||
|
||||
- `inquiry-submitted` with `{berth, source}` payload — fires on EOI form submit
|
||||
- `brochure-download` with `{brochureId}` — fires on brochure download
|
||||
- `berth-detail-viewed` with `{berthId, mooring}` — fires on `/berths/[mooring]` page view
|
||||
- `phone-revealed` / `email-revealed` — fires when contact details are exposed
|
||||
|
||||
These light up the Events tab + enable funnel analysis in Phase 5.
|
||||
|
||||
**Phase 4b — Pixel-based email open tracking (~3-4h CRM-side)**
|
||||
|
||||
Probe finding: Umami exposes pixel URLs at `https://analytics.portnimara.com/p/<slug>` — fetching the URL records an event. Use case: embed in HTML emails as a 1x1 image.
|
||||
|
||||
**Two architecture options:**
|
||||
|
||||
**Option A — One Umami pixel per email type** (simple, low fidelity)
|
||||
|
||||
- Create a pixel manually in Umami for each templated email type (`portal-invite`, `eoi-sent`, `reservation-reminder`, etc.)
|
||||
- Embed the static pixel URL in each template
|
||||
- Pro: zero CRM-side code beyond template HTML. Open rates roll up in Umami by pixel.
|
||||
- Con: can't tell _which recipient_ opened — only aggregate counts per template.
|
||||
|
||||
**Option B — One Umami pixel + CRM-side per-send tracking endpoint** (richer, recommended)
|
||||
|
||||
- Build `GET /api/public/email-pixel/:sendId.gif` in our CRM that:
|
||||
1. Returns a 1×1 transparent GIF
|
||||
2. Records the open in `document_sends.opened_at` (already a table; per CLAUDE.md "send-from accounts" section)
|
||||
3. Optionally proxies the hit to Umami via `POST /api/send` with the email type + send id as event properties for cross-correlation
|
||||
- Embed `<img src="https://crm.portnimara.com/api/public/email-pixel/{sendId}.gif" width="1" height="1" />` in every templated email
|
||||
- Pro: per-recipient open tracking + open-time + CRM-attached. Funnels by email type via Umami too.
|
||||
- Con: needs the public endpoint + a schema column (or reuse `document_sends.opened_at`).
|
||||
|
||||
**Recommendation: ship Option B.** The CRM-side hook gives us per-deal attribution ("client X opened the EOI reminder twice but hasn't signed"), and Umami still gets the aggregate.
|
||||
|
||||
| Task | File | Effort |
|
||||
| ------------------------------------------------------------------------------------------------------------- | ---------------------------------------------- | ------- |
|
||||
| New endpoint `/api/public/email-pixel/[sendId]/route.ts` returning a 1×1 GIF + recording open | new route | ~1h |
|
||||
| Migration: add `opened_at`, `open_count`, `last_opened_user_agent` to `document_sends` if not present | drizzle migration | ~30 min |
|
||||
| Email template helper: inject the pixel HTML into every transactional template | `src/lib/email/render.ts` | ~30 min |
|
||||
| UI surface: on each `document_sends` row in the activity feed, show "Opened N times, last at X" badge | `email-activity-row.tsx` | ~1h |
|
||||
| Cross-post to Umami via `trackEvent('email-opened', {emailType, sendId})` so Umami funnel data includes opens | new `trackEvent` wrapper in `umami.service.ts` | ~30 min |
|
||||
| Privacy: respect `EMAIL_REDIRECT_TO` dev gate; don't fire pixels for redirected dev emails | ditto | ~15 min |
|
||||
|
||||
**Phase 4c — Tracked redirect links (~1.5h)**
|
||||
|
||||
Umami's `/q/<slug>` endpoint is a tracked redirect — records a click then 302s to the destination URL. Use for outbound CTAs:
|
||||
|
||||
- "View brochure" links in emails → wrap via Umami link → records click → opens brochure
|
||||
- "Schedule a viewing" buttons → wrap via Umami link → click attribution
|
||||
- Marketing-site CTAs → wrap → measure engagement
|
||||
|
||||
| Task | File | Effort |
|
||||
| ------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------- | ------- |
|
||||
| Service: `createTrackedLink(name, destinationUrl)` → POST to Umami's links endpoint via authenticated API | `umami.service.ts` | ~45 min |
|
||||
| Email template helper: `<trackedLink href="..." name="...">` JSX wrapper that auto-creates the Umami link on first render + caches the slug | `src/lib/email/components/` | ~45 min |
|
||||
|
||||
---
|
||||
|
||||
### Phase 5 — Reports surfaces (Funnels, UTM, Journeys) (~6-8h)
|
||||
|
||||
| Task | File | Effort |
|
||||
| ---------------------------------------------------------------------------------------------------------------------------------------------- | ------------------ | ------ |
|
||||
| Service: `getReport(reportType, body)` POST wrapper covering `/funnel`, `/journey`, `/utm`, `/goals`, `/retention`, `/revenue`, `/attribution` | `umami.service.ts` | ~1h |
|
||||
| UI: `/website-analytics/funnels` page — admin-configurable funnel definitions (steps as event names or URL paths), per-step drop-off chart | new page | ~3h |
|
||||
| UI: `/website-analytics/utm` page — UTM source/medium/campaign breakdown with click-through to attributed sessions | new page | ~2h |
|
||||
| UI: `/website-analytics/journeys` page — top navigation paths rendered as ranked list (skip Sankey for v1) | new page | ~1.5h |
|
||||
| Defer: Goals / Retention / Revenue / Attribution to v2 (low signal for marina sales) | | |
|
||||
|
||||
**High-leverage funnels to wire as defaults:**
|
||||
|
||||
- **Inquiry funnel**: `/` → `/berths` → `/berths/[mooring]` → `inquiry-submitted` event → CRM `eoi-signed` (cross-system!) → CRM `reservation-paid` (cross-system!)
|
||||
- **Email funnel**: `email-sent` → `email-opened` (pixel) → tracked-link click → CRM action
|
||||
|
||||
The cross-system funnels require Phase 4 to be live first.
|
||||
|
||||
---
|
||||
|
||||
### Phase 6 — CRM → Umami event push for outcome attribution (~2-3h)
|
||||
|
||||
Close the funnel from "marketing site click" → "CRM closed deal" by firing CRM-side events back into Umami via `POST /api/send`:
|
||||
|
||||
| Event | Fired by | Payload |
|
||||
| ---------------------- | -------------------------------------------- | --------------------------------------- |
|
||||
| `crm-inquiry-created` | `createInterest()` in `interests.service.ts` | `{interestId, source, leadCategory}` |
|
||||
| `crm-eoi-sent` | `generateAndSign()` after EOI dispatch | `{interestId, berth, pathway}` |
|
||||
| `crm-eoi-signed` | Documenso `DOCUMENT_COMPLETED` webhook | `{interestId, berth}` |
|
||||
| `crm-reservation-paid` | manual stage advance to `deposit_paid` | `{interestId, berth, amount, currency}` |
|
||||
| `crm-contract-signed` | manual stage advance to `contract` | `{interestId, berth, amount, currency}` |
|
||||
|
||||
| Task | File | Effort |
|
||||
| ----------------------------------------------------------------------------------------- | ------------------- | -------- |
|
||||
| Service: `trackEvent(name, payload, sessionId?)` → `POST /api/send` on the Umami instance | `umami.service.ts` | ~45 min |
|
||||
| Hook into the 5 service entry points above (one event per outcome milestone) | each service file | ~1.5h |
|
||||
| Audit log entry per event sent so we can verify Umami received it | `audit_logs` insert | included |
|
||||
|
||||
**Unlock:** Umami's Revenue + Attribution reports start showing CRM outcomes attributed to marketing-site channels — closes the leadership question "which traffic sources actually generate signed deals, not just leads?"
|
||||
|
||||
---
|
||||
|
||||
### Phase 7 — Cross-cutting CRM placements (~3-4h)
|
||||
|
||||
Beyond the dedicated `/website-analytics` page, surface Umami data inside CRM context:
|
||||
|
||||
| Placement | What | Effort |
|
||||
| ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------- | ------------------------------------ |
|
||||
| **Dashboard rail tile** (already shipped) — Pageviews + active now | already done in this session | — |
|
||||
| **Inquiry detail page** — "Source attribution" card showing the inquiry's UTM params, landing page, time-on-site, pages-viewed-before-submit. Pulls from `getSession(sessionId)` if the inquiry's create payload includes a session ID (requires marketing-site change to pass it). | new `inquiry-attribution-card.tsx` | ~1.5h + marketing-site change |
|
||||
| **Client detail page** — "Website activity" card: total sessions, pageviews, last-seen, top pages visited. Requires `umami.identify({email})` on marketing site to link sessions back to clients. | new `client-web-activity-card.tsx` | ~1.5h + marketing-site identify call |
|
||||
| **Berth detail page** — "Marketing demand" card: pageviews to `/berths/{mooring}` over time + referrer breakdown. Drives "this berth is being viewed but not inquired-about — flag for outreach." | new `berth-demand-card.tsx` | ~1h |
|
||||
| **Document send activity** — pixel opens per recipient (from Phase 4b) | inline on existing `document_sends` rows | included in 4b |
|
||||
|
||||
---
|
||||
|
||||
## 2b. Library adoptions (changes the plan materially)
|
||||
|
||||
Context7 lookup surfaced three official libraries that reshape the plan. **Adopt all three.**
|
||||
|
||||
### `@umami/api-client` — official read-side client
|
||||
|
||||
Covers every read endpoint we need including all the report types. Built-in filter support, login/JWT auth handled internally, `{ok, data}` discriminated union for clean error handling.
|
||||
|
||||
**Replaces:** ~60-70% of our current `umami.service.ts` (drop `umamiFetch`, JWT cache, decrypt boilerplate; keep thin wrappers with existing signatures so consumers don't change).
|
||||
|
||||
**One-time refactor (~2h):**
|
||||
|
||||
```ts
|
||||
const clientByPort = new Map<string, UmamiApiClient>();
|
||||
|
||||
async function getClient(portId: string): Promise<UmamiApiClient | null> {
|
||||
if (clientByPort.has(portId)) return clientByPort.get(portId)!;
|
||||
const cfg = await loadUmamiConfig(portId);
|
||||
if (!cfg) return null;
|
||||
const client = new UmamiApiClient({
|
||||
apiEndpoint: `${cfg.apiUrl}/api`,
|
||||
apiKey: cfg.apiToken ?? undefined,
|
||||
});
|
||||
if (!cfg.apiToken && cfg.username && cfg.password) await client.login(cfg.username, cfg.password);
|
||||
clientByPort.set(portId, client);
|
||||
return client;
|
||||
}
|
||||
|
||||
export async function getStats(portId: string, range: DateRange) {
|
||||
const client = await getClient(portId);
|
||||
if (!client) return null;
|
||||
const { from, to } = rangeToBounds(range);
|
||||
const result = await client.getWebsiteStats(WEBSITE_ID, {
|
||||
startAt: from.getTime(),
|
||||
endAt: to.getTime(),
|
||||
});
|
||||
return result.ok ? result.data : null;
|
||||
}
|
||||
```
|
||||
|
||||
Same pattern for `getPageviewsSeries`, `getMetric`, `getActiveVisitors`, plus new ones from the SDK: `getRealtime`, `getWebsiteSessionStats`, `runFunnelReport`, `runJourneyReport`, etc.
|
||||
|
||||
### `@umami/node` — official write-side SDK
|
||||
|
||||
For Phase 6 (CRM → Umami push) and Phase 4b cross-post:
|
||||
|
||||
```ts
|
||||
const umami = new Umami({ websiteId, hostUrl });
|
||||
await umami.track({
|
||||
url: '/crm/eoi-signed',
|
||||
name: 'crm-eoi-signed',
|
||||
data: { interestId, berth, dealValue },
|
||||
});
|
||||
await umami.identify({ sessionId, email, interestId });
|
||||
```
|
||||
|
||||
**Replaces:** the planned hand-rolled `trackEvent()` wrapper. Single line per outcome milestone.
|
||||
|
||||
### `react-simple-maps` — for the world heatmap (Phase 1b)
|
||||
|
||||
Declarative SVG choropleth on d3-geo + topojson-client. SSR-safe. Use `topojson/world-atlas` (110m resolution ~30KB) cached in `public/`. Bundle ~30-50KB + topojson 30-100KB.
|
||||
|
||||
```jsx
|
||||
<ComposableMap projection="geoMercator">
|
||||
<Geographies geography="/world-110m.json">
|
||||
{({ geographies }) =>
|
||||
geographies.map((geo) => (
|
||||
<Geography
|
||||
key={geo.rsmKey}
|
||||
geography={geo}
|
||||
fill={scaleByVisitorCount(visitorsByCountry[geo.properties.iso_a2] ?? 0)}
|
||||
onClick={() => onCountryClick(geo.properties.iso_a2)}
|
||||
/>
|
||||
))
|
||||
}
|
||||
</Geographies>
|
||||
</ComposableMap>
|
||||
```
|
||||
|
||||
**Chose this over visx/Nivo/Chart.js Geo:** visx is overkill for one map; Nivo + Chart.js force a different charting idiom (we use recharts everywhere); react-simple-maps' compose-primitives shape matches our recharts pattern.
|
||||
|
||||
### Net effect on phase efforts
|
||||
|
||||
| Phase | Original estimate | Revised after library adoption |
|
||||
| ---------------------------------------- | ----------------- | --------------------------------------------------------------- |
|
||||
| Service refactor (one-time) | — | **+2h** (one-time foundation; pays back across all phases) |
|
||||
| Phase 1 — Overview parity | 3-4h | 3-4h (unchanged; api-client makes the filter additions trivial) |
|
||||
| Phase 1b — World heatmap | 4-6h | 3-4h (library choice locked in) |
|
||||
| Phase 2 — Sessions | 4-5h | 3-4h (api-client has session methods built-in) |
|
||||
| Phase 3 — Events | 3-4h | 2-3h (api-client provides) |
|
||||
| Phase 4b — Pixel hybrid | 3-4h | 2.5-3h (cross-post is one line) |
|
||||
| Phase 5 — Reports (funnels/UTM/journeys) | 6-8h | 3-4h (every report method pre-wrapped) |
|
||||
| Phase 6 — CRM → Umami push | 2-3h | 1.5h (`@umami/node` handles transport) |
|
||||
|
||||
**Total scope drops from ~30-40h to ~22-28h** with these adoptions.
|
||||
|
||||
---
|
||||
|
||||
## 3. Service-layer additions consolidated
|
||||
|
||||
Add to `src/lib/services/umami.service.ts` (each is a thin wrapper around existing `umamiFetch` / new `umamiPost`):
|
||||
|
||||
```ts
|
||||
// Sessions (Phase 2)
|
||||
getSessions(portId, range, { page?, pageSize?, query? }) → /sessions
|
||||
getSession(portId, sessionId) → /sessions/:id
|
||||
getSessionActivity(portId, sessionId, range) → /sessions/:id/activity
|
||||
getSessionProperties(portId, sessionId) → /sessions/:id/properties
|
||||
getSessionsWeekly(portId, range, timezone) → /sessions/weekly
|
||||
|
||||
// Events (Phase 3)
|
||||
getEvents(portId, range, opts) → /events
|
||||
getEventsStats(portId, range) → /events/stats
|
||||
getEventsSeries(portId, range, eventName, unit) → /events/series
|
||||
getEventDataProperties(portId, range) → /event-data/properties
|
||||
|
||||
// Realtime (Phase 1)
|
||||
getRealtime(portId, range) → /api/realtime/:id (richer than /active)
|
||||
|
||||
// Reports (Phase 5)
|
||||
getReport(portId, reportType, body) → POST /api/reports/:type (funnel/journey/utm/goals/retention/revenue/attribution)
|
||||
|
||||
// CRM → Umami (Phase 6)
|
||||
trackEvent(portId, name, payload, sessionId?) → POST /api/send
|
||||
|
||||
// Links + Pixels admin (Phase 4)
|
||||
createTrackedLink(portId, name, destinationUrl) → POST team-level /links
|
||||
createTrackingPixel(portId, name) → POST team-level /pixels
|
||||
```
|
||||
|
||||
Plus a new `umamiPost(config, path, body)` helper alongside the existing `umamiFetch` since GET-only doesn't cover reports + send.
|
||||
|
||||
---
|
||||
|
||||
## 4. Pixel-tracking answer (the user's specific question)
|
||||
|
||||
**Q: Can we use Umami's pixel tracking for email opens?**
|
||||
|
||||
**A: Yes — and recommended in hybrid form.** Direct verification on the live instance:
|
||||
|
||||
- Pixel UI at `/teams/[teamId]/pixels` lets an admin create named pixels. Each gets an auto-generated slug.
|
||||
- The pixel URL is `https://analytics.portnimara.com/p/<slug>` — fetching it records an event (no auth required from the email client side; the slug is the credential).
|
||||
- Embedded as `<img src="..." width="1" height="1" />` in HTML emails, it fires when the email is rendered (Outlook/Apple Mail/etc.).
|
||||
- Standard caveats: Apple Mail privacy protection pre-fetches images server-side → opens may be over-counted for iOS users. Some recipients block images entirely → opens under-counted. Same caveats as every email tracking pixel ever.
|
||||
|
||||
**Recommended hybrid (Phase 4b above):** build a CRM-side pixel endpoint `/api/public/email-pixel/[sendId].gif` that:
|
||||
|
||||
- Returns the 1×1 GIF
|
||||
- Records `opened_at` in `document_sends`
|
||||
- Cross-posts the hit to Umami via `POST /api/send` so the Umami Events tab + funnels include opens
|
||||
|
||||
This way: per-recipient attribution in the CRM, aggregate roll-ups in Umami, single source of truth for both.
|
||||
|
||||
---
|
||||
|
||||
## 5. Effort summary + prioritization
|
||||
|
||||
| Phase | Scope | Effort | Priority |
|
||||
| ----- | ------------------------------------------------------------------------------------ | ---------------------- | ------------------------------------------------------------ |
|
||||
| 1 | Overview parity (KPI tiles, stacked-bar chart, date arrows, browser/OS/device cards) | ~3-4h | **High** — most visible polish, no dependencies |
|
||||
| 1b | World choropleth heatmap (already queued separately) | ~4-6h | **High** if leadership wants the visual |
|
||||
| 2 | Sessions surface (table + detail sheet + weekly heatmap) | ~4-5h | **High** — biggest "wow" + actionable |
|
||||
| 3 | Events surface | ~3-4h | **Medium** — blocked on 4a |
|
||||
| 4a | Marketing-site event tracking | ~2-3h (marketing repo) | **High** — unblocks 3 + 5 |
|
||||
| 4b | Pixel-based email open tracking (hybrid) | ~3-4h | **High** — direct ask + immediate value |
|
||||
| 4c | Tracked redirect links | ~1.5h | **Medium** |
|
||||
| 5 | Reports (Funnels, UTM, Journeys) | ~6-8h | **Medium** — depends on 4a being live |
|
||||
| 6 | CRM → Umami event push for outcome attribution | ~2-3h | **Medium-high** — needed to close marketing→outcome loop |
|
||||
| 7 | Cross-cutting placements (inquiry / client / berth detail cards) | ~3-4h | **Medium** — depends on `umami.identify()` on marketing site |
|
||||
|
||||
**Recommended build order (updated 2026-05-19 per user):**
|
||||
|
||||
1. ✅ **Service refactor** — Kept hand-rolled `umamiFetch` (the official `@umami/api-client` transitively pulls `next-basics` which requires React at module-import time, breaking SSR + tsx scripts). Adopted `@umami/node` for the write side.
|
||||
2. ✅ **Phase 1** — Overview parity (KPI tiles + browser/OS/device cards + date arrows + stacked-bar chart + `compare=prev` overlay)
|
||||
3. ✅ **Phase 1b** — World heatmap. **Switched from `react-simple-maps` to ECharts + `public/world-map/echarts-world.json`** — the `world-atlas/110m` topojson has antimeridian-crossing polygons (Russia/Fiji/Antarctica) that render a horizontal line through the equator regardless of projection. ECharts' world.json is pre-cleaned.
|
||||
4. ✅ **Phase 4b** — Pixel-based email open tracking. `document_send_opens` table + `/api/public/email-pixel/[sendId]` endpoint + `injectTrackingPixel` helper wired into `performSend`. Per-port kill switch `email_open_tracking_enabled` (admin UI on `/admin/website-analytics`). Cross-posts to Umami as `email-opened`.
|
||||
5. ✅ **Phase 2** — Sessions surface. `SessionsList` (paginated, click-through to detail), `SessionDetailSheet` (full activity stream), `WeeklyHeatmap` (7×24 grid). API endpoints `sessions`, `session`, `session-activity`, `sessions-weekly`.
|
||||
6. ✅ **Phase 6** — CRM → Umami event push. `trackEvent` calls wired into `createInterest` (`interest-created`), `updateInterestStage` (`interest-stage-changed`), `setInterestOutcome` (`interest-outcome-set`).
|
||||
7. ✅ **Phase 7** — Cross-cutting placements. `email-sent` (in `performSend`), `eoi-signed` (in `handleDocumentCompleted`). Remaining placements (inquiry / berth detail attribution cards) defer until UI surfaces them.
|
||||
8. ✅ **Phase 4c** — Tracked redirect links. `tracked_links` + `tracked_link_clicks` tables + `/q/[slug]` redirect endpoint + `createTrackedLink` / `buildTrackedUrl` service helpers. Email-composer integration deferred to UI follow-up.
|
||||
9. **Phase 3 + Phase 5 — DEFERRED to the end.** Events tab is empty until marketing-site `umami.track()` calls land (Phase 4a, deferred). Funnels save for the end per user direction — pageview-only marketing funnel is the v1; richer event-based funnels come later.
|
||||
10. **Phase 4a + cross-system funnels** — when there's appetite for marketing-site repo changes, unlock Events tab + cross-system funnels.
|
||||
|
||||
**Total scope: ~22-28h** with library adoptions, of which ~13-15h is the high-priority Phases 1 + 1b + 4b + 2 + 6 that ship first.
|
||||
|
||||
Total scope: ~30-40h end-to-end for the full flesh-out.
|
||||
|
||||
---
|
||||
|
||||
## 6. What stays in Umami vs. mirrored in CRM
|
||||
|
||||
| In CRM (mirror) | In Umami only (deep-link) |
|
||||
| ----------------------------------------------------------- | ----------------------------------------------------------------- |
|
||||
| Overview / KPI tiles / trend chart | Replays (paid) |
|
||||
| Sessions list + detail | Retention (low signal) |
|
||||
| Top pages / referrers / countries / browsers / OS / devices | Saved Boards (admin power-user) |
|
||||
| Events + per-event drill | Pixels/Links admin CRUD (use Umami for setup; render data in CRM) |
|
||||
| Funnels + UTM + Journeys | Performance / Web Vitals |
|
||||
| World heatmap | Cohorts / Segments (defer until use case emerges) |
|
||||
| Email open tracking | Multi-website CRUD |
|
||||
|
||||
Every page header in the CRM analytics surface gets a small "View in Umami →" outbound link in the corner for power users who want the full feature surface.
|
||||
|
||||
---
|
||||
|
||||
## 7. Open questions for the user before implementation
|
||||
|
||||
1. **Marketing site repo access**: Phase 4a (umami.track calls), Phase 4 (umami.identify for client linkage), and Phase 7 (passing sessionId to inquiry intake) all require changes there. Confirm whoever owns the marketing site is in the loop.
|
||||
2. **Pixel hybrid vs Umami-only**: do you want per-recipient open tracking (hybrid) or just aggregate (Umami-only)? Recommended hybrid above; switch to Umami-only if the engineering cost isn't worth it.
|
||||
3. **Funnel definitions**: who defines the canonical funnels? Suggest admins set them up via a CRM-side admin page that POSTs to Umami's `/api/reports/funnel`, with the most important funnels (inquiry, email-conversion) seeded as defaults at install time.
|
||||
4. **Privacy / GDPR**: email pixel tracking + `umami.identify({email})` linkage both touch PII. Confirm consent model — likely already handled by the marketing-site cookie banner, but the email pixel needs explicit opt-out handling (e.g. don't fire pixel if the recipient is in a do-not-track list).
|
||||
Reference in New Issue
Block a user