pn-new-crm/PROGRESS.md

Port Nimara CRM - Project Progress

Last updated: 2026-03-26 Repo: https://code.letsbe.solutions/letsbe/pn-new-crm Domain: pn.letsbe.solutions Stack: Next.js 15 + TypeScript + Tailwind + Drizzle ORM + PostgreSQL + Redis + BullMQ + MinIO + Socket.io

---

What's Been Built (Layers 0-4 Complete)

Layer 0: Foundation (DONE)

• Next.js 15 project scaffold with TypeScript strict mode
• Tailwind CSS + shadcn/ui component library (full set in src/components/ui/)
• PostgreSQL via Drizzle ORM - full schema across 12 schema files
  • berths.ts, clients.ts, documents.ts, email.ts, financial.ts, interests.ts, operations.ts, ports.ts, relations.ts, system.ts, users.ts + index.ts
• Better Auth integration with multi-port middleware (src/middleware.ts)
• Redis connection (src/lib/redis.ts)
• BullMQ queue system with 8 workers: document-signing.ts, email-sync.ts, email.ts, import.ts, maintenance.ts, notifications.ts, reports.ts, webhooks.ts
• MinIO file storage service (src/lib/services/storage.ts, files.ts)
• Socket.io real-time server (src/lib/socket/server.ts, events.ts)
• Docker setup: Dockerfile (multi-stage app), Dockerfile.worker, Dockerfile.dev, docker-compose.yml, docker-compose.dev.yml, docker-compose.prod.yml
• Dashboard layout shell with sidebar, port switcher, navigation (src/app/(dashboard)/layout.tsx)
• Auth pages: login, reset-password, set-password
• Seed script (src/lib/db/seed.ts)
• ESLint + Prettier + Husky + lint-staged
• Health check endpoint (/api/health)
• Rate limiting (src/lib/rate-limit.ts)
• Encryption utilities (src/lib/utils/encryption.ts)
• Zod validators for all entities (17 validator files in src/lib/validators/)

Layer 1: Core CRUD (DONE)

• Client Management - Full CRUD, contacts, relationships, notes, tags, duplicate detection
  • Pages: list (/clients), detail (/clients/[clientId])
  • API: full REST at /api/v1/clients/... (CRUD, contacts, notes, relationships, tags, restore, export-pdf, options)
  • Service: clients.service.ts
  • Components: src/components/clients/
• Berth Management - Full CRUD, specs, tags, status management, waiting list, maintenance
  • Pages: list (/berths), detail (/berths/[berthId])
  • API: full REST at /api/v1/berths/... (CRUD, status, tags, waiting-list, maintenance, export-pdf, options)
  • Service: berths.service.ts, berth-rules-engine.ts
  • Components: src/components/berths/
• Auth Admin - User management, role builder, port management
  • Pages: admin/users, admin/roles, admin/ports, admin/onboarding
  • API: /api/v1/admin/users/, /api/v1/admin/roles/

Layer 2: Business Workflows (DONE)

• Interest/Pipeline Management - CRUD, pipeline stages, berth linking, recommendations, scoring, timeline
  • Pages: list (/interests), detail (/interests/[interestId])
  • API: full REST at /api/v1/interests/... (CRUD, stage, berth, notes, tags, recommendations, timeline, restore, export-pdf)
  • Services: interests.service.ts, interest-scoring.service.ts, recommendations.ts
  • Components: src/components/interests/
• Documents & EOI - Documenso integration, 3-party signing, webhook receiver, document templates
  • Pages: /documents
  • API: /api/v1/documents/..., /api/v1/document-templates/..., /api/webhooks/documenso/
  • Services: documents.service.ts, document-templates.service.ts, document-templates.ts, documenso-client.ts, documenso-webhook.ts, document-reminders.ts
  • Components: src/components/documents/
• Expenses & Invoices - CRUD, receipt scanner (AI), invoice generation, payment tracking, CSV/PDF export
  • Pages: /expenses, /expenses/[id], /expenses/scan, /invoices, /invoices/[id], /invoices/new
  • API: /api/v1/expenses/... (CRUD, scan-receipt, export CSV/PDF/parent-company), /api/v1/invoices/... (CRUD, generate-pdf, payment, send)
  • Services: expenses.ts, invoices.ts, expense-export.ts, receipt-scanner.ts
  • Components: src/components/expenses/, src/components/invoices/
• File Management - MinIO integration, folder management, upload/download/preview
  • API: /api/v1/files/... (CRUD, upload, download, preview, folders)
  • Components: src/components/files/
  • Store: src/stores/file-browser-store.ts

Layer 3: Operations (DONE)

• Email System - SMTP/IMAP, accounts, thread viewer, composer, AI drafts
  • Pages: /email
  • API: /api/v1/email/... (accounts, sync, threads, compose), /api/v1/ai/email-draft/...
  • Services: email-accounts.service.ts, email-compose.service.ts, email-draft.service.ts, email-threads.service.ts
  • Components: src/components/email/
• Notifications - Notification center, preferences, real-time via Socket.io
  • Pages: (integrated in layout)
  • API: /api/v1/notifications/... (CRUD, preferences, read-all, unread-count)
  • Service: notifications.service.ts
  • Components: src/components/notifications/
• Reminders - Reminder pages
  • Pages: /reminders
• Search - Global search (inline in topbar), saved views
  • API: /api/v1/search/..., /api/v1/saved-views/...
  • Service: search.service.ts, saved-views.service.ts
  • Components: src/components/search/
• Dashboard & Analytics - KPIs, pipeline summary, revenue forecast, activity feed
  • Pages: /[portSlug] (dashboard)
  • API: /api/v1/dashboard/... (kpis, pipeline, forecast, activity)
  • Service: dashboard.service.ts
  • Components: src/components/dashboard/

Layer 4: Advanced Features (DONE)

• Webhooks - CRUD, delivery tracking, event mapping, test delivery, secret regeneration
  • Pages: admin/webhooks
  • API: /api/v1/admin/webhooks/...
  • Services: webhooks.service.ts, webhook-dispatch.ts, webhook-event-map.ts
• Document Templates - Template management with versioning, preview, rollback
  • Pages: admin/templates
  • API: /api/v1/admin/templates/...
  • Service: document-templates.service.ts
• AI Features - Interest scoring, AI email drafts, receipt scanning
  • API: /api/v1/ai/...
  • Uses OpenAI SDK
• Custom Fields - Dynamic custom fields for entities
  • Pages: admin/custom-fields
  • API: /api/v1/admin/custom-fields/..., /api/v1/custom-fields/...
  • Service: custom-fields.service.ts
• Reports - Report generation with download, scheduled reports
  • Pages: /reports, admin/reports
  • API: /api/v1/reports/...
  • Services: reports.service.ts, report-generators.ts
• Admin Tools - Audit log, backup, import, settings, monitoring, forms, tags
  • Pages: full admin section
  • API: /api/v1/admin/... (connections, errors, health, queues)
  • Service: system-monitoring.service.ts
• Client Portal (separate Nuxt app in client-portal/)
  • Pages: portal login, verify, dashboard, documents, interests, invoices
  • API: /api/portal/... (auth, dashboard, documents, interests, invoices)
  • Service: portal.service.ts
• Currency - Multi-currency with conversion rates
  • API: /api/v1/currency/...
  • Service: currency.ts
• Tags - Cross-entity tagging system
  • API: /api/v1/tags/...
  • Service: tags.service.ts
• Record Export - PDF export for clients, berths, interests
  • Service: record-export.ts
• Settings & Feature Flags
  • Pages: /settings
  • API: /api/v1/settings/feature-flag/

Testing (Layer 5 - Partially Done)

• 25 E2E smoke tests (Playwright) covering all major flows
• 5 integration tests (Vitest): CRUD audit, custom fields, pipeline transitions, port scoping, webhook delivery
• 15 unit tests (Vitest): validators, encryption, security, audit, interest scoring, query plans, webhook events, etc.
• Test helpers and factories (tests/helpers/factories.ts)
• Playwright config with global setup

Infrastructure (DONE)

• Gitea Actions CI/CD workflow (.gitea/workflows/build.yml)
  • Lint + type-check on PR
  • Docker build + push to Gitea Container Registry on main
  • SSH deploy to production server
• Production docker-compose (docker-compose.prod.yml) using registry images
• Nginx config for Docker (nginx/nginx.conf)
• Nginx config for host/certbot (nginx/pn.letsbe.solutions.conf)
• Environment variable template (.env.example)

---

Project Stats

Metric	Count
TypeScript/TSX files	461
React components	133
API route handlers	120+
DB schema files	12
Service files	38
Validator files	17
BullMQ workers	8
E2E tests	25
Integration tests	5
Unit tests	15
Pages (dashboard)	37
Pages (portal)	6
Pages (auth)	3

---

What's Left To Do

Priority 1: Deployment & Go-Live

• Push to Gitea and verify CI/CD pipeline builds
• Set up server: install Docker, nginx, configure DNS for pn.letsbe.solutions
• Run certbot --nginx -d pn.letsbe.solutions for SSL
• Configure production .env on server
• Run database migrations (pnpm db:push)
• Run seed data (pnpm db:seed)
• Verify all services start and health check passes

Priority 2: Testing & Hardening (Layer 5 - Remaining)

• Run full E2E test suite against deployed instance and fix failures
• Run full unit/integration test suite and fix failures
• Security hardening pass (review SECURITY-GUIDELINES.md checklist)
• Performance audit (Core Web Vitals, API response times)
• Load testing with realistic data volumes

Priority 3: Migration (Layer 6)

• NocoDB data export scripts
• Data transformation/mapping (see NOCODB-MIGRATION-MAPPING.md)
• PostgreSQL data import
• MinIO file reorganization
• Smoke test migrated data
• DNS/proxy switchover from old system
• Old system decommission

Priority 4: Polish & Nice-to-Haves

• Dark mode toggle (infrastructure exists via next-themes)
• Mobile responsive polish pass
• Google Calendar integration for reminders
• Bulk operations UI refinement
• Email template system (MJML-based)
• Client portal deployment as separate service or subdomain

---

Key Architecture Decisions

Decision	Choice	Rationale
Framework	Next.js 15 (App Router)	Server Components, streaming, typed routes
ORM	Drizzle	Type-safe, lightweight, PostgreSQL-native
Auth	Better Auth	Multi-tenant, role-based, session management
State	Zustand + React Query	Zustand for UI state, RQ for server state
Queue	BullMQ + Redis	Background jobs, email sync, reports
Storage	MinIO	S3-compatible, self-hosted, encrypted
Real-time	Socket.io + Redis adapter	Notifications, live updates
UI	shadcn/ui + Tailwind	Accessible, customizable components
Signing	Documenso	Self-hosted document signing
AI	OpenAI	Receipt scanning, interest scoring, email drafts
Container Registry	Gitea built-in	Same host as git, simpler auth

---

File Structure Overview

src/
  app/
    (auth)/           # Login, reset-password, set-password
    (dashboard)/      # Main CRM - all [portSlug] scoped pages
    (portal)/         # Client-facing portal
    api/              # All API routes (v1/, portal/, webhooks/, auth/, health/)
  components/
    admin/            # Admin panel components
    berths/           # Berth management UI
    clients/          # Client management UI
    dashboard/        # Dashboard widgets
    documents/        # Document management UI
    email/            # Email client UI
    expenses/         # Expense tracking UI
    files/            # File browser UI
    interests/        # Interest/pipeline UI
    invoices/         # Invoice management UI
    layout/           # Sidebar, topbar, navigation
    notifications/    # Notification center
    portal/           # Client portal components
    reports/          # Report views
    search/           # Global search
    shared/           # Shared components
    ui/               # shadcn/ui base components
  hooks/              # React hooks
  jobs/               # BullMQ job definitions
  lib/
    auth/             # Better Auth config
    db/               # Drizzle schema, migrations, seed
    queue/            # Queue definitions, workers
    services/         # Business logic (38 service files)
    validators/       # Zod schemas (17 validator files)
    socket/           # Socket.io config
  providers/          # React context providers
  stores/             # Zustand stores
  types/              # TypeScript type definitions

---

How to Continue Development

# Clone and install
git clone https://code.letsbe.solutions/letsbe/pn-new-crm.git
cd pn-new-crm
pnpm install

# Start dev environment (needs Docker for PostgreSQL + Redis)
docker compose -f docker-compose.dev.yml up -d
cp .env.example .env  # then edit with real values
pnpm dev

# Run tests
pnpm exec vitest run
pnpm exec playwright test

# Build for production
pnpm build

# Database operations
pnpm db:push      # Push schema to database
pnpm db:generate  # Generate migration files
pnpm db:seed      # Seed with sample data
pnpm db:studio    # Open Drizzle Studio GUI

---

Spec Documents Reference

Doc	Purpose
01-CONSOLIDATED-SYSTEM-SPEC.md	Full system specification
02-FEATURE-INVENTORY.md	Feature keep/cut/rethink decisions
03-ARCHITECTURE-DECISIONS.md	Architecture exploration
04-ARCHITECTURE-COMPARISON.md	Stack comparison
05-FINAL-ARCHITECTURE-DECISIONS.md	Final architecture choices
06-MASTER-FEATURE-SPEC.md	Detailed feature specifications
07-DATABASE-SCHEMA.md	Database schema design
08-API-ENDPOINT-CATALOG.md	Full API endpoint list
09-BUSINESS-RULES.md	Business logic rules
10-AUTH-AND-PERMISSIONS.md	Auth & permission model
11-REALTIME-AND-BACKGROUND-JOBS.md	Socket.io & BullMQ design
12-IMPLEMENTATION-SEQUENCE.md	Build order & dependencies
13-UI-PAGE-MAP.md	Page layout specifications
14-TECHNICAL-DECISIONS.md	Technical trade-off decisions
15-DESIGN-TOKENS.md	Design system tokens
NOCODB-MIGRATION-MAPPING.md	NocoDB to PostgreSQL mapping
SECURITY-GUIDELINES.md	Security requirements