letsbe/pn-new-crm
1
0
Fork 0
Code Issues Pull Requests 1 Actions Packages Projects Releases Wiki Activity

Initial commit: Port Nimara CRM (Layers 0-4)
Some checks failed
Build & Push Docker Images / build-and-push (push) Has been cancelled
Details
Build & Push Docker Images / deploy (push) Has been cancelled
Details
Build & Push Docker Images / lint (push) Has been cancelled
Details

Browse Source 
Full CRM rebuild with Next.js 15, TypeScript, Tailwind, Drizzle ORM,
PostgreSQL, Redis, BullMQ, MinIO, and Socket.io. Includes 461 source
files covering clients, berths, interests/pipeline, documents/EOI,
expenses/invoices, email, notifications, dashboard, admin, and
client portal. CI/CD via Gitea Actions with Docker builds.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
Matt
2026-03-26 11:52:51 +01:00
commit 67d7e6e3d5
572 changed files with 86496 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

328
PROGRESS.md Normal file
View File

@@ -0,0 +1,328 @@
# Port Nimara CRM - Project Progress
**Last updated:** 2026-03-26
**Repo:** https://code.letsbe.solutions/letsbe/pn-new-crm
**Domain:** pn.letsbe.solutions
**Stack:** Next.js 15 + TypeScript + Tailwind + Drizzle ORM + PostgreSQL + Redis + BullMQ + MinIO + Socket.io
---
## What's Been Built (Layers 0-4 Complete)
### Layer 0: Foundation (DONE)
- [x] Next.js 15 project scaffold with TypeScript strict mode
- [x] Tailwind CSS + shadcn/ui component library (full set in `src/components/ui/`)
- [x] PostgreSQL via Drizzle ORM - full schema across 12 schema files
- `berths.ts`, `clients.ts`, `documents.ts`, `email.ts`, `financial.ts`, `interests.ts`, `operations.ts`, `ports.ts`, `relations.ts`, `system.ts`, `users.ts` + `index.ts`
- [x] Better Auth integration with multi-port middleware (`src/middleware.ts`)
- [x] Redis connection (`src/lib/redis.ts`)
- [x] BullMQ queue system with 8 workers: `document-signing.ts`, `email-sync.ts`, `email.ts`, `import.ts`, `maintenance.ts`, `notifications.ts`, `reports.ts`, `webhooks.ts`
- [x] MinIO file storage service (`src/lib/services/storage.ts`, `files.ts`)
- [x] Socket.io real-time server (`src/lib/socket/server.ts`, `events.ts`)
- [x] Docker setup: `Dockerfile` (multi-stage app), `Dockerfile.worker`, `Dockerfile.dev`, `docker-compose.yml`, `docker-compose.dev.yml`, `docker-compose.prod.yml`
- [x] Dashboard layout shell with sidebar, port switcher, navigation (`src/app/(dashboard)/layout.tsx`)
- [x] Auth pages: login, reset-password, set-password
- [x] Seed script (`src/lib/db/seed.ts`)
- [x] ESLint + Prettier + Husky + lint-staged
- [x] Health check endpoint (`/api/health`)
- [x] Rate limiting (`src/lib/rate-limit.ts`)
- [x] Encryption utilities (`src/lib/utils/encryption.ts`)
- [x] Zod validators for all entities (17 validator files in `src/lib/validators/`)
### Layer 1: Core CRUD (DONE)
- [x] **Client Management** - Full CRUD, contacts, relationships, notes, tags, duplicate detection
- Pages: list (`/clients`), detail (`/clients/[clientId]`)
- API: full REST at `/api/v1/clients/...` (CRUD, contacts, notes, relationships, tags, restore, export-pdf, options)
- Service: `clients.service.ts`
- Components: `src/components/clients/`
- [x] **Berth Management** - Full CRUD, specs, tags, status management, waiting list, maintenance
- Pages: list (`/berths`), detail (`/berths/[berthId]`)
- API: full REST at `/api/v1/berths/...` (CRUD, status, tags, waiting-list, maintenance, export-pdf, options)
- Service: `berths.service.ts`, `berth-rules-engine.ts`
- Components: `src/components/berths/`
- [x] **Auth Admin** - User management, role builder, port management
- Pages: admin/users, admin/roles, admin/ports, admin/onboarding
- API: `/api/v1/admin/users/`, `/api/v1/admin/roles/`
### Layer 2: Business Workflows (DONE)
- [x] **Interest/Pipeline Management** - CRUD, pipeline stages, berth linking, recommendations, scoring, timeline
- Pages: list (`/interests`), detail (`/interests/[interestId]`)
- API: full REST at `/api/v1/interests/...` (CRUD, stage, berth, notes, tags, recommendations, timeline, restore, export-pdf)
- Services: `interests.service.ts`, `interest-scoring.service.ts`, `recommendations.ts`
- Components: `src/components/interests/`
- [x] **Documents & EOI** - Documenso integration, 3-party signing, webhook receiver, document templates
- Pages: `/documents`
- API: `/api/v1/documents/...`, `/api/v1/document-templates/...`, `/api/webhooks/documenso/`
- Services: `documents.service.ts`, `document-templates.service.ts`, `document-templates.ts`, `documenso-client.ts`, `documenso-webhook.ts`, `document-reminders.ts`
- Components: `src/components/documents/`
- [x] **Expenses & Invoices** - CRUD, receipt scanner (AI), invoice generation, payment tracking, CSV/PDF export
- Pages: `/expenses`, `/expenses/[id]`, `/expenses/scan`, `/invoices`, `/invoices/[id]`, `/invoices/new`
- API: `/api/v1/expenses/...` (CRUD, scan-receipt, export CSV/PDF/parent-company), `/api/v1/invoices/...` (CRUD, generate-pdf, payment, send)
- Services: `expenses.ts`, `invoices.ts`, `expense-export.ts`, `receipt-scanner.ts`
- Components: `src/components/expenses/`, `src/components/invoices/`
- [x] **File Management** - MinIO integration, folder management, upload/download/preview
- API: `/api/v1/files/...` (CRUD, upload, download, preview, folders)
- Components: `src/components/files/`
- Store: `src/stores/file-browser-store.ts`
### Layer 3: Operations (DONE)
- [x] **Email System** - SMTP/IMAP, accounts, thread viewer, composer, AI drafts
- Pages: `/email`
- API: `/api/v1/email/...` (accounts, sync, threads, compose), `/api/v1/ai/email-draft/...`
- Services: `email-accounts.service.ts`, `email-compose.service.ts`, `email-draft.service.ts`, `email-threads.service.ts`
- Components: `src/components/email/`
- [x] **Notifications** - Notification center, preferences, real-time via Socket.io
- Pages: (integrated in layout)
- API: `/api/v1/notifications/...` (CRUD, preferences, read-all, unread-count)
- Service: `notifications.service.ts`
- Components: `src/components/notifications/`
- [x] **Reminders** - Reminder pages
- Pages: `/reminders`
- [x] **Search** - Global search (inline in topbar), saved views
- API: `/api/v1/search/...`, `/api/v1/saved-views/...`
- Service: `search.service.ts`, `saved-views.service.ts`
- Components: `src/components/search/`
- [x] **Dashboard & Analytics** - KPIs, pipeline summary, revenue forecast, activity feed
- Pages: `/[portSlug]` (dashboard)
- API: `/api/v1/dashboard/...` (kpis, pipeline, forecast, activity)
- Service: `dashboard.service.ts`
- Components: `src/components/dashboard/`
### Layer 4: Advanced Features (DONE)
- [x] **Webhooks** - CRUD, delivery tracking, event mapping, test delivery, secret regeneration
- Pages: admin/webhooks
- API: `/api/v1/admin/webhooks/...`
- Services: `webhooks.service.ts`, `webhook-dispatch.ts`, `webhook-event-map.ts`
- [x] **Document Templates** - Template management with versioning, preview, rollback
- Pages: admin/templates
- API: `/api/v1/admin/templates/...`
- Service: `document-templates.service.ts`
- [x] **AI Features** - Interest scoring, AI email drafts, receipt scanning
- API: `/api/v1/ai/...`
- Uses OpenAI SDK
- [x] **Custom Fields** - Dynamic custom fields for entities
- Pages: admin/custom-fields
- API: `/api/v1/admin/custom-fields/...`, `/api/v1/custom-fields/...`
- Service: `custom-fields.service.ts`
- [x] **Reports** - Report generation with download, scheduled reports
- Pages: `/reports`, admin/reports
- API: `/api/v1/reports/...`
- Services: `reports.service.ts`, `report-generators.ts`
- [x] **Admin Tools** - Audit log, backup, import, settings, monitoring, forms, tags
- Pages: full admin section
- API: `/api/v1/admin/...` (connections, errors, health, queues)
- Service: `system-monitoring.service.ts`
- [x] **Client Portal** (separate Nuxt app in `client-portal/`)
- Pages: portal login, verify, dashboard, documents, interests, invoices
- API: `/api/portal/...` (auth, dashboard, documents, interests, invoices)
- Service: `portal.service.ts`
- [x] **Currency** - Multi-currency with conversion rates
- API: `/api/v1/currency/...`
- Service: `currency.ts`
- [x] **Tags** - Cross-entity tagging system
- API: `/api/v1/tags/...`
- Service: `tags.service.ts`
- [x] **Record Export** - PDF export for clients, berths, interests
- Service: `record-export.ts`
- [x] **Settings & Feature Flags**
- Pages: `/settings`
- API: `/api/v1/settings/feature-flag/`
### Testing (Layer 5 - Partially Done)
- [x] 25 E2E smoke tests (Playwright) covering all major flows
- [x] 5 integration tests (Vitest): CRUD audit, custom fields, pipeline transitions, port scoping, webhook delivery
- [x] 15 unit tests (Vitest): validators, encryption, security, audit, interest scoring, query plans, webhook events, etc.
- [x] Test helpers and factories (`tests/helpers/factories.ts`)
- [x] Playwright config with global setup
### Infrastructure (DONE)
- [x] Gitea Actions CI/CD workflow (`.gitea/workflows/build.yml`)
- Lint + type-check on PR
- Docker build + push to Gitea Container Registry on main
- SSH deploy to production server
- [x] Production docker-compose (`docker-compose.prod.yml`) using registry images
- [x] Nginx config for Docker (`nginx/nginx.conf`)
- [x] Nginx config for host/certbot (`nginx/pn.letsbe.solutions.conf`)
- [x] Environment variable template (`.env.example`)
---
## Project Stats
| Metric | Count |
| -------------------- | ----- |
| TypeScript/TSX files | 461 |
| React components | 133 |
| API route handlers | 120+ |
| DB schema files | 12 |
| Service files | 38 |
| Validator files | 17 |
| BullMQ workers | 8 |
| E2E tests | 25 |
| Integration tests | 5 |
| Unit tests | 15 |
| Pages (dashboard) | 37 |
| Pages (portal) | 6 |
| Pages (auth) | 3 |
---
## What's Left To Do
### Priority 1: Deployment & Go-Live
- [ ] Push to Gitea and verify CI/CD pipeline builds
- [ ] Set up server: install Docker, nginx, configure DNS for `pn.letsbe.solutions`
- [ ] Run `certbot --nginx -d pn.letsbe.solutions` for SSL
- [ ] Configure production `.env` on server
- [ ] Run database migrations (`pnpm db:push`)
- [ ] Run seed data (`pnpm db:seed`)
- [ ] Verify all services start and health check passes
### Priority 2: Testing & Hardening (Layer 5 - Remaining)
- [ ] Run full E2E test suite against deployed instance and fix failures
- [ ] Run full unit/integration test suite and fix failures
- [ ] Security hardening pass (review SECURITY-GUIDELINES.md checklist)
- [ ] Performance audit (Core Web Vitals, API response times)
- [ ] Load testing with realistic data volumes
### Priority 3: Migration (Layer 6)
- [ ] NocoDB data export scripts
- [ ] Data transformation/mapping (see `NOCODB-MIGRATION-MAPPING.md`)
- [ ] PostgreSQL data import
- [ ] MinIO file reorganization
- [ ] Smoke test migrated data
- [ ] DNS/proxy switchover from old system
- [ ] Old system decommission
### Priority 4: Polish & Nice-to-Haves
- [ ] Dark mode toggle (infrastructure exists via `next-themes`)
- [ ] Mobile responsive polish pass
- [ ] Google Calendar integration for reminders
- [ ] Bulk operations UI refinement
- [ ] Email template system (MJML-based)
- [ ] Client portal deployment as separate service or subdomain
---
## Key Architecture Decisions
| Decision | Choice | Rationale |
| ------------------ | ------------------------- | ------------------------------------------------ |
| Framework | Next.js 15 (App Router) | Server Components, streaming, typed routes |
| ORM | Drizzle | Type-safe, lightweight, PostgreSQL-native |
| Auth | Better Auth | Multi-tenant, role-based, session management |
| State | Zustand + React Query | Zustand for UI state, RQ for server state |
| Queue | BullMQ + Redis | Background jobs, email sync, reports |
| Storage | MinIO | S3-compatible, self-hosted, encrypted |
| Real-time | Socket.io + Redis adapter | Notifications, live updates |
| UI | shadcn/ui + Tailwind | Accessible, customizable components |
| Signing | Documenso | Self-hosted document signing |
| AI | OpenAI | Receipt scanning, interest scoring, email drafts |
| Container Registry | Gitea built-in | Same host as git, simpler auth |
---
## File Structure Overview
```
src/
app/
(auth)/ # Login, reset-password, set-password
(dashboard)/ # Main CRM - all [portSlug] scoped pages
(portal)/ # Client-facing portal
api/ # All API routes (v1/, portal/, webhooks/, auth/, health/)
components/
admin/ # Admin panel components
berths/ # Berth management UI
clients/ # Client management UI
dashboard/ # Dashboard widgets
documents/ # Document management UI
email/ # Email client UI
expenses/ # Expense tracking UI
files/ # File browser UI
interests/ # Interest/pipeline UI
invoices/ # Invoice management UI
layout/ # Sidebar, topbar, navigation
notifications/ # Notification center
portal/ # Client portal components
reports/ # Report views
search/ # Global search
shared/ # Shared components
ui/ # shadcn/ui base components
hooks/ # React hooks
jobs/ # BullMQ job definitions
lib/
auth/ # Better Auth config
db/ # Drizzle schema, migrations, seed
queue/ # Queue definitions, workers
services/ # Business logic (38 service files)
validators/ # Zod schemas (17 validator files)
socket/ # Socket.io config
providers/ # React context providers
stores/ # Zustand stores
types/ # TypeScript type definitions
```
---
## How to Continue Development
```bash
# Clone and install
git clone https://code.letsbe.solutions/letsbe/pn-new-crm.git
cd pn-new-crm
pnpm install
# Start dev environment (needs Docker for PostgreSQL + Redis)
docker compose -f docker-compose.dev.yml up -d
cp .env.example .env # then edit with real values
pnpm dev
# Run tests
pnpm exec vitest run
pnpm exec playwright test
# Build for production
pnpm build
# Database operations
pnpm db:push # Push schema to database
pnpm db:generate # Generate migration files
pnpm db:seed # Seed with sample data
pnpm db:studio # Open Drizzle Studio GUI
```
---
## Spec Documents Reference
| Doc | Purpose |
| ------------------------------------ | ---------------------------------- |
| `01-CONSOLIDATED-SYSTEM-SPEC.md` | Full system specification |
| `02-FEATURE-INVENTORY.md` | Feature keep/cut/rethink decisions |
| `03-ARCHITECTURE-DECISIONS.md` | Architecture exploration |
| `04-ARCHITECTURE-COMPARISON.md` | Stack comparison |
| `05-FINAL-ARCHITECTURE-DECISIONS.md` | Final architecture choices |
| `06-MASTER-FEATURE-SPEC.md` | Detailed feature specifications |
| `07-DATABASE-SCHEMA.md` | Database schema design |
| `08-API-ENDPOINT-CATALOG.md` | Full API endpoint list |
| `09-BUSINESS-RULES.md` | Business logic rules |
| `10-AUTH-AND-PERMISSIONS.md` | Auth & permission model |
| `11-REALTIME-AND-BACKGROUND-JOBS.md` | Socket.io & BullMQ design |
| `12-IMPLEMENTATION-SEQUENCE.md` | Build order & dependencies |
| `13-UI-PAGE-MAP.md` | Page layout specifications |
| `14-TECHNICAL-DECISIONS.md` | Technical trade-off decisions |
| `15-DESIGN-TOKENS.md` | Design system tokens |
| `NOCODB-MIGRATION-MAPPING.md` | NocoDB to PostgreSQL mapping |
| `SECURITY-GUIDELINES.md` | Security requirements |