letsbe/pn-new-crm
1
0
Fork 0
Code Issues Pull Requests 1 Actions Packages Projects Releases Wiki Activity
Files
8df8ded46cb5b101dc4bc6d466a6c55f52cff8dc
pn-new-crm/PROGRESS.md
Matt 67d7e6e3d5
Some checks failed
Build & Push Docker Images / build-and-push (push) Has been cancelled
Details
Build & Push Docker Images / deploy (push) Has been cancelled
Details
Build & Push Docker Images / lint (push) Has been cancelled
Details
Initial commit: Port Nimara CRM (Layers 0-4) 
Full CRM rebuild with Next.js 15, TypeScript, Tailwind, Drizzle ORM,
PostgreSQL, Redis, BullMQ, MinIO, and Socket.io. Includes 461 source
files covering clients, berths, interests/pipeline, documents/EOI,
expenses/invoices, email, notifications, dashboard, admin, and
client portal. CI/CD via Gitea Actions with Docker builds.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-26 11:52:51 +01:00

329 lines
15 KiB
Markdown
Raw Blame History

# Port Nimara CRM - Project Progress
**Last updated:** 2026-03-26
**Repo:** https://code.letsbe.solutions/letsbe/pn-new-crm
**Domain:** pn.letsbe.solutions
**Stack:** Next.js 15 + TypeScript + Tailwind + Drizzle ORM + PostgreSQL + Redis + BullMQ + MinIO + Socket.io
---
## What's Been Built (Layers 0-4 Complete)
### Layer 0: Foundation (DONE)
- [x] Next.js 15 project scaffold with TypeScript strict mode
- [x] Tailwind CSS + shadcn/ui component library (full set in `src/components/ui/`)
- [x] PostgreSQL via Drizzle ORM - full schema across 12 schema files
- `berths.ts`, `clients.ts`, `documents.ts`, `email.ts`, `financial.ts`, `interests.ts`, `operations.ts`, `ports.ts`, `relations.ts`, `system.ts`, `users.ts` + `index.ts`
- [x] Better Auth integration with multi-port middleware (`src/middleware.ts`)
- [x] Redis connection (`src/lib/redis.ts`)
- [x] BullMQ queue system with 8 workers: `document-signing.ts`, `email-sync.ts`, `email.ts`, `import.ts`, `maintenance.ts`, `notifications.ts`, `reports.ts`, `webhooks.ts`
- [x] MinIO file storage service (`src/lib/services/storage.ts`, `files.ts`)
- [x] Socket.io real-time server (`src/lib/socket/server.ts`, `events.ts`)
- [x] Docker setup: `Dockerfile` (multi-stage app), `Dockerfile.worker`, `Dockerfile.dev`, `docker-compose.yml`, `docker-compose.dev.yml`, `docker-compose.prod.yml`
- [x] Dashboard layout shell with sidebar, port switcher, navigation (`src/app/(dashboard)/layout.tsx`)
- [x] Auth pages: login, reset-password, set-password
- [x] Seed script (`src/lib/db/seed.ts`)
- [x] ESLint + Prettier + Husky + lint-staged
- [x] Health check endpoint (`/api/health`)
- [x] Rate limiting (`src/lib/rate-limit.ts`)
- [x] Encryption utilities (`src/lib/utils/encryption.ts`)
- [x] Zod validators for all entities (17 validator files in `src/lib/validators/`)
### Layer 1: Core CRUD (DONE)
- [x] **Client Management** - Full CRUD, contacts, relationships, notes, tags, duplicate detection
- Pages: list (`/clients`), detail (`/clients/[clientId]`)
- API: full REST at `/api/v1/clients/...` (CRUD, contacts, notes, relationships, tags, restore, export-pdf, options)
- Service: `clients.service.ts`
- Components: `src/components/clients/`
- [x] **Berth Management** - Full CRUD, specs, tags, status management, waiting list, maintenance
- Pages: list (`/berths`), detail (`/berths/[berthId]`)
- API: full REST at `/api/v1/berths/...` (CRUD, status, tags, waiting-list, maintenance, export-pdf, options)
- Service: `berths.service.ts`, `berth-rules-engine.ts`
- Components: `src/components/berths/`
- [x] **Auth Admin** - User management, role builder, port management
- Pages: admin/users, admin/roles, admin/ports, admin/onboarding
- API: `/api/v1/admin/users/`, `/api/v1/admin/roles/`
### Layer 2: Business Workflows (DONE)
- [x] **Interest/Pipeline Management** - CRUD, pipeline stages, berth linking, recommendations, scoring, timeline
- Pages: list (`/interests`), detail (`/interests/[interestId]`)
- API: full REST at `/api/v1/interests/...` (CRUD, stage, berth, notes, tags, recommendations, timeline, restore, export-pdf)
- Services: `interests.service.ts`, `interest-scoring.service.ts`, `recommendations.ts`
- Components: `src/components/interests/`
- [x] **Documents & EOI** - Documenso integration, 3-party signing, webhook receiver, document templates
- Pages: `/documents`
- API: `/api/v1/documents/...`, `/api/v1/document-templates/...`, `/api/webhooks/documenso/`
- Services: `documents.service.ts`, `document-templates.service.ts`, `document-templates.ts`, `documenso-client.ts`, `documenso-webhook.ts`, `document-reminders.ts`
- Components: `src/components/documents/`
- [x] **Expenses & Invoices** - CRUD, receipt scanner (AI), invoice generation, payment tracking, CSV/PDF export
- Pages: `/expenses`, `/expenses/[id]`, `/expenses/scan`, `/invoices`, `/invoices/[id]`, `/invoices/new`
- API: `/api/v1/expenses/...` (CRUD, scan-receipt, export CSV/PDF/parent-company), `/api/v1/invoices/...` (CRUD, generate-pdf, payment, send)
- Services: `expenses.ts`, `invoices.ts`, `expense-export.ts`, `receipt-scanner.ts`
- Components: `src/components/expenses/`, `src/components/invoices/`
- [x] **File Management** - MinIO integration, folder management, upload/download/preview
- API: `/api/v1/files/...` (CRUD, upload, download, preview, folders)
- Components: `src/components/files/`
- Store: `src/stores/file-browser-store.ts`
### Layer 3: Operations (DONE)
- [x] **Email System** - SMTP/IMAP, accounts, thread viewer, composer, AI drafts
- Pages: `/email`
- API: `/api/v1/email/...` (accounts, sync, threads, compose), `/api/v1/ai/email-draft/...`
- Services: `email-accounts.service.ts`, `email-compose.service.ts`, `email-draft.service.ts`, `email-threads.service.ts`
- Components: `src/components/email/`
- [x] **Notifications** - Notification center, preferences, real-time via Socket.io
- Pages: (integrated in layout)
- API: `/api/v1/notifications/...` (CRUD, preferences, read-all, unread-count)
- Service: `notifications.service.ts`
- Components: `src/components/notifications/`
- [x] **Reminders** - Reminder pages
- Pages: `/reminders`
- [x] **Search** - Global search (inline in topbar), saved views
- API: `/api/v1/search/...`, `/api/v1/saved-views/...`
- Service: `search.service.ts`, `saved-views.service.ts`
- Components: `src/components/search/`
- [x] **Dashboard & Analytics** - KPIs, pipeline summary, revenue forecast, activity feed
- Pages: `/[portSlug]` (dashboard)
- API: `/api/v1/dashboard/...` (kpis, pipeline, forecast, activity)
- Service: `dashboard.service.ts`
- Components: `src/components/dashboard/`
### Layer 4: Advanced Features (DONE)
- [x] **Webhooks** - CRUD, delivery tracking, event mapping, test delivery, secret regeneration
- Pages: admin/webhooks
- API: `/api/v1/admin/webhooks/...`
- Services: `webhooks.service.ts`, `webhook-dispatch.ts`, `webhook-event-map.ts`
- [x] **Document Templates** - Template management with versioning, preview, rollback
- Pages: admin/templates
- API: `/api/v1/admin/templates/...`
- Service: `document-templates.service.ts`
- [x] **AI Features** - Interest scoring, AI email drafts, receipt scanning
- API: `/api/v1/ai/...`
- Uses OpenAI SDK
- [x] **Custom Fields** - Dynamic custom fields for entities
- Pages: admin/custom-fields
- API: `/api/v1/admin/custom-fields/...`, `/api/v1/custom-fields/...`
- Service: `custom-fields.service.ts`
- [x] **Reports** - Report generation with download, scheduled reports
- Pages: `/reports`, admin/reports
- API: `/api/v1/reports/...`
- Services: `reports.service.ts`, `report-generators.ts`
- [x] **Admin Tools** - Audit log, backup, import, settings, monitoring, forms, tags
- Pages: full admin section
- API: `/api/v1/admin/...` (connections, errors, health, queues)
- Service: `system-monitoring.service.ts`
- [x] **Client Portal** (separate Nuxt app in `client-portal/`)
- Pages: portal login, verify, dashboard, documents, interests, invoices
- API: `/api/portal/...` (auth, dashboard, documents, interests, invoices)
- Service: `portal.service.ts`
- [x] **Currency** - Multi-currency with conversion rates
- API: `/api/v1/currency/...`
- Service: `currency.ts`
- [x] **Tags** - Cross-entity tagging system
- API: `/api/v1/tags/...`
- Service: `tags.service.ts`
- [x] **Record Export** - PDF export for clients, berths, interests
- Service: `record-export.ts`
- [x] **Settings & Feature Flags**
- Pages: `/settings`
- API: `/api/v1/settings/feature-flag/`
### Testing (Layer 5 - Partially Done)
- [x] 25 E2E smoke tests (Playwright) covering all major flows
- [x] 5 integration tests (Vitest): CRUD audit, custom fields, pipeline transitions, port scoping, webhook delivery
- [x] 15 unit tests (Vitest): validators, encryption, security, audit, interest scoring, query plans, webhook events, etc.
- [x] Test helpers and factories (`tests/helpers/factories.ts`)
- [x] Playwright config with global setup
### Infrastructure (DONE)
- [x] Gitea Actions CI/CD workflow (`.gitea/workflows/build.yml`)
- Lint + type-check on PR
- Docker build + push to Gitea Container Registry on main
- SSH deploy to production server
- [x] Production docker-compose (`docker-compose.prod.yml`) using registry images
- [x] Nginx config for Docker (`nginx/nginx.conf`)
- [x] Nginx config for host/certbot (`nginx/pn.letsbe.solutions.conf`)
- [x] Environment variable template (`.env.example`)
---
## Project Stats
| Metric | Count |
| -------------------- | ----- |
| TypeScript/TSX files | 461 |
| React components | 133 |
| API route handlers | 120+ |
| DB schema files | 12 |
| Service files | 38 |
| Validator files | 17 |
| BullMQ workers | 8 |
| E2E tests | 25 |
| Integration tests | 5 |
| Unit tests | 15 |
| Pages (dashboard) | 37 |
| Pages (portal) | 6 |
| Pages (auth) | 3 |
---
## What's Left To Do
### Priority 1: Deployment & Go-Live
- [ ] Push to Gitea and verify CI/CD pipeline builds
- [ ] Set up server: install Docker, nginx, configure DNS for `pn.letsbe.solutions`
- [ ] Run `certbot --nginx -d pn.letsbe.solutions` for SSL
- [ ] Configure production `.env` on server
- [ ] Run database migrations (`pnpm db:push`)
- [ ] Run seed data (`pnpm db:seed`)
- [ ] Verify all services start and health check passes
### Priority 2: Testing & Hardening (Layer 5 - Remaining)
- [ ] Run full E2E test suite against deployed instance and fix failures
- [ ] Run full unit/integration test suite and fix failures
- [ ] Security hardening pass (review SECURITY-GUIDELINES.md checklist)
- [ ] Performance audit (Core Web Vitals, API response times)
- [ ] Load testing with realistic data volumes
### Priority 3: Migration (Layer 6)
- [ ] NocoDB data export scripts
- [ ] Data transformation/mapping (see `NOCODB-MIGRATION-MAPPING.md`)
- [ ] PostgreSQL data import
- [ ] MinIO file reorganization
- [ ] Smoke test migrated data
- [ ] DNS/proxy switchover from old system
- [ ] Old system decommission
### Priority 4: Polish & Nice-to-Haves
- [ ] Dark mode toggle (infrastructure exists via `next-themes`)
- [ ] Mobile responsive polish pass
- [ ] Google Calendar integration for reminders
- [ ] Bulk operations UI refinement
- [ ] Email template system (MJML-based)
- [ ] Client portal deployment as separate service or subdomain
---
## Key Architecture Decisions
| Decision | Choice | Rationale |
| ------------------ | ------------------------- | ------------------------------------------------ |
| Framework | Next.js 15 (App Router) | Server Components, streaming, typed routes |
| ORM | Drizzle | Type-safe, lightweight, PostgreSQL-native |
| Auth | Better Auth | Multi-tenant, role-based, session management |
| State | Zustand + React Query | Zustand for UI state, RQ for server state |
| Queue | BullMQ + Redis | Background jobs, email sync, reports |
| Storage | MinIO | S3-compatible, self-hosted, encrypted |
| Real-time | Socket.io + Redis adapter | Notifications, live updates |
| UI | shadcn/ui + Tailwind | Accessible, customizable components |
| Signing | Documenso | Self-hosted document signing |
| AI | OpenAI | Receipt scanning, interest scoring, email drafts |
| Container Registry | Gitea built-in | Same host as git, simpler auth |
---
## File Structure Overview
```
src/
app/
(auth)/ # Login, reset-password, set-password
(dashboard)/ # Main CRM - all [portSlug] scoped pages
(portal)/ # Client-facing portal
api/ # All API routes (v1/, portal/, webhooks/, auth/, health/)
components/
admin/ # Admin panel components
berths/ # Berth management UI
clients/ # Client management UI
dashboard/ # Dashboard widgets
documents/ # Document management UI
email/ # Email client UI
expenses/ # Expense tracking UI
files/ # File browser UI
interests/ # Interest/pipeline UI
invoices/ # Invoice management UI
layout/ # Sidebar, topbar, navigation
notifications/ # Notification center
portal/ # Client portal components
reports/ # Report views
search/ # Global search
shared/ # Shared components
ui/ # shadcn/ui base components
hooks/ # React hooks
jobs/ # BullMQ job definitions
lib/
auth/ # Better Auth config
db/ # Drizzle schema, migrations, seed
queue/ # Queue definitions, workers
services/ # Business logic (38 service files)
validators/ # Zod schemas (17 validator files)
socket/ # Socket.io config
providers/ # React context providers
stores/ # Zustand stores
types/ # TypeScript type definitions
```
---
## How to Continue Development
```bash
# Clone and install
git clone https://code.letsbe.solutions/letsbe/pn-new-crm.git
cd pn-new-crm
pnpm install
# Start dev environment (needs Docker for PostgreSQL + Redis)
docker compose -f docker-compose.dev.yml up -d
cp .env.example .env # then edit with real values
pnpm dev
# Run tests
pnpm exec vitest run
pnpm exec playwright test
# Build for production
pnpm build
# Database operations
pnpm db:push # Push schema to database
pnpm db:generate # Generate migration files
pnpm db:seed # Seed with sample data
pnpm db:studio # Open Drizzle Studio GUI
```
---
## Spec Documents Reference
| Doc | Purpose |
| ------------------------------------ | ---------------------------------- |
| `01-CONSOLIDATED-SYSTEM-SPEC.md` | Full system specification |
| `02-FEATURE-INVENTORY.md` | Feature keep/cut/rethink decisions |
| `03-ARCHITECTURE-DECISIONS.md` | Architecture exploration |
| `04-ARCHITECTURE-COMPARISON.md` | Stack comparison |
| `05-FINAL-ARCHITECTURE-DECISIONS.md` | Final architecture choices |
| `06-MASTER-FEATURE-SPEC.md` | Detailed feature specifications |
| `07-DATABASE-SCHEMA.md` | Database schema design |
| `08-API-ENDPOINT-CATALOG.md` | Full API endpoint list |
| `09-BUSINESS-RULES.md` | Business logic rules |
| `10-AUTH-AND-PERMISSIONS.md` | Auth & permission model |
| `11-REALTIME-AND-BACKGROUND-JOBS.md` | Socket.io & BullMQ design |
| `12-IMPLEMENTATION-SEQUENCE.md` | Build order & dependencies |
| `13-UI-PAGE-MAP.md` | Page layout specifications |
| `14-TECHNICAL-DECISIONS.md` | Technical trade-off decisions |
| `15-DESIGN-TOKENS.md` | Design system tokens |
| `NOCODB-MIGRATION-MAPPING.md` | NocoDB to PostgreSQL mapping |
| `SECURITY-GUIDELINES.md` | Security requirements |
Reference in New Issue View Git Blame Copy Permalink