Fix portal account creation and improve email handling

- Add explicit POST method to portal account creation API call - Improve error handling with specific messages for different failure cases - Remove SMTP verification step that was causing issues with some servers - Make email sending non-critical to portal account creation success - Add better response data handling for keycloak_id - Add integration review documentation
2025-08-09 16:13:52 +02:00 · 2025-08-09 16:13:52 +02:00 · c4a0230f42
parent 8d872f9a04
commit c4a0230f42
5 changed files with 338 additions and 27 deletions
--- a/INTEGRATION_REVIEW.md
+++ b/INTEGRATION_REVIEW.md
@ -0,0 +1,280 @@
+# MonacoUSA Portal - Integration Review & Troubleshooting Guide
+
+## SMTP Email Integration Points
+
+### 1. Email Configuration Storage
+- **Location**: `server/utils/admin-config.ts`
+- **Storage**: Encrypted in `/app/data/admin-config.json` (Docker) or `./data/admin-config.json` (local)
+- **Fields**: host, port, secure, username, password, fromAddress, fromName
+
+### 2. Email Service Implementation
+- **Location**: `server/utils/email.ts`
+- **Features**:
+  - Auto-detects security settings based on port
+  - Increased timeouts (60 seconds) for slow servers
+  - Supports STARTTLS (port 587) and SSL/TLS (port 465)
+  - Authentication type set to 'login' for compatibility
+  - Accepts self-signed certificates
+
+### 3. Email Usage Points
+- **Registration**: `server/api/registration.post.ts` - Sends welcome email with verification link
+- **Portal Account Creation**: `server/api/members/[id]/create-portal-account.post.ts` - Sends welcome email
+- **Password Reset**: `server/api/auth/forgot-password.post.ts` - Sends password reset link
+- **Email Verification Resend**: `server/api/auth/send-verification-email.post.ts`
+- **Test Email**: `server/api/admin/test-email.post.ts` - Admin panel test
+
+### 4. Common SMTP Issues & Solutions
+
+#### Issue: "500 plugin timeout" / EAUTH errors
+**Solutions**:
+1. **Port 587 (STARTTLS)**:
+   - Set SSL/TLS: OFF
+   - Username: Full email address (noreply@monacousa.org)
+   - Password: Your SMTP password (not email password if different)
+
+2. **Port 465 (SSL/TLS)**:
+   - Set SSL/TLS: ON
+   - Same credentials as above
+
+3. **Port 25 (Unencrypted)**:
+   - Set SSL/TLS: OFF
+   - May not require authentication
+   - Not recommended for production
+
+4. **Alternative Configuration** for mail.monacousa.org:
+   - Try port 587 with SSL/TLS OFF
+   - Try port 465 with SSL/TLS ON
+   - Ensure username is full email address
+   - Some servers require app-specific passwords
+
+#### Issue: Connection timeouts
+**Solutions**:
+- Timeouts already increased to 60 seconds
+- Check firewall rules allow outbound connections on SMTP port
+- Verify DNS resolution of mail server
+
+#### Issue: Certificate errors
+**Solutions**:
+- Self-signed certificates are already accepted
+- TLS minimum version set to TLSv1 for compatibility
+
+### 5. Testing SMTP Without Email
+If SMTP cannot be configured, the system gracefully handles email failures:
+- Portal accounts are still created
+- Users can use "Forgot Password" to set initial password
+- Admin sees appropriate messages about email status
+
+## Keycloak Integration Points
+
+### 1. Authentication Flow
+- **Login**: `server/api/auth/keycloak/login.get.ts` - Redirects to Keycloak
+- **Callback**: `server/api/auth/keycloak/callback.get.ts` - Handles OAuth callback
+- **Session**: `server/utils/session.ts` - Manages encrypted sessions
+- **Logout**: `server/api/auth/logout.post.ts` - Clears session and Keycloak logout
+
+### 2. User Management
+- **Admin Client**: `server/utils/keycloak-admin.ts`
+- **Features**:
+  - Create users with role-based registration
+  - Update user attributes (membership data)
+  - Password reset functionality
+  - Email verification tokens
+  - User search by email
+
+### 3. Role-Based Access
+- **Tiers**: admin, board, user
+- **Middleware**:
+  - `middleware/auth.ts` - General authentication
+  - `middleware/auth-admin.ts` - Admin only
+  - `middleware/auth-board.ts` - Board and admin
+  - `middleware/auth-user.ts` - All authenticated users
+
+### 4. Member-Portal Sync
+- **Dual Database System**:
+  - NocoDB: Member records (source of truth)
+  - Keycloak: Authentication and portal accounts
+- **Sync Points**:
+  - Registration creates both records
+  - Portal account creation links existing member to Keycloak
+  - Member updates sync to Keycloak attributes
+
+### 5. Common Keycloak Issues & Solutions
+
+#### Issue: Login redirect loops
+**Solutions**:
+- Check `NUXT_KEYCLOAK_CALLBACK_URL` matches actual domain
+- Verify Keycloak client redirect URIs include callback URL
+- Ensure session secret is set and consistent
+
+#### Issue: User creation failures
+**Solutions**:
+- Check Keycloak admin credentials in environment
+- Verify realm exists and is accessible
+- Ensure email is unique in Keycloak
+
+#### Issue: Role assignment not working
+**Solutions**:
+- Verify realm roles exist: user, board, admin
+- Check client scope mappings include roles
+- Ensure token includes role claims
+
+## Environment Variables Required
+
+### Keycloak Configuration
+```env
+NUXT_KEYCLOAK_ISSUER=https://auth.monacousa.org/realms/monacousa-portal
+NUXT_KEYCLOAK_CLIENT_ID=monacousa-portal
+NUXT_KEYCLOAK_CLIENT_SECRET=your-client-secret
+NUXT_KEYCLOAK_CALLBACK_URL=https://monacousa.org/auth/callback
+NUXT_KEYCLOAK_ADMIN_USERNAME=admin
+NUXT_KEYCLOAK_ADMIN_PASSWORD=admin-password
+```
+
+### Session Security
+```env
+NUXT_SESSION_SECRET=48-character-secret-key
+NUXT_ENCRYPTION_KEY=32-character-encryption-key
+```
+
+### Public Configuration
+```env
+NUXT_PUBLIC_DOMAIN=monacousa.org
+```
+
+## Health Check Endpoints
+
+### System Health
+- **Endpoint**: `GET /api/health`
+- **Checks**:
+  - Database connectivity (NocoDB)
+  - Keycloak connectivity
+  - Session management
+  - File storage (if configured)
+
+## Troubleshooting Workflow
+
+### For SMTP Issues:
+1. Try port 587 with SSL/TLS OFF first
+2. If fails, try port 465 with SSL/TLS ON
+3. Check credentials (use full email as username)
+4. Test with personal Gmail/Outlook account to verify code works
+5. Check firewall/network restrictions
+6. Review server logs for specific error messages
+
+### For Keycloak Issues:
+1. Verify all environment variables are set
+2. Check Keycloak server is accessible
+3. Test with direct Keycloak login first
+4. Review browser console for redirect issues
+5. Check server logs for token/session errors
+6. Verify realm and client configuration in Keycloak admin
+
+## Manual SMTP Testing
+
+To manually test SMTP settings without the portal:
+
+### Using OpenSSL (for connection test):
+```bash
+# For STARTTLS (port 587)
+openssl s_client -starttls smtp -connect mail.monacousa.org:587
+
+# For SSL/TLS (port 465)
+openssl s_client -connect mail.monacousa.org:465
+```
+
+### Using Telnet (for basic connectivity):
+```bash
+telnet mail.monacousa.org 587
+```
+
+### Using swaks (comprehensive SMTP test):
+```bash
+swaks --to test@example.com \
+      --from noreply@monacousa.org \
+      --server mail.monacousa.org:587 \
+      --auth LOGIN \
+      --auth-user noreply@monacousa.org \
+      --auth-password yourpassword \
+      --tls
+```
+
+## Alternative Email Solutions
+
+If SMTP continues to fail:
+
+### 1. Use Gmail with App Password:
+- Enable 2FA on Gmail account
+- Generate app-specific password
+- Use smtp.gmail.com:587
+- Username: your gmail address
+- Password: app-specific password
+
+### 2. Use SendGrid (Free tier available):
+- Sign up at sendgrid.com
+- Create API key
+- Use smtp.sendgrid.net:587
+- Username: apikey (literal string)
+- Password: your API key
+
+### 3. Use Local Mail Server (Development):
+- Install MailHog or MailCatcher
+- No authentication required
+- Captures all emails locally
+- Perfect for testing
+
+## System Architecture
+
+```
+┌─────────────────┐     ┌──────────────┐     ┌─────────────┐
+│                 │────▶│              │────▶│             │
+│   Frontend      │     │   Nuxt API   │     │  Keycloak   │
+│   (Vue/Vuetify) │◀────│   Routes     │◀────│   Server    │
+│                 │     │              │     │             │
+└─────────────────┘     └──────────────┘     └─────────────┘
+                               │                     ▲
+                               │                     │
+                               ▼                     │
+                        ┌──────────────┐             │
+                        │              │             │
+                        │    NocoDB    │─────────────┘
+                        │   Database   │
+                        │              │
+                        └──────────────┘
+                               │
+                               ▼
+                        ┌──────────────┐
+                        │              │
+                        │     SMTP     │
+                        │    Server    │
+                        │              │
+                        └──────────────┘
+```
+
+## Production Checklist
+
+- [ ] All environment variables set correctly
+- [ ] SSL certificates valid and configured
+- [ ] Keycloak realm and client configured
+- [ ] NocoDB database accessible and configured
+- [ ] SMTP credentials tested and working
+- [ ] Session secrets are strong and unique
+- [ ] Firewall rules allow necessary ports
+- [ ] Backup strategy in place
+- [ ] Monitoring and logging configured
+- [ ] Health check endpoint monitored
+
+## Support Resources
+
+- **Keycloak Documentation**: https://www.keycloak.org/documentation
+- **NocoDB Documentation**: https://docs.nocodb.com
+- **Nodemailer Documentation**: https://nodemailer.com
+- **Nuxt 3 Documentation**: https://nuxt.com
+
+## Contact for Issues
+
+If you continue to experience issues after following this guide:
+1. Check server logs for detailed error messages
+2. Test each component independently
+3. Verify network connectivity and DNS resolution
+4. Review firewall and security group rules
+5. Consider using alternative email providers
--- a/pages/dashboard/member-list.vue
+++ b/pages/dashboard/member-list.vue
@ -507,23 +507,41 @@ const createPortalAccount = async (member: Member) => {
  creatingPortalAccountIds.value.push(member.Id);
  
  try {
-    const response = await $fetch(`/api/members/${member.Id}/create-portal-account`) as any;
+    const response = await $fetch<any>(`/api/members/${member.Id}/create-portal-account`, {
+      method: 'POST'
+    });

    if (response?.success) {
      // Update the member in the local array to reflect the new keycloak_id
      const index = members.value.findIndex(m => m.Id === member.Id);
      if (index !== -1) {
-        members.value[index] = { ...members.value[index], keycloak_id: response.keycloak_id };
+        // Get keycloak_id from response.data
+        members.value[index] = { ...members.value[index], keycloak_id: response.data?.keycloak_id };
      }
      
      showSuccess.value = true;
-      successMessage.value = `Portal account created successfully for ${member.FullName}. They will receive an email with login instructions.`;
+      successMessage.value = response.message || `Portal account created successfully for ${member.FullName}.`;
    } else {
      throw new Error(response?.message || 'Failed to create portal account');
    }
  } catch (err: any) {
    console.error('Error creating portal account:', err);
-    error.value = err.data?.message || err.message || 'Failed to create portal account. Please try again.';
+    
+    // Better error handling
+    let errorMessage = 'Failed to create portal account. Please try again.';
+    if (err.statusCode === 409) {
+      errorMessage = 'This member already has a portal account or a user with this email already exists.';
+    } else if (err.statusCode === 400) {
+      errorMessage = 'Member must have email, first name, and last name to create a portal account.';
+    } else if (err.data?.message) {
+      errorMessage = err.data.message;
+    } else if (err.message) {
+      errorMessage = err.message;
+    }
+    
+    // Show error in snackbar
+    showSuccess.value = true; // Reuse success snackbar for errors
+    successMessage.value = errorMessage;
  } finally {
    // Remove from creating array
    const index = creatingPortalAccountIds.value.indexOf(member.Id);
--- a/server/api/admin/test-email.post.ts
+++ b/server/api/admin/test-email.post.ts
@ -51,18 +51,11 @@ export default defineEventHandler(async (event) => {
    const { getEmailService } = await import('~/server/utils/email');
    const emailService = await getEmailService();
    
-    // Try to verify connection but don't fail if verification doesn't work
-    // Some SMTP servers have issues with verify() but work fine for sending
-    try {
-      const connectionOk = await emailService.verifyConnection();
-      if (connectionOk) {
-        console.log('[api/admin/test-email.post] SMTP connection verified successfully');
-      }
-    } catch (verifyError: any) {
-      console.warn('[api/admin/test-email.post] SMTP verification failed, attempting to send anyway:', verifyError.message);
-    }
-
-    // Attempt to send test email regardless of verification result
+    // Skip verification entirely and just try to send
+    // Many SMTP servers don't support the VERIFY command
+    console.log('[api/admin/test-email.post] Attempting to send test email without verification...');
+    
+    // Attempt to send test email directly
    await emailService.sendTestEmail(body.testEmail);

    console.log('[api/admin/test-email.post] ✅ Test email sent successfully');
--- a/server/api/members/[id]/create-portal-account.post.ts
+++ b/server/api/members/[id]/create-portal-account.post.ts
@ -117,12 +117,13 @@ export default defineEventHandler(async (event) => {
    await updateMember(memberId, { keycloak_id: keycloakId });

    // 9. Send welcome/verification email using our custom email system
-    console.log('[api/members/[id]/create-portal-account.post] Sending welcome/verification email...');
+    console.log('[api/members/[id]/create-portal-account.post] Attempting to send welcome/verification email...');
+    let emailSent = false;
    try {
      const { getEmailService } = await import('~/server/utils/email');
      const { generateEmailVerificationToken } = await import('~/server/utils/email-tokens');
      
-        const emailService = await getEmailService();
+      const emailService = await getEmailService();
      const verificationToken = await generateEmailVerificationToken(keycloakId, member.email);
      const config = useRuntimeConfig();
      const verificationLink = `${config.public.domain}/api/auth/verify-email?token=${verificationToken}`;
@ -134,6 +135,7 @@ export default defineEventHandler(async (event) => {
        memberId: memberId
      });

+      emailSent = true;
      console.log('[api/members/[id]/create-portal-account.post] Welcome email sent successfully');
    } catch (emailError: any) {
      console.error('[api/members/[id]/create-portal-account.post] Failed to send welcome email:', emailError.message);
@ -144,12 +146,15 @@ export default defineEventHandler(async (event) => {

    return {
      success: true,
-      message: 'Portal account created successfully. The member will receive an email to verify their account and set their password.',
+      message: emailSent 
+        ? 'Portal account created successfully. The member will receive an email to verify their account and set their password.'
+        : 'Portal account created successfully. Email sending is not configured - the member will need to request a password reset to access their account.',
      data: {
        keycloak_id: keycloakId,
        member_id: memberId,
        email: member.email,
-        name: `${member.first_name} ${member.last_name}`
+        name: `${member.first_name} ${member.last_name}`,
+        email_sent: emailSent
      }
    };

--- a/server/utils/email.ts
+++ b/server/utils/email.ts
@ -84,11 +84,14 @@ export class EmailService {
        host: this.config.host,
        port: this.config.port,
        secure: useSecure,
-        // Connection timeout settings
-        connectionTimeout: 30000, // 30 seconds
-        greetingTimeout: 30000,
-        socketTimeout: 30000,
-        // Debug logging
+        // Increased timeout settings to handle slow servers
+        connectionTimeout: 60000, // 60 seconds
+        greetingTimeout: 60000,
+        socketTimeout: 60000,
+        // Pool configuration for better connection management
+        pool: false,
+        maxConnections: 1,
+        // Debug logging (can be enabled for troubleshooting)
        logger: false,
        debug: false
      };
@ -96,20 +99,32 @@ export class EmailService {
      // Add requireTLS if needed (for STARTTLS)
      if (requireTLS && !useSecure) {
        transporterOptions.requireTLS = true;
+        transporterOptions.opportunisticTLS = true;
      }

      // Configure TLS options
      transporterOptions.tls = {
        rejectUnauthorized: false, // Accept self-signed certificates
-        // Don't specify minVersion or ciphers to allow auto-negotiation
+        // Allow various TLS versions for compatibility
+        minVersion: 'TLSv1',
+        // Don't specify ciphers to allow auto-negotiation
      };

      // Add authentication only if credentials are provided
      if (this.config.username && this.config.password) {
        transporterOptions.auth = {
          user: this.config.username,
-          pass: this.config.password
+          pass: this.config.password,
+          // Try different auth methods for compatibility
+          type: 'login' // Can be 'oauth2', 'login', or omitted for auto-detection
        };
+        
+        // For some servers, disabling STARTTLS can help
+        if (this.config.port === 587) {
+          transporterOptions.ignoreTLS = false;
+          transporterOptions.secure = false;
+          transporterOptions.requireTLS = true;
+        }
      }

      this.transporter = nodemailer.createTransport(transporterOptions);