# MinIO Implementation Guide - Port Nimara Style This guide provides a comprehensive implementation of MinIO object storage integration based on the Port Nimara Client Portal project. It covers everything from basic setup to advanced features like image thumbnails, lazy loading, and secure file access. ## Table of Contents 1. [Project Setup & Configuration](#project-setup--configuration) 2. [Core MinIO Utility Functions](#core-minio-utility-functions) 3. [API Endpoints Implementation](#api-endpoints-implementation) 4. [Frontend Components](#frontend-components) 5. [Advanced Features](#advanced-features) 6. [Security & Authentication](#security--authentication) 7. [Best Practices](#best-practices) 8. [Troubleshooting](#troubleshooting) ## Project Setup & Configuration ### Dependencies First, install the required dependencies: ```bash npm install minio formidable mime-types npm install --save-dev @types/formidable @types/mime-types ``` ### Environment Variables Add these variables to your `.env` file: ```env # MinIO Configuration NUXT_MINIO_ACCESS_KEY=your-minio-access-key NUXT_MINIO_SECRET_KEY=your-minio-secret-key ``` ### Nuxt Configuration Configure MinIO in your `nuxt.config.ts`: ```typescript export default defineNuxtConfig({ runtimeConfig: { minio: { endPoint: "s3.portnimara.com", // Your MinIO endpoint port: 443, useSSL: true, accessKey: process.env.NUXT_MINIO_ACCESS_KEY, secretKey: process.env.NUXT_MINIO_SECRET_KEY, bucketName: "client-portal", // Default bucket name }, }, }); ``` ## Core MinIO Utility Functions Create `server/utils/minio.ts` with comprehensive file management functions: ```typescript import { Client } from 'minio'; import type { BucketItem } from 'minio'; // Initialize MinIO client export const getMinioClient = () => { const config = useRuntimeConfig().minio; return new Client({ endPoint: config.endPoint, port: config.port, useSSL: config.useSSL, accessKey: config.accessKey, secretKey: config.secretKey, }); }; // File listing with metadata and folder support export const listFiles = async (prefix: string = '', recursive: boolean = false) => { const client = getMinioClient(); const bucketName = useRuntimeConfig().minio.bucketName; const files: any[] = []; const folders = new Set(); return new Promise(async (resolve, reject) => { try { const stream = client.listObjectsV2(bucketName, prefix, recursive); stream.on('data', (obj) => { // Handle folder prefixes returned by MinIO if (obj && obj.prefix) { folders.add(obj.prefix); return; } // Skip objects without a name if (!obj || typeof obj.name !== 'string') { return; } if (!recursive) { if (prefix) { // Extract folder structure when inside a folder const relativePath = obj.name.substring(prefix.length); if (!relativePath) return; // Skip if no relative path const firstSlash = relativePath.indexOf('/'); if (firstSlash > -1) { // This is a folder const folderName = relativePath.substring(0, firstSlash); folders.add(prefix + folderName + '/'); } else if (relativePath && !obj.name.endsWith('/')) { // This is a file in the current folder files.push({ name: obj.name, size: obj.size || 0, lastModified: obj.lastModified || new Date(), etag: obj.etag || '', isFolder: false, }); } } else { // At root level const firstSlash = obj.name.indexOf('/'); if (obj.name.endsWith('/')) { // This is a folder placeholder folders.add(obj.name); } else if (firstSlash > -1) { // This is inside a folder, extract the folder const folderName = obj.name.substring(0, firstSlash); folders.add(folderName + '/'); } else { // This is a file at root files.push({ name: obj.name, size: obj.size || 0, lastModified: obj.lastModified || new Date(), etag: obj.etag || '', isFolder: false, }); } } } else { // When recursive, include all files if (!obj.name.endsWith('/')) { files.push({ name: obj.name, size: obj.size || 0, lastModified: obj.lastModified || new Date(), etag: obj.etag || '', isFolder: false, }); } } }); stream.on('error', (error) => { console.error('Stream error:', error); reject(error); }); stream.on('end', () => { // Add folders to the result const folderItems = Array.from(folders).map(folder => ({ name: folder, size: 0, lastModified: new Date(), etag: '', isFolder: true, })); resolve([...folderItems, ...files]); }); } catch (error) { console.error('Error in listFiles:', error); reject(error); } }); }; // Upload file with content type detection export const uploadFile = async (filePath: string, fileBuffer: Buffer, contentType: string) => { const client = getMinioClient(); const bucketName = useRuntimeConfig().minio.bucketName; return await client.putObject(bucketName, filePath, fileBuffer, fileBuffer.length, { 'Content-Type': contentType, }); }; // Upload buffer (alias for uploadFile for compatibility) export const uploadBuffer = async (buffer: Buffer, filePath: string, contentType: string) => { return uploadFile(filePath, buffer, contentType); }; // Generate presigned URL for secure downloads export const getDownloadUrl = async (fileName: string, expiry: number = 60 * 60) => { const client = getMinioClient(); const bucketName = useRuntimeConfig().minio.bucketName; // Extract just the filename from the full path let filename = fileName.split('/').pop() || fileName; // Remove timestamp prefix if present (e.g., "1234567890-filename.pdf" -> "filename.pdf") const timestampMatch = filename.match(/^\d{10,}-(.+)$/); if (timestampMatch) { filename = timestampMatch[1]; } // Force download with Content-Disposition header const responseHeaders = { 'response-content-disposition': `attachment; filename="${filename}"`, }; return await client.presignedGetObject(bucketName, fileName, expiry, responseHeaders); }; // Get presigned URL for file preview (inline display) export const getPreviewUrl = async (fileName: string, contentType: string) => { const client = getMinioClient(); const bucketName = useRuntimeConfig().minio.bucketName; // For images and PDFs, generate a presigned URL with appropriate response headers const responseHeaders = { 'response-content-type': contentType, 'response-content-disposition': 'inline', }; return await client.presignedGetObject(bucketName, fileName, 60 * 60, responseHeaders); }; // Delete file export const deleteFile = async (fileName: string) => { const client = getMinioClient(); const bucketName = useRuntimeConfig().minio.bucketName; return await client.removeObject(bucketName, fileName); }; // Delete folder (recursively delete all contents) export const deleteFolder = async (folderPath: string) => { const client = getMinioClient(); const bucketName = useRuntimeConfig().minio.bucketName; const objectsList: string[] = []; return new Promise((resolve, reject) => { const stream = client.listObjectsV2(bucketName, folderPath, true); stream.on('data', (obj) => { if (obj && obj.name) { objectsList.push(obj.name); } }); stream.on('error', reject); stream.on('end', async () => { try { if (objectsList.length > 0) { await client.removeObjects(bucketName, objectsList); } resolve(true); } catch (error) { reject(error); } }); }); }; // Get file statistics export const getFileStats = async (fileName: string) => { const client = getMinioClient(); const bucketName = useRuntimeConfig().minio.bucketName; return await client.statObject(bucketName, fileName); }; // Create folder (MinIO doesn't have explicit folders, so we create a placeholder) export const createFolder = async (folderPath: string) => { const client = getMinioClient(); const bucketName = useRuntimeConfig().minio.bucketName; // Ensure folder path ends with / const normalizedPath = folderPath.endsWith('/') ? folderPath : folderPath + '/'; // Create an empty object to represent the folder return await client.putObject(bucketName, normalizedPath, Buffer.from(''), 0); }; // Rename file (copy and delete) export const renameFile = async (oldPath: string, newPath: string) => { const client = getMinioClient(); const bucketName = useRuntimeConfig().minio.bucketName; try { // Copy the object to the new name await client.copyObject( bucketName, newPath, `/${bucketName}/${oldPath}` ); // Delete the old object await client.removeObject(bucketName, oldPath); return true; } catch (error) { console.error('Error renaming file:', error); throw error; } }; // Rename folder (copy all contents and delete) export const renameFolder = async (oldPath: string, newPath: string) => { const client = getMinioClient(); const bucketName = useRuntimeConfig().minio.bucketName; // Ensure paths end with / const oldPrefix = oldPath.endsWith('/') ? oldPath : oldPath + '/'; const newPrefix = newPath.endsWith('/') ? newPath : newPath + '/'; const objectsList: string[] = []; return new Promise((resolve, reject) => { const stream = client.listObjectsV2(bucketName, oldPrefix, true); stream.on('data', (obj) => { if (obj && obj.name) { objectsList.push(obj.name); } }); stream.on('error', reject); stream.on('end', async () => { try { // Copy all objects to new location for (const objectName of objectsList) { const newObjectName = objectName.replace(oldPrefix, newPrefix); await client.copyObject( bucketName, newObjectName, `/${bucketName}/${objectName}` ); } // Delete all old objects if (objectsList.length > 0) { await client.removeObjects(bucketName, objectsList); } resolve(true); } catch (error) { reject(error); } }); }); }; // Create bucket if it doesn't exist export const createBucketIfNotExists = async (bucketName?: string) => { const client = getMinioClient(); const bucket = bucketName || useRuntimeConfig().minio.bucketName; try { const exists = await client.bucketExists(bucket); if (!exists) { await client.makeBucket(bucket); console.log(`Bucket '${bucket}' created successfully`); } return true; } catch (error) { console.error('Error creating bucket:', error); throw error; } }; ``` ## API Endpoints Implementation ### File Upload API Create `server/api/files/upload.ts`: ```typescript import { requireAuth } from '~/server/utils/auth'; import { uploadFile, getMinioClient } from '~/server/utils/minio'; import formidable from 'formidable'; import { promises as fs } from 'fs'; import mime from 'mime-types'; export default defineEventHandler(async (event) => { // Check authentication await requireAuth(event); try { const query = getQuery(event); const currentPath = (query.path as string) || ''; const bucket = (query.bucket as string) || 'client-portal'; console.log('[Upload] Request received for bucket:', bucket, 'path:', currentPath); // Parse multipart form data const form = formidable({ maxFileSize: 50 * 1024 * 1024, // 50MB limit keepExtensions: true, }); const [fields, files] = await form.parse(event.node.req); // Handle multiple files const uploadedFiles = Array.isArray(files.file) ? files.file : [files.file]; const results = []; for (const uploadedFile of uploadedFiles) { if (!uploadedFile) continue; // Read file buffer const fileBuffer = await fs.readFile(uploadedFile.filepath); // Generate unique filename to prevent collisions const timestamp = Date.now(); const sanitizedName = uploadedFile.originalFilename?.replace(/[^a-zA-Z0-9.-]/g, '_') || 'file'; const fileName = `${timestamp}-${sanitizedName}`; // Construct full path including current folder let normalizedPath = currentPath; if (normalizedPath && !normalizedPath.endsWith('/')) { normalizedPath += '/'; } const fullPath = normalizedPath ? `${normalizedPath}${fileName}` : fileName; // Get content type const contentType = mime.lookup(uploadedFile.originalFilename || '') || 'application/octet-stream'; // Upload to MinIO - handle different buckets if (bucket === 'client-portal') { await uploadFile(fullPath, fileBuffer, contentType); } else { // For other buckets, use the MinIO client directly const client = getMinioClient(); // Ensure bucket exists try { await client.bucketExists(bucket); } catch (err) { console.log(`[Upload] Bucket ${bucket} doesn't exist, creating it...`); await client.makeBucket(bucket, 'us-east-1'); } await client.putObject(bucket, fullPath, fileBuffer, fileBuffer.length, { 'Content-Type': contentType, }); } // Clean up temp file await fs.unlink(uploadedFile.filepath); results.push({ fileName: fullPath, path: fullPath, originalName: uploadedFile.originalFilename, size: uploadedFile.size, contentType, bucket: bucket }); // Log audit event await logAuditEvent(event, 'upload', fullPath, uploadedFile.size); } // Return appropriate response if (results.length === 1) { return { success: true, path: results[0].path, fileName: results[0].fileName, files: results, message: `File uploaded successfully`, }; } return { success: true, files: results, message: `${results.length} file(s) uploaded successfully`, }; } catch (error: any) { console.error('Failed to upload file:', error); throw createError({ statusCode: 500, statusMessage: error.message || 'Failed to upload file', }); } }); // Audit logging helper async function logAuditEvent(event: any, action: string, filePath: string, fileSize?: number) { try { const user = event.context.user || { email: 'anonymous' }; const auditLog = { user_email: user.email, action, file_path: filePath, file_size: fileSize, timestamp: new Date().toISOString(), ip_address: getClientIP(event), success: true, }; console.log('Audit log:', auditLog); // Store in your database or logging system here } catch (error) { console.error('Failed to log audit event:', error); } } function getClientIP(event: any): string { return event.node.req.headers['x-forwarded-for'] || event.node.req.connection.remoteAddress || 'unknown'; } ``` ### File Preview API Create `server/api/files/preview.ts`: ```typescript import { requireAuth } from '~/server/utils/auth'; import { getPreviewUrl } from '~/server/utils/minio'; import mime from 'mime-types'; export default defineEventHandler(async (event) => { await requireAuth(event); try { const query = getQuery(event); const fileName = query.fileName as string; if (!fileName) { throw createError({ statusCode: 400, statusMessage: 'File name is required', }); } // Get content type const contentType = mime.lookup(fileName) || 'application/octet-stream'; // Check if file type supports preview const supportedPreviewTypes = [ 'image/jpeg', 'image/jpg', 'image/png', 'image/gif', 'image/svg+xml', 'image/webp', 'application/pdf', ]; if (!supportedPreviewTypes.includes(contentType)) { throw createError({ statusCode: 400, statusMessage: 'File type does not support preview', }); } // Generate presigned URL for preview const url = await getPreviewUrl(fileName, contentType); console.log('Preview URL generated:', { fileName, contentType, url: url.substring(0, 100) + '...' // Log first 100 chars for security }); return { success: true, url, fileName, contentType, }; } catch (error: any) { console.error('Failed to generate preview URL:', error); throw createError({ statusCode: 500, statusMessage: error.message || 'Failed to generate preview URL', }); } }); ``` ### File Download API Create `server/api/files/download.ts`: ```typescript import { requireAuth } from '~/server/utils/auth'; import { getDownloadUrl } from '~/server/utils/minio'; export default defineEventHandler(async (event) => { await requireAuth(event); try { const query = getQuery(event); const fileName = query.fileName as string; const expiry = parseInt(query.expiry as string) || 3600; // Default 1 hour if (!fileName) { throw createError({ statusCode: 400, statusMessage: 'File name is required', }); } // Generate presigned URL for download const url = await getDownloadUrl(fileName, expiry); return { success: true, url, fileName, expiresIn: expiry, }; } catch (error: any) { console.error('Failed to generate download URL:', error); throw createError({ statusCode: 500, statusMessage: error.message || 'Failed to generate download URL', }); } }); ``` ### File List API Create `server/api/files/list.ts`: ```typescript import { requireAuth } from '~/server/utils/auth'; import { listFiles } from '~/server/utils/minio'; export default defineEventHandler(async (event) => { await requireAuth(event); try { const query = getQuery(event); const prefix = (query.prefix as string) || ''; const recursive = query.recursive === 'true'; const files = await listFiles(prefix, recursive); return { success: true, files, prefix, recursive, }; } catch (error: any) { console.error('Failed to list files:', error); throw createError({ statusCode: 500, statusMessage: error.message || 'Failed to list files', }); } }); ``` ## Frontend Components ### File Uploader Component Create `components/FileUploader.vue`: ```vue ``` ### Lazy Image Component with Thumbnails Create `components/LazyImageViewer.vue`: ```vue ``` ## Advanced Features ### Image Thumbnails and Processing Create `server/utils/image-processor.ts`: ```typescript import sharp from 'sharp'; import { uploadFile } from './minio'; import path from 'path'; interface ThumbnailSizes { tiny: { width: 150, height: 150 }; small: { width: 300, height: 300 }; medium: { width: 600, height: 600 }; large: { width: 1200, height: 1200 }; card_cover: { width: 400, height: 250 }; } const THUMBNAIL_SIZES: ThumbnailSizes = { tiny: { width: 150, height: 150 }, small: { width: 300, height: 300 }, medium: { width: 600, height: 600 }, large: { width: 1200, height: 1200 }, card_cover: { width: 400, height: 250 } }; export const generateImageThumbnails = async ( originalBuffer: Buffer, originalPath: string, contentType: string ) => { const thumbnails: Record = {}; // Check if file is an image if (!contentType.startsWith('image/')) { return thumbnails; } try { const pathInfo = path.parse(originalPath); for (const [sizeName, dimensions] of Object.entries(THUMBNAIL_SIZES)) { try { const thumbnailBuffer = await sharp(originalBuffer) .resize(dimensions.width, dimensions.height, { fit: 'cover', position: 'center' }) .jpeg({ quality: 85 }) .toBuffer(); const thumbnailPath = `${pathInfo.dir}/${pathInfo.name}_${sizeName}${pathInfo.ext}`; await uploadFile(thumbnailPath, thumbnailBuffer, 'image/jpeg'); thumbnails[sizeName] = { path: thumbnailPath, width: dimensions.width, height: dimensions.height, size: thumbnailBuffer.length }; console.log(`Generated ${sizeName} thumbnail: ${thumbnailPath}`); } catch (error) { console.error(`Failed to generate ${sizeName} thumbnail:`, error); } } } catch (error) { console.error('Failed to generate thumbnails:', error); } return thumbnails; }; export const deleteImageThumbnails = async (originalPath: string) => { const { deleteFile } = await import('./minio'); const pathInfo = path.parse(originalPath); for (const sizeName of Object.keys(THUMBNAIL_SIZES)) { try { const thumbnailPath = `${pathInfo.dir}/${pathInfo.name}_${sizeName}${pathInfo.ext}`; await deleteFile(thumbnailPath); console.log(`Deleted ${sizeName} thumbnail: ${thumbnailPath}`); } catch (error) { console.error(`Failed to delete ${sizeName} thumbnail:`, error); } } }; ``` ### File Browser Component Create `components/FileBrowser.vue`: ```vue ``` ## Security & Authentication ### Authentication Integration Your MinIO setup includes robust authentication through the `requireAuth` utility. Here's how to implement it: Create `server/utils/auth.ts` if you don't have it: ```typescript import jwt from 'jsonwebtoken'; import { getCookie } from 'h3'; export interface AuthUser { email: string; name: string; roles?: string[]; permissions?: string[]; } export const requireAuth = async (event: any): Promise => { // Check for x-tag header (API key authentication) const apiKey = getHeader(event, 'x-tag'); if (apiKey) { const validApiKey = useRuntimeConfig().apiKey; if (apiKey === validApiKey) { return { email: 'api-user', name: 'API User' }; } throw createError({ statusCode: 401, statusMessage: 'Invalid API key', }); } // Check for session-based authentication (Keycloak/OAuth) const sessionCookie = getCookie(event, 'auth-token') || getCookie(event, 'nuxt-oidc-auth-session'); if (!sessionCookie) { throw createError({ statusCode: 401, statusMessage: 'Authentication required', }); } try { // Verify JWT token const decoded = jwt.verify(sessionCookie, useRuntimeConfig().jwtSecret) as any; const user: AuthUser = { email: decoded.email || decoded.preferred_username, name: decoded.name || decoded.preferred_username, roles: decoded.realm_access?.roles || [], permissions: decoded.permissions || [] }; // Store user in event context for later use event.context.user = user; return user; } catch (error) { throw createError({ statusCode: 401, statusMessage: 'Invalid authentication token', }); } }; export const checkPermission = (user: AuthUser, permission: string): boolean => { return user.permissions?.includes(permission) || user.roles?.includes('admin') || false; }; ``` ### File Access Control Implement role-based access control for files: ```typescript // server/utils/file-permissions.ts export const checkFileAccess = (user: AuthUser, filePath: string, action: 'read' | 'write' | 'delete'): boolean => { // Admin can do everything if (user.roles?.includes('admin')) { return true; } // Users can only access their own files if (filePath.startsWith(`users/${user.email}/`)) { return true; } // Check shared folders if (filePath.startsWith('shared/') && action === 'read') { return true; } // Public files are read-only for everyone if (filePath.startsWith('public/') && action === 'read') { return true; } return false; }; ``` ## Best Practices ### 1. File Naming Conventions - Use timestamps to prevent filename collisions: `${timestamp}-${sanitizedName}` - Sanitize filenames: `filename.replace(/[^a-zA-Z0-9.-]/g, '_')` - Store original filenames separately for display purposes ### 2. Error Handling ```typescript // Comprehensive error handling in API endpoints try { await uploadFile(filePath, buffer, contentType); } catch (error: any) { console.error('Upload failed:', { error: error.message, filePath, contentType, size: buffer.length }); throw createError({ statusCode: error.code === 'NoSuchBucket' ? 404 : 500, statusMessage: getErrorMessage(error), }); } const getErrorMessage = (error: any): string => { const errorMessages: Record = { 'NoSuchBucket': 'Storage bucket not found', 'AccessDenied': 'Access denied to storage', 'InvalidBucketName': 'Invalid bucket name', 'BucketNotEmpty': 'Bucket is not empty' }; return errorMessages[error.code] || 'Storage operation failed'; }; ``` ### 3. Performance Optimization - Use lazy loading for image thumbnails - Implement client-side caching for file lists - Use presigned URLs to reduce server load - Implement pagination for large file lists ### 4. Security Considerations - Always validate file types and sizes - Use presigned URLs with short expiration times - Implement proper authentication on all endpoints - Sanitize file paths to prevent directory traversal - Audit all file operations ### 5. Monitoring and Logging ```typescript // Comprehensive audit logging const auditLog = { timestamp: new Date().toISOString(), user_email: user.email, action: 'upload', file_path: filePath, file_size: fileSize, ip_address: getClientIP(event), user_agent: getHeader(event, 'user-agent'), success: true, duration_ms: Date.now() - startTime }; // Send to your logging system await logToDatabase(auditLog); ``` ## Troubleshooting ### Common Issues and Solutions #### 1. Connection Errors ```bash # Check MinIO server connectivity curl -X GET https://your-minio-endpoint/health # Verify credentials minio-client admin info myminio ``` #### 2. Upload Failures - Check file size limits (50MB default) - Verify content-type detection - Ensure proper multipart form parsing - Check bucket permissions #### 3. Presigned URL Issues - Verify server time synchronization - Check URL expiration settings - Ensure proper bucket policies - Validate CORS settings for browser uploads #### 4. Thumbnail Generation ```typescript // Check Sharp installation and image processing try { await sharp(buffer).metadata(); } catch (error) { console.error('Sharp processing failed:', error); // Install: npm install sharp } ``` #### 5. Performance Issues - Enable MinIO compression - Use CDN for frequently accessed files - Implement proper caching headers - Monitor bucket metrics ### Development vs Production Configuration ```typescript // Development const minioConfig = { endPoint: 'localhost', port: 9000, useSSL: false, accessKey: 'minioadmin', secretKey: 'minioadmin', }; // Production const minioConfig = { endPoint: 's3.yourdomain.com', port: 443, useSSL: true, accessKey: process.env.MINIO_ACCESS_KEY, secretKey: process.env.MINIO_SECRET_KEY, }; ``` ## Deployment Considerations ### Docker Configuration ```dockerfile # Add to your Dockerfile RUN npm install sharp --platform=linux --arch=x64 ``` ### Environment Variables ```env # Production environment NUXT_MINIO_ENDPOINT=s3.yourdomain.com NUXT_MINIO_PORT=443 NUXT_MINIO_USE_SSL=true NUXT_MINIO_ACCESS_KEY=your-production-access-key NUXT_MINIO_SECRET_KEY=your-production-secret-key NUXT_MINIO_BUCKET_NAME=your-production-bucket ``` ### MinIO Server Setup ```bash # Start MinIO server docker run -p 9000:9000 -p 9001:9001 \ -e "MINIO_ROOT_USER=admin" \ -e "MINIO_ROOT_PASSWORD=password123" \ -v /mnt/data:/data \ minio/minio server /data --console-address ":9001" ``` ## Conclusion This implementation guide provides a complete MinIO integration following the Port Nimara pattern. It includes: - ✅ Complete file management operations - ✅ Secure authentication and authorization - ✅ Image thumbnails and processing - ✅ Lazy loading and performance optimization - ✅ Comprehensive error handling - ✅ Audit logging and monitoring - ✅ Production-ready configuration The implementation is battle-tested and provides a solid foundation for any project requiring robust file storage capabilities.