import { z } from 'zod' import { TRPCError } from '@trpc/server' import { router, protectedProcedure, adminProcedure, juryProcedure } from '../trpc' import { logAudit } from '@/server/utils/audit' import { notifyAdmins, NotificationTypes } from '../services/in-app-notification' import { processEvaluationReminders } from '../services/evaluation-reminders' import { generateSummary } from '@/server/services/ai-evaluation-summary' export const evaluationRouter = router({ /** * Get evaluation for an assignment */ get: protectedProcedure .input(z.object({ assignmentId: z.string() })) .query(async ({ ctx, input }) => { // Verify ownership or admin const assignment = await ctx.prisma.assignment.findUniqueOrThrow({ where: { id: input.assignmentId }, include: { round: true }, }) if ( ctx.user.role === 'JURY_MEMBER' && assignment.userId !== ctx.user.id ) { throw new TRPCError({ code: 'FORBIDDEN' }) } return ctx.prisma.evaluation.findUnique({ where: { assignmentId: input.assignmentId }, include: { form: true, }, }) }), /** * Start an evaluation (creates draft) */ start: protectedProcedure .input( z.object({ assignmentId: z.string(), }) ) .mutation(async ({ ctx, input }) => { // Verify assignment ownership const assignment = await ctx.prisma.assignment.findUniqueOrThrow({ where: { id: input.assignmentId }, include: { round: { include: { evaluationForms: { where: { isActive: true }, take: 1 }, }, }, }, }) if (assignment.userId !== ctx.user.id) { throw new TRPCError({ code: 'FORBIDDEN' }) } // Get active form const form = assignment.round.evaluationForms[0] if (!form) { throw new TRPCError({ code: 'BAD_REQUEST', message: 'No active evaluation form for this round', }) } // Check if evaluation exists const existing = await ctx.prisma.evaluation.findUnique({ where: { assignmentId: input.assignmentId }, }) if (existing) return existing return ctx.prisma.evaluation.create({ data: { assignmentId: input.assignmentId, formId: form.id, status: 'DRAFT', }, }) }), /** * Autosave evaluation (debounced on client) */ autosave: protectedProcedure .input( z.object({ id: z.string(), criterionScoresJson: z.record(z.union([z.number(), z.string(), z.boolean()])).optional(), globalScore: z.number().int().min(1).max(10).optional().nullable(), binaryDecision: z.boolean().optional().nullable(), feedbackText: z.string().optional().nullable(), }) ) .mutation(async ({ ctx, input }) => { const { id, ...data } = input // Verify ownership and status const evaluation = await ctx.prisma.evaluation.findUniqueOrThrow({ where: { id }, include: { assignment: true }, }) if (evaluation.assignment.userId !== ctx.user.id) { throw new TRPCError({ code: 'FORBIDDEN' }) } if ( evaluation.status === 'SUBMITTED' || evaluation.status === 'LOCKED' ) { throw new TRPCError({ code: 'BAD_REQUEST', message: 'Cannot edit submitted evaluation', }) } return ctx.prisma.evaluation.update({ where: { id }, data: { ...data, status: 'DRAFT', }, }) }), /** * Submit evaluation (final) */ submit: protectedProcedure .input( z.object({ id: z.string(), criterionScoresJson: z.record(z.union([z.number(), z.string(), z.boolean()])), globalScore: z.number().int().min(1).max(10), binaryDecision: z.boolean(), feedbackText: z.string().min(10), }) ) .mutation(async ({ ctx, input }) => { const { id, ...data } = input // Verify ownership const evaluation = await ctx.prisma.evaluation.findUniqueOrThrow({ where: { id }, include: { assignment: { include: { round: true }, }, }, }) if (evaluation.assignment.userId !== ctx.user.id) { throw new TRPCError({ code: 'FORBIDDEN' }) } // Check voting window const round = evaluation.assignment.round const now = new Date() if (round.status !== 'ACTIVE') { throw new TRPCError({ code: 'BAD_REQUEST', message: 'Round is not active', }) } // Check for grace period const gracePeriod = await ctx.prisma.gracePeriod.findFirst({ where: { roundId: round.id, userId: ctx.user.id, OR: [ { projectId: null }, { projectId: evaluation.assignment.projectId }, ], extendedUntil: { gte: now }, }, }) const effectiveEndDate = gracePeriod?.extendedUntil ?? round.votingEndAt if (round.votingStartAt && now < round.votingStartAt) { throw new TRPCError({ code: 'BAD_REQUEST', message: 'Voting has not started yet', }) } if (effectiveEndDate && now > effectiveEndDate) { throw new TRPCError({ code: 'BAD_REQUEST', message: 'Voting window has closed', }) } // Submit evaluation and mark assignment as completed atomically const [updated] = await ctx.prisma.$transaction([ ctx.prisma.evaluation.update({ where: { id }, data: { ...data, status: 'SUBMITTED', submittedAt: now, }, }), ctx.prisma.assignment.update({ where: { id: evaluation.assignmentId }, data: { isCompleted: true }, }), ]) // Audit log await logAudit({ prisma: ctx.prisma, userId: ctx.user.id, action: 'EVALUATION_SUBMITTED', entityType: 'Evaluation', entityId: id, detailsJson: { projectId: evaluation.assignment.projectId, roundId: evaluation.assignment.roundId, globalScore: data.globalScore, binaryDecision: data.binaryDecision, }, ipAddress: ctx.ip, userAgent: ctx.userAgent, }) return updated }), /** * Get aggregated stats for a project (admin only) */ getProjectStats: adminProcedure .input(z.object({ projectId: z.string() })) .query(async ({ ctx, input }) => { const evaluations = await ctx.prisma.evaluation.findMany({ where: { status: 'SUBMITTED', assignment: { projectId: input.projectId }, }, }) if (evaluations.length === 0) { return null } const globalScores = evaluations .map((e) => e.globalScore) .filter((s): s is number => s !== null) const yesVotes = evaluations.filter( (e) => e.binaryDecision === true ).length return { totalEvaluations: evaluations.length, averageGlobalScore: globalScores.length > 0 ? globalScores.reduce((a, b) => a + b, 0) / globalScores.length : null, minScore: globalScores.length > 0 ? Math.min(...globalScores) : null, maxScore: globalScores.length > 0 ? Math.max(...globalScores) : null, yesVotes, noVotes: evaluations.length - yesVotes, yesPercentage: (yesVotes / evaluations.length) * 100, } }), /** * Get all evaluations for a round (admin only) */ listByRound: adminProcedure .input( z.object({ roundId: z.string(), status: z.enum(['NOT_STARTED', 'DRAFT', 'SUBMITTED', 'LOCKED']).optional(), }) ) .query(async ({ ctx, input }) => { return ctx.prisma.evaluation.findMany({ where: { assignment: { roundId: input.roundId }, ...(input.status && { status: input.status }), }, include: { assignment: { include: { user: { select: { id: true, name: true, email: true } }, project: { select: { id: true, title: true } }, }, }, }, orderBy: { updatedAt: 'desc' }, }) }), /** * Get my past evaluations (read-only for jury) */ myPastEvaluations: protectedProcedure .input(z.object({ roundId: z.string().optional() })) .query(async ({ ctx, input }) => { return ctx.prisma.evaluation.findMany({ where: { assignment: { userId: ctx.user.id, ...(input.roundId && { roundId: input.roundId }), }, status: 'SUBMITTED', }, include: { assignment: { include: { project: { select: { id: true, title: true } }, round: { select: { id: true, name: true } }, }, }, }, orderBy: { submittedAt: 'desc' }, }) }), // ========================================================================= // Conflict of Interest (COI) Endpoints // ========================================================================= /** * Declare a conflict of interest for an assignment */ declareCOI: protectedProcedure .input( z.object({ assignmentId: z.string(), hasConflict: z.boolean(), conflictType: z.string().optional(), description: z.string().optional(), }) ) .mutation(async ({ ctx, input }) => { // Look up the assignment to get projectId, roundId, userId const assignment = await ctx.prisma.assignment.findUniqueOrThrow({ where: { id: input.assignmentId }, include: { project: { select: { title: true } }, round: { select: { name: true } }, }, }) // Verify ownership if (assignment.userId !== ctx.user.id) { throw new TRPCError({ code: 'FORBIDDEN' }) } // Upsert COI record const coi = await ctx.prisma.conflictOfInterest.upsert({ where: { assignmentId: input.assignmentId }, create: { assignmentId: input.assignmentId, userId: ctx.user.id, projectId: assignment.projectId, roundId: assignment.roundId, hasConflict: input.hasConflict, conflictType: input.hasConflict ? input.conflictType : null, description: input.hasConflict ? input.description : null, }, update: { hasConflict: input.hasConflict, conflictType: input.hasConflict ? input.conflictType : null, description: input.hasConflict ? input.description : null, declaredAt: new Date(), }, }) // Notify admins if conflict declared if (input.hasConflict) { await notifyAdmins({ type: NotificationTypes.JURY_INACTIVE, title: 'Conflict of Interest Declared', message: `${ctx.user.name || ctx.user.email} declared a conflict of interest (${input.conflictType || 'unspecified'}) for project "${assignment.project.title}" in ${assignment.round.name}.`, linkUrl: `/admin/rounds/${assignment.roundId}/coi`, linkLabel: 'Review COI', priority: 'high', metadata: { assignmentId: input.assignmentId, userId: ctx.user.id, projectId: assignment.projectId, roundId: assignment.roundId, conflictType: input.conflictType, }, }) } // Audit log await logAudit({ prisma: ctx.prisma, userId: ctx.user.id, action: 'COI_DECLARED', entityType: 'ConflictOfInterest', entityId: coi.id, detailsJson: { assignmentId: input.assignmentId, projectId: assignment.projectId, roundId: assignment.roundId, hasConflict: input.hasConflict, conflictType: input.conflictType, }, ipAddress: ctx.ip, userAgent: ctx.userAgent, }) return coi }), /** * Get COI status for an assignment */ getCOIStatus: protectedProcedure .input(z.object({ assignmentId: z.string() })) .query(async ({ ctx, input }) => { return ctx.prisma.conflictOfInterest.findUnique({ where: { assignmentId: input.assignmentId }, }) }), /** * List COI declarations for a round (admin only) */ listCOIByRound: adminProcedure .input( z.object({ roundId: z.string(), hasConflictOnly: z.boolean().optional(), }) ) .query(async ({ ctx, input }) => { return ctx.prisma.conflictOfInterest.findMany({ where: { roundId: input.roundId, ...(input.hasConflictOnly && { hasConflict: true }), }, include: { user: { select: { id: true, name: true, email: true } }, assignment: { include: { project: { select: { id: true, title: true } }, }, }, reviewedBy: { select: { id: true, name: true, email: true } }, }, orderBy: { declaredAt: 'desc' }, }) }), /** * Review a COI declaration (admin only) */ reviewCOI: adminProcedure .input( z.object({ id: z.string(), reviewAction: z.enum(['cleared', 'reassigned', 'noted']), }) ) .mutation(async ({ ctx, input }) => { const coi = await ctx.prisma.conflictOfInterest.update({ where: { id: input.id }, data: { reviewedById: ctx.user.id, reviewedAt: new Date(), reviewAction: input.reviewAction, }, }) // Audit log await logAudit({ prisma: ctx.prisma, userId: ctx.user.id, action: 'COI_REVIEWED', entityType: 'ConflictOfInterest', entityId: input.id, detailsJson: { reviewAction: input.reviewAction, assignmentId: coi.assignmentId, userId: coi.userId, projectId: coi.projectId, }, ipAddress: ctx.ip, userAgent: ctx.userAgent, }) return coi }), // ========================================================================= // Reminder Triggers // ========================================================================= /** * Manually trigger reminder check for a specific round (admin only) */ triggerReminders: adminProcedure .input(z.object({ roundId: z.string() })) .mutation(async ({ ctx, input }) => { const result = await processEvaluationReminders(input.roundId) await logAudit({ prisma: ctx.prisma, userId: ctx.user.id, action: 'REMINDERS_TRIGGERED', entityType: 'Round', entityId: input.roundId, detailsJson: { sent: result.sent, errors: result.errors, }, ipAddress: ctx.ip, userAgent: ctx.userAgent, }) return result }), // ========================================================================= // AI Evaluation Summary Endpoints // ========================================================================= /** * Generate an AI-powered evaluation summary for a project (admin only) */ generateSummary: adminProcedure .input( z.object({ projectId: z.string(), roundId: z.string(), }) ) .mutation(async ({ ctx, input }) => { return generateSummary({ projectId: input.projectId, roundId: input.roundId, userId: ctx.user.id, prisma: ctx.prisma, }) }), /** * Get an existing evaluation summary for a project (admin only) */ getSummary: adminProcedure .input( z.object({ projectId: z.string(), roundId: z.string(), }) ) .query(async ({ ctx, input }) => { return ctx.prisma.evaluationSummary.findUnique({ where: { projectId_roundId: { projectId: input.projectId, roundId: input.roundId, }, }, }) }), /** * Generate summaries for all projects in a round with submitted evaluations (admin only) */ generateBulkSummaries: adminProcedure .input(z.object({ roundId: z.string() })) .mutation(async ({ ctx, input }) => { // Find all projects in the round with at least 1 submitted evaluation const projects = await ctx.prisma.project.findMany({ where: { roundId: input.roundId, assignments: { some: { evaluation: { status: 'SUBMITTED', }, }, }, }, select: { id: true }, }) let generated = 0 const errors: Array<{ projectId: string; error: string }> = [] // Generate summaries sequentially to avoid rate limits for (const project of projects) { try { await generateSummary({ projectId: project.id, roundId: input.roundId, userId: ctx.user.id, prisma: ctx.prisma, }) generated++ } catch (error) { errors.push({ projectId: project.id, error: error instanceof Error ? error.message : 'Unknown error', }) } } return { total: projects.length, generated, errors, } }), // ========================================================================= // Side-by-Side Comparison (F4) // ========================================================================= /** * Get multiple projects with evaluations for side-by-side comparison */ getMultipleForComparison: juryProcedure .input( z.object({ projectIds: z.array(z.string()).min(2).max(3), roundId: z.string(), }) ) .query(async ({ ctx, input }) => { // Verify all projects are assigned to current user in this round const assignments = await ctx.prisma.assignment.findMany({ where: { userId: ctx.user.id, roundId: input.roundId, projectId: { in: input.projectIds }, }, include: { project: { select: { id: true, title: true, teamName: true, description: true, country: true, tags: true, files: { select: { id: true, fileName: true, fileType: true, size: true, }, }, }, }, evaluation: true, }, }) if (assignments.length !== input.projectIds.length) { throw new TRPCError({ code: 'FORBIDDEN', message: 'You are not assigned to all requested projects in this round', }) } return assignments.map((a) => ({ project: a.project, evaluation: a.evaluation, assignmentId: a.id, })) }), // ========================================================================= // Peer Review & Discussion (F13) // ========================================================================= /** * Get anonymized peer evaluation summary for a project */ getPeerSummary: juryProcedure .input( z.object({ projectId: z.string(), roundId: z.string(), }) ) .query(async ({ ctx, input }) => { // Verify user has submitted their own evaluation first const userAssignment = await ctx.prisma.assignment.findFirst({ where: { userId: ctx.user.id, projectId: input.projectId, roundId: input.roundId, }, include: { evaluation: true }, }) if (!userAssignment || userAssignment.evaluation?.status !== 'SUBMITTED') { throw new TRPCError({ code: 'PRECONDITION_FAILED', message: 'You must submit your own evaluation before viewing peer summaries', }) } // Check round settings for peer review const round = await ctx.prisma.round.findUniqueOrThrow({ where: { id: input.roundId }, }) const settings = (round.settingsJson as Record) || {} if (!settings.peer_review_enabled) { throw new TRPCError({ code: 'FORBIDDEN', message: 'Peer review is not enabled for this round', }) } // Get all submitted evaluations for this project const evaluations = await ctx.prisma.evaluation.findMany({ where: { status: 'SUBMITTED', assignment: { projectId: input.projectId, roundId: input.roundId, }, }, include: { assignment: { include: { user: { select: { id: true, name: true } }, }, }, }, }) if (evaluations.length === 0) { return { aggregated: null, individualScores: [], totalEvaluations: 0 } } // Calculate average and stddev per criterion const criterionData: Record = {} evaluations.forEach((e) => { const scores = e.criterionScoresJson as Record | null if (scores) { Object.entries(scores).forEach(([key, val]) => { if (typeof val === 'number') { if (!criterionData[key]) criterionData[key] = [] criterionData[key].push(val) } }) } }) const aggregated: Record }> = {} Object.entries(criterionData).forEach(([key, scores]) => { const avg = scores.reduce((a, b) => a + b, 0) / scores.length const variance = scores.reduce((sum, s) => sum + Math.pow(s - avg, 2), 0) / scores.length const stddev = Math.sqrt(variance) const distribution: Record = {} scores.forEach((s) => { const bucket = Math.round(s) distribution[bucket] = (distribution[bucket] || 0) + 1 }) aggregated[key] = { average: avg, stddev, count: scores.length, distribution } }) // Anonymize individual scores based on round settings const anonymizationLevel = (settings.anonymization_level as string) || 'fully_anonymous' const individualScores = evaluations.map((e) => { let jurorLabel: string if (anonymizationLevel === 'named') { jurorLabel = e.assignment.user.name || 'Juror' } else if (anonymizationLevel === 'show_initials') { const name = e.assignment.user.name || '' jurorLabel = name .split(' ') .map((n) => n[0]) .join('') .toUpperCase() || 'J' } else { jurorLabel = `Juror ${evaluations.indexOf(e) + 1}` } return { jurorLabel, globalScore: e.globalScore, binaryDecision: e.binaryDecision, criterionScoresJson: e.criterionScoresJson, } }) return { aggregated, individualScores, totalEvaluations: evaluations.length, } }), /** * Get or create a discussion for a project evaluation */ getDiscussion: juryProcedure .input( z.object({ projectId: z.string(), roundId: z.string(), }) ) .query(async ({ ctx, input }) => { // Get or create discussion let discussion = await ctx.prisma.evaluationDiscussion.findUnique({ where: { projectId_roundId: { projectId: input.projectId, roundId: input.roundId, }, }, include: { comments: { include: { user: { select: { id: true, name: true } }, }, orderBy: { createdAt: 'asc' }, }, }, }) if (!discussion) { discussion = await ctx.prisma.evaluationDiscussion.create({ data: { projectId: input.projectId, roundId: input.roundId, }, include: { comments: { include: { user: { select: { id: true, name: true } }, }, orderBy: { createdAt: 'asc' }, }, }, }) } // Anonymize comments based on round settings const round = await ctx.prisma.round.findUniqueOrThrow({ where: { id: input.roundId }, }) const settings = (round.settingsJson as Record) || {} const anonymizationLevel = (settings.anonymization_level as string) || 'fully_anonymous' const anonymizedComments = discussion.comments.map((c, idx) => { let authorLabel: string if (anonymizationLevel === 'named' || c.userId === ctx.user.id) { authorLabel = c.user.name || 'Juror' } else if (anonymizationLevel === 'show_initials') { const name = c.user.name || '' authorLabel = name .split(' ') .map((n) => n[0]) .join('') .toUpperCase() || 'J' } else { authorLabel = `Juror ${idx + 1}` } return { id: c.id, authorLabel, isOwn: c.userId === ctx.user.id, content: c.content, createdAt: c.createdAt, } }) return { id: discussion.id, status: discussion.status, createdAt: discussion.createdAt, closedAt: discussion.closedAt, comments: anonymizedComments, } }), /** * Add a comment to a project evaluation discussion */ addComment: juryProcedure .input( z.object({ projectId: z.string(), roundId: z.string(), content: z.string().min(1).max(2000), }) ) .mutation(async ({ ctx, input }) => { // Check max comment length from round settings const round = await ctx.prisma.round.findUniqueOrThrow({ where: { id: input.roundId }, }) const settings = (round.settingsJson as Record) || {} const maxLength = (settings.max_comment_length as number) || 2000 if (input.content.length > maxLength) { throw new TRPCError({ code: 'BAD_REQUEST', message: `Comment exceeds maximum length of ${maxLength} characters`, }) } // Get or create discussion let discussion = await ctx.prisma.evaluationDiscussion.findUnique({ where: { projectId_roundId: { projectId: input.projectId, roundId: input.roundId, }, }, }) if (!discussion) { discussion = await ctx.prisma.evaluationDiscussion.create({ data: { projectId: input.projectId, roundId: input.roundId, }, }) } if (discussion.status === 'closed') { throw new TRPCError({ code: 'BAD_REQUEST', message: 'This discussion has been closed', }) } const comment = await ctx.prisma.discussionComment.create({ data: { discussionId: discussion.id, userId: ctx.user.id, content: input.content, }, }) // Audit log try { await logAudit({ prisma: ctx.prisma, userId: ctx.user.id, action: 'DISCUSSION_COMMENT_ADDED', entityType: 'DiscussionComment', entityId: comment.id, detailsJson: { discussionId: discussion.id, projectId: input.projectId, roundId: input.roundId, }, ipAddress: ctx.ip, userAgent: ctx.userAgent, }) } catch { // Never throw on audit failure } return comment }), /** * Close a discussion (admin only) */ closeDiscussion: adminProcedure .input(z.object({ discussionId: z.string() })) .mutation(async ({ ctx, input }) => { const discussion = await ctx.prisma.evaluationDiscussion.update({ where: { id: input.discussionId }, data: { status: 'closed', closedAt: new Date(), closedById: ctx.user.id, }, }) // Audit log try { await logAudit({ prisma: ctx.prisma, userId: ctx.user.id, action: 'DISCUSSION_CLOSED', entityType: 'EvaluationDiscussion', entityId: input.discussionId, detailsJson: { projectId: discussion.projectId, roundId: discussion.roundId, }, ipAddress: ctx.ip, userAgent: ctx.userAgent, }) } catch { // Never throw on audit failure } return discussion }), })