# 07. Testing, Observability, And Release Gates

## 1) QA Strategy
The redesign is policy-heavy. Testing must validate:
1. Functional correctness
2. Policy compliance
3. Audit completeness
4. Role-based access integrity
5. Operational resilience under load and deadline pressure

## 2) Test Pyramid

### 2.1 Unit Tests
Target modules:
- context resolver
- policy resolver
- assignment policy evaluator (hard/soft/buffer/category-bias weighting)
- round bundle lock evaluator
- final confirmation rule evaluator

### 2.2 Integration Tests
Target flows:
- stage transitions with purpose validation
- filtering outcomes + override
- assignment run + unassigned queue + manual assignment
- document upload/edit lock transitions
- mentor promotion pipeline
- live voting -> confirmation -> result lock

### 2.3 End-to-End Tests
Target user journeys:
- applicant R1 -> eligibility -> Jury1 -> R2 -> Jury2 -> finals prep -> finals -> result
- admin stage manager live finals operations
- award mode A and mode B paths
- admin invite -> onboarding -> jury assignment materialization
- team invite -> accept invite -> team collaboration

## 3) Monaco Flow Test Matrix

| Flow Step | Required Test Cases |
|---|---|
| Submission R1 | open/close behavior, late policy modes, required docs enforcement |
| Eligibility filter | deterministic pass/fail, AI banding, manual queue, admin override trail |
| Jury1 | cap modes, category-bias preference behavior (suggestive only), deadline reminder triggers, score submission |
| Submission R2 | R1 read-only for applicants, R2 editable, admin full override |
| Jury2 | visibility of R1+R2 docs, finalist recommendation and override |
| Awards | separate pool, dual track, admin-confirmed pull-out behavior, jury overlap, single-judge decision mode |
| Mentoring | private file sharing, comments, promotion to official slot with provenance |
| Jury3 live | stage manager controls, voting windows, notes, audience vote mode |
| Final confirmation | unanimous jury agreement on active quorum (or configured rule), juror replacement/absence handling, single-judge award exception flow, admin finalize, admin override path, result lock |

## 4) Invite/Member/Onboarding Test Matrix (Critical)

| Workflow | Must Validate |
|---|---|
| Admin member invite (`/admin/members/invite`) | creates user + jury membership + selected pre-assignment mode (intent and/or direct policy-validated assignment); respects role authorization |
| Bulk invite CSV | invalid rows, duplicate handling, unauthorized role attempts, staged pre-assignment mode selection and materialization |
| Invite acceptance (`/accept-invite`) | token validity, role-context preview, routing to correct onboarding path |
| Jury onboarding | cap/bias override onboarding when enabled, including explicit disclosure that category bias is a suggestion not a rule |
| Team invite (`/applicant/team`) | lead-only invite/remove, account setup vs existing account behavior |
| Cross-context conflict | same user invited as jury and team member still resolves correct permissions by context |

## 5) Regression Coverage Extension Against Existing Test Suite
Current suite already has useful coverage (`tests/unit/*`, `tests/integration/*`). Extend with:

1. `tests/unit/assignment-policy-hard-soft.test.ts`
2. `tests/unit/final-confirmation-rules.test.ts`
3. `tests/integration/invite-jury-membership-intent.test.ts`
4. `tests/integration/submission-bundle-locking.test.ts`
5. `tests/integration/mentor-promotion-provenance.test.ts`
6. `tests/integration/award-mode-routing.test.ts`
7. `tests/integration/live-to-final-lock.test.ts`

## 6) Audit Completeness Checks
Add automated assertions that every critical operation emits both domain event and audit record:
- eligibility override
- assignment exception
- stage advancement override
- award winner override
- promotion event
- final confirmation finalize/lock

Failure to emit required audit evidence should fail tests.

## 7) Observability Blueprint

## 7.1 Metrics

### Competition Lifecycle Metrics
- `projects_submitted_total` by category and stage
- `eligibility_pass_rate`
- `manual_review_rate`
- `assignment_coverage_percent`
- `unassigned_projects_count`
- `over_cap_exception_count`
- `submission_late_rate`
- `mentor_promotion_count`
- `live_vote_completion_rate`
- `final_confirmation_duration_seconds`

### Reliability Metrics
- API latency by router
- background job success/failure rates (filtering/assignment/award jobs)
- reminder send success/failure
- invitation send success/failure

## 7.2 Logs And Tracing
- Correlation id per operation across router -> service -> DB writes
- Structured logs include `programId`, `stageId`, `stagePurposeKey`, `juryId`, `projectId`, `userId`

## 7.3 Alerts
- assignment run leaves residual without queue record
- final confirmation attempted without satisfied quorum requirement
- result lock write failure
- result unlock attempted by non-super-admin
- invite acceptance failures spike
- live voting session state mismatch (open with no active project)

## 8) Release Gates (Must Pass)

## Gate A: Schema + Backfill Readiness
- all migrations idempotent in staging
- no unresolved backfill anomalies

## Gate B: Policy Engine Correctness
- hard/soft cap test pass
- ratio policy test pass
- access resolver test pass

## Gate C: End-To-End Flow Simulation
- full Monaco dry run with seeded data and mixed categories
- award mode A/B dry runs

## Gate D: Invite/Onboarding Stability
- admin invite + team invite + acceptance flows pass under load

## Gate E: Finals And Confirmation Integrity
- live voting and final confirmation produce correct lock artifacts
- super-admin unlock/relock path preserves complete audit integrity

## Gate F: Operational Readiness
- runbooks approved
- dashboards and alerts active
- support playbooks distributed

## 9) Performance And Capacity Scenarios
- Large intake window close with bulk late submissions
- Filtering and assignment jobs across high project volume
- Live finals peak voting concurrency (jury + audience)
- Bulk invitation campaigns with email retries

## 10) Sign-Off Criteria
Release is approved only when:
1. All phase gates pass.
2. No Sev-1/Sev-2 defects remain open in critical competition flows.
3. Audit completeness checks pass for all critical operations.
4. Operators can execute runbook drills successfully.