Skip to content

Feature/event lifecycle queue status - #9

Merged
SURYANSH-RAI merged 75 commits into
feature/retire-legacy-matchingfrom
feature/event-lifecycle-queue-status
Nov 13, 2026
Merged

SURYANSH-RAI merged 75 commits into
feature/retire-legacy-matchingfrom
feature/event-lifecycle-queue-status

Conversation

@SURYANSH-RAI

Copy link
Copy Markdown
Collaborator

R08: Event lifecycle permissions and real queue status

Draft, open, and closed events now behave consistently across every action, and participants and organizers can tell pending, running, stuck, failed, empty, and ready results apart.

Lifecycle policy (src/domain/shared/event-lifecycle.ts)

Action Draft Open Closed
Save profile, search, refresh, request intro, share zone ✗ ✓ ✗
View matches / accepted contacts ✗ ✓ ✓
Reply to pending intros ✗ ✓ ✓
Organizer: compute matches opens the event, then queues ✓ ✗ refused
Worker: process a job skip ✓ skip
  • Every participant action checks the table. Pages the status does not allow redirect to the event landing page, which explains draft vs closed.
  • "Compute matches" no longer silently reopens a closed event.
  • Workers mark jobs for events that are not open as done and skip them.

Queue status

  • Migration 0005_job_status.sql: matching_jobs.started_at (set on claim) plus an index for each participant's latest job.
  • States: the latest job maps to none / queued / running / stuck (running over 10 minutes) / failed / done. A finished embed job counts as queued, since its match job follows.
  • Participant matches page: a message per state, with "Try again" where useful. It refreshes itself every 10 seconds for up to 3 minutes while queued or running. Existing matches stay visible with an "updating" or "last update failed" notice.
  • Organizer matching page: counts of queued, running, stuck, failed, and finished participants. The compute button label reflects the event status.
  • Stuck jobs: enqueue() releases a participant's job stuck "running" over 10 minutes, so "Try again" can queue a new one. Full crash recovery (leases, reaper, backoff) remains R21.

Only profile saves and search checked that the event was open; refresh,
intros, zone sharing and match views ignored the status, and "compute
matches" silently reopened closed events.

- domain/shared/event-lifecycle.ts: one table of what draft, open and
  closed events allow, with messages explaining why something is off
- Open only: profile, search, refresh, new intros, zone sharing
- Open or closed: viewing matches/connections, replying to intros
- Compute matches opens a draft first and refuses a closed event
- Participant pages an event does not allow redirect to the event
  landing page, which now explains draft vs closed
- Workers complete and skip jobs whose event is not open (statusOf)
- Migration 0005: matching_jobs.started_at (set when a worker claims a
  job) and an index for each participant's latest job
- domain/matching/match-status.ts: a participant's latest job maps to
  none / queued / running / stuck (running over 10 minutes) / failed /
  done; a finished embed job counts as queued since its match job
  follows; summarizeQueue counts states across an event
- MatchQueue port with a Postgres adapter and get-match-status use
  cases, registered in the container
- Unit tests for the states and a real-Postgres test that the latest
  job is chosen per participant and events stay separate
- Matches page tells apart: not started, queued, running, stuck,
  failed, and finished with nobody to match, each with its own message;
  stuck/failed/empty offer "Try again"; existing matches stay visible
  with an "updating" or "last update failed" notice
- While matching is queued or running the page refreshes itself every
  10 seconds for up to 3 minutes (AutoRefresh)
- Closed events: matches stay readable with a notice, but intro
  buttons, refresh, profile/search links and zone sharing are hidden
- Organizer matching page: per-participant queue counts (queued,
  running, stuck, failed, finished); compute button reflects draft /
  open / closed instead of always "Open registration"
- enqueue() releases that participant's job stuck "running" over 10
  minutes, which otherwise blocked "Try again" via the one-active-job
  index; crash recovery proper stays in R21
Both limits were check-then-act, so requests sent at the same moment
got past them (the new tests fail on the old code):

- Issuing: the per-e-mail count and the insert now run in one
  transaction under a per-e-mail advisory lock (createWithinLimit),
  so ten simultaneous requests with a limit of three create three
- Guessing: an attempt is spent in one UPDATE that re-checks
  attempts < max under the row lock *before* the code is compared
  (reserveAttempt), so parallel guesses get at most five comparisons
- countRecent/create/findLiveByEmail/incrementAttempts are replaced
  by these two functions
…art 2)

- clientIp took the first X-Forwarded-For entry, which the caller
  writes; it now takes the address our proxy appended, counting
  TRUSTED_PROXY_HOPS from the end (1 behind the ALB, set in Terraform;
  0 locally trusts no header). Validated in env.ts, documented in
  .env.example
- Per-IP limits on the shared database limiter: 10 code requests and
  30 code checks per IP per 15 minutes (429 with Retry-After); the
  per-e-mail limits are unchanged
- /api/auth/lookup is retired (410): it returned any attendee's name
  and e-mail for a phone or address with no limit, and the login page
  never used it
- Middleware rejects non-GET /api requests whose Origin or Referer is
  another site (403), blocking login/logout CSRF; requests with neither
  header (non-browser clients) still pass. The JSON sign-in routes also
  refuse non-JSON bodies (415), so a plain HTML form cannot post to them
loginChallenges.deleteExpired existed but nothing called it, so expired
login codes, dead rate-limit buckets and expired retrieval-cache rows
accumulated forever.

- housekeeping.sweepExpired deletes login challenges 24 hours after
  they expire, rate-limit buckets older than a day, and expired
  retrieval-cache rows
- The existing worker runs it on start and then at most once an hour;
  a failed sweep is logged and never stops matching
- Real-Postgres test that only dead rows go, plus a timing unit test
- Web and worker images take the release tag as APP_VERSION (build
  arg, declared late so cached layers survive); /api/health reports it
  and the worker logs it on start, so a smoke test or rollback can tell
  which version is serving
- CI runs on every pull request (any target branch) and on pushes to
  main and dev; previously only PRs into main were checked
- CI is callable (workflow_call) so the deploy can run the same checks;
  build_images=false skips its image builds when the deploy builds its
  own. The concurrency group includes the workflow name so the deploy's
  call and the standalone run on the same push do not cancel each other
…rt 2)

- The deploy runs ci.yml (lint, typecheck, unit and real-Postgres
  integration tests, Terraform validate) before building; previously it
  re-ran only typecheck and unit tests
- The migrate task and Keycloak are pinned to the release tag like web
  and worker: running the migrate family started its newest revision
  (the mutable `latest` image), and Keycloak's forced redeploy reused
  its old revision
- After the services settle, each one's PRIMARY deployment must be the
  revision just registered; the circuit breaker's rollback otherwise
  looked "stable" and the old version passed the smoke test
- The smoke test is mandatory (fails without APP_BASE_URL) and requires
  /api/health to report the release tag plus /api/health/ready to pass
- A failed rollout restores the web and worker revisions that were
  serving before; workflow_dispatch rollback_tag re-deploys an existing
  tag (checked in ECR) without checks, build or migrations
- The web pinning step is unchanged apart from its tag source, so the
  R02 e-mail guard still applies cleanly
canReturnToPath read /orgs/<org>/events/new as an event called "new",
found no such event, and refused the sign-in, so an organization owner
following the "create event" link was told they were "not on the
attendee list". The organization's event list and new-event pages now
check organization membership instead.
PostgresRateLimiter passed the limit as $2 but the SQL never used $2,
so Postgres rejected every call with "could not determine data type of
parameter $2". Search failed for every participant, and in production
(TRUSTED_PROXY_HOPS=1) every sign-in would have failed at the per-IP
limit. Unit tests mocked the limiter, so it was never exercised.

The query now takes only the key and window. A new real-Postgres test
covers the limit, simultaneous requests and window reset; it fails on
the old code.
A signed-in participant or organizer had no sign-out control: the only
Sign out button was on the "not on this roster" page, while open events
redirect straight to matches/profile and no other page offered one.

A shared SignOutButton (the existing signOutCurrentUser action) now
ends every participant and organizer page nav, and the closed/draft
event page has one too. The organizer event list and new-event pages
get a nav for it.
The R03 fix (expired/tampered cookies loop on /login, lost return
paths) was committed on a side branch from R01 and never reached the
R04-R10 line, so a stale cookie again bounced users from /login to
/e/demo and a reused magic link left them on the wrong event.

Brings in "Fix expired-session login dead end and keep return paths
(R03)" (cherry picked from commit e6f2743). The middleware conflict is
resolved by keeping R09's same-origin check for /api requests and
R03's session-aware /login redirect, on the nodejs runtime; the R09
middleware test now awaits the async middleware.
…t 1)

- Web startup refuses to boot without RESEND_API_KEY unless
  EMAIL_DEV_FALLBACK=true, and validates EMAIL_FROM once Resend is set
- Offer "Sign in with Google" only when GOOGLE_CLIENT_ID is configured;
  the start route redirects to a clear error otherwise
- Login page reads the flag per request (server component wrapper)
- Pass GOOGLE_CLIENT_ID to the web container in Compose and ECS

(cherry picked from commit 36a13d6)
- New `npm run keycloak:sync` applies KEYCLOAK_CLIENT_SECRET and the
  Google client id/secret from the environment to an already-imported
  realm via the admin API (realm import only runs on first boot)
- Deploy workflow rejects a web task that enables EMAIL_DEV_FALLBACK or
  has no RESEND_API_KEY secret
- Document activating Resend and Google, locally and in AWS

(cherry picked from commit 5516503)
- DATABASE_SSL=verify checks the server certificate against
  DATABASE_CA_CERT or DATABASE_CA_CERT_FILE and refuses to run without
  one; "require" stays encrypted-but-unverified. ensure-database uses
  the same rules instead of always skipping verification
- Production startup (web and worker) refuses a remote database
  without verify + CA, and the web app refuses a non-https
  APP_BASE_URL; local hosts stay exempt so docker-compose still runs
- HSTS is sent when APP_BASE_URL is https
- Web, worker, migrate and Keycloak images ship Amazon's RDS CA bundle
- ensure-roles.ts (run by the migration task as the master user):
  a runtime login with row-level read/write and no DDL, and a Keycloak
  login that owns the Keycloak database, including tables an earlier
  deploy created as master (handed over object by object, since
  REASSIGN OWNED would also move databases). CONNECT is revoked from
  PUBLIC on both databases. Re-running rotates passwords
- Real-Postgres test of what each login can and cannot do
- An environment = "prod" plan fails unless domain_name and
  acm_certificate_arn are set, so production never serves over HTTP
- /admin on Keycloak is no longer routed publicly: a 404 rule blocks
  it at the load balancer, with an allow rule only for
  var.keycloak_admin_cidrs (empty by default). The app's admin API
  calls already go over Cloud Map inside the VPC
- Web and worker connect as app_runtime (row access only), Keycloak as
  its own keycloak login over JDBC sslmode=verify-full, and only the
  migration task holds the master login; it runs ensure-roles.ts after
  migrations to create/rotate the other two
- Secrets split into app (web/worker), identity (Keycloak) and migrate
  (master login and the passwords it sets)
- Every task uses DATABASE_SSL=verify with the RDS bundle in the image
- deploy.yml pins images onto the family's newest task definition
  revision instead of the one the service is running, so environment
  and secret changes applied by Terraform (these, and R09's
  TRUSTED_PROXY_HOPS) actually reach the services
- Migration 0006: event_participants gains invited_at, invite_count,
  invite_error, claimed_at and revoked_at; existing profiles count as
  claimed
- Revoked attendees lose event sign-in (isParticipant), page and action
  access (hasEventAccess, so existing sessions too), matching
  (participantIds) and every results path (ELIGIBLE_CANDIDATE).
  Re-importing the roster does not silently restore them
- The first signed-in visit to the event records the claim time
- sendInvitations e-mails everyone never invited, or one attendee as a
  resend (at most every 10 minutes, never after claiming or removal);
  each attempt is recorded, failures with their reason
- setAttendeeAccess removes or restores access
- Invitation e-mail template and NotificationPort.invitation
- Analytics count invited, claimed and removed attendees separately
- Roster page: each attendee's state (not invited, invited, invitation
  failed with the reason, signed in, profile complete, access removed)
  with times and counts; "Send invitations" for everyone not yet
  invited; per-row Resend (after 10 minutes), Remove access, Restore
- Organizer actions for invitations and access; both use cases check
  the caller organizes the event
- A signed-in attendee without access is sent to the event page, which
  says whether the organizer removed them or they are on a different
  account (sign out and use the invited e-mail or phone), instead of a
  bare 404
- The event page also records the claim on first visit
- Analytics shows invitations sent, claimed accounts and removed count
Session cookies were trusted until they expired, so disabling or
deleting someone had no effect on a browser already signed in.

- Migration 0007: registered_attendees.disabled_at and
  sessions_revoked_at
- isSessionActive (packages/auth) runs on every request in getSession
  and in the /login middleware (otherwise a revoked session would loop
  between /login and the page): the account must exist and not be
  disabled, the session must be newer than sessions_revoked_at, and
  the Keycloak user must exist and be enabled (cached 5 minutes per
  replica; a Keycloak outage does not sign everyone out, the database
  checks still apply)
- Every sign-in path refuses a disabled account (canReturnToPath)
- npm run account:disable | account:enable | account:revoke-sessions
  update the database and the Keycloak user (disable also ends its
  Keycloak sessions)
- Tests: unit rules, and the commands against the real database and
  Keycloak realm, including that repeated sign-ins keep one identity
The visually hidden "Actions" header added to the roster table in R12 is
absolutely positioned, and the table's scroll wrapper was not, so the
header escaped the wrapper and widened the page by 175px on a phone.
The wrapper is now the positioning context.
…(R13 part 2)

- OTP then Google hit Keycloak's default first-login flow, which asks
  the existing account (created by e-mail sign-in, no password) to
  confirm with a password: a dead end. keycloak:sync now installs
  "mr-bridge link existing account" (create the user if the e-mail is
  new, otherwise link it; no review-profile form) and binds the Google
  provider to it. Google addresses are verified (trustEmail), and the
  app still rejects unverified ones. Re-running changes nothing
- Sign out (button and /api/auth/logout) ends the Keycloak session of a
  Google sign-in through the admin API instead of sending the browser
  to Keycloak's "Do you want to log out?" page
- A mr-bridge login theme (child of keycloak.v2) gives the remaining
  Keycloak screens the site's colours; shipped in the Keycloak image,
  mounted in compose, selected by keycloak:sync when installed
- Supported shared-domain hostname setup: strict hostname (the
  KC_HOSTNAME_STRICT=false override is gone) and
  KC_HOSTNAME_BACKCHANNEL_DYNAMIC for the app's Cloud Map calls
- README documents keycloak:sync and the account commands
…ctive (R14)

Roles existed but every organizer could do everything, and imports
rewrote people's names, phones and details across organizations in
batches that could fail halfway.

Roles (domain/shared/organizer-permissions.ts):
- Organization owners/admins manage every event in the organization
- Event owners/admins configure the event, import and remove attendees
- An event "organizer" views, sends invitations and starts matching
- Use cases authorize per action; pages hide what the role cannot use
  (read-only settings, no import or remove controls)
- Event lists show only events the viewer may open; organizers of one
  event may open the list (not the new-event page)

Imports (events.importRoster):
- One transaction with a per-event lock: all applied or none, and
  retrying the same file changes nothing
- Existing people keep their name, phone and details; the sheet only
  fills empty fields, and the summary says how many were kept
- A new person whose phone belongs to someone else is imported without
  it instead of failing the whole file
- The roster page reports added, already on the roster, skipped, kept
  and phone conflicts
R14 lets organization owners and admins configure every event in their
organization, but events.update still required an event_organizers row
in its own SQL, so an organization admin who did not create the event
got "Event not found" when saving its settings. The query now applies
the same rule as the use case. Covered by a real-Postgres test that
fails on the old query.
events.form_config existed, but the participant form was fixed.

- domain/participant/profile-form.ts: a catalog of the profile fields
  matching already reads (role, domain, about, focus, build, years,
  interests). Per event, organizers set each field's label, whether it
  is shown and whether it is required. A required field must be shown,
  and at least one descriptive field stays required so matching always
  has text. Each save bumps the form version
- Organizer dashboard: "Profile form" settings (owners/admins); errors
  keep the edits on screen
- The participant form renders from the config, and the server
  validates against the same config with its labels
- Hidden fields keep what was saved before, and every stored field
  still feeds matching, so changing the form never wipes profiles or
  changes embeddings
- Migration 0008: profile_form_version per participant; someone saved
  under an older form is asked for newly required answers, while their
  matches keep working
Search replaced the searcher's profile embedding with the query's, so
query relevance and personal compatibility were blurred together, and
without embeddings (no API key) retrieval was alphabetical, so every
query returned the same people. There were no filters.

- domain/matching/search-query.ts: query words, query relevance
  (semantic when both vectors exist, else word-prefix matches), and a
  final score of 60% query relevance and 40% personal compatibility;
  both appear in the score breakdown
- The pipeline's search path keeps the searcher's own profile for
  compatibility and uses the query only for retrieval and relevance,
  so the same query ranks differently for different people
- Without a query embedding, retrieval uses Postgres full-text search
  (any word, prefix match) ranked by ts_rank
- Intent and domain filters, validated against the event's intents and
  the domains present among eligible participants; applied in SQL and
  again on shared-cache hits, and part of the cache key with the
  normalized query and retrieval kind
- Search page: filter selects with their own labels
mutualInterest was set from intent compatibility alone, so any two
people with compatible goals were shown "Your event goals align"
however poorly they fit. Every scorer was also symmetric, so a
two-direction rule would have meant nothing.

- domain/matching/scoring-config.ts: scorer weights, the mutual
  interest threshold and a version in one place; the version is part
  of the persisted computation version (v2.s2), so older results can be
  found and recomputed
- Interests are now directional (how much of what the viewer cares
  about the candidate shares), so score(A->B) and score(B->A) can differ
- ScoreStage computes both directions; the flag needs both to reach the
  threshold (0.65, calibrated on documented examples). The reverse
  score is stored in the breakdown
- Wording: "Likely a strong fit for each other"; the fallback reason no
  longer presents intent compatibility as a prediction
- Calibration tests: both high flags; compatible goals alone, or a
  one-way fit, do not; works with and without embeddings
Every rerank asked the model to explain the whole shortlist again,
even when nothing about the pairs had changed.

- Migration 0009: pair_explanations (cache key, event, viewer,
  candidate, reason, expiry), deleted with the event
- domain/matching/explanation-key.ts: the key covers event, the
  viewer -> candidate direction, both profile revisions (last saved),
  model, prompt version, scoring version, and the context: "profile"
  or the normalized search query
- LlmRerankStage reads cached reasons first. When every shortlisted
  pair has one, the model is not called and the scored order stands: a
  cached reason never decides a shortlist's order. Otherwise the model
  ranks as before and its new reasons are cached for a week. If the
  model fails, cached reasons are used where they exist; a cache outage
  never blocks matching
- ExplanationCache port with a Postgres adapter; LlmProvider exposes a
  prompt version; the hourly sweep removes expired explanations
- A declined request is final for the event: re-requesting used to
  reset it to pending, letting someone keep asking
- If the other person already asked, the requester is pointed to their
  request instead of creating a second one; accepted pairs are refused
- At most 20 intro requests an hour and 25 waiting for an answer per
  person and event; self-requests are refused in the use case too
- An attendee removed from the event disappears from other people's
  connections and their contact is no longer shown; restoring access
  brings both back. Accepting a request from a removed person fails
- Participants can hide their profile from matches and search; hidden,
  they keep browsing and their accepted connections (consent already
  given), and nobody new can request them
- Tests: request rules, abuse limits, and real-Postgres checks of
  decline finality and removal
- Migration 0010: audit_events (event, actor, action, subject, details
  with counts and field names only). Recorded: roster imports,
  invitations, access removed/restored, settings and profile-form
  changes, introductions accepted (contacts shared), and account
  disable/enable/revoke/erase. Recording failures are logged and never
  block the action. Organizers see "Recent activity" on the analytics
  page. Records are kept for a year by the hourly sweep and survive
  their event's deletion
- Removing someone from an event deletes cached explanations written
  about or for them; hiding a profile deletes explanations of it
- npm run account:erase -- <email> --yes for deletion requests:
  deletes the Keycloak user first (so a failure leaves data to retry),
  then the attendee and everything cascading from it, cached
  explanations and login challenges; refuses without --yes
- Tests: real Postgres for records, sweep, retention on removal and
  hiding, and erasure against the local Keycloak realm
The worker reports queue backlog, waiting time and permanent failures
once a minute with a heartbeat; every search reports its latency and
failures; every model call reports its outcome. They are written as
CloudWatch embedded metrics (no agent or new service) when
METRICS_NAMESPACE is set. Terraform adds alarms for a stopped worker,
matching and notification backlogs, failed jobs and notifications, slow
or failing search and a sustained model failure rate, plus a dashboard.
Each alarm names its runbook section.
Two workers saving match lists that named each other deadlocked: each
locked its own profile row FOR UPDATE, then its inserts took key-share
locks on the other's row through the foreign keys. Found by the capacity
check (300 attendees, 2 workers). The revision check now locks FOR NO KEY
UPDATE, which still blocks a concurrent profile save but not those
foreign-key locks.
Retrieval took the 50 nearest people and only then dropped intents the
viewer does not welcome, so a mentor whose nearest neighbours were all
mentors got no matches even with a mentee in the event. Found by the
capacity check (1 of 300 attendees). Retrieval now applies the viewer's
welcomed intents in the query, before the limit, for profile matches and
search (part of the shared search cache key; retrieval version r3).
npm run capacity:check creates a throwaway event of the chosen size,
saves every profile, runs the real embed and match workers until the
queue is quiet, runs concurrent searches through the real use case, and
reports time to first match lists, search p50/p95, failures and model
calls per attendee and search against the budgets in the runbook, then
removes everything. It uses a free deterministic embedding by default
(EMBEDDING_PROVIDER=hash, refused in production) or the deployment's real
providers with --providers configured. Locally: 300 attendees matched in
6 s with search p95 83 ms; 1000 attendees in 25 s with search p95 169 ms.
npm run db:verify checks a database is usable: pgvector, every migration
applied from the same file (either line ending), core tables present, and
row counts against a source with --compare. npm run db:restore-rehearsal
dumps the compose database, restores it into a scratch database, verifies
it and removes it, timing each step (locally 1.3 s). CI now refuses a
change to a released migration or a new migration that would break the
previous image (drop, rename, type change, truncate) unless the line is
marked as a deliberate contract step.
The GitHub deploy role could push to the web, Keycloak and migrate image
repositories but not the worker's, while every release pushes a worker
image, so a real deploy would stop at "Build & push worker" with access
denied. The worker repository is now in the push statement.
The deploy workflow takes an environment (production or staging). Staging
is its own Terraform stack (match-maker-staging-*, example variables and
separate state key), its jobs run in the GitHub 'staging' environment and
refuse to start unless that environment names the staging stack, so they
never fall back to production settings. promote_tag copies the images a
staging deploy tested into production unchanged (buildx imagetools, same
account) and then migrates and rolls out as usual; production's deploy
role may read the staging repositories when image_promotion_source is
set. Pushes to main still deploy to production as before.
docs/OPERATIONS.md: the service targets and measured capacity, the
release path through staging and promotion, migration rules and
rollback, what to do for every alarm, database restore with
verification, rotation of each secret, running tools against the private
database, and the launch checklist. The worker image now carries the
operator tools so account commands and the Keycloak sync can run as
one-off tasks inside the VPC.
TypeScript 6 (used by the editor) deprecates baseUrl and reported it on
apps/web and packages/auth (inherited from tsconfig.base.json). Path
aliases do not need it: the entries are now relative to the file that
declares them. Type-checks clean with TypeScript 5.7 and 6.0.
@SURYANSH-RAI
SURYANSH-RAI changed the base branch from dev to feature/retire-legacy-matching November 2, 2026 18:17
…requests

Feature/notifications location requests
@SURYANSH-RAI
SURYANSH-RAI merged commit 30af9f1 into feature/retire-legacy-matching Nov 13, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant