Skip to content

feat(regulations): emanate through the GitHub App and notify the Soci by email and Telegram - #117

Draft
lorenzocorallo wants to merge 13 commits into
feat/regulations-difffrom
feat/regulations-emanation
Draft

lorenzocorallo wants to merge 13 commits into
feat/regulations-difffrom
feat/regulations-emanation

Conversation

@lorenzocorallo

@lorenzocorallo lorenzocorallo commented Nov 2, 2026 •

Copy link
Copy Markdown
Member

Stacked on #116. Emanation and notifications (spec decision 10.4 and 12), plus the deleghe emails.

Emanation (emanation/service.ts)

emanation.start({ draftId }) ("Emana e notifica"):

  • modifica: regulations:manage, approved, with a ready diff.
  • rettifica: regulations:rectify or an active delega. It skips approval: it freezes the live text (it must
    parse), records the rettifica delibera (today; onBehalfOf = the Segretario when acting on a delega) and
    generates the diff first. On any failure it undoes all of this.

Then a resumable state machine, every step idempotent:

  1. pr: commit the generated sources (applyDelibera sets \approvazione / \rettifica[delegato]) under the
    document dir, deleting sources that disappeared, on regolamenti/<doc>-<draft>. Open the PR (title,
    delibera, LLM categories, "non modificare a mano") and enable auto-merge (squash; if the checks have
    already passed, merge directly). No review: the approval happened in the dashboard.
  2. checks, then merge: wait for CI and the merge (a webhook, or the 30 s cron). A CI failure or a PR closed
    without merging fails the step.
  3. tag: <tagPrefix>vYYYY.MM.DD from the delibera date, .2, .3… when it's taken. It's pushed with the
    installation token, so it starts the legal release workflow. Creates the version (generated sources,
    commit, assets) and sets the draft emanata.
  4. release: wait for the release with the official PDF (legal.json outputs.pdf), upload
    differenze-<tag>.pdf (re-rendered with the tag and the delibera), and keep the official PDF as the version's
    PDF (versions.pdf).
  5. email: to REGULATIONS_EMAIL_TO (soci@) from REGULATIONS_EMAIL_SENDER (still to be decided; defaults to
    the existing sender), with Reply-To REGULATIONS_EMAIL_REPLY_TO (direttivo@). React Email template: delibera,
    tag, release link, dashboard link, categories, the 10-day deadline for a modifica, and the differences PDF
    attached. Its time is notifiedAt, which starts the opposition window. For a rettifica on a delega, the
    Segretario who granted it gets a notice.
  6. telegram: postRegulation to REGULATIONS_TELEGRAM_CHAT_ID/_TOPIC_ID (pending from Lorenzo; skipped while
    unset) with the official PDF and the differences PDF.
  • Every send goes through sendOnce: recorded before (pending) and after (sent: time, recipients, attempts), keyed,
    so a resumed step never sends twice. emanation.get shows the steps (state, url, at, detail) and the
    notifications. emanation.retry resets failed steps and resumes.
  • Deleghe: creating or revoking one emails REGULATIONS_DIRETTIVO_EMAIL (Statuto, art. "Direttivo", comma 9).
  • The diff can't be regenerated once the emanation has started.

GitHub (emanation/github.ts)

  • githubAppClient: Octokit App from REGULATIONS_GITHUB_APP_ID / _PRIVATE_KEY (the App isn't created yet),
    finding the installation on REGULATIONS_GITHUB_REPO. It uses the git data API (blobs, tree, commit, ref),
    pulls, check runs, the GraphQL enablePullRequestAutoMerge, tags, releases, and asset upload and download.
  • githubLegalSource: documents.sync reads legal.json and the document from GitHub in production.
  • POST /github/webhook checks the HMAC (REGULATIONS_GITHUB_WEBHOOK_SECRET) and resumes the emanation on
    pull_request, check_suite and release events. The backend is in-cluster only, so the webhook is optional; the
    30 s poll covers it.
  • fakeGitHub: an in-memory repository. CI passes after a delay, auto-merge merges, and a tag gets a release
    whose "CI" compiles the official PDF with the worker in the dev stack. It's the default outside production, and
    keeps its state across restarts in development (REGULATIONS_FAKE_GITHUB_STATE).

Telegram (websocket/telegram.ts)

The new postRegulation command, with the shape agreed with the telegram thread (PostRegulation: chat, topic,
Telegram HTML text, structured fields, files as Uint8Array). The bot acks with cb(null, { messageIds }) and a
failure is retried. A small registry lets the regulations module reach the socket server without importing the
entrypoint.

Verified

  • tests/regulations-emanation.fixture.ts (Postgres + fake GitHub, wrapped by regulations-emanation.test.ts):

    • Modifica: the Direttivo is refused; the Presidente starts it; the PR auto-merges after CI; main has the
      edited sources and \approvazione{direttivo}{2026-11-09}; tag sample-v2026.11.09; the release gets
      differenze-…pdf; the version has the PDF and release URL; the draft is emanata with the opposition
      deadline.
    • Bot down: the email is sent once (Reply-To direttivo@, 1 attachment) and Telegram fails. retry posts both
      PDFs and doesn't email again. The records show email sent ×1, telegram sent after 2 attempts.
    • Rettifica by a delegate the same day: tag .2; \rettifica[Davide Delegato]{…} next to the kept
      approval; the delega created/used emails; no opposition line.
    • It caught a real bug: Bun returns date columns as Date, which gave the tag "Mon Nov 09".
  • Live, on the local stack (build(regulations): run the editor's whole stack locally #115) with the real RRI: draft A (approved through the admin UI) emanated over HTTP:

    1. PR feat: separate npm package from the backend code #1;
    2. CI;
    3. auto-merge;
    4. tag rri-v2026.11.02;
    5. release whose official PDF the worker compiled ("Approvato dal Consiglio Direttivo il 02/11/2026" printed by
      legal's \delibera);
    6. email via the dev mailer;
    7. Telegram skipped (not configured).

    The version list shows v2026.11.02 with its PDF.

  • Not verified:

    • githubAppClient against GitHub: the App doesn't exist yet, and it needs Lorenzo.
    • postRegulation against the real bot: that's the telegram thread's side.
    • Graph email from the production sender.

Added while the admin thread tested end to end

  • Superseded base: a draft whose lineage starts from an older version than the latest one, without that
    version merged in, would undo what was published since. Found by the admin thread on a real draft. Sending to
    approval and emanating (rettifiche included) are now refused with "La bozza parte da una versione superata (…): unisci prima la versione in vigore …", shown as the transition's blockedBy.
  • can.emanateBlockedBy on drafts.get (a contract addition): why emanation is unavailable right now
    (superseded base, text that doesn't parse, missing or pending diff).
  • emanation.retry on a waiting emanation re-checks GitHub immediately.
  • The email links to the dashboard's diff view, …/dashboard/regulations/drafts/{draftId}/diff (overridable with
    REGULATIONS_WEB_URL).
  • tests/regulations-github.fixture.ts drives the real githubAppClient against a local imitation of the GitHub
    API:
    • an RSA App key, then an installation token;
    • a git data API commit that deletes removed sources and isn't repeated;
    • the PR, then squash auto-merge via GraphQL;
    • the tag;
    • release assets uploaded and downloaded byte for byte.
  • On the local stack, the admin thread emanated two more drafts through the dashboard: a modifica merged up to
    the version in force (rri-v2026.11.03) and a delegate's rettifica (rri-v2026.11.03.2). Both finished,
    including after backend reloads (the fake GitHub persists to disk; the 30 s poll resumes emanations).

Public routes (decided by Lorenzo) and Telegram follow-ups

  • /public (src/public): the backend's only public surface, routed by polinetwork-cd feat: option to hide groups from search #32 (api.polinetwork.org,
    PathPrefix(/public/) with a plain-segment regex).
    • One registry (buildPublicRoutes). Each route has its own auth, body limit and rate limits (per minute, total
      and per client). tests/public-routes.test.ts lists them, so a new public route fails the tests until it's
      listed.
    • A guard runs before every other route. A path that touches /public must be the exact plain path of a declared
      route.
    • Bun resolves .. and %2e%2e before the app sees the path, so /public/../api/trpc arrives as /api/trpc.
      For that case, requests marked by the public ingress (X-PoliNetwork-Ingress, set by Traefik; requested on
      feat: option to hide groups from search #32) only ever reach declared routes.
  • POST /public/webhooks/github:
    • X-Hub-Signature-256 HMAC compared in constant time with REGULATIONS_GITHUB_WEBHOOK_SECRET; 404 while the
      secret is unset;
    • X-GitHub-Delivery handled once, recorded in common_webhook_deliveries (migration 0034); a failed one is
      forgotten, so GitHub's redelivery works;
    • 1 MiB body limit; 120/min total and 60/min per client;
    • drives emanations on pull_request, check_suite, check_run and release.
  • Polling is now a reconciliation every REGULATIONS_RECONCILE_SECONDS (default 300; 15 in the dev stack, whose
    fake GitHub sends no webhooks). Deliveries older than a week are pruned daily.
  • Telegram (telegram#161):
    • postRegulation files are now base64 (files[].base64). A Uint8Array in the payload never reaches the bot
      through this Socket.IO engine (binary attachment frames are lost; verified with socket.io-client acting as the
      bot, on websocket and polling).
    • Each post carries idempotencyKey = the notification row id, the same on every retry.
    • The ack is read from socket.timeout(60_000).emit(…, (timeoutErr, error, result) => …).
  • Verified:
    • tests/public-routes.test.ts:
      • the route list;
      • 12 hostile paths;
      • a signed delivery handled once, the replay ignored;
      • wrong, missing and truncated signatures rejected;
      • redelivery after a failure;
      • 413 and 429 limits;
      • 404 without a secret.
    • tests/public-webhook.fixture.ts: a real Bun server, Postgres, raw request targets.
    • tests/regulations-telegram.fixture.ts: the real WebSocketServer and a bot client; a 400 KB file arrives intact
      with the key; the ack returns the message ids; the bot's error surfaces.
    • The emanation fixture checks that a failed post and its retry carry the same key.

@coderabbitai

coderabbitai Bot commented Nov 2, 2026 •

Copy link
Copy Markdown
Contributor

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

  • Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

CodeRabbit Verify

  • Run CodeRabbit Verify to test the changes in this pull request.

… by email and Telegram

"Emana e notifica" opens a PR on the legal repository with the generated
sources (the delibera macro set), enables auto-merge so only CI gates it, and
after the merge creates the date-based tag, waits for the release the legal CI
builds, attaches the differences PDF and keeps the official PDF as the new
published version. Then it emails soci@ (Reply-To direttivo@, differences PDF
attached) and posts to the Soci group through the new postRegulation command,
recording every send. Each step is idempotent and resumes from the GitHub
webhook, a 30-second poll or emanation.retry.

A rettifica skips approval: the Segretario or a delegate emanates it, the
sources say who rectified it, and the Segretario is told about each one made
on their delega. Creating or revoking a delega emails direttivo@.

Without the App (not created yet) an in-memory GitHub whose CI compiles with
the worker stands in, outside production. The diff can no longer be
regenerated once the emanation started.
@lorenzocorallo
lorenzocorallo force-pushed the feat/regulations-emanation branch from af09b41 to 1c3ac87 Compare November 2, 2026 23:02
…ing it

A draft started from a version that has since been superseded would send the
older text to approval and emanation, silently reverting what was published in
between. Sending to approval and emanating (rettifiche included) are refused,
and the transition shows why, until the version in force is merged in.
…e-check a waiting emanation

drafts.get adds can.emanateBlockedBy (superseded base, text that doesn't
parse, missing differences), so a delegate sees why there is no button.
emanation.retry on an emanation waiting on GitHub checks again right away
instead of at the next poll.
…f the GitHub API

App JWT to installation token, a git data API commit that deletes removed
sources and is not repeated, the PR, squash auto-merge through GraphQL, the
tag, and release assets uploaded and downloaded byte for byte.
…rseded base

Merging needs regulations:write: a delegate is told to start a new rettifica
from the version in force, others to ask the Consiglio Direttivo.
…and prebuild the font cache

PORT and WORKDIR work as well as WORKER_PORT and WORKER_WORK_DIR, and the
image builds the fontconfig cache, since its root filesystem is read-only at
runtime.
…idempotency key

Uint8Array attachments never reach the bot through the backend's Socket.IO
engine (binary frames are lost, checked with a client acting as the bot), so
files travel as base64 in the payload. Each post carries the notification's id
as idempotencyKey, the same on every retry, and the bot's ack returns the
posted message ids.
…GitHub webhook

The backend is in-cluster only except for /public, routed by polinetwork-cd.
Every public route is declared in one registry with its own authentication,
body limit and rate limits, and a test lists them. A guard runs before every
other route: paths under /public must be the exact plain path of a declared
route, and a request marked by the public ingress reaches nothing else, since
Bun resolves dot segments before the app sees the path.

POST /public/webhooks/github verifies X-Hub-Signature-256 in constant time,
handles each X-GitHub-Delivery once (a failed one can be redelivered), and
drives emanations on pull_request, check_suite, check_run and release events.
Polling becomes a reconciliation every REGULATIONS_RECONCILE_SECONDS (5 min).
…e it a uid of its own

RLIMIT_NPROC counts every task of the uid on the whole node, so with a uid
shared with other apps xelatex could not start xdvipdfmx in production. The
worker keeps the per-process address-space limit, the Pod's pids limit bounds
processes, and the image documents that it must run as a dedicated uid.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant