Repository navigation
feat(regulations): emanate through the GitHub App and notify the Soci by email and Telegram - #117
Draft
lorenzocorallo wants to merge 13 commits into
Draft
lorenzocorallo wants to merge 13 commits into
lorenzocorallo wants to merge 13 commits into
Conversation
Contributor
|
Important Draft PR not reviewedDraft PRs are not automatically reviewed by default.
To automatically review draft PRs, update your CodeRabbit configuration: reviews:
auto_review:
drafts: true
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. CodeRabbit Verify
|
… by email and Telegram "Emana e notifica" opens a PR on the legal repository with the generated sources (the delibera macro set), enables auto-merge so only CI gates it, and after the merge creates the date-based tag, waits for the release the legal CI builds, attaches the differences PDF and keeps the official PDF as the new published version. Then it emails soci@ (Reply-To direttivo@, differences PDF attached) and posts to the Soci group through the new postRegulation command, recording every send. Each step is idempotent and resumes from the GitHub webhook, a 30-second poll or emanation.retry. A rettifica skips approval: the Segretario or a delegate emanates it, the sources say who rectified it, and the Segretario is told about each one made on their delega. Creating or revoking a delega emails direttivo@. Without the App (not created yet) an in-memory GitHub whose CI compiles with the worker stands in, outside production. The diff can no longer be regenerated once the emanation started.
lorenzocorallo
force-pushed
the
feat/regulations-diff
branch
from
November 2, 2026 23:02
9887f37 to
cbb60ac
Compare
lorenzocorallo
force-pushed
the
feat/regulations-emanation
branch
from
November 2, 2026 23:02
af09b41 to
1c3ac87
Compare
…ing it A draft started from a version that has since been superseded would send the older text to approval and emanation, silently reverting what was published in between. Sending to approval and emanating (rettifiche included) are refused, and the transition shows why, until the version in force is merged in.
…e-check a waiting emanation drafts.get adds can.emanateBlockedBy (superseded base, text that doesn't parse, missing differences), so a delegate sees why there is no button. emanation.retry on an emanation waiting on GitHub checks again right away instead of at the next poll.
…f the GitHub API App JWT to installation token, a git data API commit that deletes removed sources and is not repeated, the PR, squash auto-merge through GraphQL, the tag, and release assets uploaded and downloaded byte for byte.
…rseded base Merging needs regulations:write: a delegate is told to start a new rettifica from the version in force, others to ask the Consiglio Direttivo.
…and prebuild the font cache PORT and WORKDIR work as well as WORKER_PORT and WORKER_WORK_DIR, and the image builds the fontconfig cache, since its root filesystem is read-only at runtime.
…idempotency key Uint8Array attachments never reach the bot through the backend's Socket.IO engine (binary frames are lost, checked with a client acting as the bot), so files travel as base64 in the payload. Each post carries the notification's id as idempotencyKey, the same on every retry, and the bot's ack returns the posted message ids.
…GitHub webhook The backend is in-cluster only except for /public, routed by polinetwork-cd. Every public route is declared in one registry with its own authentication, body limit and rate limits, and a test lists them. A guard runs before every other route: paths under /public must be the exact plain path of a declared route, and a request marked by the public ingress reaches nothing else, since Bun resolves dot segments before the app sees the path. POST /public/webhooks/github verifies X-Hub-Signature-256 in constant time, handles each X-GitHub-Delivery once (a failed one can be redelivered), and drives emanations on pull_request, check_suite, check_run and release events. Polling becomes a reconciliation every REGULATIONS_RECONCILE_SECONDS (5 min).
…e it a uid of its own RLIMIT_NPROC counts every task of the uid on the whole node, so with a uid shared with other apps xelatex could not start xdvipdfmx in production. The worker keeps the per-process address-space limit, the Pod's pids limit bounds processes, and the image documents that it must run as a dedicated uid.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #116. Emanation and notifications (spec decision 10.4 and 12), plus the deleghe emails.
Emanation (
emanation/service.ts)emanation.start({ draftId })("Emana e notifica"):regulations:manage, approved, with a ready diff.regulations:rectifyor an active delega. It skips approval: it freezes the live text (it mustparse), records the rettifica delibera (today;
onBehalfOf= the Segretario when acting on a delega) andgenerates the diff first. On any failure it undoes all of this.
Then a resumable state machine, every step idempotent:
applyDeliberasets\approvazione/\rettifica[delegato]) under thedocument
dir, deleting sources that disappeared, onregolamenti/<doc>-<draft>. Open the PR (title,delibera, LLM categories, "non modificare a mano") and enable auto-merge (squash; if the checks have
already passed, merge directly). No review: the approval happened in the dashboard.
without merging fails the step.
<tagPrefix>vYYYY.MM.DDfrom the delibera date,.2,.3… when it's taken. It's pushed with theinstallation token, so it starts the legal release workflow. Creates the version (generated sources,
commit, assets) and sets the draft
emanata.outputs.pdf), uploaddifferenze-<tag>.pdf(re-rendered with the tag and the delibera), and keep the official PDF as the version'sPDF (
versions.pdf).REGULATIONS_EMAIL_TO(soci@) fromREGULATIONS_EMAIL_SENDER(still to be decided; defaults tothe existing sender), with Reply-To
REGULATIONS_EMAIL_REPLY_TO(direttivo@). React Email template: delibera,tag, release link, dashboard link, categories, the 10-day deadline for a modifica, and the differences PDF
attached. Its time is
notifiedAt, which starts the opposition window. For a rettifica on a delega, theSegretario who granted it gets a notice.
postRegulationtoREGULATIONS_TELEGRAM_CHAT_ID/_TOPIC_ID(pending from Lorenzo; skipped whileunset) with the official PDF and the differences PDF.
sendOnce: recorded before (pending) and after (sent: time, recipients, attempts), keyed,so a resumed step never sends twice.
emanation.getshows the steps (state, url, at, detail) and thenotifications.
emanation.retryresets failed steps and resumes.REGULATIONS_DIRETTIVO_EMAIL(Statuto, art. "Direttivo", comma 9).GitHub (
emanation/github.ts)githubAppClient: OctokitAppfromREGULATIONS_GITHUB_APP_ID/_PRIVATE_KEY(the App isn't created yet),finding the installation on
REGULATIONS_GITHUB_REPO. It uses the git data API (blobs, tree, commit, ref),pulls, check runs, the GraphQL
enablePullRequestAutoMerge, tags, releases, and asset upload and download.githubLegalSource:documents.syncreads legal.json and the document from GitHub in production.POST /github/webhookchecks the HMAC (REGULATIONS_GITHUB_WEBHOOK_SECRET) and resumes the emanation onpull_request, check_suite and release events. The backend is in-cluster only, so the webhook is optional; the
30 s poll covers it.
fakeGitHub: an in-memory repository. CI passes after a delay, auto-merge merges, and a tag gets a releasewhose "CI" compiles the official PDF with the worker in the dev stack. It's the default outside production, and
keeps its state across restarts in development (
REGULATIONS_FAKE_GITHUB_STATE).Telegram (
websocket/telegram.ts)The new
postRegulationcommand, with the shape agreed with the telegram thread (PostRegulation: chat, topic,Telegram HTML text, structured fields, files as Uint8Array). The bot acks with
cb(null, { messageIds })and afailure is retried. A small registry lets the regulations module reach the socket server without importing the
entrypoint.
Verified
tests/regulations-emanation.fixture.ts(Postgres + fake GitHub, wrapped byregulations-emanation.test.ts):mainhas theedited sources and
\approvazione{direttivo}{2026-11-09}; tagsample-v2026.11.09; the release getsdifferenze-…pdf; the version has the PDF and release URL; the draft isemanatawith the oppositiondeadline.
retryposts bothPDFs and doesn't email again. The records show
email sent ×1,telegram sent after 2 attempts..2;\rettifica[Davide Delegato]{…}next to the keptapproval; the delega created/used emails; no opposition line.
datecolumns asDate, which gave the tag "Mon Nov 09".Live, on the local stack (build(regulations): run the editor's whole stack locally #115) with the real RRI: draft A (approved through the admin UI) emanated over HTTP:
rri-v2026.11.02;legal's
\delibera);The version list shows
v2026.11.02with its PDF.Not verified:
githubAppClientagainst GitHub: the App doesn't exist yet, and it needs Lorenzo.postRegulationagainst the real bot: that's the telegram thread's side.Added while the admin thread tested end to end
version merged in, would undo what was published since. Found by the admin thread on a real draft. Sending to
approval and emanating (rettifiche included) are now refused with
"La bozza parte da una versione superata (…): unisci prima la versione in vigore …", shown as the transition'sblockedBy.can.emanateBlockedByondrafts.get(a contract addition): why emanation is unavailable right now(superseded base, text that doesn't parse, missing or pending diff).
emanation.retryon awaitingemanation re-checks GitHub immediately.…/dashboard/regulations/drafts/{draftId}/diff(overridable withREGULATIONS_WEB_URL).tests/regulations-github.fixture.tsdrives the realgithubAppClientagainst a local imitation of the GitHubAPI:
the version in force (
rri-v2026.11.03) and a delegate's rettifica (rri-v2026.11.03.2). Both finished,including after backend reloads (the fake GitHub persists to disk; the 30 s poll resumes emanations).
Public routes (decided by Lorenzo) and Telegram follow-ups
/public(src/public): the backend's only public surface, routed by polinetwork-cd feat: option to hide groups from search #32 (api.polinetwork.org,PathPrefix(/public/)with a plain-segment regex).buildPublicRoutes). Each route has its own auth, body limit and rate limits (per minute, totaland per client).
tests/public-routes.test.tslists them, so a new public route fails the tests until it'slisted.
/publicmust be the exact plain path of a declaredroute.
..and%2e%2ebefore the app sees the path, so/public/../api/trpcarrives as/api/trpc.For that case, requests marked by the public ingress (
X-PoliNetwork-Ingress, set by Traefik; requested onfeat: option to hide groups from search #32) only ever reach declared routes.
POST /public/webhooks/github:X-Hub-Signature-256HMAC compared in constant time withREGULATIONS_GITHUB_WEBHOOK_SECRET; 404 while thesecret is unset;
X-GitHub-Deliveryhandled once, recorded incommon_webhook_deliveries(migration 0034); a failed one isforgotten, so GitHub's redelivery works;
pull_request,check_suite,check_runandrelease.REGULATIONS_RECONCILE_SECONDS(default 300; 15 in the dev stack, whosefake GitHub sends no webhooks). Deliveries older than a week are pruned daily.
postRegulationfiles are now base64 (files[].base64). A Uint8Array in the payload never reaches the botthrough this Socket.IO engine (binary attachment frames are lost; verified with socket.io-client acting as the
bot, on websocket and polling).
idempotencyKey= the notification row id, the same on every retry.socket.timeout(60_000).emit(…, (timeoutErr, error, result) => …).tests/public-routes.test.ts:tests/public-webhook.fixture.ts: a real Bun server, Postgres, raw request targets.tests/regulations-telegram.fixture.ts: the real WebSocketServer and a bot client; a 400 KB file arrives intactwith the key; the ack returns the message ids; the bot's error surfaces.