Skip to content

feat(regulations): compile drafts in a sandboxed LaTeX worker - #112

Draft
lorenzocorallo wants to merge 2 commits into
feat/regulations-latexfrom
feat/regulations-worker
Draft

lorenzocorallo wants to merge 2 commits into
feat/regulations-latexfrom
feat/regulations-worker

Conversation

@lorenzocorallo

Copy link
Copy Markdown
Member

Stacked on #111. The PDF preview compiler (spec decision 4): a separate, sandboxed worker service plus the
DB-backed compiler that the collab service drives. The backend never runs xelatex in-process.

Worker (src/regulations/worker, docker/latex-worker.Dockerfile)

  • Image FROM ghcr.io/polinetworkorg/legal-texlive (build arg TEXLIVE_IMAGE). It adds only the bun binary and
    one bundled worker.js. CI builds ghcr.io/polinetworkorg/latex-worker on main for amd64 and arm64. Since
    legal is private, its package must grant this repository read access.
  • HTTP: POST /compile, PUT /assets/<hash>, GET /health, with a bearer secret (WORKER_SECRET). There's no
    database and no egress. Sources come in the request. The asset bundle (fonts, shared/*.cls, anything else
    that isn't a source) is uploaded once per content hash and cached on tmpfs. The client re-uploads it when the
    worker answers assets_missing, e.g. after a restart.
  • One compile per key at a time, latest wins: a waiting request is answered superseded.
    WORKER_CONCURRENCY overall.
  • Warm workdirs: one per draft, so .aux survives. Only changed sources are rewritten. LRU eviction.
  • xelatex sandbox: -no-shell-escape -halt-on-error -file-line-error, openin_any=p, openout_any=p,
    shell_escape=f, a clean environment, prlimit on address space, and a total timeout (SIGKILL). It reruns
    while the log asks, up to 3 passes. Errors are parsed with file and line.
  • Defense in depth: workdir names are HMAC'd with a per-process secret, so one draft's LaTeX can't \input
    another's. The process is made non-dumpable (prctl), so xelatex can't read its /proc/<pid>/environ.

Compiler (src/regulations/compile)

  • createCompiler: one in flight per draft, latest wins, cached by sourceHash (entry + sources + assets
    hash). Results are shared across drafts and viewers. It stores regulations_compiles (with the PDF in
    regulations_blobs), the per-draft regulations_draft_compiles (state, hash, lastGoodHash) and
    compile_refs, and emits events for the collab service to broadcast. An unreachable worker shows up as an
    error but isn't cached.
  • readCompileStatus / readCompilePdf back compile.status / compile.pdf. A PDF is served only for hashes
    compiled for that draft.
  • readDocument splits a legal checkout's document directory into sources (legal.json sources globs) and the
    asset bundle.

Verified

  • tests/regulations-compile.test.ts: queue, hashing and path checks, log parsing, compiler cache and
    latest-wins with a fake worker. If xelatex is installed, it also runs a bun fixture against a real worker:
    it compiles, uploads missing assets, reports errors with files, refuses writes outside the workdir, runs no
    shell escape, and refuses requests without the secret.
  • By hand: built the legal thread's TeX Live image locally (legal/docker/texlive) and this worker on top of it,
    then ran it with --read-only, tmpfs /work and /tmp, --cap-drop ALL, no-new-privileges, uid 1000,
    1.5 GiB / 2 CPUs, on an --internal network:
    • The real RRI (from the legal thread's migration) compiles to 19 pages: cold 2.5 s (3 passes), warm edit
      0.75 s (1 pass). The edit is in the PDF.
    • \input{/proc/1/environ}, \input{/etc/passwd} and the \@@input primitive on absolute paths are all
      refused. /proc/1/environ is permission-denied for the worker's own user. No egress (EAI_AGAIN).
    • Note: on macOS TinyTeX (TeX Live 2026), openin_any=p did not stop the \input primitive from reading
      absolute paths. That's why the HMAC'd workdirs and the non-dumpable process are there.

A separate service and image (FROM ghcr.io/polinetworkorg/legal-texlive) that
runs xelatex without shell escape, with reads and writes confined to the
workdir (openin_any/openout_any paranoid), a timeout and a memory limit. It
keeps a warm workdir per draft so .aux survives, runs one compile per draft
(latest wins) and a few in parallel, and needs no database or network: the
sources come in the request and the asset bundle (fonts, the shared class) is
uploaded once per hash. Workdir names are keyed with a per-process secret and
the worker is non-dumpable, so a draft's LaTeX can read neither another draft
nor the worker's environment.

The compiler caches results by source hash, records each draft's compile state
for compile.status and serves only the PDFs compiled for that draft.
@coderabbitai

coderabbitai Bot commented Nov 2, 2026

Copy link
Copy Markdown
Contributor

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

  • Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant