Skip to content

Harden file I/O and widen the VIR corpus - #349

Merged
jserv merged 5 commits into
masterfrom
refine
Oct 21, 2026
Merged

jserv merged 5 commits into
masterfrom
refine

Conversation

@jserv

@jserv jserv commented Oct 21, 2026 •

Copy link
Copy Markdown
Collaborator

Source files are now read through end of input instead of being sized with fseek() and ftell(), so a program or header streamed through a pipe or /dev/stdin compiles, and a read that fails is reported rather than compiled as a short file. Failed writes and a failed close of the output ELF now exit with a diagnostic instead of reporting success for a truncated image. The VIR corpus no longer skips a fixture the compiler rejects or that lacks main: each fixture that cannot run standalone is listed with the harness that owns it, which brings the corpus from 168 to 282 programs on x86-64 and from about 77 to 282 on Arm and RISC-V. The remaining commits fold vir-baseline into vir-opt-baseline, stop check-sanitizer from deleting out/shecc, run the optimization policy checks at stage 2, and drop helpers nothing called.

Verified with make check-all-targets, which passed on arm, arm64, riscv and x64; make bootstrap on all four targets (stage 1 and stage 2 byte-identical); make bootstrap DYNLINK=1 and check-stage2 DYNLINK=1 on Arm (3073 passed); and make check-sanitizer on x64. Each commit bootstraps on x64 on its own.

cubic-dev-ai[bot]

This comment was marked as resolved.

jserv added 5 commits October 22, 2026 00:53
op_is_integer_binary(), op_is_scalar_unary() and op_is_scalar_cast()
had no callers, so their property bits go with them. The same holds
for vir_type_is_i32_or_i64_or_pointer(), vir_type_is_pointer_or_i64(),
the AArch64 SXTW extend constant, the name of the ARM opcode enum, and
a variadic callback the indirect-call fixture declared but never
called.
A source file was sized with fseek() and ftell() and then read in one
request. A pipe or /dev/stdin has no such length, so its text was
lost, and a read that failed was taken as a short file. Read instead
into a growing buffer until read() reports end of input, and report a
failed read as "cannot read source file" rather than compiling
whatever arrived. An #include of a file that opens but cannot be read
is reported at the directive, and a quoted #include of an absolute
path is no longer joined to the including file's directory.

On the output side, a short or failed write of the ELF image was
ignored, and so was the result of fclose(), where a buffered runtime
may only then discover that the device is full. Either now exits with
a diagnostic, and the embedded libc's fclose() returns -1 when close()
fails instead of always 0.

The driver streams a program through /dev/stdin, directly and through
#include, at sizes around the reader's first refill and a later
growth; passes a directory to check the read diagnostic; and writes a
small dynamic and a large static image to /dev/full.
Both runners compiled a list of workloads repeatedly, required
identical output and stats, and could time the builds. vir-opt-baseline
also covers both optimization levels and both compiler stages, so it
takes over the four workloads only vir-baseline had, the branch,
pointer, arithmetic and fib programs, along with its check that the
stats carry the layout and lowering lines. vir-baseline.sh, its list,
the vir-baseline and check-vir-baseline targets, and the timing helper
only it used are removed.
The corpus skipped any tests/*.c the compiler rejected or that had no
main, so a fixture that stopped building dropped out of the run
without a failure. Both are now failures. A fixture that cannot build
standalone, or that a dedicated harness already runs at both levels,
is listed with its owner in tests/vir-corpus-exclusions.txt. An
optional third field names the one target where such a fixture does
build: the arm64 call and frame fixtures pass more than eight
arguments, so the corpus compares them there.

The runtime fixtures vir-native-regressions.sh runs at both levels
move from a word list in the script to tests/vir-runtime-workloads.txt,
which the corpus also skips. Three volatile and typedef fixtures leave
that list, since vir-qualifiers.sh already runs them at both levels.
The two lists must be disjoint and name existing files.

Both harnesses read their case lists on fd 3, so a fixture that reads
stdin cannot consume them, and vir-qualifiers.sh compiles each fixture
once per level instead of once per checked function.
check-sanitizer copied the sanitizer build over out/shecc, ran the
driver, and removed out/shecc afterwards. A passing run left the
other suites without a stage-0 compiler, and a failing one left the
sanitizer build in its place. The driver now takes its stage-0
compiler from SHECC_STAGE0 when set, and check-sanitizer passes the
sanitizer build that way.

The frontend optimization policy checks ran only with the stage-0
compiler, so a policy the self-compiled compiler applied differently
went unnoticed. make check now runs them at stage 2 too.
@jserv
jserv merged commit 563b3d7 into master Oct 21, 2026
55 checks passed
@jserv
jserv deleted the refine branch October 21, 2026 17:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant