Skip to content

test(bats): add ACQ_BATS_SHARD to run one shard of the offline suite - #651

Open
mogul wants to merge 1 commit into
mainfrom
ci/bats-shard-script
Open

mogul wants to merge 1 commit into
mainfrom
ci/bats-shard-script

Conversation

@mogul

@mogul mogul commented Oct 29, 2026

Copy link
Copy Markdown
Contributor

🤖: AI agent (First Mate), AI-assisted change (OpenCode). Human review required before merge.

Context

The bash 3.2 offline-suite job takes 8.5 to 16 min for identical code depending on runner speed, and a merge-queue run of PR 611 was cancelled at the old limit with no failing test. This PR adds the shard-selection plumbing so the job can be split across a matrix. It does not change any workflow; the workflow change (commit 203c7b8) could not be pushed with the available token (no workflow scope) and follows separately.

Plan / what changed

  • scripts/bats-shard I/N [DIR]: new. Deterministic LPT balancing by @test count, recomputed from the .bats files on disk on every run (no checked-in table; new files join a shard automatically). Fails closed (exit 2) on a malformed or out-of-range spec, missing dir, no files, or an empty shard.
  • scripts/test-acq-bats: honors ACQ_BATS_SHARD=I/N on the parallel, xargs-fallback and serial paths. Explicit file args pass straight through. Invalid spec exits 2 before any test runs.
  • test/bats/helper.bash: acq_setup_stubs unsets ACQ_BATS_SHARD so a CI leg's spec cannot leak into runner tests (found in adversarial review).
  • Tests: test/bats/02-bats-shard.bats (11 tests: partition complete, no dupes, deterministic, add/remove stable, weighting, spaces in paths, fail-closed specs) and 5 runner tests in 01-test-runner.bats.
  • .pre-commit-config.yaml (trigger regex) and CONTRIBUTING.md (docs).

Verification

  • scripts/bats-shard for N=1..5 on the real suite: union equals all 60 files, no duplicates. N=3 gives 20/20/21 files and 359/358/358 tests.
  • ShellCheck 0.9.0 (scripts/run-shellcheck) clean; fix-exec-bits --check OK; npm run lint:md clean.
  • ACQ_BATS_JOBS=1 scripts/test-acq-bats test/bats/01-test-runner.bats test/bats/02-bats-shard.bats all pass, also with ACQ_BATS_SHARD=1/3 exported.
  • Shard 2/3 run locally (xargs path): 357 of 358 ok on a heavily loaded host. The one failure (103-kit-fetch-lifecycle, a fixed 3 s subprocess timeout) passes when rerun alone; it is load-sensitive, unrelated to sharding.
  • Not run locally: bash 3.2 (not available here). The scripts avoid mapfile, associative arrays and other post-3.2 features; CI will exercise it.

Rollback

Revert this commit. Nothing consumes ACQ_BATS_SHARD until the workflow change lands; unsharded behavior is unchanged.

Security impact

None to auth, secrets or data handling. No workflow, dependency or network changes in this PR. New script only reads *.bats under test/bats and prints paths.

Follow-up (tracked separately)

Workflow matrix change (.github/workflows/bash32-compat.yml, 3 shards, fail-fast: false, same gate job and required-check name acq offline suite under bash 3.2) needs a push by someone with workflow scope.

Add scripts/bats-shard, which selects shard I of N of test/bats/*.bats
deterministically (LPT balancing by @test count, derived from the files on
disk each run, so a new test file joins a shard automatically). It fails
closed (exit 2) on a malformed or out-of-range spec, a missing directory,
no .bats files, or an empty shard.

scripts/test-acq-bats honors ACQ_BATS_SHARD=I/N on the GNU parallel, xargs
fallback and serial paths; explicit file arguments still pass straight
through. acq_setup_stubs clears ACQ_BATS_SHARD so a CI leg's shard spec
cannot leak into the runner contract tests.

Tests: test/bats/02-bats-shard.bats (partition completeness, determinism,
add/remove stability, weighting, fail-closed specs) and runner tests in
01-test-runner.bats.

Co-authored-by: OpenCode Agent <user@gsa.gov>
@mogul
mogul requested a review from a team as a code owner October 29, 2026 00:19

@basiliskus basiliskus left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict: request changes — f7069c9202c9214c3f005b49c9a1b7340d3c6cc2

One compatibility-gate gap needs fixing before merge:

  • .github/workflows/bash32-compat.yml:133-143: add scripts/bats-shard to the relevance filter. Otherwise, a future PR changing only this Bash-3.2-sensitive script will skip the real Bash 3.2 suite while the required gate reports success.

Assisted-by: OpenCode [gpt-5.6-sol-latest-guardrails-defaultv2] skill:change-review

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants