Skip to content

Fix the CI, test and template follow-ups of the M4 and M6 review - #3127

Merged
pournasserian merged 2 commits into
mainfrom
3074-ci-test-template-followups
Nov 15, 2026
Merged

pournasserian merged 2 commits into
mainfrom
3074-ci-test-template-followups

Conversation

@pournasserian

@pournasserian pournasserian commented Nov 15, 2026 •

Copy link
Copy Markdown
Member

Closes #3074

The CI, test and template follow-ups of the M4 and M6 review. The fast checks now run the tool's tests and the host's commands; a test holds the template's Blog copy to the sample; the plugins' cleanup of a deleted site runs on each provider; five flaky tests are fixed at their cause; the templates keep secrets out of the Docker build context and the plugin template's note limit holds under concurrent posts.

Dependencies: none.

Items

  1. Fast checks: Feature CI (both OS) and the Windows job of CI on pull requests and pushes now also run FluentCMS.Cli.Tests and the Commands namespace of FluentCMS.Core.IntegrationTests and below it (backup, restore, export and import on SQLite). That filtered step takes --minimum-expected-tests 40, since the test projects ignore exit code 8 and a filter that matched nothing would pass; zero tests then exit 9. The required check names are unchanged. The fast checks' timeout goes from 10 to 15 minutes.
  2. TemplateSampleTests (architecture tests, so in the fast checks too) compares templates/fluentcms/Blog with samples/Blog with the namespace replaced. Allowed to differ: BlogSample.cs. Only in the sample: Blog.csproj, BlogExamples.cs.
  3. DeletedSitePluginTests (data tests) writes rows of the webhooks, MCP, media (with files), search and multi-site plugins in two sites, deletes one and drains the outbox, and checks that only the other site's rows and files stay. Webhook deliveries and media variants are found by the id of their webhook and asset, since the plugins' tables have no foreign keys. The four plugins get InternalsVisibleTo for the data tests, as search has.
  4. NestedReadOnlyTests on SQLite: 60 runs of the class (76 cases each, 4,560 cases), four at a time and beside a full solution build: no failure. Nothing to record on the issue.
  5. Template hardening: the .dockerignore leaves out .env, .env.*, *.pfx, *.p12, *.key, *.local.json, *.Local.json and secrets.json, and test-templates.sh checks it; the note endpoint counts and adds under the plugin's IDistributedLock, with a template test that sends ten posts at once with MaxNotes 3 and checks that at most three notes are added, one for each 201; a post that waits more than ten seconds for the lock answers 503 and adds nothing. Without the lock the posts added 6 to 10 notes in three local runs.

Decisions

  • Item 1: the tool's tests and the commands, not a path-triggered full CI. It catches a crash of the tool (the Fix the refused-key case of the MCP proxy #3040 one was in the mcp proxy) and of export and import on every issue pull request, for about a minute on Linux, without a new action or a second copy of ci.yml. The data tests of transfer (TransferTests, BackupCommandTests, SchemaApplyTests) stay in full CI: they took over four minutes locally. They are added to the Windows job of CI too, so the fast tests are one set and the tool runs on Windows before the nightly run.
  • Item 5: one lock name for the task ({name}.notes), not one per site, as IDistributedLock asks for a fixed set of names; a post that does not get it within ten seconds answers 503.
  • The same single-pass wait as the lease-lapse test was in five manual-clock tests of JobRunnerTests; they get the same fix (JobFixture.FirstRunAsync).
  • The job sweep (JobStore.SweepAsync) takes its batch in id order. Every stuck job is swept in some pass, so the order changes nothing it handles, and EF Core no longer warns of a Take without OrderBy there. The prune jobs of the outbox and the job retention have the same Take and stay as they are.

Declined

  • User Id=sa in the SQL Server host's appsettings.Development.json: no dev compose runs SQL Server (the template ships none, B-99, and the repository's compose.yaml has PostgreSQL and MinIO), and the SQL Server image creates only sa, with no hook to add a login at start, so a non-sa default would fail at the first run for most users. The password stays out of the file as before. The template README now says to use a login of the application's own outside development.

Causes of the flaky tests

  • JobRunnerTests.AFailingJobDoesNotTakeAWorkerDown (PostgreSQL): the four jobs had one run_at, so the claim's order (priority, run_at, id) fell to the ids, and version 7 GUIDs made in the same millisecond are in random order. good could run before unreadable, and the test stopped once good was done. good is now due a second later.
  • SchemaHistoryTests.TheNewestRowIsTheLastAddedWhateverTheClocksSay (SQLite): the log capture counted every category but the startup report, and the started host logs in the background. A repro of 30 runs caught EF Core's warning of a Take without OrderBy, which the first host of a process logs when it compiles such a query of the job runner. The capture now counts only MigrationRunner's logger, the one the history writes to, which fixes the class's other log assertions too.
  • PublishingAccessTests.HiddenValuesAreInTheSnapshotAndInNoVersionReadOrDiffForAnyone (SQLite): the hidden value s1 was searched for case-insensitively in JSON that holds random 12-character block ids. The values are now top secret 1 and top secret 2, whose space no id has.
  • JobFailureTests.WhenALeaseLapsesMidJobTheTwoRunnersNeverOverlap (SQL Server 2019, run 43724851071): it timed out waiting for the first run (JobFailureTests.cs:435). On the manual clock the worker gets one pass at start; the job was enqueued on another instance, so no wake reached it, and a first pass that missed the job (such as a claim chosen as a deadlock victim beside the runner's own maintenance reads) waited for a poll on a clock that does not move. The wait now signals the instance's wake, which sends the worker back without moving the clock.
  • AiClientFactoryTests.TheClientEndsASlowCallOnlyWhenItsCallerCancelsIt: it waited 3 seconds of real time per case. A client timeout runs on the real clock, so no test clock can stand in; the test now waits for the request to reach the stub, checks that the call is still open 250 ms past requestTimeout (1 second, its least), then cancels it. A client with a 1-second network timeout fails it.

Tests

  • dotnet build -c Release -warnaserror; SDK, core unit and architecture tests; DeletedSitePluginTests, PublishingAccessTests, JobRunnerTests, JobFailureTests, SchemaHistoryTests, AiClientFactoryTests and TemplateSampleTests on SQLite; markdownlint; actionlint on both workflows.
  • The plugin template from a local pack: built with warnings as errors, its nine tests pass.
  • Checked that the new tests fail: without the outbox drain, without the delete of webhook deliveries (WebhookService.cs:193) and without the delete of variant rows (DeletedSiteMedia.cs:79) for the site cleanup, with a changed file in the Blog copy, without the note lock, and with a 1-second client timeout; and that the commands step fails with a filter that matches nothing.
  • PostgreSQL and SQL Server run in the full CI run below.

Full CI run after the review fixes: 43737128113, success. The first run, 43730998175, passed on attempt 2: on attempt 1 the integration shard reached its 20-minute timeout on a slow runner (every test slower, Core.IntegrationTests 18.5 minutes against about 5 in the other manual runs of the day), and the rerun of that job passed in 5 minutes. A manual run takes the full Windows suite; this pull request's own CI ran the new Windows fast steps (the tool in 41 seconds, the commands in 26).

@pournasserian pournasserian added type:chore Repository, tooling or housekeeping area:build Solution, CI and packaging labels Nov 15, 2026
@pournasserian
pournasserian merged commit a692025 into main Nov 15, 2026
37 checks passed
@pournasserian
pournasserian deleted the 3074-ci-test-template-followups branch November 15, 2026 17:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:build Solution, CI and packaging type:chore Repository, tooling or housekeeping

Projects

None yet

Development

Successfully merging this pull request may close these issues.

CI, test and template follow-ups from the review of M4 and M6

1 participant