Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,9 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
* **BREAKING** Bumped `MINIMUM_QSV_VERSION` from `20.0.0` to `20.1.0` (and bumped the qsv version installed by `Dockerfile.worker`, `.github/workflows/ci.yml`, and `.github/workflows/main.yml` to match). Operators must upgrade their `qsv` binary at the path configured by `ckanext.datapusher_plus.qsv_bin` before deploying this version — DP+ will refuse to start otherwise with `JobError: At least qsv version 20.1.0 required. Found 20.0.0.`. qsv 20.1.0 itself introduces no breaking changes against 20.0.0 (per the [20.1.0 release notes](https://github.com/dathere/qsv/releases/tag/20.1.0): "pipelines built on 20.0.0 will upgrade in place"), so the upgrade is binary-swap-only — no data re-ingestion is required. The reason this is still flagged as **BREAKING** is the minimum-version gate, not the qsv behavior. User-visible improvement justifying the floor bump: qsv-dateparser 0.14 → 0.15 in qsv 20.1.0 adds recognition for ISO 8601 `T`-separated datetimes without a timezone suffix (e.g. `2024-10-11T14:30:00`) — qsv 20.0.0 misclassified these as `String` during `qsv stats --infer-dates`, which in DP+ surfaced as a date-typed column being demoted to text on certain CSV shapes (one of the gaps documented in the issue #173 regression test, now closed). The regression-test matrix in `tests/test_issue_173_date_format_inference.py` is updated to the new baseline.
* **BREAKING** Bumped `MINIMUM_QSV_VERSION` from `20.1.0` to `25.0.0` (and bumped the qsv version installed by `Dockerfile.worker`, `.github/workflows/ci.yml`, `.github/workflows/main.yml` and `.github/workflows/test.yml` to match). Operators must upgrade their `qsv` binary at the path configured by `ckanext.datapusher_plus.qsv_bin` before deploying this version — DP+ will refuse to start otherwise with `JobError: At least qsv version 25.0.0 required. Found 20.1.0.`. qsv [25.0.0](https://github.com/dathere/qsv/releases/tag/25.0.0) is the first release with what `QSVCommand` reads through `qsv-client`: structured JSON errors (`QSV_ERROR_FORMAT=json`, [qsv#4765](https://github.com/dathere/qsv/pull/4765)), so a failed qsv step reports its error kind and message instead of raw stderr, and `qsv --capabilities` for the version check. On older qsv, both fell back to parsing plain text. It also redacts API keys from `describegpt`'s attribution ([qsv#4766](https://github.com/dathere/qsv/pull/4766)). Date inference is unchanged: the issue #173 matrix in `tests/test_issue_173_date_format_inference.py` gives the same type for every column under qsv 25.0.0 as under 20.1.0, and is now pinned to 25.0.x.

### Fixed
* Pinned `fastapi<0.144`. Prefect 3.7 only requires `fastapi<1.0`, and the FastAPI releases of 2026-10-18 to 23 break it twice: from 0.155 a fresh install fails on `import prefect` with `TypeError: Body() got an unexpected keyword argument 'min_items'` (unit suite, `prefect server start` and the integration CI all die), and from 0.144 the server starts but rejects the Prefect client's `workers/filter` request with `422 Input should be a valid integer` for `limit: null`, so `ckan datapusher_plus prefect-deploy` fails. 0.143.x is what the last green CI run installed. The pin goes away once a Prefect release in range works on current FastAPI (issue #353).

### qsv 20.0.0 migration notes (operator-facing)
The new minimum brings the following user-visible changes from qsv 4.0.0 → 20.0.0 that may affect existing CKAN deployments:

Expand Down
3 changes: 3 additions & 0 deletions requirements.txt
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,9 @@ pandas==2.2.3
shapely==2.1.0
pyproj>=3.7.1
prefect>=3.7,<3.8
# fastapi 0.155+ breaks `import prefect` (removed min_items); 0.144-0.154 reject
# the Prefect client's `limit: null` with 422.
fastapi<0.144
# importlib_metadata: declared explicitly because prefect 3.7.1's
# ``workers/base.py`` imports it unconditionally but doesn't list it
# as a direct dep — it was being satisfied transitively by
Expand Down
Loading