Support Python 3.14: runtime fixes and protobuf>=5 by fellhorn · Pull Request #3478 · flyteorg/flytekit

fellhorn · 2026-10-25T17:01:43Z

Tracking issue

Related to flyteorg/flyte#6993
Closes flyteorg/flyte#5448

Why are the changes needed?

flytekit cannot be installed or used on Python 3.14:

requires-python is capped at <3.13.
import flytekit fails: TypeEngine.register raises "already registered" because typing.Union and types.UnionType are the same object on 3.14, so UnionTransformer registers the same key twice.
DefaultImages.find_image_for() raises ValueError: (3, 14) is not a valid PythonVersion, which breaks pyflyte, FlyteRemote, ImageSpec and the sqlalchemy / openai-batch plugin tasks.
The dev and CI environments pin protobuf<5. protobuf 4.x cannot be imported on 3.14 (TypeError: Metaclasses with custom tp_new are not supported from the upb extension), so the test suite does not even start.

What changes were proposed in this pull request?

flytekit/core/type_engine.py: treat re-registering the identical transformer instance for a type as a no-op instead of a conflict.
pyproject.toml: drop the upper bound of requires-python (>=3.10). uv.lock regenerated.
PythonVersion enum and default image prefixes: add 3.13 and 3.14 (core, flytekit-sqlalchemy, flytekit-openai batch).
tests/flytekit/unit/core/test_unions.py: accept the A | B repr that 3.14 uses for typing.Union[A, B] in the error message.
Port of Upgrade CI to use latest protobuf version #2946: remove the protobuf<5 / types-protobuf<5 pins from dev-requirements.in and the plugin CI job, key the plugin pip cache on dev-requirements.in, delete the stale dev-requirements.txt. All Python versions now resolve the same protobuf (6.x) instead of a per-version split.

Not part of this PR (follow-up in #3479): adding 3.13/3.14 to the CI and image matrices.

Note: flyteidl still declares requires-python <3.14 (flyteorg/flyte, flyteidl/pyproject.toml). uv ignores the upper bound, pip needs --ignore-requires-python until a flyteidl release lifts it.

How was this patch tested?

We have already been running this patch for a few weeks in our internal systems.

Besides this: make unit_test (parallel + serial) and unit_test_extras on Python 3.12.13 and 3.14.6 from #3479:

	unit (parallel)	serial	extras
3.12	1742 passed	17 passed	101 passed
3.14	1742 passed	17 passed	92 passed

The number of extra tests dropped as tensorflow does not provide Python 3.14 wheels.

Setup process

uv venv -p 3.14 .venv314
SETUPTOOLS_SCM_PRETEND_VERSION=1.999.0dev0 VIRTUAL_ENV=.venv314 uv pip install -r dev-requirements.in
VIRTUAL_ENV=.venv314 uv pip uninstall pandas pyarrow
.venv314/bin/pytest -n auto --dist=loadfile -m "not (serial or sandbox_test or hypothesis)" tests/flytekit/unit/ \
  --ignore=tests/flytekit/unit/extras/ --ignore=tests/flytekit/unit/models --ignore=tests/flytekit/unit/extend

Check all the applicable boxes

I updated the documentation accordingly.
All new and existing tests passed.
All commits are signed-off.

Related PRs

Upgrade CI to use latest protobuf version #2946 (protobuf 5 in CI, ported here)
Add Python 3.13 support #3465, Bump Python version to 3.13 #3401: Python 3.13 support
Add Python 3.13 and 3.14 support #3368: Python 3.13 and 3.14 support

Docs link

N/A

fellhorn · 2026-10-25T17:05:34Z

These changes are cherry-picked from #2946

fellhorn · 2026-10-25T17:06:06Z

Cherry picked from #2946

codecov · 2026-10-25T17:16:43Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 83.58%. Comparing base (d69b3fb) to head (f251201).
⚠️ Report is 5 commits behind head on master.

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #3478      +/-   ##
==========================================
- Coverage   86.67%   83.58%   -3.09%     
==========================================
  Files         443        3     -440     
  Lines       37040      195   -36845     
  Branches     3036        0    -3036     
==========================================
- Hits        32106      163   -31943     
+ Misses       4052       32    -4020     
+ Partials      882        0     -882

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

fellhorn · 2026-10-25T20:31:28Z

-plugin_requires = ["flytekit", "skl2onnx>=1.10.3,<1.19.0", "networkx<3.2; python_version<'3.9'", "onnx<1.18.0"]
+# flytekit>=1.16.29: first release whose lazy_module does not execute skl2onnx.common.data_types twice, which
+# skl2onnx>=1.19 needs.
+plugin_requires = ["flytekit>=1.16.29", "skl2onnx>=1.10.3", "networkx<3.2; python_version<'3.9'"]


This change was needed because I dropped the protobuf < 5 requirement, which brings in onnxruntime 1.31 next to onnx<1.18, and that combination fails

fellhorn · 2026-10-25T22:51:07Z

Cherry picked from #2946

…3.14 On Python 3.14 typing.Union and types.UnionType are the same object, so UnionTransformer's registration (which passes both) collapses to a single _REGISTRY key and TypeEngine.register raised "already registered". Skip the duplicate when the identical transformer instance is already registered. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> (cherry picked from commit 7e2e0a0) (cherry picked from commit 907fc5a) Signed-off-by: Dennis Keck <26092524+fellhorn@users.noreply.github.com> build: allow Python 3.13 and 3.14 Drop the <3.13 upper cap on requires-python so the package resolves on Python 3.13/3.14. Pairs with the typing.Union registration fix needed for 3.14. Upper bound removed to match consuming projects that are unbounded. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Signed-off-by: Dennis Keck <26092524+fellhorn@users.noreply.github.com> fix: make unit tests pass on Python 3.14 - Add PYTHON_3_13/PYTHON_3_14 to PythonVersion and the default image maps (core, sqlalchemy, openai-batch); find_image_for() raised ValueError on 3.13+, breaking pyflyte, remote and image_spec code paths. - Accept the "A | B" union repr used by Python 3.14 in test_unions. - Only pin protobuf<5 below 3.14; protobuf 4.x fails to import on 3.14. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Dennis Keck <26092524+fellhorn@users.noreply.github.com> Cherry pick flyteorg#2946 Signed-off-by: Dennis Keck <26092524+fellhorn@users.noreply.github.com>

lazy_module() checks sys.modules before importlib.util.find_spec(), but for a dotted name find_spec() imports the parent packages, and a parent's __init__ may import the requested module itself. lazy_module() then overwrote that real module in sys.modules with a LazyLoader placeholder, so the module was executed a second time on first use and produced a second set of class objects. skl2onnx>=1.19 imports skl2onnx.common.data_types from its own __init__, which made flytekitplugins-onnxscikitlearn fail with "shape must be a DataType": the FloatTensorType the user passes in came from the second execution, the DataType skl2onnx checks against from the first. The same duplication affected other plugins that lazy-load a submodule the parent imports (pyspark.sql.dataframe, pandas.io.sql, fsspec.utils), just without a visible failure. - lazy_module: re-check sys.modules after find_spec() and return the module the parent import created. - flytekit-onnx-scikitlearn: drop the onnx<1.18 and skl2onnx<1.19 caps from flyteorg#3257. They were coupled (skl2onnx 1.18 imports helpers that onnx 1.18 removed) and onnx<1.18 has no Python 3.13 wheels, which broke the plugin job on 3.13. Require flytekit>=1.16.29, the first release with this fix, so an upgrade of the plugin alone cannot combine a new skl2onnx with an old flytekit. - Regression test with a temporary package whose __init__ imports the lazily requested submodule. Verified by replaying the build-plugins job for the plugin on 3.12 and 3.13 with onnx 1.22 / skl2onnx 1.20 and with the previously pinned onnx 1.17 / skl2onnx 1.18: all pass. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Signed-off-by: bochinski <30434375+bochinski@users.noreply.github.com> (cherry picked from commit 87dd7b9) Signed-off-by: Dennis Keck <26092524+fellhorn@users.noreply.github.com>

fellhorn requested review from cosmicbboy, davidmirror-ops, kumare3, machichima, pingsutw, samhita-alla and wild-endeavor as code owners October 25, 2026 17:01

fellhorn force-pushed the dennis/py314-v1.16.28 branch from 2be03c7 to f251201 Compare October 25, 2026 17:04

fellhorn commented Oct 25, 2026

View reviewed changes

Comment thread .github/workflows/pythonbuild.yml

fellhorn Oct 25, 2026

Copy link
Copy Markdown

Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These changes are cherry-picked from #2946

fellhorn commented Oct 25, 2026

View reviewed changes

Comment thread dev-requirements.in

fellhorn Oct 25, 2026

Copy link
Copy Markdown

Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cherry picked from #2946

fellhorn marked this pull request as draft October 25, 2026 17:38

fellhorn commented Oct 25, 2026

View reviewed changes

fellhorn force-pushed the dennis/py314-v1.16.28 branch from fa1108a to 44a3470 Compare October 25, 2026 20:59

fellhorn mentioned this pull request Oct 25, 2026

Python 3.13 and 3.14 in CI and published images #3479

Draft

2 of 3 tasks

fellhorn force-pushed the dennis/py314-v1.16.28 branch from 44a3470 to 466d451 Compare October 25, 2026 21:42

fellhorn commented Oct 25, 2026

View reviewed changes

Comment thread dev-requirements.txt Outdated

fellhorn Oct 25, 2026

Copy link
Copy Markdown

Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cherry picked from #2946

fellhorn and others added 2 commits October 25, 2026 23:51

fellhorn force-pushed the dennis/py314-v1.16.28 branch from 466d451 to 392f6db Compare October 25, 2026 22:52

fellhorn marked this pull request as ready for review October 25, 2026 22:58

fellhorn changed the title ~~Support Python 3.14: runtime fixes and protobuf>=5 in dev/CI~~ Support Python 3.14: runtime fixes and protobuf>=5 Oct 25, 2026

kumare3 approved these changes Oct 25, 2026

View reviewed changes

pingsutw approved these changes Oct 26, 2026

View reviewed changes

pingsutw merged commit d0c64d5 into flyteorg:master Oct 26, 2026
57 checks passed

Navigation Menu

Repository navigation

Support Python 3.14: runtime fixes and protobuf>=5 - #3478

Support Python 3.14: runtime fixes and protobuf>=5#3478
pingsutw merged 2 commits into
flyteorg:masterfrom
fellhorn:dennis/py314-v1.16.28

fellhorn commented Oct 25, 2026 •

edited

Loading

Uh oh!

fellhorn Oct 25, 2026

Uh oh!

fellhorn Oct 25, 2026

Uh oh!

codecov Bot commented Oct 25, 2026

Uh oh!

fellhorn Oct 25, 2026

Uh oh!

fellhorn Oct 25, 2026

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Conversation

fellhorn commented Oct 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Tracking issue

Why are the changes needed?

What changes were proposed in this pull request?

How was this patch tested?

Setup process

Check all the applicable boxes

Related PRs

Docs link

Uh oh!

fellhorn Oct 25, 2026

Choose a reason for hiding this comment

Uh oh!

fellhorn Oct 25, 2026

Choose a reason for hiding this comment

Uh oh!

codecov Bot commented Oct 25, 2026

Codecov Report

Uh oh!

fellhorn Oct 25, 2026

Choose a reason for hiding this comment

Uh oh!

fellhorn Oct 25, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

fellhorn commented Oct 25, 2026 •

edited

Loading