Skip to content

Remove unused imports before sorting in pydabs-codegen - #7375

Merged
janniklasrose merged 1 commit into
mainfrom
janniklasrose/pydabs-codegen-import-order
Nov 5, 2026
Merged

janniklasrose merged 1 commit into
mainfrom
janniklasrose/pydabs-codegen-import-order

Conversation

@janniklasrose

Copy link
Copy Markdown
Member

Why

./task pydabs-codegen fixes the generated code with a single ruff check --fix. In that run the unused-import removal (F401) and the import sort (I001) overlap, and ruff defers whichever comes second to the next iteration. Which one runs first depends on the neighbouring statements: a whole-statement deletion swallows its newline, so the fix for the statement right after it waits a round, and by then the sort runs first. A long import sorted while it still carries its unused names is wrapped with a trailing comma. The trim keeps the comma, and isort's split-on-trailing-comma plus the formatter's magic trailing comma then keep it multi-line for good.

The output is deterministic for a given input, but not local: adding one import to a package can rewrap unrelated imports in the same file. #7366 shows the effect, with depends_on imports flipping between one line and multi-line across packages for reasons unrelated to that change.

Changes

  • pydabs-codegen runs ruff check --fix --select F401 on its own before the existing full fix and format steps, so wrapping depends only on the final line.
  • Regenerated models: 32 files collapse imports that were wrapped this way. No other line changes.

Tests

  • Running the task a second time produces no further changes.
  • ./task lint-python, ruff format --check, the pydabs test suite (530 tests) and pyright pass locally.

This pull request and its description were written by Isaac.

In a single `ruff check --fix`, the unused-import removal (F401) and the
import sort (I001) overlap, and ruff defers whichever comes second to the
next iteration. Which one runs first depends on the neighbouring
statements: a whole-statement deletion swallows the newline, so the fix
for the statement right after it waits a round, and by then the sort runs
first. A long import sorted while it still carries its unused names gets
wrapped with a trailing comma, which survives the trim and is never folded
back. Adding one import could therefore rewrap unrelated ones.

Run F401 on its own first so wrapping depends only on the final line. The
regenerated models collapse the imports in 32 files that were wrapped this
way; a second run of the task produces no further changes.

Co-authored-by: Isaac <no-reply@databricks.com>
@github-actions github-actions Bot added the PyDABs label Nov 5, 2026
@janniklasrose
janniklasrose marked this pull request as ready for review November 5, 2026 10:30
@janniklasrose
janniklasrose requested review from a team as code owners November 5, 2026 10:30
@eng-dev-ecosystem-bot

Copy link
Copy Markdown
Collaborator

Integration test report

Commit: bd88e4a

Run: 42011099281

Env ✅​pass 🙈​skip Time
✅​ aws linux-2core-8gb 271 15 3:55
✅​ aws-windows-latest-4core-16gb 273 13 3:21
✅​ azure linux-2core-8gb 270 15 3:46
✅​ azure-windows-latest-4core-16gb 272 13 3:15
✅​ gcp linux-2core-8gb 271 15 4:01
✅​ gcp-windows-latest-4core-16gb 273 13 3:10
Top 3 slowest tests (at least 2 minutes):
duration env testname
3:15 aws-windows-latest-4core-16gb TestAccept
3:10 azure-windows-latest-4core-16gb TestAccept
3:04 gcp-windows-latest-4core-16gb TestAccept

@janniklasrose
janniklasrose added this pull request to the merge queue Nov 5, 2026
Merged via the queue into main with commit 523290c Nov 5, 2026
31 checks passed
@janniklasrose
janniklasrose deleted the janniklasrose/pydabs-codegen-import-order branch November 5, 2026 11:09
@eng-dev-ecosystem-bot

Copy link
Copy Markdown
Collaborator

Integration test report

Commit: 523290c

Run: 42016743509

Env ❌​FAIL 🔄​flaky ✅​pass 🙈​skip Time
❌​ aws linux-2core-8gb 7 2 1508 1256 173:31
❌​ aws-windows-latest-4core-16gb 7 2 1427 1283 195:43
🔄​ azure linux-2core-8gb 4 1360 1308 139:04
✅​ azure-windows-latest-4core-16gb 1283 1335 146:13
✅​ gcp linux-2core-8gb 1351 1313 145:51
✅​ gcp-windows-latest-4core-16gb 1270 1340 156:52
15 interesting tests: 8 flaky, 7 FAIL
Test Name aws linux-2core-8gb aws-windows-latest-4core-16gb azure linux-2core-8gb
❌​ TestAccept ❌​F ❌​F ✅​p
🔄​ TestAccept/bundle/deployment/bind/job/job-spark-python-task ✅​p ✅​p 🔄​f
🔄​ TestAccept/bundle/deployment/bind/job/job-spark-python-task/DMS= ✅​p ✅​p 🔄​f
🔄​ TestAccept/bundle/invariant/no_drift ✅​p 🔄​f ✅​p
🔄​ TestAccept/bundle/invariant/no_drift/DMS=true/INPUT_CONFIG=cluster.yml.tmpl/READPLAN=1 ✅​p 🔄​f ✅​p
🔄​ TestAccept/bundle/resources/apps/lifecycle-started 🔄​f ✅​p ✅​p
🔄​ TestAccept/bundle/resources/apps/lifecycle-started/DMS=true 🔄​f ✅​p ✅​p
🔄​ TestAccept/bundle/resources/jobs/cluster_policy/default_flag ✅​p ✅​p 🔄​f
🔄​ TestAccept/bundle/resources/jobs/cluster_policy/default_flag/DMS=true ✅​p ✅​p 🔄​f
❌​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/with_permissions ❌​F ❌​F 🙈​s
❌​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/with_permissions/DMS= ❌​F ❌​F
❌​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/with_permissions/DMS=true ❌​F ❌​F
❌​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/without_permissions ❌​F ❌​F 🙈​s
❌​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/without_permissions/DMS= ❌​F ❌​F
❌​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/without_permissions/DMS=true ❌​F ❌​F
Top 50 slowest tests (at least 2 minutes):
duration env testname
14:22 gcp linux-2core-8gb TestAccept/bundle/resources/clusters/lifecycle-started/DMS=
13:50 aws linux-2core-8gb TestAccept/bundle/resources/clusters/lifecycle-started/DMS=
11:29 aws-windows-latest-4core-16gb TestAccept/bundle/invariant/no_drift/DMS=true/INPUT_CONFIG=cluster.yml.tmpl/READPLAN=
11:25 aws-windows-latest-4core-16gb TestAccept/bundle/invariant/no_drift/DMS=true/INPUT_CONFIG=cluster_apply_policy_default_values.yml.tmpl/READPLAN=
10:52 gcp-windows-latest-4core-16gb TestAccept/bundle/resources/apps/lifecycle-started/DMS=true
10:31 aws-windows-latest-4core-16gb TestAccept/bundle/invariant/no_drift/DMS=true/INPUT_CONFIG=cluster_libraries.yml.tmpl/READPLAN=1
10:18 azure linux-2core-8gb TestAccept/bundle/resources/clusters/lifecycle-started-toggle/DMS=true
10:09 gcp linux-2core-8gb TestAccept/bundle/resources/clusters/lifecycle-started-toggle/DMS=
10:03 aws linux-2core-8gb TestAccept/bundle/invariant/no_drift/DMS=/INPUT_CONFIG=cluster_libraries.yml.tmpl/READPLAN=1
10:02 aws linux-2core-8gb TestAccept/bundle/invariant/no_drift/DMS=/INPUT_CONFIG=cluster_libraries.yml.tmpl/READPLAN=
9:55 azure linux-2core-8gb TestAccept/bundle/resources/apps/lifecycle-started/DMS=true
9:55 gcp linux-2core-8gb TestAccept/bundle/resources/clusters/lifecycle-started-toggle/DMS=true
9:34 aws linux-2core-8gb TestAccept/bundle/invariant/no_drift/DMS=/INPUT_CONFIG=cluster_apply_policy_default_values.yml.tmpl/READPLAN=1
9:32 aws linux-2core-8gb TestAccept/bundle/invariant/no_drift/DMS=/INPUT_CONFIG=cluster.yml.tmpl/READPLAN=
9:27 aws-windows-latest-4core-16gb TestAccept/bundle/resources/clusters/lifecycle-started/DMS=
9:26 gcp-windows-latest-4core-16gb TestAccept/bundle/resources/apps/lifecycle-started/DMS=
9:25 aws linux-2core-8gb TestAccept/bundle/invariant/no_drift/DMS=/INPUT_CONFIG=cluster_apply_policy_default_values.yml.tmpl/READPLAN=
9:11 gcp linux-2core-8gb TestAccept/bundle/resources/clusters/lifecycle-started/DMS=true
9:11 aws linux-2core-8gb TestAccept/bundle/resources/clusters/lifecycle-started/DMS=true
8:33 aws-windows-latest-4core-16gb TestAccept/bundle/resources/apps/lifecycle-started/DMS=true
8:33 aws-windows-latest-4core-16gb TestAccept/bundle/resources/clusters/lifecycle-started/DMS=true
8:31 gcp linux-2core-8gb TestAccept/bundle/resources/clusters/deploy/local_ssd_count/DMS=true
8:29 aws linux-2core-8gb TestAccept/bundle/invariant/no_drift/DMS=true/INPUT_CONFIG=cluster.yml.tmpl/READPLAN=
8:29 aws linux-2core-8gb TestAccept/bundle/resources/apps/lifecycle-started/DMS=
8:26 azure-windows-latest-4core-16gb TestAccept/bundle/resources/apps/lifecycle-started/DMS=true
8:21 azure-windows-latest-4core-16gb TestAccept/bundle/resources/apps/lifecycle-started/DMS=
8:18 aws linux-2core-8gb TestAccept/bundle/resources/apps/lifecycle-started/DMS=true
8:12 aws-windows-latest-4core-16gb TestAccept/bundle/resources/apps/lifecycle-started/DMS=
8:10 azure linux-2core-8gb TestAccept/bundle/resources/clusters/resize-terminated-fallback/DMS=
8:10 gcp linux-2core-8gb TestAccept/bundle/resources/apps/lifecycle-started/DMS=true
8:05 aws-windows-latest-4core-16gb TestAccept/bundle/invariant/no_drift/DMS=/INPUT_CONFIG=cluster_apply_policy_default_values.yml.tmpl/READPLAN=1
7:58 aws linux-2core-8gb TestAccept/bundle/invariant/no_drift/DMS=/INPUT_CONFIG=cluster.yml.tmpl/READPLAN=1
7:58 azure linux-2core-8gb TestAccept/bundle/invariant/destroy_idempotent/DMS=/INPUT_CONFIG=cluster_libraries.yml.tmpl/READPLAN=1
7:55 azure linux-2core-8gb TestAccept/bundle/resources/apps/lifecycle-started/DMS=
7:52 azure linux-2core-8gb TestAccept/bundle/resources/clusters/resize-terminated-fallback/DMS=true
7:48 aws-windows-latest-4core-16gb TestAccept/bundle/invariant/no_drift/DMS=true/INPUT_CONFIG=cluster_libraries.yml.tmpl/READPLAN=
7:43 aws linux-2core-8gb TestAccept/bundle/resources/clusters/resize-terminated-fallback/DMS=true
7:43 gcp-windows-latest-4core-16gb TestAccept/bundle/resources/clusters/deploy/local_ssd_count/DMS=true
7:42 aws linux-2core-8gb TestAccept/bundle/resources/clusters/lifecycle-started-toggle/DMS=
7:37 gcp-windows-latest-4core-16gb TestAccept/bundle/resources/clusters/lifecycle-started/DMS=true
7:37 aws-windows-latest-4core-16gb TestAccept/bundle/invariant/no_drift/DMS=/INPUT_CONFIG=cluster_libraries.yml.tmpl/READPLAN=1
7:28 aws linux-2core-8gb TestAccept/bundle/resources/clusters/lifecycle-started-toggle/DMS=true
7:24 aws-windows-latest-4core-16gb TestAccept/bundle/invariant/no_drift/DMS=/INPUT_CONFIG=cluster_apply_policy_default_values.yml.tmpl/READPLAN=
7:22 azure linux-2core-8gb TestAccept/bundle/resources/clusters/lifecycle-started/DMS=true
7:15 gcp-windows-latest-4core-16gb TestAccept/bundle/resources/clusters/lifecycle-started/DMS=
7:07 gcp-windows-latest-4core-16gb TestAccept/bundle/resources/clusters/deploy/local_ssd_count/DMS=
7:06 aws linux-2core-8gb TestAccept/bundle/config-remote-sync/multiple_resources/DMS=true
6:57 azure linux-2core-8gb TestAccept/bundle/resources/clusters/lifecycle-started/DMS=
6:53 gcp linux-2core-8gb TestAccept/bundle/config-remote-sync/multiple_resources/DMS=true
6:50 gcp linux-2core-8gb TestAccept/bundle/resources/clusters/deploy/update-after-create/DMS=

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants