Skip to content

Wait for and print the results of databricks pipelines test - #7298

Merged
tharunbhuk-db merged 3 commits into
mainfrom
tharun/pipelines-test-pr2-output
Nov 17, 2026
Merged

tharunbhuk-db merged 3 commits into
mainfrom
tharun/pipelines-test-pr2-output

Conversation

@tharunbhuk-db

@tharunbhuk-db tharunbhuk-db commented Nov 2, 2026 •

Copy link
Copy Markdown
Contributor

🥞 Stacked PR


  • Wait for the test update, print the results, and exit 0 only when at least one test ran and none failed or errored.
  • --no-wait returns after the update starts.
  • The wait follows the bundle pipeline runner's loop.
  • "No tests were collected" names the selection, or the pipeline's test sources, and points to the --pytest-arg filters when they were given.
  • The changelog entry for the results output is in this PR.

Tests: unit tests; acceptance in acceptance/pipelines/test/results/*, with the service stubbed per case.

This pull request and its description were written by Isaac.

@github-actions

github-actions Bot commented Nov 2, 2026 •

Copy link
Copy Markdown
Contributor

Commit: 6dd223360 | Workflow run

Regression test report

Selected 8 changed tests (limit=20, 0 not selected)

8 tests: regression=7 unreleased=0 coverage=1 failing=0
test branch main (865ee2a) latest (v1.26.0)
TestAccept/pipelines/test/arguments ✅ ❌ ❌
TestAccept/pipelines/test/immutable ✅ ❌ ❌
TestAccept/pipelines/test/pipeline-id ✅ ❌ ❌
TestAccept/pipelines/test/results/failed ✅ ❌ ❌
TestAccept/pipelines/test/results/incomplete ✅ ❌ ❌
TestAccept/pipelines/test/results/passed ✅ ❌ ❌
TestAccept/pipelines/test/selection ✅ ❌ ❌
TestAccept/pipelines/test/submit-error ✅ ✅ ✅
TestAccept/pipelines/test/arguments: branch ✅ | main (865ee2a) ❌ | latest (v1.26.0) ❌

main (865ee2a):

acceptance_test.go:1380: Diff:
        --- pipelines/test/arguments/output.txt
        +++ /tmp/TestAcceptpipelinestestarguments2852006134/002/output.txt
        @@ -23,16 +23,10 @@
         
           databricks pipelines test my_pipeline --pytest-arg=-k --pytest-arg='orders and not slow'
         
        -The command waits for the results. It exits with code 0 only if at least one
        -test was collected and none failed or errored, so a run whose tests were all
        -skipped succeeds; otherwise it exits with code 1. --no-wait returns after the
        -update starts. To stop the update, run 'pipelines stop'.
        -
         Usage:
           databricks pipelines test [flags] [KEY] [TEST_SELECTION...]
         
         Pipeline Test Flags:
        -      --no-wait[=true|false]     Don't wait for the tests to complete.
               --pipeline-id string       ID of the pipeline to test, instead of a KEY.
               --pytest-arg stringArray   Argument to pass to pytest (repeatable). Only -k and -m are supported.

latest (v1.26.0):

acceptance_test.go:1380: Diff:
        --- pipelines/test/arguments/output.txt
        +++ /tmp/TestAcceptpipelinestestarguments3783427661/002/output.txt
        @@ -23,28 +23,22 @@
         
           databricks pipelines test my_pipeline --pytest-arg=-k --pytest-arg='orders and not slow'
         
        -The command waits for the results. It exits with code 0 only if at least one
        -test was collected and none failed or errored, so a run whose tests were all
        -skipped succeeds; otherwise it exits with code 1. --no-wait returns after the
        -update starts. To stop the update, run 'pipelines stop'.
        -
         Usage:
           databricks pipelines test [flags] [KEY] [TEST_SELECTION...]
         
         Pipeline Test Flags:
        -      --no-wait[=true|false]     Don't wait for the tests to complete.
               --pipeline-id string       ID of the pipeline to test, instead of a KEY.
               --pytest-arg stringArray   Argument to pass to pytest (repeatable). Only -k and -m are supported.
         
         Flags:
        -  -h, --help[=true|false]   help for test
        +  -h, --help   help for test
         
         Global Flags:
        -      --debug[=true|false]   enable debug logging
        -  -o, --output type          output type: text or json (default text)
        -  -p, --profile string       ~/.databrickscfg profile
        -  -t, --target string        bundle target to use (if applicable)
        -      --var strings          set values for variables defined in project config. Example: --var="foo=bar"
        +      --debug            enable debug logging
        +  -o, --output type      output type: text or json (default text)
        +  -p, --profile string   ~/.databrickscfg profile
        +  -t, --target string    bundle target to use (if applicable)
        +      --var strings      set values for variables defined in project config. Example: --var="foo=bar"
         
         === The command is experimental and off by default
TestAccept/pipelines/test/immutable: branch ✅ | main (865ee2a) ❌ | latest (v1.26.0) ❌

main (865ee2a):

acceptance_test.go:1380: Diff:
        --- pipelines/test/immutable/output.txt
        +++ /tmp/TestAcceptpipelinestestimmutable3740050379/002/output.txt
        @@ -4,7 +4,7 @@
         Created pipelines.my_pipeline
         Files: 0 uploaded, 0 deleted
         Resources: 2 created, 0 changed, 0 deleted, 0 unchanged
        -View your pipeline my_pipeline here: [DATABRICKS_URL]/pipelines/[UUID][0]?w=[WORKSPACE_ID]
        +View your pipeline my_pipeline here: [DATABRICKS_URL]/pipelines/[UUID]?w=[WORKSPACE_ID]
         
         === Relative test paths are rejected with workspace.sync_mode immutable
         
        @@ -18,38 +18,24 @@
         === Absolute workspace paths and running all tests still work
         
         >>> MSYS_NO_PATHCONV=1 [CLI] pipelines test my_pipeline /Workspace/Users/someone@example.com/tests/test_orders.py::test_backfill --no-wait
        -Recommendation: This command runs the last deployed version of the code
        +Error: unknown flag: --no-wait
         
        -If you've made local changes, run 'databricks pipelines deploy' first to ensure they are included.
        +Usage:
        +  databricks pipelines test [flags] [KEY] [TEST_SELECTION...]
         
        -Pipeline: my_pipeline ([UUID][0])
        -Update: [UUID][1]
        +Pipeline Test Flags:
        +      --pipeline-id string       ID of the pipeline to test, instead of a KEY.
        +      --pytest-arg stringArray   Argument to pass to pytest (repeatable). Only -k and -m are supported.
         
        ->>> [CLI] pipelines test my_pipeline --no-wait
        -Recommendation: This command runs the last deployed version of the code
        +Flags:
        +  -h, --help[=true|false]   help for test
         
        -If you've made local changes, run 'databricks pipelines deploy' first to ensure they are included.
        +Global Flags:
        +      --debug[=true|false]   enable debug logging
        +  -o, --output type          output type: text or json (default text)
        +  -p, --profile string       ~/.databrickscfg profile
        +  -t, --target string        bundle target to use (if applicable)
        +      --var strings          set values for variables defined in project config. Example: --var="foo=bar"
         
        -Pipeline: my_pipeline ([UUID][0])
        -Update: [UUID][2]
         
        ->>> print_requests.py //updates
        -{
        -  "method": "POST",
        -  "path": "/api/2.0/pipelines/[UUID][0]/updates",
        -  "body": {
        -    "test_only": true,
        -    "test_options": {
        -      "test_selection": [
        -        "/Workspace/Users/someone@example.com/tests/test_orders.py::test_backfill"
        -      ]
        -    }
        -  }
        -}
        -{
        -  "method": "POST",
        -  "path": "/api/2.0/pipelines/[UUID][0]/updates",
        -  "body": {
        -    "test_only": true
        -  }
        -}
        +Exit code: 1
        
FAIL

latest (v1.26.0):

acceptance_test.go:1380: Diff:
        --- pipelines/test/immutable/output.txt
        +++ /tmp/TestAcceptpipelinestestimmutable1254231683/002/output.txt
        @@ -4,7 +4,7 @@
         Created pipelines.my_pipeline
         Files: 0 uploaded, 0 deleted
         Resources: 2 created, 0 changed, 0 deleted, 0 unchanged
        -View your pipeline my_pipeline here: [DATABRICKS_URL]/pipelines/[UUID][0]?w=[WORKSPACE_ID]
        +View your pipeline my_pipeline here: [DATABRICKS_URL]/pipelines/[UUID]?w=[WORKSPACE_ID]
         
         === Relative test paths are rejected with workspace.sync_mode immutable
         
        @@ -13,43 +13,29 @@
         
         If you've made local changes, run 'databricks pipelines deploy' first to ensure they are included.
         
        -Error: cannot resolve test selection "tests/test_orders.py::test_backfill": relative test paths need synced local files, but workspace.sync_mode is immutable; pass the /Workspace path
        +Error: cannot resolve test selection "tests/test_orders.py::test_backfill": local test paths are not supported with workspace.sync_mode immutable; pass an absolute workspace path
         
         === Absolute workspace paths and running all tests still work
         
         >>> MSYS_NO_PATHCONV=1 [CLI] pipelines test my_pipeline /Workspace/Users/someone@example.com/tests/test_orders.py::test_backfill --no-wait
        -Recommendation: This command runs the last deployed version of the code
        +Error: unknown flag: --no-wait
         
        -If you've made local changes, run 'databricks pipelines deploy' first to ensure they are included.
        +Usage:
        +  databricks pipelines test [flags] [KEY] [TEST_SELECTION...]
         
        -Pipeline: my_pipeline ([UUID][0])
        -Update: [UUID][1]
        +Pipeline Test Flags:
        +      --pipeline-id string       ID of the pipeline to test, instead of a KEY.
        +      --pytest-arg stringArray   Argument to pass to pytest (repeatable). Only -k and -m are supported.
         
        ->>> [CLI] pipelines test my_pipeline --no-wait
        -Recommendation: This command runs the last deployed version of the code
        +Flags:
        +  -h, --help   help for test
         
        -If you've made local changes, run 'databricks pipelines deploy' first to ensure they are included.
        +Global Flags:
        +      --debug            enable debug logging
        +  -o, --output type      output type: text or json (default text)
        +  -p, --profile string   ~/.databrickscfg profile
        +  -t, --target string    bundle target to use (if applicable)
        +      --var strings      set values for variables defined in project config. Example: --var="foo=bar"
         
        -Pipeline: my_pipeline ([UUID][0])
        -Update: [UUID][2]
         
        ->>> print_requests.py //updates
        -{
        -  "method": "POST",
        -  "path": "/api/2.0/pipelines/[UUID][0]/updates",
        -  "body": {
        -    "test_only": true,
        -    "test_options": {
        -      "test_selection": [
        -        "/Workspace/Users/someone@example.com/tests/test_orders.py::test_backfill"
        -      ]
        -    }
        -  }
        -}
        -{
        -  "method": "POST",
        -  "path": "/api/2.0/pipelines/[UUID][0]/updates",
        -  "body": {
        -    "test_only": true
        -  }
        -}
        +Exit code: 1
        
FAIL
TestAccept/pipelines/test/pipeline-id: branch ✅ | main (865ee2a) ❌ | latest (v1.26.0) ❌

main (865ee2a):

acceptance_test.go:1380: Diff:
        --- pipelines/test/pipeline-id/output.txt
        +++ /tmp/TestAcceptpipelinestestpipeline-id2762444292/002/output.txt
        @@ -1,53 +1,25 @@
         
         === Relative test selections resolve against the pipeline root_path
         
        ->>> MSYS_NO_PATHCONV=1 [CLI] pipelines test --pipeline-id [UUID][0] tests/test_orders.py::test_backfill /Workspace/Users/someone@example.com/other/tests --no-wait
        -Pipeline: [UUID][0]
        -Update: [UUID][1]
        +>>> MSYS_NO_PATHCONV=1 [CLI] pipelines test --pipeline-id [UUID] tests/test_orders.py::test_backfill /Workspace/Users/someone@example.com/other/tests --no-wait
        +Error: unknown flag: --no-wait
         
        ->>> print_requests.py //updates
        -{
        -  "method": "POST",
        -  "path": "/api/2.0/pipelines/[UUID][0]/updates",
        -  "body": {
        -    "test_only": true,
        -    "test_options": {
        -      "test_selection": [
        -        "/Workspace/Users/someone@example.com/orders/tests/test_orders.py::test_backfill",
        -        "/Workspace/Users/someone@example.com/other/tests"
        -      ]
        -    }
        -  }
        -}
        +Usage:
        +  databricks pipelines test [flags] [KEY] [TEST_SELECTION...]
         
        -=== Without a root_path, test selections must be workspace paths
        +Pipeline Test Flags:
        +      --pipeline-id string       ID of the pipeline to test, instead of a KEY.
        +      --pytest-arg stringArray   Argument to pass to pytest (repeatable). Only -k and -m are supported.
         
        ->>> musterr [CLI] pipelines test --pipeline-id [UUID][2] tests
        -Error: cannot resolve test selection "tests": the pipeline has no root_path; use an absolute workspace path such as /Workspace/Users/someone@example.com/tests
        +Flags:
        +  -h, --help[=true|false]   help for test
         
        ->>> MSYS_NO_PATHCONV=1 [CLI] pipelines test --pipeline-id [UUID][2] /Workspace/Users/someone@example.com/tests/test_other.py::test_x --no-wait
        -Pipeline: [UUID][2]
        -Update: [UUID][3]
        +Global Flags:
        +      --debug[=true|false]   enable debug logging
        +  -o, --output type          output type: text or json (default text)
        +  -p, --profile string       ~/.databrickscfg profile
        +  -t, --target string        bundle target to use (if applicable)
        +      --var strings          set values for variables defined in project config. Example: --var="foo=bar"
         
        ->>> print_requests.py //updates
        -{
        -  "method": "POST",
        -  "path": "/api/2.0/pipelines/[UUID][2]/updates",
        -  "body": {
        -    "test_only": true,
        -    "test_options": {
        -      "test_selection": [
        -        "/Workspace/Users/someone@example.com/tests/test_other.py::test_x"
        -      ]
        -    }
        -  }
        -}
         
        -=== Outside a project, --pipeline-id is required
        -
        ->>> musterr [CLI] pipelines test tests
        -Error: Unable to locate the bundle root: databricks.yml not found.
        -
        -Run this command from a directory that contains a bundle, or set the DATABRICKS_BUNDLE_ROOT environment variable to the bundle root directory.
        -Alternatively, create a new bundle with 'databricks bundle init'.
        -
        +Exit code: 1

latest (v1.26.0):

acceptance_test.go:1380: Diff:
        --- pipelines/test/pipeline-id/output.txt
        +++ /tmp/TestAcceptpipelinestestpipeline-id2495436109/002/output.txt
        @@ -1,53 +1,25 @@
         
         === Relative test selections resolve against the pipeline root_path
         
        ->>> MSYS_NO_PATHCONV=1 [CLI] pipelines test --pipeline-id [UUID][0] tests/test_orders.py::test_backfill /Workspace/Users/someone@example.com/other/tests --no-wait
        -Pipeline: [UUID][0]
        -Update: [UUID][1]
        +>>> MSYS_NO_PATHCONV=1 [CLI] pipelines test --pipeline-id [UUID] tests/test_orders.py::test_backfill /Workspace/Users/someone@example.com/other/tests --no-wait
        +Error: unknown flag: --no-wait
         
        ->>> print_requests.py //updates
        -{
        -  "method": "POST",
        -  "path": "/api/2.0/pipelines/[UUID][0]/updates",
        -  "body": {
        -    "test_only": true,
        -    "test_options": {
        -      "test_selection": [
        -        "/Workspace/Users/someone@example.com/orders/tests/test_orders.py::test_backfill",
        -        "/Workspace/Users/someone@example.com/other/tests"
        -      ]
        -    }
        -  }
        -}
        +Usage:
        +  databricks pipelines test [flags] [KEY] [TEST_SELECTION...]
         
        -=== Without a root_path, test selections must be workspace paths
        +Pipeline Test Flags:
        +      --pipeline-id string       ID of the pipeline to test, instead of a KEY.
        +      --pytest-arg stringArray   Argument to pass to pytest (repeatable). Only -k and -m are supported.
         
        ->>> musterr [CLI] pipelines test --pipeline-id [UUID][2] tests
        -Error: cannot resolve test selection "tests": the pipeline has no root_path; use an absolute workspace path such as /Workspace/Users/someone@example.com/tests
        +Flags:
        +  -h, --help   help for test
         
        ->>> MSYS_NO_PATHCONV=1 [CLI] pipelines test --pipeline-id [UUID][2] /Workspace/Users/someone@example.com/tests/test_other.py::test_x --no-wait
        -Pipeline: [UUID][2]
        -Update: [UUID][3]
        +Global Flags:
        +      --debug            enable debug logging
        +  -o, --output type      output type: text or json (default text)
        +  -p, --profile string   ~/.databrickscfg profile
        +  -t, --target string    bundle target to use (if applicable)
        +      --var strings      set values for variables defined in project config. Example: --var="foo=bar"
         
        ->>> print_requests.py //updates
        -{
        -  "method": "POST",
        -  "path": "/api/2.0/pipelines/[UUID][2]/updates",
        -  "body": {
        -    "test_only": true,
        -    "test_options": {
        -      "test_selection": [
        -        "/Workspace/Users/someone@example.com/tests/test_other.py::test_x"
        -      ]
        -    }
        -  }
        -}
         
        -=== Outside a project, --pipeline-id is required
        -
        ->>> musterr [CLI] pipelines test tests
        -Error: Unable to locate the bundle root: databricks.yml not found.
        -
        -Run this command from a directory that contains a bundle, or set the DATABRICKS_BUNDLE_ROOT environment variable to the bundle root directory.
        -Alternatively, create a new bundle with 'databricks bundle init'.
        -
        +Exit code: 1
TestAccept/pipelines/test/results/failed: branch ✅ | main (865ee2a) ❌ | latest (v1.26.0) ❌

main (865ee2a):

acceptance_test.go:1380: 
        	Error Trace:	/home/runner/work/cli/cli/libs/testdiff/testdiff.go:22
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:1380
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:1121
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:666
        	Error:      	Not equal: 
        	            	expected: "\n=== Failed, errored and skipped tests\n\n>>> musterr [CLI] pipelines test --pipeline-id [UUID][0]\nPipeline: [UUID][0]\nUpdate: [UUID][1]\nRunning tests...\n1 of 4 PASSED tests/test_orders.py::test_valid_order ........................... [PASSED]\n2 of 4 FAILED tests/test_orders.py::test_backfill .............................. [FAILED in 2.14s]\n3 of 4 SKIPPED tests/test_orders.py::test_legacy_format ........................ [SKIPPED]\n4 of 4 ERROR tests/test_orders.py::test_fixture ................................ [ERROR]\n\nCompleted with 1 failed and 1 errored tests:\n\nFailure in tests/test_orders.py::test_backfill\n  AssertionError: assert 100 == 150\n\nError in tests/test_orders.py::test_fixture\n  RuntimeError: fixture orders setup failed\n\nDone. PASSED=1 FAILED=1 SKIPPED=1 ERROR=1 TOTAL=4\n"
        	            	actual  : "\n=== Failed, errored and skipped tests\n\n>>> musterr [CLI] pipelines test --pipeline-id [UUID][0]\nPipeline: [UUID][0]\nUpdate: [UUID][1]\n\nUnexpected success\n\nExit code: 1\n"
        	            	
        	            	Diff:
        	            	--- Expected
        	            	+++ Actual
        	            	@@ -6,17 +6,6 @@
        	            	 Update: [UUID][1]
        	            	-Running tests...
        	            	-1 of 4 PASSED tests/test_orders.py::test_valid_order ........................... [PASSED]
        	            	-2 of 4 FAILED tests/test_orders.py::test_backfill .............................. [FAILED in 2.14s]
        	            	-3 of 4 SKIPPED tests/test_orders.py::test_legacy_format ........................ [SKIPPED]
        	            	-4 of 4 ERROR tests/test_orders.py::test_fixture ................................ [ERROR]
        	            	 
        	            	-Completed with 1 failed and 1 errored tests:
        	            	+Unexpected success
        	            	 
        	            	-Failure in tests/test_orders.py::test_backfill
        	            	-  AssertionError: assert 100 == 150
        	            	+Exit code: 1
        	            	 
        	            	-Error in tests/test_orders.py::test_fixture
        	            	-  RuntimeError: fixture orders setup failed
        	            	-
        	            	-Done. PASSED=1 FAILED=1 SKIPPED=1 ERROR=1 TOTAL=4
        	            	-
        	Test:       	TestAccept/pipelines/test/results/failed
        	Messages:   	pipelines/test/results/failed/output.txt vs /tmp/TestAcceptpipelinestestresultsfailed2305771195/002/output.txt

latest (v1.26.0): same output as main (865ee2a)

TestAccept/pipelines/test/results/incomplete: branch ✅ | main (865ee2a) ❌ | latest (v1.26.0) ❌

main (865ee2a):

acceptance_test.go:1380: 
        	Error Trace:	/home/runner/work/cli/cli/libs/testdiff/testdiff.go:22
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:1380
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:1121
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:666
        	Error:      	Not equal: 
        	            	expected: "\n=== Compute ends before all results are received\n\n>>> musterr [CLI] pipelines test --pipeline-id [UUID][0]\nPipeline: [UUID][0]\nUpdate: [UUID][1]\nRunning tests...\n1 of 2 PASSED tests/test_compute.py::test_small_order .......................... [PASSED]\n\nRUN INCOMPLETE: the update failed before all test results were received.\nDone. PASSED=1 FAILED=0 SKIPPED=0 ERROR=0 TOTAL=2\nError: update [UUID][1] is FAILED: ComputeTerminatedException: Compute terminated unexpectedly.\n"
        	            	actual  : "\n=== Compute ends before all results are received\n\n>>> musterr [CLI] pipelines test --pipeline-id [UUID][0]\nPipeline: [UUID][0]\nUpdate: [UUID][1]\n\nUnexpected success\n\nExit code: 1\n"
        	            	
        	            	Diff:
        	            	--- Expected
        	            	+++ Actual
        	            	@@ -6,8 +6,6 @@
        	            	 Update: [UUID][1]
        	            	-Running tests...
        	            	-1 of 2 PASSED tests/test_compute.py::test_small_order .......................... [PASSED]
        	            	 
        	            	-RUN INCOMPLETE: the update failed before all test results were received.
        	            	-Done. PASSED=1 FAILED=0 SKIPPED=0 ERROR=0 TOTAL=2
        	            	-Error: update [UUID][1] is FAILED: ComputeTerminatedException: Compute terminated unexpectedly.
        	            	+Unexpected success
        	            	 
        	            	+Exit code: 1
        	            	+
        	Test:       	TestAccept/pipelines/test/results/incomplete
        	Messages:   	pipelines/test/results/incomplete/output.txt vs /tmp/TestAcceptpipelinestestresultsincomplete1607590767/002/output.txt

latest (v1.26.0): same output as main (865ee2a)

TestAccept/pipelines/test/results/passed: branch ✅ | main (865ee2a) ❌ | latest (v1.26.0) ❌

main (865ee2a):

acceptance_test.go:1380: 
        	Error Trace:	/home/runner/work/cli/cli/libs/testdiff/testdiff.go:22
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:1380
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:1121
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:666
        	Error:      	Not equal: 
        	            	expected: "\n=== All tests pass\n\n>>> [CLI] pipelines test --pipeline-id [UUID][0]\nPipeline: [UUID][0]\nUpdate: [UUID][1]\nRunning tests...\n1 of 2 PASSED tests/test_orders.py::test_valid_order ........................... [PASSED in 1.84s]\n2 of 2 PASSED tests/test_orders.py::test_backfill .............................. [PASSED in 1.00s]\n\nDone. PASSED=2 FAILED=0 SKIPPED=0 ERROR=0 TOTAL=2\n\n>>> print_requests.py --get //events\n{\n  \"method\": \"GET\",\n  \"path\": \"/api/2.0/pipelines/[UUID][0]/events\",\n  \"q\": {\n    \"filter\": \"update_id = '[UUID][1]'\",\n    \"max_results\": \"250\",\n    \"order_by\": \"timestamp asc\"\n  }\n}\n{\n  \"method\": \"GET\",\n  \"path\": \"/api/2.0/pipelines/[UUID][0]/events\",\n  \"q\": {\n    \"filter\": \"update_id = '[UUID][1]'\",\n    \"max_results\": \"250\",\n    \"order_by\": \"timestamp asc\"\n  }\n}\n"
        	            	actual  : "\n=== All tests pass\n\n>>> [CLI] pipelines test --pipeline-id [UUID][0]\nPipeline: [UUID][0]\nUpdate: [UUID][1]\n\n>>> print_requests.py --get //events\n"
        	            	
        	            	Diff:
        	            	--- Expected
        	            	+++ Actual
        	            	@@ -6,27 +6,4 @@
        	            	 Update: [UUID][1]
        	            	-Running tests...
        	            	-1 of 2 PASSED tests/test_orders.py::test_valid_order ........................... [PASSED in 1.84s]
        	            	-2 of 2 PASSED tests/test_orders.py::test_backfill .............................. [PASSED in 1.00s]
        	            	-
        	            	-Done. PASSED=2 FAILED=0 SKIPPED=0 ERROR=0 TOTAL=2
        	            	 
        	            	 >>> print_requests.py --get //events
        	            	-{
        	            	-  "method": "GET",
        	            	-  "path": "/api/2.0/pipelines/[UUID][0]/events",
        	            	-  "q": {
        	            	-    "filter": "update_id = '[UUID][1]'",
        	            	-    "max_results": "250",
        	            	-    "order_by": "timestamp asc"
        	            	-  }
        	            	-}
        	            	-{
        	            	-  "method": "GET",
        	            	-  "path": "/api/2.0/pipelines/[UUID][0]/events",
        	            	-  "q": {
        	            	-    "filter": "update_id = '[UUID][1]'",
        	            	-    "max_results": "250",
        	            	-    "order_by": "timestamp asc"
        	            	-  }
        	            	-}
        	            	 
        	Test:       	TestAccept/pipelines/test/results/passed
        	Messages:   	pipelines/test/results/passed/output.txt vs /tmp/TestAcceptpipelinestestresultspassed3515673290/002/output.txt

latest (v1.26.0): same output as main (865ee2a)

TestAccept/pipelines/test/selection: branch ✅ | main (865ee2a) ❌ | latest (v1.26.0) ❌

main (865ee2a):

acceptance_test.go:1380: Diff:
        --- pipelines/test/selection/output.txt
        +++ /tmp/TestAcceptpipelinestestselection802882921/002/output.txt
        @@ -4,84 +4,29 @@
         Created pipelines.my_pipeline
         Files: 3 uploaded, 0 deleted
         Resources: 1 created, 0 changed, 0 deleted, 0 unchanged
        -View your pipeline my_pipeline here: [DATABRICKS_URL]/pipelines/[UUID][0]?w=[WORKSPACE_ID]
        +View your pipeline my_pipeline here: [DATABRICKS_URL]/pipelines/[UUID]?w=[WORKSPACE_ID]
         
         === Run all tests of the only pipeline
         
         >>> [CLI] pipelines test --no-wait
        -Recommendation: This command runs the last deployed version of the code
        +Error: unknown flag: --no-wait
         
        -If you've made local changes, run 'databricks pipelines deploy' first to ensure they are included.
        +Usage:
        +  databricks pipelines test [flags] [KEY] [TEST_SELECTION...]
         
        -Pipeline: my_pipeline ([UUID][0])
        -Update: [UUID][1]
        +Pipeline Test Flags:
        +      --pipeline-id string       ID of the pipeline to test, instead of a KEY.
        +      --pytest-arg stringArray   Argument to pass to pytest (repeatable). Only -k and -m are supported.
         
        ->>> print_requests.py //updates
        -{
        -  "method": "POST",
        -  "path": "/api/2.0/pipelines/[UUID][0]/updates",
        -  "body": {
        -    "test_only": true
        -  }
        -}
        +Flags:
        +  -h, --help[=true|false]   help for test
         
        -=== Select tests by relative path and test ID, with pytest arguments in both --pytest-arg forms
        +Global Flags:
        +      --debug[=true|false]   enable debug logging
        +  -o, --output type          output type: text or json (default text)
        +  -p, --profile string       ~/.databrickscfg profile
        +  -t, --target string        bundle target to use (if applicable)
        +      --var strings          set values for variables defined in project config. Example: --var="foo=bar"
         
        ->>> MSYS_NO_PATHCONV=1 [CLI] pipelines test my_pipeline tests tests/test_orders.py::test_backfill /Workspace/Users/someone@example.com/tests/test_other.py --pytest-arg=-k --pytest-arg=orders and not slow --pytest-arg -m --pytest-arg smoke,fast --no-wait
        -Recommendation: This command runs the last deployed version of the code
         
        -If you've made local changes, run 'databricks pipelines deploy' first to ensure they are included.
        -
        -Pipeline: my_pipeline ([UUID][0])
        -Update: [UUID][2]
        -
        ->>> print_requests.py //updates
        -{
        -  "method": "POST",
        -  "path": "/api/2.0/pipelines/[UUID][0]/updates",
        -  "body": {
        -    "test_only": true,
        -    "test_options": {
        -      "pytest_args": [
        -        "-k",
        -        "orders and not slow",
        -        "-m",
        -        "smoke,fast"
        -      ],
        -      "test_selection": [
        -        "/Workspace/Users/[USERNAME]/.bundle/test-pipeline-test-selection/default/files/tests",
        -        "/Workspace/Users/[USERNAME]/.bundle/test-pipeline-test-selection/default/files/tests/test_orders.py::test_backfill",
        -        "/Workspace/Users/someone@example.com/tests/test_other.py"
        -      ]
        -    }
        -  }
        -}
        -
        -=== Relative paths resolve from the working directory
        -
        ->>> [CLI] pipelines test my_pipeline test_orders.py::test_valid_order --no-wait
        -Recommendation: This command runs the last deployed version of the code
        -
        -If you've made local changes, run 'databricks pipelines deploy' first to ensure they are included.
        -
        -Pipeline: my_pipeline ([UUID][0])
        -Update: [UUID][3]
        -
        ->>> print_requests.py //updates
        -{
        -  "method": "POST",
        -  "path": "/api/2.0/pipelines/[UUID][0]/updates",
        -  "body": {
        -    "test_only": true,
        -    "test_options": {
        -      "test_selection": [
        -        "/Workspace/Users/[USERNAME]/.bundle/test-pipeline-test-selection/default/files/tests/test_orders.py::test_valid_order"
        -      ]
        -    }
        -  }
        -}
        -
        -=== Test selections require the pipeline KEY first
        -
        ->>> musterr [CLI] pipelines test tests/test_orders.py
        -Error: "tests/test_orders.py" is not a pipeline in this bundle (pipelines: my_pipeline). Usage: databricks pipelines test my_pipeline tests/test_orders.py
        +Exit code: 1

latest (v1.26.0):

acceptance_test.go:1380: Diff:
        --- pipelines/test/selection/output.txt
        +++ /tmp/TestAcceptpipelinestestselection2655896794/002/output.txt
        @@ -4,84 +4,29 @@
         Created pipelines.my_pipeline
         Files: 3 uploaded, 0 deleted
         Resources: 1 created, 0 changed, 0 deleted, 0 unchanged
        -View your pipeline my_pipeline here: [DATABRICKS_URL]/pipelines/[UUID][0]?w=[WORKSPACE_ID]
        +View your pipeline my_pipeline here: [DATABRICKS_URL]/pipelines/[UUID]?w=[WORKSPACE_ID]
         
         === Run all tests of the only pipeline
         
         >>> [CLI] pipelines test --no-wait
        -Recommendation: This command runs the last deployed version of the code
        +Error: unknown flag: --no-wait
         
        -If you've made local changes, run 'databricks pipelines deploy' first to ensure they are included.
        +Usage:
        +  databricks pipelines test [flags] [KEY] [TEST_SELECTION...]
         
        -Pipeline: my_pipeline ([UUID][0])
        -Update: [UUID][1]
        +Pipeline Test Flags:
        +      --pipeline-id string       ID of the pipeline to test, instead of a KEY.
        +      --pytest-arg stringArray   Argument to pass to pytest (repeatable). Only -k and -m are supported.
         
        ->>> print_requests.py //updates
        -{
        -  "method": "POST",
        -  "path": "/api/2.0/pipelines/[UUID][0]/updates",
        -  "body": {
        -    "test_only": true
        -  }
        -}
        +Flags:
        +  -h, --help   help for test
         
        -=== Select tests by relative path and test ID, with pytest arguments in both --pytest-arg forms
        +Global Flags:
        +      --debug            enable debug logging
        +  -o, --output type      output type: text or json (default text)
        +  -p, --profile string   ~/.databrickscfg profile
        +  -t, --target string    bundle target to use (if applicable)
        +      --var strings      set values for variables defined in project config. Example: --var="foo=bar"
         
        ->>> MSYS_NO_PATHCONV=1 [CLI] pipelines test my_pipeline tests tests/test_orders.py::test_backfill /Workspace/Users/someone@example.com/tests/test_other.py --pytest-arg=-k --pytest-arg=orders and not slow --pytest-arg -m --pytest-arg smoke,fast --no-wait
        -Recommendation: This command runs the last deployed version of the code
         
        -If you've made local changes, run 'databricks pipelines deploy' first to ensure they are included.
        -
        -Pipeline: my_pipeline ([UUID][0])
        -Update: [UUID][2]
        -
        ->>> print_requests.py //updates
        -{
        -  "method": "POST",
        -  "path": "/api/2.0/pipelines/[UUID][0]/updates",
        -  "body": {
        -    "test_only": true,
        -    "test_options": {
        -      "pytest_args": [
        -        "-k",
        -        "orders and not slow",
        -        "-m",
        -        "smoke,fast"
        -      ],
        -      "test_selection": [
        -        "/Workspace/Users/[USERNAME]/.bundle/test-pipeline-test-selection/default/files/tests",
        -        "/Workspace/Users/[USERNAME]/.bundle/test-pipeline-test-selection/default/files/tests/test_orders.py::test_backfill",
        -        "/Workspace/Users/someone@example.com/tests/test_other.py"
        -      ]
        -    }
        -  }
        -}
        -
        -=== Relative paths resolve from the working directory
        -
        ->>> [CLI] pipelines test my_pipeline test_orders.py::test_valid_order --no-wait
        -Recommendation: This command runs the last deployed version of the code
        -
        -If you've made local changes, run 'databricks pipelines deploy' first to ensure they are included.
        -
        -Pipeline: my_pipeline ([UUID][0])
        -Update: [UUID][3]
        -
        ->>> print_requests.py //updates
        -{
        -  "method": "POST",
        -  "path": "/api/2.0/pipelines/[UUID][0]/updates",
        -  "body": {
        -    "test_only": true,
        -    "test_options": {
        -      "test_selection": [
        -        "/Workspace/Users/[USERNAME]/.bundle/test-pipeline-test-selection/default/files/tests/test_orders.py::test_valid_order"
        -      ]
        -    }
        -  }
        -}
        -
        -=== Test selections require the pipeline KEY first
        -
        ->>> musterr [CLI] pipelines test tests/test_orders.py
        -Error: "tests/test_orders.py" is not a pipeline in this bundle (pipelines: my_pipeline). Usage: databricks pipelines test my_pipeline tests/test_orders.py
        +Exit code: 1

@eng-dev-ecosystem-bot

eng-dev-ecosystem-bot commented Nov 2, 2026 •

Copy link
Copy Markdown
Collaborator

✅ Integration test report

Commit: b45969f

Run: 42728311165

Env ✅​pass 🙈​skip Time
✅​ aws linux-2core-8gb 271 23 5:05
✅​ aws-windows-latest-4core-16gb 273 21 4:58
✅​ azure linux-2core-8gb 270 23 4:45
✅​ azure-windows-latest-4core-16gb 272 21 5:48
✅​ gcp linux-2core-8gb 271 23 5:22
✅​ gcp-windows-latest-4core-16gb 273 21 6:48
Top 3 slowest tests (at least 2 minutes):
duration env testname
3:14 gcp-windows-latest-4core-16gb TestAccept
3:06 aws-windows-latest-4core-16gb TestAccept
3:03 azure-windows-latest-4core-16gb TestAccept

@tharunbhuk-db
tharunbhuk-db force-pushed the tharun/pipelines-test-pr2-output branch from 8542365 to cb0a526 Compare November 2, 2026 23:22
@tharunbhuk-db
tharunbhuk-db force-pushed the tharun/pipelines-test-pr2-output branch 2 times, most recently from 025649f to 23e0bb0 Compare November 3, 2026 17:34
Comment thread cmd/pipelines/test.go Outdated
if incompleteReason == "" {
summary := run.summaries[0]
testsFailed := summary.Total == 0 || summary.Failed > 0 || summary.Errors > 0
if !testsFailed && state == pipelines.UpdateInfoStateCompleted {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A COMPLETED update whose tests all passed returns nil here without checking run.fatalError, so an update that also reported a fatal error exits 0. Could we add run.fatalError == "" to this condition, with a TestTestRunError case for it?

Related: a run where every test is SKIPPED also returns nil here. That matches the design (an all-skipped run completes), but CI and agents will read it as green, so the help should say so explicitly.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done. testRunError also checks run.fatalError == "", with a test case. The help says an all-skipped run exits 0.

Comment thread cmd/pipelines/test.go
UpdateId: tw.updateID,
})
if err != nil {
return "", err

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A GetUpdate error (and the same for the events read just below) ends the wait with the raw error and no --resume hint, even though the update keeps running. The SDK client already retries transient HTTP errors, so no extra retry logic is needed here. Could the error just include the same resume command as the timeout path?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll do this in a follow-up.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in #7426 (3b224ed), where --resume exists. A failed read of the update or its events now ends with the --resume command, as the timeout does.

Comment thread cmd/pipelines/test.go Outdated
updateID := resumeUpdateID
if !resume {
req := newStartTestUpdateRequest(target.selections, pytestArgs)
updateID, err = startTestUpdate(ctx, target.w, target.pipelineID, req)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This POST uses the signal context, so a Ctrl+C during Start Update loses the update ID, and the update runs with no way to find it from this output. Could the POST use cmd.Context()?

Also on this line: when the service rejects the start because an update is already active, a short static hint after its message would give agents the next step, for example "Run 'databricks pipelines stop KEY' to cancel it, or wait for it to finish." There's no need to parse the ID out of the message.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in #7426: Start Update uses cmd.Context(), tested by TestInterruptDuringStartUpdateKeepsTheUpdateID. The already-active hint: I'll do that in a follow-up.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The active-update hint is fixed in #7426 (3b224ed). When Start Update returns 409, the error says to wait, to run pipelines stop KEY for a bundle pipeline, or to read the update with --resume. The ID is not parsed from the message.

Comment thread cmd/pipelines/test_render.go Outdated
fmt.Fprintln(out)

if incompleteReason != "" {
fmt.Fprintf(out, "RUN INCOMPLETE: %s.\n", incompleteReason)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The most likely cause of INCOMPLETE is events still arriving after the terminal state, and the update can still be read later. Could we print the --resume command here too?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll do this in a follow-up.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in #7426 (3b224ed). An incomplete result now prints the --resume command before the summary line.


// testVerdictErrorPrefix starts the message of the error by which the service reports that tests
// failed, such as "[PIPELINE_TEST_FAILED] 1 of the pipeline's tests failed (1 passed, 0 skipped).".
const testVerdictErrorPrefix = "[PIPELINE_TEST_FAILED]"

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Recognizing the service verdict by message prefix is fragile. The exit code doesn't depend on it, only the output does. The SDK's ErrorDetail has no error class yet, so a comment saying this is temporary until a class is exposed would help.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in #7432. A comment says the message is matched because ErrorDetail has no error class yet.

Comment thread cmd/pipelines/test.go
// update ended in another way or reports an error of its own. The service's own verdict,
// such as "[PIPELINE_TEST_FAILED] 1 of the pipeline's tests failed", is not.
ended := state == pipelines.UpdateInfoStateFailed || state == pipelines.UpdateInfoStateCompleted
if testsFailed && ended && run.fatalError == "" {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Every failure path exits 1. Test failures and zero-collected runs print nothing to stderr, while infra failures print Error:, so an agent tells "fix the code" from "retry" only from the output. pipelines run behaves the same, so distinct exit codes would be a CLI-wide decision. For this stack, could the help just say what exit 0 and 1 mean? Small related nit: with --junit-xml, the JUnit report: line prints after the summary, so tail -1 misses the summary; printing it before the summary would keep the summary last.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done. The help says what exit 0 and 1 mean. The JUnit report: line prints before the summary from #7434.

Comment thread cmd/pipelines/test_render.go Outdated
summary := run.summaries[0]
switch summary.Total {
case 0:
fmt.Fprintln(out, "No tests were collected.")

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agents hit this often, and the message gives no hint. Naming what was run would make it self-correcting, e.g. No tests were collected from <testfile>. Check the test selection and any --pytest-arg filters. (I'd avoid restating pytest's naming rules here, since projects can change them in their pytest configuration.) But if we can give a bit better message that would be nice.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll do this in a follow-up.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 86122f3. The message names the selection, or the pipeline's test sources, and points to the --pytest-arg filters when they were given. It does not restate pytest's naming rules.

testCasePassed testCaseStatus = "PASSED"
testCaseFailed testCaseStatus = "FAILED"
testCaseSkipped testCaseStatus = "SKIPPED"
testCaseError testCaseStatus = "ERROR"

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should this be errored? I see we do error and errored at some places would be good if we can do consistent.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These follow the proto: the case status is ERROR and the summary field is errored. The summary line uses the status names (ERROR=).

@@ -0,0 +1,799 @@
# 200 cases emit 601 events, more than two pages of List Pipeline Events.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Non-blocking: instead of committing 800 lines of identical tests, could the script generate this file before pipelines deploy? The fake collects tests with a regex over the deployed files (libs/testserver/pipelines_testing.go:106), so this works the same and runs on bash 3.2 and Git Bash:

for i in $(seq 0 199); do printf 'def test_case_%03d():\n    assert True\n\n' "$i"; done > tests/test_many.py

Also worth checking how many cases you actually need. The CLI doesn't send max_results, so the fake pages at its default of 25 events (pipelines_testing.go:332), and the output shows 8 pages read. Fewer cases may cover the same paths. The page math reads better in the script than as a comment in a generated file.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done. The unit test TestReadTestEvents200Cases in #7432 replaces it. The reader sends max_results=250.

assert.Equal(t, 400, response.StatusCode)
}

func TestPipelineStartUpdateRejectsInvalidTestOptions(t *testing.T) {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Non-blocking: this file tests the fake more deeply than the CLI contract needs, and a few could go:

  • TestPipelineStartUpdateRejectsInvalidTestOptions and TestPipelineStartUpdateRejectsTestOptionsWithoutTestOnly, together with the fake's pipelineTestOptionsError. The CLI validates --pytest-arg and makes selections absolute before any request, and it always sends test_only: true, so pipelines test never reaches this validation.
  • TestPipelineFiniteUpdateIsActiveUntilItEnds, TestPipelineUpdateThatIsNeverPolledEndsWithItsFirstState and TestPipelineStopHidesEventsOfStatesNotReached. They pin the fake's state timing, and what the CLI sees (409, CANCELED) is golden in run-control and submit-error.
  • TestPipelineEventsPages. Paging is covered by the CLI's own reader test and many-cases.

TestCollectPipelineTests, TestPipelineTestUpdate and TestPipelineTestUpdateConflictAndStop cover what the acceptance tests rely on. Also, line 242 repeats the assertion on line 241.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done. The stack no longer changes libs/testserver.

@tharunbhuk-db
tharunbhuk-db marked this pull request as ready for review November 5, 2026 16:09
@tharunbhuk-db tharunbhuk-db changed the title [WIP] Wait for and print the results of databricks pipelines test Wait for and print the results of databricks pipelines test Nov 5, 2026
@tharunbhuk-db
tharunbhuk-db marked this pull request as draft November 9, 2026 23:57
@tharunbhuk-db
tharunbhuk-db force-pushed the tharun/pipelines-test-pr1-command branch from 66e7f7e to 62049a1 Compare November 10, 2026 00:01
@tharunbhuk-db
tharunbhuk-db changed the base branch from tharun/pipelines-test-pr1-command to tharun/pipelines-test-a2-fake-server November 10, 2026 00:25
@tharunbhuk-db
tharunbhuk-db force-pushed the tharun/pipelines-test-pr2-output branch from 23e0bb0 to b45969f Compare November 10, 2026 00:25
@tharunbhuk-db
tharunbhuk-db force-pushed the tharun/pipelines-test-print-results branch from 2036088 to 42a9409 Compare November 10, 2026 17:13
@tharunbhuk-db
tharunbhuk-db force-pushed the tharun/pipelines-test-pr2-output branch from e7de143 to d0d7138 Compare November 10, 2026 17:13

@pranshupand-db pranshupand-db left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Follow-ups for the stack are in the #7297 review: #7297

idelafuente-oc pushed a commit to idelafuente-oc/cli that referenced this pull request Nov 11, 2026
## 🥞 Stacked PR
Use this [link](https://github.com/databricks/cli/pull/7297/files) to
review incremental changes.
-
[**stack/pipelines-test-pr1-command**](databricks#7297)
[[Files changed](https://github.com/databricks/cli/pull/7297/files)] ←
_this PR_
-
[stack/pipelines-test-bundle-key](databricks#7431)
[[Files changed](https://github.com/databricks/cli/pull/7431/files)]
-
[stack/pipelines-test-read-events](databricks#7432)
[[Files changed](https://github.com/databricks/cli/pull/7432/files)]
-
[stack/pipelines-test-print-results](databricks#7433)
[[Files changed](https://github.com/databricks/cli/pull/7433/files)]
-
[stack/pipelines-test-pr2-output](databricks#7298)
[[Files changed](https://github.com/databricks/cli/pull/7298/files)]
-
[stack/pipelines-test-a4-run-control](databricks#7426)
[[Files changed](https://github.com/databricks/cli/pull/7426/files)]
-
[stack/pipelines-test-pr3-junit](databricks#7299)
[[Files changed](https://github.com/databricks/cli/pull/7299/files)]
-
[stack/pipelines-test-junit-flag](databricks#7434)
[[Files changed](https://github.com/databricks/cli/pull/7434/files)]

---------
- Add `databricks pipelines test --pipeline-id <id>
[TEST_SELECTION...]`: starts a test-only update and prints its IDs.
- Experimental: runs only with `DATABRICKS_CLI_PIPELINES_TEST=true`.
- Selections are workspace paths or paths relative to `root_path`, sent
as typed. `--pytest-arg` accepts only `-k` and `-m`.

Tests: unit tests; acceptance in
`acceptance/pipelines/test/{arguments,pipeline-id,submit-error}`.

This pull request and its description were written by Isaac.

Co-authored-by: Isaac <no-reply@databricks.com>
idelafuente-oc pushed a commit to idelafuente-oc/cli that referenced this pull request Nov 11, 2026
## 🥞 Stacked PR
-
[stack/pipelines-test-pr1-command](databricks#7297)
[[Files changed](https://github.com/databricks/cli/pull/7297/files)]
-
[**stack/pipelines-test-bundle-key**](databricks#7431)
[[Files changed](https://github.com/databricks/cli/pull/7431/files)] ←
_this PR_
-
[stack/pipelines-test-read-events](databricks#7432)
[[Files changed](https://github.com/databricks/cli/pull/7432/files)]
-
[stack/pipelines-test-print-results](databricks#7433)
[[Files changed](https://github.com/databricks/cli/pull/7433/files)]
-
[stack/pipelines-test-pr2-output](databricks#7298)
[[Files changed](https://github.com/databricks/cli/pull/7298/files)]
-
[stack/pipelines-test-a4-run-control](databricks#7426)
[[Files changed](https://github.com/databricks/cli/pull/7426/files)]
-
[stack/pipelines-test-pr3-junit](databricks#7299)
[[Files changed](https://github.com/databricks/cli/pull/7299/files)]
-
[stack/pipelines-test-junit-flag](databricks#7434)
[[Files changed](https://github.com/databricks/cli/pull/7434/files)]

---------
- In a bundle, identify the pipeline by KEY: `pipelines test [flags]
[KEY] [TEST_SELECTION...]`.
- Relative selections are local paths, mapped to their deployed paths.
- A first argument that is not a KEY gets an error that names the
bundle's pipelines and the working command.

Tests: unit tests; acceptance in
`acceptance/pipelines/test/{selection,immutable}`.

This pull request and its description were written by Isaac.

Co-authored-by: Isaac <no-reply@databricks.com>
@tharunbhuk-db
tharunbhuk-db force-pushed the tharun/pipelines-test-print-results branch from 42a9409 to 8ff858f Compare November 16, 2026 22:19
@tharunbhuk-db
tharunbhuk-db force-pushed the tharun/pipelines-test-pr2-output branch from d0d7138 to cdc73e0 Compare November 16, 2026 22:19
@tharunbhuk-db
tharunbhuk-db force-pushed the tharun/pipelines-test-print-results branch from 8ff858f to 40a204e Compare November 17, 2026 00:17
@tharunbhuk-db
tharunbhuk-db force-pushed the tharun/pipelines-test-pr2-output branch from cdc73e0 to 86122f3 Compare November 17, 2026 00:17
tharunbhuk-db added a commit that referenced this pull request Nov 17, 2026
The changelog fragments of #7298, #7426 and #7434 are left out of this
merge: they belong to those PRs, and this merge goes away when they land.

Co-authored-by: Isaac <no-reply@databricks.com>
@tharunbhuk-db
tharunbhuk-db force-pushed the tharun/pipelines-test-print-results branch from 40a204e to f2268c2 Compare November 17, 2026 07:00
@tharunbhuk-db
tharunbhuk-db force-pushed the tharun/pipelines-test-pr2-output branch from 86122f3 to ae6fcdc Compare November 17, 2026 07:00
@tharunbhuk-db
tharunbhuk-db force-pushed the tharun/pipelines-test-print-results branch from f2268c2 to 6c5c865 Compare November 17, 2026 07:28
Base automatically changed from tharun/pipelines-test-print-results to main November 17, 2026 07:55
tharunbhuk-db and others added 2 commits November 17, 2026 07:56
The command waits for the test-only update, prints each test case as it
finishes, then the failure details and a summary line. It exits with code 0
only for a complete run where at least one test ran and none failed or errored,
and a completed update that reports a fatal error does not pass. Test IDs are
shown relative to the project folder. With the no-wait flag, the command returns
after the update starts.

The wait takes the steps of the bundle pipeline runner's wait loop. The
acceptance tests stub the service per case, as the other pipelines tests do.

Co-authored-by: Isaac <no-reply@databricks.com>
The message names the test selection, or the pipeline's test sources, and
points to the --pytest-arg filters when they were given. It does not restate
pytest's naming rules, because a project can change them. The changelog
entry for the results output moves to this PR.

Co-authored-by: Isaac <no-reply@databricks.com>
@tharunbhuk-db
tharunbhuk-db force-pushed the tharun/pipelines-test-pr2-output branch from ae6fcdc to cfe688b Compare November 17, 2026 07:56
The entry is added with the other pipeline testing entries in #7439.

Co-authored-by: Isaac <no-reply@databricks.com>
tharunbhuk-db added a commit that referenced this pull request Nov 17, 2026
…and JUnit PRs

They move here from #7298, #7426 and #7434, so that all pipeline testing
entries are in one PR.

Co-authored-by: Isaac <no-reply@databricks.com>
@tharunbhuk-db
tharunbhuk-db added this pull request to the merge queue Nov 17, 2026
Merged via the queue into main with commit 4e503ea Nov 17, 2026
24 checks passed
@tharunbhuk-db
tharunbhuk-db deleted the tharun/pipelines-test-pr2-output branch November 17, 2026 08:46
@eng-dev-ecosystem-bot

Copy link
Copy Markdown
Collaborator

❌ Integration test report

Commit: 4e503ea

Run: 44049649553

Env ❌​FAIL 🟨​KNOWN 💚​RECOVERED ✅​pass 🙈​skip Time
❌​ aws linux-2core-8gb 83 6 7 1539 1339 261:03
❌​ aws-windows-latest-4core-16gb 91 1 12 1450 1366 265:15
❌​ azure linux-2core-8gb 67 1 12 1535 1347 255:18
❌​ azure-windows-latest-4core-16gb 92 1 12 1429 1374 266:52
❌​ gcp linux-2core-8gb 98 3 10 1511 1345 284:19
❌​ gcp-windows-latest-4core-16gb 93 1 12 1435 1372 298:04
132 interesting tests: 119 FAIL, 7 RECOVERED, 6 KNOWN
Test Name aws linux-2core-8gb aws-windows-latest-4core-16gb azure linux-2core-8gb azure-windows-latest-4core-16gb gcp linux-2core-8gb gcp-windows-latest-4core-16gb
🟨​ TestAccept 🟨​K 🟨​K 🟨​K 🟨​K 🟨​K 🟨​K
❌​ TestAccept/bundle/config-remote-sync/cluster_policy_cross_target ❌​F 🙈​s ❌​F 🙈​s ❌​F 🙈​s
❌​ TestAccept/bundle/config-remote-sync/cluster_policy_cross_target/DMS= ❌​F ❌​F ❌​F
❌​ TestAccept/bundle/config-remote-sync/cluster_policy_cross_target/DMS=true ❌​F ❌​F ❌​F
❌​ TestAccept/bundle/config-remote-sync/cluster_policy_remote_addition ❌​F 🙈​s ❌​F 🙈​s ❌​F 🙈​s
❌​ TestAccept/bundle/config-remote-sync/cluster_policy_remote_addition/DMS= ❌​F ❌​F ❌​F
❌​ TestAccept/bundle/config-remote-sync/cluster_policy_remote_addition/DMS=true ❌​F ❌​F ❌​F
❌​ TestAccept/bundle/config-remote-sync/policy_injected_cluster_fields ❌​F 🙈​s ❌​F 🙈​s ❌​F 🙈​s
❌​ TestAccept/bundle/config-remote-sync/policy_injected_cluster_fields/DMS= ❌​F ❌​F ❌​F
❌​ TestAccept/bundle/config-remote-sync/policy_injected_cluster_fields/DMS=true ❌​F ❌​F ❌​F
❌​ TestAccept/bundle/deploy/spark-jar-task ❌​F ❌​F ❌​F ❌​F ❌​F ❌​F
❌​ TestAccept/bundle/deploy/spark-jar-task/DMS= ❌​F ❌​F ❌​F ❌​F ❌​F ❌​F
❌​ TestAccept/bundle/deploy/spark-jar-task/DMS=true ❌​F ❌​F ❌​F ❌​F ❌​F ❌​F
❌​ TestAccept/bundle/dms/declined-deploy ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F
❌​ TestAccept/bundle/dms/declined-deploy/DMS=true ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F
❌​ TestAccept/bundle/dms/deployment-metadata-change ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F
❌​ TestAccept/bundle/dms/deployment-metadata-change/DMS=true ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F
❌​ TestAccept/bundle/dms/record-failure ✅​p ✅​p ✅​p ❌​F ✅​p ✅​p
❌​ TestAccept/bundle/dms/record-failure/DMS=true ✅​p ✅​p ✅​p ❌​F ✅​p ✅​p
❌​ TestAccept/bundle/invariant/continue_293 ❌​F ❌​F ❌​F ❌​F ❌​F ❌​F
❌​ TestAccept/bundle/invariant/continue_293/DMS=/INPUT_CONFIG=cluster.yml.tmpl ❌​F ❌​F ❌​F ❌​F ❌​F ❌​F
❌​ TestAccept/bundle/invariant/continue_293/DMS=/INPUT_CONFIG=cluster_apply_policy_default_values.yml.tmpl ❌​F ❌​F ❌​F ❌​F ❌​F ❌​F
❌​ TestAccept/bundle/invariant/continue_293/DMS=/INPUT_CONFIG=job_apply_policy_default_values_for_each_task.yml.tmpl ❌​F ❌​F ❌​F ❌​F ❌​F ❌​F
❌​ TestAccept/bundle/invariant/continue_293/DMS=/INPUT_CONFIG=job_apply_policy_default_values_job_cluster.yml.tmpl ❌​F ❌​F ❌​F ❌​F ❌​F ❌​F
❌​ TestAccept/bundle/invariant/continue_293/DMS=/INPUT_CONFIG=job_apply_policy_default_values_task_cluster.yml.tmpl ❌​F ❌​F ❌​F ❌​F ❌​F ❌​F
❌​ TestAccept/bundle/invariant/continue_293/DMS=/INPUT_CONFIG=job_with_task.yml.tmpl ❌​F ❌​F ❌​F ❌​F ❌​F ❌​F
❌​ TestAccept/bundle/invariant/delete_idempotent ❌​F ❌​F ❌​F ❌​F ❌​F ❌​F
❌​ TestAccept/bundle/invariant/delete_idempotent/DMS=/INPUT_CONFIG=cluster.yml.tmpl/READPLAN= ❌​F ❌​F ❌​F ❌​F ❌​F ❌​F
❌​ TestAccept/bundle/invariant/delete_idempotent/DMS=/INPUT_CONFIG=cluster.yml.tmpl/READPLAN=1 ❌​F ❌​F ❌​F ❌​F ❌​F ❌​F
❌​ TestAccept/bundle/invariant/delete_idempotent/DMS=/INPUT_CONFIG=cluster_apply_policy_default_values.yml.tmpl/READPLAN= ❌​F ❌​F ❌​F ❌​F ❌​F ❌​F
❌​ TestAccept/bundle/invariant/delete_idempotent/DMS=/INPUT_CONFIG=cluster_apply_policy_default_values.yml.tmpl/READPLAN=1 ❌​F ❌​F ❌​F ❌​F ❌​F ❌​F
❌​ TestAccept/bundle/invariant/delete_idempotent/DMS=/INPUT_CONFIG=cluster_libraries.yml.tmpl/READPLAN= ❌​F ❌​F ❌​F ❌​F ❌​F ❌​F
❌​ TestAccept/bundle/invariant/delete_idempotent/DMS=/INPUT_CONFIG=cluster_libraries.yml.tmpl/READPLAN=1 ❌​F ❌​F ❌​F ❌​F ❌​F ❌​F
❌​ TestAccept/bundle/invariant/delete_idempotent/DMS=/INPUT_CONFIG=job_apply_policy_default_values_for_each_task.yml.tmpl/READPLAN= ❌​F ❌​F ❌​F ❌​F ❌​F ❌​F
❌​ TestAccept/bundle/invariant/delete_idempotent/DMS=/INPUT_CONFIG=job_apply_policy_default_values_for_each_task.yml.tmpl/READPLAN=1 ❌​F ❌​F ❌​F ❌​F ❌​F ❌​F
❌​ TestAccept/bundle/invariant/delete_idempotent/DMS=/INPUT_CONFIG=job_apply_policy_default_values_job_cluster.yml.tmpl/READPLAN= ❌​F ❌​F ❌​F ❌​F ❌​F ❌​F
❌​ TestAccept/bundle/invariant/delete_idempotent/DMS=/INPUT_CONFIG=job_apply_policy_default_values_job_cluster.yml.tmpl/READPLAN=1 ❌​F ❌​F ❌​F ❌​F ❌​F ❌​F
❌​ TestAccept/bundle/invariant/delete_idempotent/DMS=/INPUT_CONFIG=job_apply_policy_default_values_task_cluster.yml.tmpl/READPLAN= ❌​F ❌​F ❌​F ❌​F ❌​F ❌​F
❌​ TestAccept/bundle/invariant/delete_idempotent/DMS=/INPUT_CONFIG=job_apply_policy_default_values_task_cluster.yml.tmpl/READPLAN=1 ❌​F ❌​F ❌​F ❌​F ❌​F ❌​F
❌​ TestAccept/bundle/invariant/delete_idempotent/DMS=/INPUT_CONFIG=job_with_task.yml.tmpl/READPLAN= ❌​F ❌​F ❌​F ❌​F ❌​F ❌​F
❌​ TestAccept/bundle/invariant/delete_idempotent/DMS=/INPUT_CONFIG=job_with_task.yml.tmpl/READPLAN=1 ❌​F ❌​F ❌​F ❌​F ❌​F ❌​F
❌​ TestAccept/bundle/invariant/destroy_idempotent ❌​F ❌​F ❌​F ❌​F ❌​F ❌​F
❌​ TestAccept/bundle/invariant/destroy_idempotent/DMS=/INPUT_CONFIG=cluster.yml.tmpl/READPLAN= ❌​F ❌​F ❌​F ❌​F ❌​F ❌​F
❌​ TestAccept/bundle/invariant/destroy_idempotent/DMS=/INPUT_CONFIG=cluster.yml.tmpl/READPLAN=1 ❌​F ❌​F ❌​F ❌​F ❌​F ❌​F
❌​ TestAccept/bundle/invariant/destroy_idempotent/DMS=/INPUT_CONFIG=cluster_apply_policy_default_values.yml.tmpl/READPLAN= ❌​F ❌​F ❌​F ❌​F ❌​F ❌​F
❌​ TestAccept/bundle/invariant/destroy_idempotent/DMS=/INPUT_CONFIG=cluster_apply_policy_default_values.yml.tmpl/READPLAN=1 ❌​F ❌​F ❌​F ❌​F ❌​F ❌​F
❌​ TestAccept/bundle/invariant/destroy_idempotent/DMS=/INPUT_CONFIG=cluster_libraries.yml.tmpl/READPLAN= ❌​F ❌​F ❌​F ❌​F ❌​F ❌​F
❌​ TestAccept/bundle/invariant/destroy_idempotent/DMS=/INPUT_CONFIG=cluster_libraries.yml.tmpl/READPLAN=1 ❌​F ❌​F ❌​F ❌​F ❌​F ❌​F
❌​ TestAccept/bundle/invariant/destroy_idempotent/DMS=/INPUT_CONFIG=job_apply_policy_default_values_for_each_task.yml.tmpl/READPLAN= ❌​F ❌​F ❌​F ❌​F ❌​F ❌​F
❌​ TestAccept/bundle/invariant/destroy_idempotent/DMS=/INPUT_CONFIG=job_apply_policy_default_values_for_each_task.yml.tmpl/READPLAN=1 ❌​F ❌​F ❌​F ❌​F ❌​F ❌​F
❌​ TestAccept/bundle/invariant/destroy_idempotent/DMS=/INPUT_CONFIG=job_apply_policy_default_values_job_cluster.yml.tmpl/READPLAN= ❌​F ❌​F ❌​F ❌​F ❌​F ❌​F
❌​ TestAccept/bundle/invariant/destroy_idempotent/DMS=/INPUT_CONFIG=job_apply_policy_default_values_job_cluster.yml.tmpl/READPLAN=1 ❌​F ❌​F ❌​F ❌​F ❌​F ❌​F
❌​ TestAccept/bundle/invariant/destroy_idempotent/DMS=/INPUT_CONFIG=job_apply_policy_default_values_task_cluster.yml.tmpl/READPLAN= ❌​F ❌​F ❌​F ❌​F ❌​F ❌​F
❌​ TestAccept/bundle/invariant/destroy_idempotent/DMS=/INPUT_CONFIG=job_apply_policy_default_values_task_cluster.yml.tmpl/READPLAN=1 ❌​F ❌​F ❌​F ❌​F ❌​F ❌​F
❌​ TestAccept/bundle/invariant/destroy_idempotent/DMS=/INPUT_CONFIG=job_pydabs_1000_tasks.yml.tmpl/READPLAN= ✅​p ✅​p ✅​p ❌​F ✅​p ✅​p
❌​ TestAccept/bundle/invariant/destroy_idempotent/DMS=/INPUT_CONFIG=job_with_task.yml.tmpl/READPLAN= ❌​F ❌​F ❌​F ❌​F ❌​F ❌​F
❌​ TestAccept/bundle/invariant/destroy_idempotent/DMS=/INPUT_CONFIG=job_with_task.yml.tmpl/READPLAN=1 ❌​F ❌​F ❌​F ❌​F ❌​F ❌​F
❌​ TestAccept/bundle/invariant/no_drift ❌​F ❌​F ❌​F ❌​F ❌​F ❌​F
❌​ TestAccept/bundle/invariant/no_drift/DMS=/INPUT_CONFIG=cluster.yml.tmpl/READPLAN= ❌​F ❌​F ❌​F ❌​F ❌​F ❌​F
❌​ TestAccept/bundle/invariant/no_drift/DMS=/INPUT_CONFIG=cluster.yml.tmpl/READPLAN=1 ❌​F ❌​F ❌​F ❌​F ❌​F ❌​F
❌​ TestAccept/bundle/invariant/no_drift/DMS=/INPUT_CONFIG=cluster_apply_policy_default_values.yml.tmpl/READPLAN= ❌​F ❌​F ❌​F ❌​F ❌​F ❌​F
❌​ TestAccept/bundle/invariant/no_drift/DMS=/INPUT_CONFIG=cluster_apply_policy_default_values.yml.tmpl/READPLAN=1 ❌​F ❌​F ❌​F ❌​F ❌​F ❌​F
❌​ TestAccept/bundle/invariant/no_drift/DMS=/INPUT_CONFIG=cluster_libraries.yml.tmpl/READPLAN= ❌​F ❌​F ❌​F ❌​F ❌​F ❌​F
❌​ TestAccept/bundle/invariant/no_drift/DMS=/INPUT_CONFIG=cluster_libraries.yml.tmpl/READPLAN=1 ❌​F ❌​F ❌​F ❌​F ❌​F ❌​F
❌​ TestAccept/bundle/invariant/no_drift/DMS=/INPUT_CONFIG=job_apply_policy_default_values_for_each_task.yml.tmpl/READPLAN= ❌​F ❌​F ❌​F ❌​F ❌​F ❌​F
❌​ TestAccept/bundle/invariant/no_drift/DMS=/INPUT_CONFIG=job_apply_policy_default_values_for_each_task.yml.tmpl/READPLAN=1 ❌​F ❌​F ❌​F ❌​F ❌​F ❌​F
❌​ TestAccept/bundle/invariant/no_drift/DMS=/INPUT_CONFIG=job_apply_policy_default_values_job_cluster.yml.tmpl/READPLAN= ❌​F ❌​F ❌​F ❌​F ❌​F ❌​F
❌​ TestAccept/bundle/invariant/no_drift/DMS=/INPUT_CONFIG=job_apply_policy_default_values_job_cluster.yml.tmpl/READPLAN=1 ❌​F ❌​F ❌​F ❌​F ❌​F ❌​F
❌​ TestAccept/bundle/invariant/no_drift/DMS=/INPUT_CONFIG=job_apply_policy_default_values_task_cluster.yml.tmpl/READPLAN= ❌​F ❌​F ❌​F ❌​F ❌​F ❌​F
❌​ TestAccept/bundle/invariant/no_drift/DMS=/INPUT_CONFIG=job_apply_policy_default_values_task_cluster.yml.tmpl/READPLAN=1 ❌​F ❌​F ❌​F ❌​F ❌​F ❌​F
❌​ TestAccept/bundle/invariant/no_drift/DMS=/INPUT_CONFIG=job_with_task.yml.tmpl/READPLAN= ❌​F ❌​F ❌​F ❌​F ❌​F ❌​F
❌​ TestAccept/bundle/invariant/no_drift/DMS=/INPUT_CONFIG=job_with_task.yml.tmpl/READPLAN=1 ❌​F ❌​F ❌​F ❌​F ❌​F ❌​F
❌​ TestAccept/bundle/invariant/no_drift/DMS=true/INPUT_CONFIG=cluster.yml.tmpl/READPLAN= ❌​F ❌​F ❌​F ❌​F ❌​F ❌​F
❌​ TestAccept/bundle/invariant/no_drift/DMS=true/INPUT_CONFIG=cluster.yml.tmpl/READPLAN=1 ❌​F ❌​F ❌​F ❌​F ❌​F ❌​F
❌​ TestAccept/bundle/invariant/no_drift/DMS=true/INPUT_CONFIG=cluster_apply_policy_default_values.yml.tmpl/READPLAN= ❌​F ❌​F ✅​p ❌​F ❌​F ❌​F
❌​ TestAccept/bundle/invariant/no_drift/DMS=true/INPUT_CONFIG=cluster_apply_policy_default_values.yml.tmpl/READPLAN=1 ❌​F ❌​F ❌​F ❌​F ❌​F ❌​F
❌​ TestAccept/bundle/invariant/no_drift/DMS=true/INPUT_CONFIG=cluster_libraries.yml.tmpl/READPLAN= ❌​F ❌​F ✅​p ❌​F ❌​F ❌​F
❌​ TestAccept/bundle/invariant/no_drift/DMS=true/INPUT_CONFIG=cluster_libraries.yml.tmpl/READPLAN=1 ❌​F ❌​F ✅​p ❌​F ❌​F ❌​F
❌​ TestAccept/bundle/invariant/no_drift/DMS=true/INPUT_CONFIG=job_apply_policy_default_values_for_each_task.yml.tmpl/READPLAN= ❌​F ❌​F ✅​p ❌​F ❌​F ❌​F
❌​ TestAccept/bundle/invariant/no_drift/DMS=true/INPUT_CONFIG=job_apply_policy_default_values_for_each_task.yml.tmpl/READPLAN=1 ❌​F ❌​F ✅​p ❌​F ❌​F ❌​F
❌​ TestAccept/bundle/invariant/no_drift/DMS=true/INPUT_CONFIG=job_apply_policy_default_values_job_cluster.yml.tmpl/READPLAN= ❌​F ❌​F ✅​p ❌​F ❌​F ❌​F
❌​ TestAccept/bundle/invariant/no_drift/DMS=true/INPUT_CONFIG=job_apply_policy_default_values_job_cluster.yml.tmpl/READPLAN=1 ❌​F ❌​F ✅​p ❌​F ❌​F ❌​F
❌​ TestAccept/bundle/invariant/no_drift/DMS=true/INPUT_CONFIG=job_apply_policy_default_values_task_cluster.yml.tmpl/READPLAN= ❌​F ❌​F ✅​p ❌​F ❌​F ❌​F
❌​ TestAccept/bundle/invariant/no_drift/DMS=true/INPUT_CONFIG=job_apply_policy_default_values_task_cluster.yml.tmpl/READPLAN=1 ❌​F ❌​F ✅​p ❌​F ❌​F ❌​F
❌​ TestAccept/bundle/invariant/no_drift/DMS=true/INPUT_CONFIG=job_with_task.yml.tmpl/READPLAN= ✅​p ❌​F ✅​p ❌​F ❌​F ❌​F
❌​ TestAccept/bundle/invariant/no_drift/DMS=true/INPUT_CONFIG=job_with_task.yml.tmpl/READPLAN=1 ✅​p ❌​F ✅​p ❌​F ❌​F ❌​F
🟨​ TestAccept/bundle/resources/apps/lifecycle-started 🟨​K 💚​R 💚​R 💚​R 💚​R 💚​R
💚​ TestAccept/bundle/resources/apps/lifecycle-started-omitted 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R
💚​ TestAccept/bundle/resources/apps/lifecycle-started-omitted/DMS= 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R
💚​ TestAccept/bundle/resources/apps/lifecycle-started-omitted/DMS=true 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R
💚​ TestAccept/bundle/resources/apps/lifecycle-started-toggle 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R
💚​ TestAccept/bundle/resources/apps/lifecycle-started-toggle/DMS= 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R
💚​ TestAccept/bundle/resources/apps/lifecycle-started-toggle/DMS=true 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R
💚​ TestAccept/bundle/resources/apps/lifecycle-started/DMS= 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R
🟨​ TestAccept/bundle/resources/apps/lifecycle-started/DMS=true 🟨​K 💚​R 💚​R 💚​R 💚​R 💚​R
❌​ TestAccept/bundle/resources/clusters/deploy/data_security_mode ✅​p ❌​F ✅​p ❌​F ❌​F ❌​F
❌​ TestAccept/bundle/resources/clusters/deploy/data_security_mode/DMS= ✅​p ❌​F ✅​p ❌​F ❌​F ❌​F
❌​ TestAccept/bundle/resources/clusters/deploy/data_security_mode/DMS=true ✅​p ❌​F ✅​p ❌​F ❌​F ❌​F
❌​ TestAccept/bundle/resources/clusters/deploy/simple ✅​p ❌​F ✅​p ❌​F ❌​F ❌​F
❌​ TestAccept/bundle/resources/clusters/deploy/simple/DMS= ✅​p ❌​F ✅​p ❌​F ❌​F ❌​F
❌​ TestAccept/bundle/resources/clusters/deploy/simple/DMS=true ✅​p ❌​F ✅​p ❌​F ❌​F ❌​F
❌​ TestAccept/bundle/resources/clusters/deploy/update-after-create ✅​p ❌​F ✅​p ❌​F ❌​F ❌​F
❌​ TestAccept/bundle/resources/clusters/deploy/update-after-create/DMS= ✅​p ❌​F ✅​p ❌​F ❌​F ❌​F
❌​ TestAccept/bundle/resources/clusters/deploy/update-after-create/DMS=true ✅​p ❌​F ✅​p ❌​F ❌​F ❌​F
❌​ TestAccept/bundle/resources/clusters/lifecycle-started ✅​p ❌​F ✅​p ❌​F ❌​F ❌​F
❌​ TestAccept/bundle/resources/clusters/lifecycle-started-toggle ✅​p ❌​F ✅​p ❌​F ❌​F ❌​F
❌​ TestAccept/bundle/resources/clusters/lifecycle-started-toggle/DMS= ✅​p ❌​F ✅​p ❌​F ❌​F ❌​F
❌​ TestAccept/bundle/resources/clusters/lifecycle-started-toggle/DMS=true ✅​p ❌​F ✅​p ❌​F ❌​F ❌​F
❌​ TestAccept/bundle/resources/clusters/lifecycle-started/DMS= ✅​p ❌​F ✅​p ❌​F ❌​F ❌​F
❌​ TestAccept/bundle/resources/clusters/lifecycle-started/DMS=true ✅​p ❌​F ✅​p ❌​F ❌​F ❌​F
❌​ TestAccept/bundle/resources/clusters/resize-terminated-fallback ✅​p ❌​F ✅​p ❌​F ❌​F ❌​F
❌​ TestAccept/bundle/resources/clusters/resize-terminated-fallback/DMS= ✅​p ❌​F ✅​p ❌​F ❌​F ❌​F
❌​ TestAccept/bundle/resources/clusters/resize-terminated-fallback/DMS=true ✅​p ❌​F ✅​p ❌​F ❌​F ❌​F
❌​ TestAccept/bundle/resources/materialization_sets/removal_scope ❌​F ✅​p ✅​p ✅​p ✅​p ✅​p
❌​ TestAccept/bundle/resources/materialization_sets/removal_scope/DMS= ❌​F ✅​p ✅​p ✅​p ✅​p ✅​p
❌​ TestAccept/bundle/resources/mcp_services/lifecycle ❌​F ✅​p ✅​p ✅​p ✅​p ✅​p
❌​ TestAccept/bundle/resources/mcp_services/lifecycle/DMS= ❌​F ✅​p ✅​p ✅​p ✅​p ✅​p
❌​ TestAccept/bundle/resources/mcp_services/lifecycle/DMS=true ❌​F ✅​p ✅​p ✅​p ✅​p ✅​p
❌​ TestAccept/bundle/resources/model_provider_services/lifecycle ❌​F ✅​p ✅​p ✅​p ✅​p ✅​p
❌​ TestAccept/bundle/resources/model_provider_services/lifecycle/DMS=true ❌​F ✅​p ✅​p ✅​p ✅​p ✅​p
❌​ TestAccept/bundle/resources/postgres_branches/recreate ✅​p ✅​p ✅​p ✅​p ❌​F ✅​p
❌​ TestAccept/bundle/resources/postgres_branches/recreate/DMS= ✅​p ✅​p ✅​p ✅​p ❌​F ✅​p
🟨​ TestAccept/bundle/resources/postgres_synced_tables/recreate 🟨​K 💚​R 💚​R 💚​R 🟨​K 💚​R
🟨​ TestAccept/bundle/resources/postgres_synced_tables/recreate/DMS= 🟨​K 💚​R 💚​R 💚​R 💚​R 💚​R
🟨​ TestAccept/bundle/resources/postgres_synced_tables/recreate/DMS=true 🟨​K 💚​R 💚​R 💚​R 🟨​K 💚​R
❌​ TestAccept/bundle/resources/schemas/rename_then_change ✅​p ❌​F ✅​p ❌​F ✅​p ✅​p
❌​ TestAccept/bundle/resources/schemas/rename_then_change/DMS= ✅​p ❌​F ✅​p ✅​p ✅​p ✅​p
❌​ TestAccept/bundle/resources/schemas/rename_then_change/DMS=true ✅​p ✅​p ✅​p ❌​F ✅​p ✅​p
❌​ TestAccept/bundle/resources/schemas/rename_with_volume ✅​p ❌​F ✅​p ✅​p ✅​p ✅​p
❌​ TestAccept/bundle/resources/schemas/rename_with_volume/DMS=true/READPLAN=1 ✅​p ❌​F ✅​p ✅​p ✅​p ✅​p
❌​ TestAccept/bundle/resources/volumes/rename_case_only ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F
❌​ TestAccept/bundle/resources/volumes/rename_case_only/DMS= ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F
Top 50 slowest tests (at least 2 minutes):
duration env testname
10:45 azure-windows-latest-4core-16gb TestAccept/bundle/resources/apps/lifecycle-started/DMS=
10:31 aws-windows-latest-4core-16gb TestAccept/bundle/resources/apps/lifecycle-started/DMS=true
9:50 aws-windows-latest-4core-16gb TestAccept/bundle/resources/apps/lifecycle-started/DMS=
9:32 azure linux-2core-8gb TestAccept/bundle/resources/apps/lifecycle-started/DMS=
9:06 azure-windows-latest-4core-16gb TestAccept/bundle/resources/apps/lifecycle-started/DMS=true
9:01 azure linux-2core-8gb TestAccept/bundle/resources/apps/lifecycle-started/DMS=true
8:57 aws linux-2core-8gb TestAccept/bundle/resources/apps/lifecycle-started/DMS=
8:08 aws linux-2core-8gb TestAccept/bundle/config-remote-sync/multiple_resources/DMS=true
7:57 gcp linux-2core-8gb TestAccept/bundle/resources/apps/lifecycle-started/DMS=true
7:48 gcp linux-2core-8gb TestAccept/bundle/resources/clusters/deploy/local_ssd_count/DMS=true
7:42 gcp-windows-latest-4core-16gb TestAccept/bundle/resources/apps/lifecycle-started/DMS=
7:40 aws linux-2core-8gb TestAccept/bundle/config-remote-sync/multiple_resources/DMS=
7:32 gcp linux-2core-8gb TestAccept/bundle/resources/clusters/deploy/local_ssd_count/DMS=
7:29 gcp-windows-latest-4core-16gb TestAccept/bundle/resources/clusters/deploy/local_ssd_count/DMS=true
7:19 gcp linux-2core-8gb TestAccept/bundle/resources/apps/lifecycle-started/DMS=
7:06 gcp-windows-latest-4core-16gb TestAccept/bundle/resources/clusters/deploy/local_ssd_count/DMS=
6:54 gcp-windows-latest-4core-16gb TestAccept/bundle/resources/apps/lifecycle-started/DMS=true
6:48 azure linux-2core-8gb TestAccept/bundle/config-remote-sync/multiple_resources/DMS=
6:45 aws linux-2core-8gb TestAccept/bundle/resources/clusters/lifecycle-started/DMS=true
6:40 gcp linux-2core-8gb TestAccept/bundle/apps/job_permissions/DMS=true
6:32 azure linux-2core-8gb TestAccept/bundle/resources/clusters/lifecycle-started/DMS=true
6:29 azure linux-2core-8gb TestAccept/bundle/config-remote-sync/multiple_resources/DMS=true
6:24 gcp linux-2core-8gb TestAccept/bundle/resources/apps/lifecycle-started-toggle/DMS=true
6:16 aws linux-2core-8gb TestAccept/bundle/resources/clusters/lifecycle-started/DMS=
6:10 gcp linux-2core-8gb TestAccept/bundle/config-remote-sync/multiple_resources/DMS=
6:05 aws linux-2core-8gb TestAccept/bundle/resources/clusters/lifecycle-started-toggle/DMS=true
6:03 gcp linux-2core-8gb TestAccept/bundle/config-remote-sync/multiple_resources/DMS=true
5:56 aws-windows-latest-4core-16gb TestAccept/bundle/resources/schemas/rename_then_change/DMS=true
5:35 aws-windows-latest-4core-16gb TestAccept/bundle/resources/apps/lifecycle-started-omitted/DMS=
5:28 azure-windows-latest-4core-16gb TestAccept/pipelines/run/refresh-flags
5:25 azure linux-2core-8gb TestAccept/bundle/resources/clusters/lifecycle-started/DMS=
5:14 gcp-windows-latest-4core-16gb TestAccept/bundle/templates/default-python/combinations/classic/DLT=no/DMS=/NBOOK=no/PY=yes/READPLAN=1
5:04 azure linux-2core-8gb TestAccept/pipelines/run/refresh-flags
4:41 aws linux-2core-8gb TestAccept/bundle/resources/apps/lifecycle-started-toggle/DMS=true
4:41 aws linux-2core-8gb TestAccept/bundle/resources/materialization_sets/immutable/DMS=
4:40 gcp-windows-latest-4core-16gb TestAccept/pipelines/run/refresh-flags
4:38 aws-windows-latest-4core-16gb TestAccept/pipelines/run/refresh-flags
4:37 aws-windows-latest-4core-16gb TestAccept/bundle/resources/apps/lifecycle-started-toggle/DMS=true
4:35 azure-windows-latest-4core-16gb TestAccept/bundle/resources/schemas/rename_then_change/DMS=
4:32 aws linux-2core-8gb TestAccept/bundle/resources/materialization_sets/immutable/DMS=true
4:32 gcp-windows-latest-4core-16gb TestAccept/bundle/invariant/delete_idempotent/DMS=/INPUT_CONFIG=schema_uppercase_name.yml.tmpl/READPLAN=1
4:31 aws linux-2core-8gb TestAccept/bundle/resources/clusters/lifecycle-started-toggle/DMS=
4:31 gcp-windows-latest-4core-16gb TestAccept/bundle/invariant/delete_idempotent/DMS=/INPUT_CONFIG=vector_search_endpoint.yml.tmpl/READPLAN=1
4:28 aws linux-2core-8gb TestAccept/bundle/resources/materialization_sets/batch/DMS=true
4:28 azure-windows-latest-4core-16gb TestAccept/bundle/deploy/immutable/breakglass/DMS=
4:27 aws-windows-latest-4core-16gb TestAccept/bundle/resources/apps/lifecycle-started-toggle/DMS=
4:27 azure linux-2core-8gb TestAccept/bundle/resources/clusters/lifecycle-started-toggle/DMS=true
4:26 gcp linux-2core-8gb TestAccept/bundle/invariant/destroy_idempotent/DMS=/INPUT_CONFIG=job_pydabs_1000_tasks.yml.tmpl/READPLAN=
4:24 gcp-windows-latest-4core-16gb TestAccept/bundle/templates/default-python/combinations/serverless/DLT=yes/DMS=/NBOOK=no/PY=yes/READPLAN=
4:24 azure-windows-latest-4core-16gb TestAccept/bundle/resources/apps/lifecycle-started-toggle/DMS=true

tharunbhuk-db added a commit that referenced this pull request Nov 17, 2026
…and JUnit PRs

They move here from #7298, #7426 and #7434, so that all pipeline testing
entries are in one PR.

Co-authored-by: Isaac <no-reply@databricks.com>
tharunbhuk-db added a commit that referenced this pull request Nov 17, 2026
…and JUnit PRs

They move here from #7298, #7426 and #7434, so that all pipeline testing
entries are in one PR.

Co-authored-by: Isaac <no-reply@databricks.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants