Repository navigation
Deploy the testing property of pipelines and show remote changes in bundle plan - #7438
tharunbhuk-db wants to merge 2 commits into
Conversation
|
Commit: Regression test reportSelected 4 changed tests (limit=20, 0 not selected) 8 tests: regression=8 unreleased=0 coverage=0 failing=0
TestAccept/bundle/environments/dependencies/DMS=: branch ✅ | main (2f5d1a6) ❌ | latest (v1.26.0) ❌main (2f5d1a6): latest (v1.26.0): same output as main (2f5d1a6) TestAccept/bundle/environments/dependencies/DMS=true: branch ✅ | main (2f5d1a6) ❌ | latest (v1.26.0) ❌main (2f5d1a6): latest (v1.26.0): same output as main (2f5d1a6) TestAccept/bundle/resources/pipelines/testing-dynamic-version/DMS=: branch ✅ | main (2f5d1a6) ❌ | latest (v1.26.0) ❌main (2f5d1a6): latest (v1.26.0): TestAccept/bundle/resources/pipelines/testing-dynamic-version/DMS=true: branch ✅ | main (2f5d1a6) ❌ | latest (v1.26.0) ❌main (2f5d1a6): latest (v1.26.0): TestAccept/bundle/resources/pipelines/testing-same-name-libraries/DMS=: branch ✅ | main (2f5d1a6) ❌ | latest (v1.26.0) ❌main (2f5d1a6): latest (v1.26.0): TestAccept/bundle/resources/pipelines/testing-same-name-libraries/DMS=true: branch ✅ | main (2f5d1a6) ❌ | latest (v1.26.0) ❌main (2f5d1a6): latest (v1.26.0): TestAccept/bundle/resources/pipelines/testing/DMS=: branch ✅ | main (2f5d1a6) ❌ | latest (v1.26.0) ❌main (2f5d1a6): latest (v1.26.0): TestAccept/bundle/resources/pipelines/testing/DMS=true: branch ✅ | main (2f5d1a6) ❌ | latest (v1.26.0) ❌main (2f5d1a6): latest (v1.26.0): |
✅ Integration test reportCommit: 4a61012
Top 22 slowest tests (at least 2 minutes):
|
48e4a95 to
fd9fedb
Compare
1fd5e11 to
3af4a58
Compare
fd9fedb to
b10a21e
Compare
3af4a58 to
7498e1b
Compare
b10a21e to
e5ff44c
Compare
7498e1b to
e79333b
Compare
| trace print_requests.py //api/2.0/pipelines --nostamp | jq '{method, path, testing: .body.testing}' | ||
|
|
||
| title "No changes after the deploy\n" | ||
| trace $CLI bundle plan |
There was a problem hiding this comment.
Just to double check: on dogfood, did bundle plan show 0 changes right after deploying a pipeline with testing? The description covers existing pipelines and removing it, but the fake server echoes the body back, so it can't tell us whether the backend normalizes glob.include or the -e /Workspace/...[test] dependency on read.
There was a problem hiding this comment.
Yes. Pranshu checked it on a real workspace (see his review on #7300): after a deploy with testing, the next plan showed 0 changes. The fake server cannot show this, as you say.
| } | ||
|
|
||
| func (r *ResourcePipeline) DoRead(ctx context.Context, id string) (*PipelineRemote, error) { | ||
| if env.ExperimentalPipelineTesting(ctx) { |
There was a problem hiding this comment.
Why gate the read on the env var when create and update are gated on config (config.Testing != nil)? The raw GET is the same request, so always reading spec.testing would be free, and as is two people running bundle plan on a pipeline that got testing added in the UI see different plans depending on their env.
There was a problem hiding this comment.
This is out of date. This PR has no raw read any more: the Go SDK reads spec.testing in DoRead for everyone, with or without the variable, as main already does.
|
|
||
| // doRaw sends a request that the Go SDK cannot express yet because it lacks the testing field. | ||
| func (r *ResourcePipeline) doRaw(ctx context.Context, method, path string, request, response any) error { | ||
| apiClient, err := client.New(r.client.Config) |
There was a problem hiding this comment.
nit: this builds a new API client on every call, and DoRead runs once per pipeline per plan. Can we create it once in New like the cmd/air callers do?
There was a problem hiding this comment.
This is out of date. The raw client is gone; Create, Edit and Get use the SDK client.
| update_file.py databricks.yml "$(sed -n '/^ testing:/,$p' databricks.yml)" "" | ||
| trace $CLI bundle plan | ||
| trace $CLI bundle deploy | ||
| trace print_requests.py //api/2.0/pipelines --nostamp | jq '{method, path, testing: .body.testing}' |
There was a problem hiding this comment.
nit: null here can mean either an absent key or "testing": null in the body. .body | has("testing") would make the assertion explicit.
There was a problem hiding this comment.
Done in 7401411. The step now prints has_testing: (.body | has("testing")), which is false.
e79333b to
35031d3
Compare
858d4df to
52dc912
Compare
35031d3 to
2126212
Compare
52dc912 to
650fc96
Compare
2126212 to
d98a141
Compare
650fc96 to
2189384
Compare
d98a141 to
7401411
Compare
2189384 to
1c74f67
Compare
7401411 to
36406cf
Compare
…emote changes in bundle plan bundle deploy sends a pipeline's testing on Create and Edit, and bundle plan reads it back, so a change made outside the bundle shows in the plan and the next deploy undoes it. The Go SDK has the field, so the direct engine and the test server already handle it. This change adds the acceptance test for it. Co-authored-by: Isaac <no-reply@databricks.com>
The last step of the acceptance test now asserts that the request body has no testing key, instead of printing testing as null, which can also mean an explicit null. Co-authored-by: Isaac <no-reply@databricks.com>
36406cf to
4a61012
Compare
🥞 Stacked PR
bundle deploysendstesting, andbundle planshows a change made to it outside the bundle; the next deploy undoes it.Tests: acceptance
bundle/resources/pipelines/testing(deploy, plan after a remote edit, update, removal).This pull request and its description were written by Isaac.