Repository navigation
Use .workspace.URL for the host in built-in templates - #7350
Open
pavloKozlov wants to merge 9 commits into
Open
pavloKozlov wants to merge 9 commits into
pavloKozlov wants to merge 9 commits into
Conversation
pavloKozlov
force-pushed
the
workspace-context-builtin-templates
branch
from
November 4, 2026 15:30
abf32b2 to
d6dcf5e
Compare
Contributor
|
Commit: Regression test reportSelected 3 changed tests (limit=20, 0 not selected) 4 tests: regression=4 unreleased=0 coverage=0 failing=0
TestAccept/bundle/templates-machinery/workspace_variable/runtime/DMS=: branch ✅ | main (663e1f7) ❌ | latest (v1.24.0) ❌main (663e1f7): latest (v1.24.0): TestAccept/bundle/templates-machinery/workspace_variable/runtime/DMS=true: branch ✅ | main (663e1f7) ❌ | latest (v1.24.0) ❌main (663e1f7): latest (v1.24.0): TestAccept/bundle/templates/dbt-sql-workspace-id/DMS=: branch ✅ | main (663e1f7) ❌ | latest (v1.24.0) ❌main (663e1f7): latest (v1.24.0): TestAccept/bundle/templates/default-minimal/profiles/DMS=: branch ✅ | main (663e1f7) ❌ | latest (v1.24.0) ❌main (663e1f7): latest (v1.24.0): |
pavloKozlov
force-pushed
the
workspace-context-template-data
branch
from
November 4, 2026 15:41
b931f85 to
5f2f6db
Compare
pavloKozlov
force-pushed
the
workspace-context-builtin-templates
branch
from
November 4, 2026 15:41
d6dcf5e to
e702534
Compare
Collaborator
✅ Integration test reportCommit: 684c444
|
pavloKozlov
force-pushed
the
workspace-context-builtin-templates
branch
4 times, most recently
from
November 5, 2026 14:04
93ee68b to
494b311
Compare
pavloKozlov
marked this pull request as ready for review
November 5, 2026 14:06
denik
approved these changes
Nov 6, 2026
andrewnester
reviewed
Nov 6, 2026
| default: {{.http_path}} | ||
| {{- /* dbt selects the workspace on a host shared by several workspaces with ?o=<id> in the | ||
| HTTP path. Use the ID from the workspace URL, which has it when the profile sets it. */}} | ||
| {{- $id := (regexp `[?&]w=([^&#]+)`).FindStringSubmatch .workspace.URL}} |
Contributor
There was a problem hiding this comment.
Parsing the ID back out of the URL we just rendered feels roundabout: workspaceContext already knows which ID URL adds (urlWorkspaceID in libs/template/workspace_context.go:51). Can we expose that on .workspace instead of re-deriving it with a regex here? Could go into #7349. I assume .workspace.ID doesn't fit because it also returns the host-metadata ID?
Contributor
Author
There was a problem hiding this comment.
Agree. Switched to .workspace.URLID.
| @@ -0,0 +1,33 @@ | |||
| # The test server is reachable as both 127.0.0.1 and localhost. The second | |||
| # stands in for a host shared by several workspaces. | |||
| CUSTOM_HOST=${DATABRICKS_HOST/127.0.0.1/localhost} | |||
Contributor
There was a problem hiding this comment.
This setup (localhost swap, .well-known/databricks-config, hand-written profiles) is now the fourth copy after acceptance/bundle/strict_profile_match/script.prepare and acceptance/bundle/templates-machinery/workspace_variable/script.prepare, and dbt-sql-workspace-id/script repeats it again. Maybe it should be a helper in acceptance/bin instead?
| # when the profile sets the ID, like ?w= in the bundle host. | ||
| for profile in no_workspace_id custom_host; do | ||
| title "Profile $profile\n" | ||
| $CLI bundle init dbt-sql --config-file ./input.json --profile $profile > /dev/null 2>&1 |
Contributor
There was a problem hiding this comment.
nit: why hide the init output here? If init fails we only see grep failing on a missing file, and the default-minimal test traces it.
Contributor
Author
There was a problem hiding this comment.
Good catch, it traces bundle init now
Built-in templates, except dbt-sql, write the workspace URL to workspace.host. It has ?w=<id> when the profile or an environment variable sets the workspace ID. Other profiles keep the same host as before. The value is quoted so the YAML stays valid for any URL. Template schemas keep workspace_host. Co-authored-by: Isaac <no-reply@databricks.com>
The welcome messages showed workspace_host, which has no workspace ID. They now show the URL that the bundle uses, so they match workspace.host in the generated databricks.yml. Messages that printed "workspace_host:" now print "workspace_url:". Co-authored-by: Isaac <no-reply@databricks.com>
The bundle host now comes from .workspace.URL like in the other built-in templates. dbt jobs get their host from Databricks, so they are not affected. The dbt profile default uses .workspace.Hostname, which gives the same value as stripping the scheme from workspace_host. No built-in template uses workspace_host now. Co-authored-by: Isaac <no-reply@databricks.com>
The dbt profile default no longer has the port of the test server. Databricks hosts don't have an explicit port, so for them the default is the same as before. Co-authored-by: Isaac <no-reply@databricks.com>
The host is no longer quoted. A URL with ?w= is a valid plain YAML value, and the quotes were only needed for test outputs, which no longer have ?w=. Welcome messages keep their "workspace_host:" label. The ?w= targets in strict_profile_match/host and the extra-workspace-input test are removed: the profiles test and the input tests in templates-machinery cover the same cases. Template outputs are now the same as before for auth without a workspace ID. Co-authored-by: Isaac <no-reply@databricks.com>
The bundle host comes from .workspace.URL like in the other built-in templates. Deployed dbt jobs get their host from Databricks, so they don't read it. The HTTP path suggested for dbt init gets ?o=<id> when the workspace URL has the ID, because dbt selects the workspace on a host shared by several workspaces with ?o= in the HTTP path. The host suggested for dbt init is .workspace.Hostname. Co-authored-by: Isaac <no-reply@databricks.com>
The dbt-sql profile default for dbt init is host:port again, so its output is the same as before. Co-authored-by: Isaac <no-reply@databricks.com>
pavloKozlov
force-pushed
the
workspace-context-builtin-templates
branch
from
November 6, 2026 14:14
850ed27 to
684b87b
Compare
pavloKozlov
changed the base branch from
main
to
workspace-context-review-fixes
November 6, 2026 14:14
dbt-sql adds ?o= from .workspace.URLID, escaped, instead of parsing the rendered URL with a regex. The profiles and dbt-sql-workspace-id tests use setup_custom_host instead of their own copy of that setup, and the dbt-sql-workspace-id test traces bundle init so a failed init shows its error. Co-authored-by: Isaac <no-reply@databricks.com>
pavloKozlov
force-pushed
the
workspace-context-builtin-templates
branch
from
November 6, 2026 14:18
684b87b to
684c444
Compare
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #7388.
Changes
Built-in templates write
host: {{.workspace.URL}}instead of{{workspace_host}}. With a profile that setsworkspace_id, the host gets?w=<id>. Other profiles get the same host as before. Welcome messages show.workspace.URL. Indbt-sql, the HTTP path suggested fordbt initgets?o=<id>from.workspace.URLIDin the same case, because dbt reads the workspace fromo.Why
On a host shared by several workspaces, the bundle needs the workspace ID in the host to deploy to the right workspace, and local dbt needs
?o=in the HTTP path to connect. Deployed dbt jobs get their host from Databricks and connect without?o=, so their profile doesn't change.Tests
New acceptance tests create bundles with a profile without
workspace_idand with one that has it. One validates with strict profile matching and deploys, the other checks thedbt-sqlhost and HTTP path. Other template outputs don't change. I also connected with dbt-databricks 1.12 and 1.13 to a workspace through a shared host and through its own host, with and without?o=.This pull request and its description were written by Isaac.