[farazshaikh]

Summary

commit_handler_from_url resolves the commit handler from the URL scheme string. Recognized schemes (s3, gs, az, memory, file, ...) get ConditionalPutCommitHandler, but every other scheme fell through to UnsafeCommitHandler, which blind-writes the manifest with no create-if-not-exists guard. Concurrent writers to such a scheme can all target the same next version, overwrite each other, and report success while silently losing updates.

This changes the default for unrecognized schemes to ConditionalPutCommitHandler (put-if-not-exists).

Why

• Most object stores (including out-of-tree providers registered at runtime through the ObjectStore registry added in feat(python): expose ObjectStoreProvider registration in pylance #8522) back onto storage that supports atomic create-if-not-exists.
• With this default, concurrent commits to an unrecognized scheme fail loudly with a conflict instead of silently clobbering each other.
• A store that genuinely lacks conditional-put support can still opt back into UnsafeCommitHandler via a caller-supplied commit handler.

Test

Extended test_commit_handler_from_url_conditional_put_schemes with a custom_scheme case asserting an unrecognized scheme now routes to ConditionalPutCommitHandler.

Context

Follow-up to #8522 per the review discussion. This is the first of two follow-ups; a subsequent change will let a provider declare its commit handler (e.g. put-if-not-exists vs unsafe) so selection is driven by the provider rather than a hardcoded scheme list.

[lance-gatekeeper]

⚠️ Gate recommendation: approve with a non-blocking risk.

Unknown schemes now use conditional creation instead of unconditional manifest overwrites. Providers that honor PutMode::Create detect conflicts, while unsupported operations return errors; explicit caller handlers retain precedence. This implements the fallback change proposed in #8522.

Older Lance writers still overwrite manifests unconditionally for unknown schemes, so mixed-version concurrent writes remain unsafe. Quiesce writers during the upgrade or coordinate every writer through a shared lock. No further code change is requested here.

[Xuanwo]

The updated head includes current main, and the full applicable CI suite passes. The change preserves explicit commit handlers, all existing known-scheme routing, and the COS safeguard while defaulting unknown schemes to conditional creation. Providers must honor PutMode::Create; mixed-version concurrency remains unsafe until older writers are quiesced or coordinated through a shared lock.