Skip to content

refactor: Reliability improvements to the Chroma migration - #375

Merged
anush008 merged 1 commit into
mainfrom
chroma-updates
Oct 27, 2026
Merged

anush008 merged 1 commit into
mainfrom
chroma-updates

Conversation

@anush008

@anush008 anush008 commented Oct 26, 2026 •

Copy link
Copy Markdown
Member

Gist

The Chroma source now calls the Chroma HTTP API directly, in place of the chroma-go client. This improves data precision and adds Chroma to the standalone binary because it no longer needs CGO..

Fixes

  • Records with a null document, array metadata don't err.
  • Integer metadata stays an integer in Qdrant. Before, it became a float.
  • Chroma URIs are now migrated, to --qdrant.uri-field (default uri).

Improvements

  • The migration fails if the target has fewer points than the source, and keeps the offsets for a resume.
  • Chroma requests retry network errors, 429, and 5xx responses.
  • Resume offsets use the Chroma collection ID, so collections with the same name in different databases stay apart.
  • Sparse vectors are skipped with a warning. Sparse vectors are Chroma Cloud only and we don't have access.

@anush008
anush008 requested a review from generall as a code owner October 26, 2026 21:07
@anush008
anush008 requested review from a team and bashofmann and removed request for generall October 26, 2026 21:07
@anush008
anush008 enabled auto-merge (squash) October 26, 2026 21:21
@anush008 anush008 changed the title refactor: Reliability improvements for the Chroma migration refactor: Reliability improvements to the Chroma migration Oct 26, 2026
@anush008

Copy link
Copy Markdown
Member Author

@generall, the CI failure is unrelated.

Trivy Vulnerability Scanner

2026-10-27T05:31:18Z FATAL Fatal error run error: init error: DB error: failed to download vulnerability DB: OCI artifact error: failed to download vulnerability DB: failed to download artifact from any source: 1 error occurred:
* OCI repository error: 1 error occurred:
* GET https://public.ecr.aws/v2/aquasecurity/trivy-db/manifests/2: TOOMANYREQUESTS: Data limit exceeded

Could you please bypass the CI requirements and merge?

@anush008
anush008 merged commit a175863 into main Oct 27, 2026
19 of 22 checks passed
@anush008
anush008 deleted the chroma-updates branch October 27, 2026 09:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants