Repository navigation
[NA] [DOCS] feat: replace manage_data notebooks with uv scripts and add S3/GCS remote artifact example - #289
Merged
Merged
Conversation
…cts.ipynb 24 of the first 200 DOTA images carry no annotations, so the annotation_map.get(id, []) path is exercised on every run.
…cloud_storage.py An empty prefix left artifact.assets empty and the script crashed with IndexError before printing its own message. A single-object URI synced correctly but was reported as NOT synced because its link equals the input URI. The synced-vs-fallback decision now rests only on the SDK's synced metadata flag.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changed
guides/manage_data/is now auvproject with three plain Python scripts and a README, replacing the two notebooks:artifacts_intro.py(wasIntroduction_to_Artifacts.ipynb): upload a dataset Artifact, fetch it by alias, download, train, log the model as an Artifact, log a new dataset version. Fixes the scaled-data step, which wrote the unscaled frames to the*-scaled.csvfiles.remote_artifact_links.py(wasA_Guide_to_Remote_Artifacts.ipynb): generic remote assets (https links with per-asset metadata), capped at 200 images, then reads the links back and fetches one file.remote_artifact_cloud_storage.py(new):s3:///gs://synced remote assets anddownload(), with a non-zero exit when the SDK falls back to a plain link because the client library or cloud credentials are missing.README.md: what each script shows, what the SDK can do with generic links, S3 and GCS, prerequisites, docs links..github/workflows/test-examples.yml: the two notebook entries are replaced by the two scripts that run withCOMET_API_KEYonly. The cloud script is not in CI (no AWS/GCP secrets).guides/get-started/Comet_Quickstart.ipynb: the one link to the deleted intro notebook now points atartifacts_intro.py.Why
The docs page Register remote artifacts is about S3/GCS sync and download, but the notebook it linked only logged https links, so the example never showed what the page describes. The intro notebook carried the scaled-data bug and inline screenshots. New examples in this repo use
.py+uvperAGENTS.md.A follow-up PR in
comet-ml/documentationswaps the Colab badges on the two artifact guide pages for links to these scripts and fixesgcs://togs://.How it was verified
uv run python artifacts_intro.pytwice in the same workspace (second run exercisesoverwrite_strategy="PRESERVE"); scaled CSV has zero-mean features and no index column.uv run python remote_artifact_links.py: 200 remote assets logged, 24 of them with empty annotation lists, one image fetched back.remote_artifact_cloud_storage.py:gcs://rejected with exit 2;s3://andgs://without credentials exit 1 with aNOT syncedmessage and no experiment;s3://noaa-ghcn-pds/csv.gz/by_year/176(public bucket, 7 objects) with AWS credentials synced, logged and downloaded all 7 files. The GCS success path was not run (no GCP credentials); it follows the same SDK code path (Artifact._add_gs_assets,LoggedArtifact.download).requirements.txtisuv export --no-hashes;pip install -r requirements.txton Python 3.10 +python remote_artifact_links.pypassed locally.artifacts_intro.pyon 3.10 is covered by the CI matrix job here (the scipy 1.15 macOS wheel does not load on the local machine).