Repository navigation
Conversation
|
@heydryft Any chance you could grant some credits so we can test it fully? Runcrate account: andrey.cheptsov@gmail.com |
|
Done: andrey.cheptsov@gmail.com now has $50 of credit on Runcrate, so you can test it end to end. Thanks for taking the time to try it properly. If anything in the offer data looks off, tell me here and I'll fix it. |
|
@peterschmidt85 a quick pointer in case it helps the test: H200 is available right now in 1, 2, 4 and 8 GPU configs. A 1x H200 is the cheapest way to check that the offers gpuhunt lists actually launch. The $50 credit covers about 10 hours of that, and stopping the instance ends the billing. Anything you'd like changed in the provider, just say. |
|
Thank you, @heydryft! We,ll be looking at it this week. Sorry for the delay. |
Runcrate is an online provider: offers are queried live from GET https://www.runcrate.ai/api/v1/instances/types with RUNCRATE_API_KEY. Each offer is a fixed on-demand VM shape in one region; instance_name is the offer id the create instance API takes as instance_type_id.
0d004e5 to
766928b
Compare
|
No worries, thanks! I rebased on main so it picks up the VRAM normalization, and mapped RTX 4090 and 5090, which were being skipped. Tests pass locally; CI just needs the run approved. |
|
@heydryft Which deployment type would you recommend integrating into dstack—VMs or containers—for the best GPU availability now and long term? Our live API comparison today found these configurations across the 1/2/4/8-GPU queries, deduplicated by offer ID:
Containers had 3.5× as many listed configurations for this GPU set. These are configuration counts; free-machine counts are not exposed by the instances API. |
|
@peterschmidt85 thanks for measuring it properly. Your numbers match ours, and VM is genuinely narrower today, not hidden: the listing shows everything a supplier reports as launchable right now. You're also right that these are configurations, not free-machine counts. We don't expose capacity depth. My recommendation: start with containers. That's where H200 SXM (1/2/4/8), B200 (1/2/4/8) and B300 are, and over the last week container launches reached running in about a minute (median 63 s). Add VMs after that for what containers can't cover: H200 NVL and most H100 are VM-only, and VMs suit jobs that need a full OS (Docker inside, driver or kernel settings). A few things that will matter for dstack:
Right now the provider lists both deployment types as if they were all VMs. I'll update it to read |
|
@heydryft We'd target containers. GPU access, root SSH and startup scripts worked in our tests. For dstack, container creation also needs to support:
The documented We want the runner directly inside the chosen image, without nested Docker. |
|
@peterschmidt85 sorry for the slow answer, it should have come days ago. Not today, for all three. The /instances API has no field for a custom image, registry credentials, an entrypoint override or a run-as-user choice. We start the instance and you SSH into it rather than us running your container image, so there is no entrypoint to override and the user is the instance's own. What works now: a launch script or startup commands that run docker login and docker run on a VM with Docker (the Devbox template ships it), so dstack could start its runner that way. A proper image plus credentials field would be real work on our side and I can't promise it or a date. If that route fits dstack's backend I'm glad to help test it. |
|
Thanks for clarifying, @heydryft, and for helping us test. We'll hold off on the dstack backend integration for now: VM availability in our tests was limited for the GPUs we're targeting, while the container path needs native custom-image and private-registry support. We'd be glad to revisit if either changes. Please keep us posted! |
|
Thanks for testing it properly, really appreciate it. I'll tell you here if either of those changes. |
Summary
Adds Runcrate as an online catalog provider.
GET https://www.runcrate.ai/api/v1/instances/types(docs), needs an API key, so this is an online provider likeseewebandhotaisle:RuncrateProvider.from_env()readsRUNCRATE_API_KEYthroughget_creds_env, anddefault_catalog()skips the provider when the variable is not set.{"data": [...], "meta": {"total": N}}). OneCatalogItemper offer:instance_nameis the offerid, which the create instance API takes asinstance_type_id, so a future dstack backend can launch straight from the offer. Ids change as capacity changes, so they are meant to be used shortly after the query, which is how online providers are used anyway.locationisregionas the API returns it (the same string the create API accepts),priceishourly_ratefor the whole VM,cpu,memoryanddisk_sizearecpu_cores,memory_gbandstorage_gb.spotis alwaysFalse.provider_datacarries the Runcrategpu_type, which the create API also accepts together withgpu_countandregion.gpu_typeis mapped toKNOWN_NVIDIA_GPUSnames and memory through an explicitGPU_MAP, asdaytonadoes:A100_80GtoA10080 GB,A100toA10040 GB,V100_32GtoV10032 GB,V100toV10016 GB,RTXPro6000toRTXPRO6000, and the other 17 types by the same name (B300 uses the registered 270 GB). Every GPU type the live API returned on 2026-11-03 is covered, includingRTX4090andRTX5090, added after the rebase. An unknown type is skipped with a warning, and the integrity tests warn about unknown types in the live response (liketest_real_world_lium_gpu_models).GH200offers getcpu_arch=armthroughis_nvidia_superchip, asvultrdoes.gpu_type: "CPU",gpu_count: 0) are published with no GPU fields.raise_for_status(), an unexpected response shape raisesValueError, and a malformed or incomplete offer is skipped with a warning.ONLINE_PROVIDERS,ONLINE_PROVIDER_MODULES, the CLI (python -m gpuhunt runcrate), the README and the catalogs workflow (atest-runcrateintegrity job using aRUNCRATE_API_KEYsecret, liketest-seeweb).The structure follows the two most recent provider PRs: #259 (Lium: online provider, unit tests against a recorded response, integrity tests that warn about unmapped GPU names, workflow job) and #263 (Daytona: explicit GPU map, API key read from the environment,
test_default.pyupdate), plus #246 (Seeweb) for the authenticatedfrom_env.Testing
All rerun on this branch on 2026-11-03 after rebasing on main (#279, #282), Python 3.14,
uv sync --all-extras:uv run pytest src/tests: 417 passed (49 new insrc/tests/providers/test_runcrate.py, run against a response recorded from the live API on 2026-10-25 and trimmed to nine offers, with offer ids replaced by placeholders; no network)uv run pytest --doctest-modules src/gpuhunt: 4 passeduv run pytest src/integrity_tests/test_runcrate.pywithRUNCRATE_API_KEYset: 11 passed against the live API, no unknown-GPU warningsuv run pyright: 0 errorspre-commit run --all-files(ruff, ruff-format): passedpython -m gpuhunt runcrate --output runcrate.csv: 155 offersgpuhunt.query(provider=["runcrate"])withRUNCRATE_API_KEYset: 155 offers (111 GPU, 44 CPU) across 41 regions and 19 GPU types;gpuhunt.query(provider=["runcrate"], gpu_name=["A100"], min_gpu_memory=80, max_gpu_count=1)returns only 80 GB A100 offersThe new integrity job needs a
RUNCRATE_API_KEYrepository secret. Reading offers does not need a balance on the account; I can provide a key for it.Gaps
These matter for a dstack backend more than for the catalog, but to be clear up front:
spot=True.AI assistance
This PR was prepared with AI assistance (Claude) and tested end to end against the live Runcrate API: the unit tests, the live integrity tests, the CLI dump and
gpuhunt.queryresults above all come from real runs. I work on Runcrate and will maintain the provider; happy to adjust anything to your conventions.