Skip to content

Add Runcrate catalog provider - #280

Open
heydryft wants to merge 2 commits into
dstackai:mainfrom
heydryft:pr_add_runcrate_provider
Open

heydryft wants to merge 2 commits into
dstackai:mainfrom
heydryft:pr_add_runcrate_provider

Conversation

@heydryft

@heydryft heydryft commented Oct 26, 2026 •

Copy link
Copy Markdown

Summary

Adds Runcrate as an online catalog provider.

  • Runcrate rents on-demand GPU and CPU VMs. Its offer list, GET https://www.runcrate.ai/api/v1/instances/types (docs), needs an API key, so this is an online provider like seeweb and hotaisle: RuncrateProvider.from_env() reads RUNCRATE_API_KEY through get_creds_env, and default_catalog() skips the provider when the variable is not set.
  • The endpoint returns every offer that can be launched right now in one unpaginated response ({"data": [...], "meta": {"total": N}}). One CatalogItem per offer:
    • instance_name is the offer id, which the create instance API takes as instance_type_id, so a future dstack backend can launch straight from the offer. Ids change as capacity changes, so they are meant to be used shortly after the query, which is how online providers are used anyway.
    • location is region as the API returns it (the same string the create API accepts), price is hourly_rate for the whole VM, cpu, memory and disk_size are cpu_cores, memory_gb and storage_gb. spot is always False.
    • provider_data carries the Runcrate gpu_type, which the create API also accepts together with gpu_count and region.
  • gpu_type is mapped to KNOWN_NVIDIA_GPUS names and memory through an explicit GPU_MAP, as daytona does: A100_80G to A100 80 GB, A100 to A100 40 GB, V100_32G to V100 32 GB, V100 to V100 16 GB, RTXPro6000 to RTXPRO6000, and the other 17 types by the same name (B300 uses the registered 270 GB). Every GPU type the live API returned on 2026-11-03 is covered, including RTX4090 and RTX5090, added after the rebase. An unknown type is skipped with a warning, and the integrity tests warn about unknown types in the live response (like test_real_world_lium_gpu_models).
  • GH200 offers get cpu_arch=arm through is_nvidia_superchip, as vultr does.
  • CPU-only offers (gpu_type: "CPU", gpu_count: 0) are published with no GPU fields.
  • Errors follow the sibling providers: an HTTP error (for example a 401 for a bad key) propagates through raise_for_status(), an unexpected response shape raises ValueError, and a malformed or incomplete offer is skipped with a warning.
  • Registered in ONLINE_PROVIDERS, ONLINE_PROVIDER_MODULES, the CLI (python -m gpuhunt runcrate), the README and the catalogs workflow (a test-runcrate integrity job using a RUNCRATE_API_KEY secret, like test-seeweb).

The structure follows the two most recent provider PRs: #259 (Lium: online provider, unit tests against a recorded response, integrity tests that warn about unmapped GPU names, workflow job) and #263 (Daytona: explicit GPU map, API key read from the environment, test_default.py update), plus #246 (Seeweb) for the authenticated from_env.

Testing

All rerun on this branch on 2026-11-03 after rebasing on main (#279, #282), Python 3.14, uv sync --all-extras:

  • uv run pytest src/tests: 417 passed (49 new in src/tests/providers/test_runcrate.py, run against a response recorded from the live API on 2026-10-25 and trimmed to nine offers, with offer ids replaced by placeholders; no network)
  • uv run pytest --doctest-modules src/gpuhunt: 4 passed
  • uv run pytest src/integrity_tests/test_runcrate.py with RUNCRATE_API_KEY set: 11 passed against the live API, no unknown-GPU warnings
  • uv run pyright: 0 errors
  • pre-commit run --all-files (ruff, ruff-format): passed
  • python -m gpuhunt runcrate --output runcrate.csv: 155 offers
  • gpuhunt.query(provider=["runcrate"]) with RUNCRATE_API_KEY set: 155 offers (111 GPU, 44 CPU) across 41 regions and 19 GPU types; gpuhunt.query(provider=["runcrate"], gpu_name=["A100"], min_gpu_memory=80, max_gpu_count=1) returns only 80 GB A100 offers

The new integrity job needs a RUNCRATE_API_KEY repository secret. Reading offers does not need a balance on the account; I can provide a key for it.

Gaps

These matter for a dstack backend more than for the catalog, but to be clear up front:

  • Terminate only: Runcrate instances cannot be stopped and started again.
  • No spot or interruptible capacity: every offer is on-demand, so the provider never emits spot=True.
  • Single-node: no multi-node clusters with private networking between instances yet.

AI assistance

This PR was prepared with AI assistance (Claude) and tested end to end against the live Runcrate API: the unit tests, the live integrity tests, the CLI dump and gpuhunt.query results above all come from real runs. I work on Runcrate and will maintain the provider; happy to adjust anything to your conventions.

@peterschmidt85

Copy link
Copy Markdown
Contributor

@heydryft Any chance you could grant some credits so we can test it fully? Runcrate account: andrey.cheptsov@gmail.com

@heydryft

Copy link
Copy Markdown
Author

Done: andrey.cheptsov@gmail.com now has $50 of credit on Runcrate, so you can test it end to end. Thanks for taking the time to try it properly. If anything in the offer data looks off, tell me here and I'll fix it.

@heydryft

heydryft commented Nov 3, 2026

Copy link
Copy Markdown
Author

@peterschmidt85 a quick pointer in case it helps the test: H200 is available right now in 1, 2, 4 and 8 GPU configs. A 1x H200 is the cheapest way to check that the offers gpuhunt lists actually launch. The $50 credit covers about 10 hours of that, and stopping the instance ends the billing. Anything you'd like changed in the provider, just say.

@peterschmidt85

Copy link
Copy Markdown
Contributor

Thank you, @heydryft! We,ll be looking at it this week. Sorry for the delay.

Runcrate is an online provider: offers are queried live from
GET https://www.runcrate.ai/api/v1/instances/types with RUNCRATE_API_KEY.
Each offer is a fixed on-demand VM shape in one region; instance_name is
the offer id the create instance API takes as instance_type_id.
@heydryft
heydryft force-pushed the pr_add_runcrate_provider branch from 0d004e5 to 766928b Compare November 3, 2026 16:57
@heydryft

heydryft commented Nov 3, 2026

Copy link
Copy Markdown
Author

No worries, thanks! I rebased on main so it picks up the VRAM normalization, and mapped RTX 4090 and 5090, which were being skipped. Tests pass locally; CI just needs the run approved.

@peterschmidt85

peterschmidt85 commented Nov 4, 2026 •

Copy link
Copy Markdown
Contributor

@heydryft Which deployment type would you recommend integrating into dstack—VMs or containers—for the best GPU availability now and long term?

Our live API comparison today found these configurations across the 1/2/4/8-GPU queries, deduplicated by offer ID:

GPU VM offers Container offers
H200 SXM 2 17
H200 NVL 4 0
B200 2 8
B300 0 3
Total 8 28

Containers had 3.5× as many listed configurations for this GPU set. These are configuration counts; free-machine counts are not exposed by the instances API.

@heydryft

heydryft commented Nov 4, 2026

Copy link
Copy Markdown
Author

@peterschmidt85 thanks for measuring it properly. Your numbers match ours, and VM is genuinely narrower today, not hidden: the listing shows everything a supplier reports as launchable right now. You're also right that these are configurations, not free-machine counts. We don't expose capacity depth.

My recommendation: start with containers. That's where H200 SXM (1/2/4/8), B200 (1/2/4/8) and B300 are, and over the last week container launches reached running in about a minute (median 63 s). Add VMs after that for what containers can't cover: H200 NVL and most H100 are VM-only, and VMs suit jobs that need a full OS (Docker inside, driver or kernel settings).

A few things that will matter for dstack:

  • Containers give you root over SSH inside the container, on a mapped host port (not a fixed 22); the port comes back on the instance. We haven't verified nested or privileged Docker inside containers, so for now treat jobs that need Docker inside as VM jobs. Containers are terminate-only (no stop/start), and /root survives a container restart but not a terminate.
  • On H200 NVL VMs, "running" arrives about 80 s before SSH and the driver are ready, so please gate on SSH rather than status. H200 NVL is NVLink in groups of four, so an 8x is two groups.
  • Container stock moves by the minute, so offer ids should be used soon after the query (as the provider docstring says).

Right now the provider lists both deployment types as if they were all VMs. I'll update it to read deployment_type from the API. Would you rather it return containers only for this first integration, or both with the type exposed? I'll follow whatever fits dstack's backend model.

@peterschmidt85

peterschmidt85 commented Nov 4, 2026 •

Copy link
Copy Markdown
Contributor

@heydryft We'd target containers. GPU access, root SSH and startup scripts worked in our tests.

For dstack, container creation also needs to support:

  • A user-chosen Docker image and private-registry credentials.
  • An entrypoint/command override to start dstack-runner.
  • Running as root even when the image declares another USER.

The documented /instances API doesn't expose these controls, and /templates only lists existing templates. Can you provide them directly or through an API for custom templates?

We want the runner directly inside the chosen image, without nested Docker.

@heydryft

Copy link
Copy Markdown
Author

@peterschmidt85 sorry for the slow answer, it should have come days ago.

Not today, for all three. The /instances API has no field for a custom image, registry credentials, an entrypoint override or a run-as-user choice. We start the instance and you SSH into it rather than us running your container image, so there is no entrypoint to override and the user is the instance's own.

What works now: a launch script or startup commands that run docker login and docker run on a VM with Docker (the Devbox template ships it), so dstack could start its runner that way. A proper image plus credentials field would be real work on our side and I can't promise it or a date.

If that route fits dstack's backend I'm glad to help test it.

@peterschmidt85

Copy link
Copy Markdown
Contributor

Thanks for clarifying, @heydryft, and for helping us test. We'll hold off on the dstack backend integration for now: VM availability in our tests was limited for the GPUs we're targeting, while the container path needs native custom-image and private-registry support.

We'd be glad to revisit if either changes. Please keep us posted!

@heydryft

Copy link
Copy Markdown
Author

Thanks for testing it properly, really appreciate it. I'll tell you here if either of those changes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants