diff --git a/agent-setup.mdx b/agent-setup.mdx index 77d6719a0..b5d620397 100644 --- a/agent-setup.mdx +++ b/agent-setup.mdx @@ -15,7 +15,7 @@ Every command below is safe to re-run on a machine where setup already happened. ## Install Runpod Skills and MCP server -Every route installs the same skills: the `runpod` router, the lane skills (`runpod-mcp`, `runpodctl`, `flash`, `runpod-usage`, `runpod-templates`, `companion-clis`, `runpod-migrate`), and the journey skills that `runpod-mcp` routes to (`discovery`, `lifecycle-crud`, `serverless-deploy`, `pod-deploy`, `pod-doctor`, `endpoint-ops`, `cost-audit`). It also wires up the hosted Runpod MCP server (`https://mcp.runpod.io/`) for control-plane tools. The skills install the Runpod CLI and Flash SDK later, on demand, the first time a task needs them. Do not install those now. +Every route installs the same skills: the `runpod` router, the lane skills (`runpod-mcp`, `runpodctl`, `flash`, `runpod-usage`, `runpod-templates`, `runpod-build-template`, `companion-clis`, `runpod-migrate`), and the journey skills that `runpod-mcp` routes to (`discovery`, `lifecycle-crud`, `serverless-deploy`, `pod-deploy`, `pod-doctor`, `endpoint-ops`, `cost-audit`). It also wires up the hosted Runpod MCP server (`https://mcp.runpod.io/`) for control-plane tools. The skills install the Runpod CLI and Flash SDK later, on demand, the first time a task needs them. Do not install those now. Use the correct section for your agent below. @@ -36,6 +36,8 @@ claude plugin list | grep -A2 'runpod@runpod' Expect `Status: ✔ enabled`. If the plugin is listed but disabled, run `claude plugin enable runpod@runpod`. +If your agent isn't Claude Code but loads Claude Code plugins from `~/.claude`, run these same two commands. + If `marketplace add` fails with an SSH `known_hosts` error, the machine usually can't reach `github.com` at all (common on networks that block GitHub). Check with `curl -sI https://github.com` before debugging SSH. > **User step.** Ask the user to run `/reload-plugins` inside Claude Code to activate the plugin, then sign in to the MCP server with `/mcp` → **runpod** → **Sign in with Runpod**. This uses OAuth, so no API key is created or stored. Wait for the user to confirm before reporting the MCP server as connected. @@ -44,9 +46,11 @@ If `marketplace add` fails with an SSH `known_hosts` error, the machine usually ## Install for other agents +**Codex:** skip step 1. The Codex plugin in step 2 installs the skills and the MCP server together and doesn't need Node.js. + ### 1. Install the skills -This step needs Node.js 22.20 or newer. Check with `node --version`; if it's older, stop and report it instead of continuing. +This step needs Node.js 22.20 or newer. Check with `node --version`. If it's older, skip this step and go to [step 2](#2-register-the-mcp-server). The MCP server serves the same playbooks as the skills, so setup still works without them. Tell the user the skills weren't installed because Node.js is too old. Run the command below, replacing `` with your agent's id from the table. Always pass `--agent`: without it, the installer targets every agent it knows about and prints a `Failed to install` block for agents that don't apply to you. @@ -69,8 +73,13 @@ npx -y skills add runpod/runpod-plugins-official --skill '*' --agent | Zed | `zed` | | Kilo Code | `kilo` | | Amp | `amp` | +| Grok Build | `grok` | + +To find the id for another agent, run the command with an id that doesn't exist, such as `--agent list-ids`. The installer installs nothing and prints every valid id under `Valid agents`. (`npx -y skills add --help` doesn't list them.) If your agent isn't there, use `--agent universal`, which installs into the shared `~/.agents/skills` directory that many agents read. If your agent reads skills only from its own directory, clone `https://github.com/runpod/runpod-plugins-official` and copy each folder under `plugins/runpod/skills/` into that directory. + +`--global` installs the skills for your user account. To install them into the current project only, leave out `--global`. The skills then go in the project's `.agents/skills/`, and you verify with `npx -y skills list` without `--global`. -For any other agent, run `npx -y skills add --help` and use the id it lists for your agent. +On Windows `cmd`, single quotes are passed through literally, so write `--skill "*"`. If the installer still reports `No matching skills found`, run `npx -y skills add runpod/runpod-plugins-official --list` and pass the skill names, comma-separated, to `--skill`. Verify: @@ -78,11 +87,13 @@ Verify: npx -y skills list --global ``` -Expect the `runpod` router and the lane and journey skills listed above. If the output also mentions PromptScript or another agent you aren't, ignore it; it doesn't affect your install. +Expect the `runpod` router and the lane and journey skills listed above. Each skill's folder should contain a `SKILL.md` file. If one is empty, run the install command again. If the output also mentions PromptScript or another agent you aren't, ignore it; it doesn't affect your install. ### 2. Register the MCP server -Use the section for your agent. Every agent connects to the same hosted server, `https://mcp.runpod.io/`, and signs in with OAuth on first Runpod tool use. +Use the section for your agent. Every agent connects to the same hosted server, `https://mcp.runpod.io/`, and signs in with OAuth on first Runpod tool use. Older configs and installers may name `https://mcp.getrunpod.io/`: it's the same server under its old name, and it still works. Register only one of them. Each address has its own sign-in, so registering both gives you two separate entries. + +Don't open the server URL in a browser to sign in: it's an MCP endpoint, not a web page, and returns HTTP `405`. Sign-in always starts from your agent. #### Codex @@ -91,12 +102,16 @@ codex plugin marketplace add https://github.com/runpod/runpod-plugins-official.g codex plugin add runpod@runpod ``` -If the `runpod` MCP tools don't appear after install, add the hosted server directly: +The plugin installs the skills and registers the hosted MCP server. Don't also add the server with `codex mcp add`, or you'll end up with a second entry that needs its own sign-in. + +If Codex says `codex plugin add` isn't a command, or reports `plugin not found`, your Codex is too old or has plugins turned off. Update Codex, or check that `features.plugins` isn't set to `false` in `~/.codex/config.toml`, then retry. If you can't install the plugin, install the skills with step 1 (`--agent codex`) and register the server directly: ``` codex mcp add runpod --url https://mcp.runpod.io/ ``` +`codex mcp add` opens the OAuth sign-in immediately and waits until the user approves it in the browser. Before you run it, tell the user to expect the browser prompt. With no user present, use an API key instead (see [No user in the loop](#no-user-in-the-loop)). + In Codex Desktop, the user must fully quit and reopen the app before the plugin and MCP tools appear. #### Gemini CLI @@ -113,9 +128,28 @@ Gemini can alternatively install natively through the plugin's bundled `gemini-e pi mcp add runpod --url https://mcp.runpod.io/ ``` +#### Hermes + +``` +hermes config set mcp_servers.runpod.url https://mcp.runpod.io/ +hermes config set mcp_servers.runpod.auth oauth +``` + +> **User step.** Ask the user to run `hermes mcp login runpod` and approve the sign-in in the browser. Then check with `hermes mcp test runpod`. + +If Hermes gets a `403` error before the browser opens, connect with an API key instead (see [If OAuth sign-in fails](#if-oauth-sign-in-fails)). + +#### Grok Build + +``` +grok mcp add --transport http runpod https://mcp.runpod.io/ +``` + +> **User step.** In Grok, the user opens `/mcps`, selects **runpod**, and presses `i` to sign in. + #### Cursor, GitHub Copilot, Windsurf, Cline, opencode, and all other agents -For Cursor, Windsurf, VS Code (GitHub Copilot), opencode, and Claude Desktop, run the installer with your agent's id. It writes the config without prompts: +For Cursor, Windsurf, VS Code (GitHub Copilot), opencode, Pi, and Claude Desktop, run the installer with your agent's id. It writes the config without prompts: ``` npx @runpod/mcp-server@latest add --client @@ -127,14 +161,57 @@ npx @runpod/mcp-server@latest add --client | Windsurf | `windsurf` | | VS Code (GitHub Copilot) | `vscode` | | opencode | `opencode` | +| Pi | `pi` | | Claude Desktop | `claude-desktop` | -The installer needs Node 20 or later. It exits `0` when every named client was configured, `1` when one failed, and `2` on bad usage. Run `npx @runpod/mcp-server@latest add --list` to see which clients it detects and where each config lives. Always pass `--client`: without it and without a terminal, the installer exits `2` and configures nothing. After it writes the config, it prints the sign-in step for each client (for example, restart Cursor, or run `opencode mcp auth runpod`); do that step or relay it to the user. +The installer needs Node.js 20.12 or later. It exits `0` when every named client was configured, `1` when one failed, and `2` on bad usage. Run `npx @runpod/mcp-server@latest add --list` to see which clients it detects and where each config lives. Always pass `--client`: without it and without a terminal, the installer exits `2` and configures nothing. After it writes the config, it prints the sign-in step for each client (for example, restart Cursor, or run `opencode mcp auth runpod`); do that step or relay it to the user. + +To set up opencode by hand instead, add this to `opencode.json`. Note that opencode uses the `mcp` key, not `mcpServers`, and needs `"type": "remote"`: + +```json +{ + "mcp": { + "runpod": { "type": "remote", "url": "https://mcp.runpod.io/" } + } +} +``` + +Then ask the user to run `opencode mcp auth runpod` to sign in. If no browser window opens, the sign-in link is in the opencode log. + +For any other agent, such as Cline, add a server entry named `runpod` with the HTTP (streamable HTTP) transport and URL `https://mcp.runpod.io/` to your agent's MCP configuration. Most clients use this shape: -For any other agent, such as Cline, add a server entry named `runpod` with the HTTP (streamable HTTP) transport and URL `https://mcp.runpod.io/` to your agent's MCP configuration. Merge it into the existing file; do not overwrite other servers. See your agent's MCP documentation in [Resources](#resources) for the file location and format. +```json +{ + "mcpServers": { + "runpod": { "url": "https://mcp.runpod.io/" } + } +} +``` + +Merge it into the existing file; do not overwrite other servers. See your agent's MCP documentation in [Resources](#resources) for the file location and key names. After registering, the user approves the OAuth sign-in in the browser the first time you call a Runpod tool. +#### If OAuth sign-in fails + +OAuth sign-in works in Claude, ChatGPT, Codex, and Cursor, and in any client that receives the sign-in on a local address (`http://localhost` or `http://127.0.0.1`). Other clients may fail with `redirect_uri is not allowed`, or get a `403` error before the browser opens. If that happens, connect with an API key instead, even when a user is present: + +1. Ask the user to create a key on the [Credentials page](https://console.runpod.io/user/settings) and set it as `RUNPOD_API_KEY` in the environment your agent starts from. +2. Send it as a header on the server entry, for example: + + ```json + { + "mcpServers": { + "runpod": { + "url": "https://mcp.runpod.io/", + "headers": { "Authorization": "Bearer ${RUNPOD_API_KEY}" } + } + } + } + ``` + + `${RUNPOD_API_KEY}` reads the key from the environment, but clients write this differently, so check your client's docs. If your client can't read environment variables, run the server locally instead, with `RUNPOD_API_KEY` set: `npx -y @runpod/mcp-server@latest`. + If your agent can't load skills, the MCP server serves the same playbooks: `list-guides` and `read-guide` return them, and `lookup-concept` and `search-concepts` answer questions about how Runpod works. --- @@ -143,7 +220,7 @@ If your agent can't load skills, the MCP server serves the same playbooks: `list When no user is present (CI, a scheduled job, an agent SDK run), nobody can do the **User steps** or the OAuth sign-in. Use a Runpod API key instead: -1. Read the key from the `RUNPOD_API_KEY` environment variable. The `runpodctl` CLI and the Flash SDK read the same variable. Do not ask for the key, print it, or write it to a file. If it is not set, stop and report that it is missing. +1. Read the key from the `RUNPOD_API_KEY` environment variable. The `runpodctl` CLI and the Flash SDK read the same variable. Do not ask for the key, print it, or write it to a file. If it is not set, stop and report that it is missing. The variable must be set in the environment the agent starts from, for example as a CI secret, in a service unit, or in a shell profile. Clients that store only the variable's name, such as `pi mcp add --bearer-token-env-var RUNPOD_API_KEY`, read the key from there each time they start. 2. Install the skills for your agent as above. Skip `/reload-plugins`: the plugin loads the next time the agent starts. 3. Connect the hosted MCP server with the key instead of OAuth: @@ -151,7 +228,7 @@ When no user is present (CI, a scheduled job, an agent SDK run), nobody can do t claude mcp add --transport http runpod -s user https://mcp.runpod.io/ --header "Authorization: Bearer $RUNPOD_API_KEY" ``` - For other clients, add the same `Authorization` header to the server entry (for Gemini CLI: `gemini mcp add -s user -t http -H "Authorization: Bearer $RUNPOD_API_KEY" runpod https://mcp.runpod.io/`), or run the server locally over stdio with `RUNPOD_API_KEY` set: `npx -y @runpod/mcp-server@latest`. + For other clients, add the same `Authorization` header to the server entry, as in the JSON under [If OAuth sign-in fails](#if-oauth-sign-in-fails). For Gemini CLI, run `gemini mcp add -s user -t http -H "Authorization: Bearer $RUNPOD_API_KEY" runpod https://mcp.runpod.io/`. You can also run the server locally with `RUNPOD_API_KEY` set: `npx -y @runpod/mcp-server@latest`. 4. In headless Claude Code (`claude -p`), allow the Runpod skill and the MCP tools your task needs up front, for example `--allowedTools "Skill(runpod:runpod-mcp)"`. Without `Skill(runpod:runpod-mcp)`, the `runpod-mcp` skill is refused. 5. Write the result banner below to your log. Mark anything that still needs a person with `✗`. diff --git a/api-reference-v2/openapi.json b/api-reference-v2/openapi.json index 79c5ce7fb..9226ff204 100644 --- a/api-reference-v2/openapi.json +++ b/api-reference-v2/openapi.json @@ -2680,7 +2680,7 @@ ] }, "direct": { - "description": "Connection straight to the pod's sshd over its published `22/tcp` mapping. Supports the full SSH feature set. Null unless `22/tcp` is in `ports` and the running pod has been assigned a public port for it — so it is absent while the pod is provisioning or stopped. Also null when listing pods with `GET /v2/pods`; fetch a single pod by ID to obtain the live mapping.", + "description": "Connection straight to the pod's sshd over its published `22/tcp` mapping. Supports the full SSH feature set. Null unless `22/tcp` is in `ports` and the running pod has been assigned a public port for it — so it is usually absent while the pod is provisioning or stopped. It can still appear on a stopped pod, so check that `status` is `RUNNING` before connecting. Also null when listing pods with `GET /v2/pods`; fetch a single pod by ID to obtain the live mapping.", "anyOf": [ { "$ref": "#/components/schemas/PodSshEndpoint" @@ -2974,7 +2974,7 @@ "cost": { "type": "number", "format": "float", - "description": "Current cost in USD per hour (0.0 when EXITED or TERMINATED)", + "description": "The Pod's hourly compute rate in USD. It's returned in every status, including `EXITED`, so it isn't what you're paying right now: a stopped Pod isn't billed for compute, only for its volume disk.", "examples": [ 0.35 ] @@ -3031,7 +3031,7 @@ "required": [ "name" ], - "description": "Request body for creating a pod. Exactly one of `gpu` or `cpu`\nmust be set — enforced at the handler layer. For CPU pods, memory\nis derived by the API from the selected flavor's RAM multiplier;\nclients provide only CPU flavor and vCPU count. CPU pods support\ncontainer disk and network volumes only; `mounts.persistent` is\ninvalid when `cpu` is set.\n\n`image` is required unless `templateId` is set.\n", + "description": "Request body for creating a pod. Exactly one of `gpu` or `cpu`\nmust be set — enforced at the handler layer. For CPU pods, memory\nis derived by the API from the selected flavor's RAM multiplier;\nclients provide only CPU flavor and vCPU count. CPU pods support\ncontainer disk, network volumes, and a global volume; `mounts.persistent`\nis invalid when `cpu` is set.\n\n`image` is required unless `templateId` is set.\n", "properties": { "name": { "type": "string", @@ -5716,9 +5716,9 @@ }, "RateLimitPolicyHeader": { "type": "string", - "description": "Static per-window quota policy. Optional — omitted for rate-limit-exempt\ncallers.\n\nA structured-field list with one member per window (`minute`, `hour`,\n`day`), each carrying the quota `q` and the window length in seconds `w`.\nReturned on responses to authenticated requests, not only on 429s.\n", + "description": "Static per-window quota policy. Optional — omitted for rate-limit-exempt\ncallers.\n\nA structured-field list with one member per window (`minute`, `hour`,\n`day`), each carrying the quota `q` and the window length in seconds `w`.\nReturned on responses to authenticated requests, not only on 429s.\n\nLimits are per user, and every `/v2` route counts toward the same\nlimits; creating pods has no separate limit. Read the current quotas from\nthis header instead of hard-coding them, because they can change and some\naccounts have higher limits. A 429 only slows you down: wait for the\n`Retry-After` time and retry. Your account isn't suspended.\n", "examples": [ - "\"minute\";q=60;w=60, \"hour\";q=3000;w=3600, \"day\";q=50000;w=86400" + "\"minute\";q=180;w=60, \"hour\";q=3600;w=3600, \"day\";q=43200;w=86400" ] } } diff --git a/get-started.mdx b/get-started.mdx index 82e8d0a42..d3b25742f 100644 --- a/get-started.mdx +++ b/get-started.mdx @@ -59,6 +59,8 @@ You can also click **Pods** in the left sidebar. Use the search box, **Network volume** filter, **Filter** button, or **Sort by** dropdown to narrow the list. Click **Compare GPUs** to ask the Runpod assistant for a comparison of the selected GPUs. + To choose between Secure Cloud and Community Cloud, click **Filter** and set **Cloud type** to **Secure** or **Community**. Organization accounts can deploy only to Secure Cloud, so the option isn't shown for them. + The **Network volume** filter narrows the GPU list to compatible GPUs by filtering for GPUs that are in the same data center as the selected network volume. Select any network volume, and GPU availability updates to match it. GPUs that aren't compatible with the selected volume move to the incompatible section of the **All** tab. If the GPU you want shows **Out of capacity**, you can still select it and deploy it once capacity frees up. Selecting an out-of-capacity GPU changes the final [Deploy](#deploy) step: instead of deploying right away, you can subscribe to deploy when the GPU becomes available. diff --git a/get-started/agent-skills.mdx b/get-started/agent-skills.mdx index 972b7716f..6ba75faf4 100644 --- a/get-started/agent-skills.mdx +++ b/get-started/agent-skills.mdx @@ -11,31 +11,28 @@ It works with [Claude Code](https://code.claude.com/docs), [Codex](https://githu ## Quick start -Install the skills with a single command that works with every agent: + +**Setting this up from a coding agent?** Point it at [agents.runpod.io](https://agents.runpod.io/) (also at [/agent-setup](/agent-setup)). It's the single onboarding path for agents, with exact commands for each client. + + +Install the skills. Pass your agent's id with `--agent` (for example `cursor`, `codex`, or `opencode`) so the installer targets only that agent: ```bash -npx skills add runpod/runpod-plugins-official +npx skills add runpod/runpod-plugins-official --skill '*' --agent --global ``` -Next, install the Runpod CLI that the skills rely on (skip this if you already have it): +In Claude Code, install the plugin instead, which also connects the MCP server: `claude plugin marketplace add runpod/runpod-plugins-official`, then `claude plugin install runpod@runpod`. -```bash -curl -sSL https://cli.runpod.net | bash -# or with Homebrew: -brew install runpod/runpodctl/runpodctl -``` +Then connect the hosted MCP server and sign in with Runpod (see [Connecting the MCP server on other agents](#connecting-the-mcp-server-on-other-agents)). Sign-in uses OAuth, so you don't need an API key to start. -Then authenticate with your [Runpod API key](/get-started/credentials#api-keys). The same key works for the CLI, Flash, and the bundled MCP server: +You don't need to install the Runpod CLI or Flash now. The skills install them the first time a task needs them, such as a file transfer, SSH, or a Flash deploy, and ask for a [Runpod API key](/get-started/credentials#api-keys) at that point. The same key works for the CLI, Flash, and the MCP server. To set it up ahead of time: ```bash -# Set the key for the current shell (add to ~/.zshrc or ~/.bashrc to persist): -export RUNPOD_API_KEY= - -# Or save it permanently to ~/.runpod/config.toml: -runpodctl doctor +curl -sSL https://cli.runpod.net | bash # or: brew install runpod/runpodctl/runpodctl +export RUNPOD_API_KEY= # add to ~/.zshrc or ~/.bashrc to persist ``` -That's it — restart your agent and [get started](#getting-started). +That's it. Restart your agent and [get started](#getting-started). ## Getting started @@ -43,7 +40,7 @@ There are no commands to memorize: describe what you want, and the router picks - "List my Runpod endpoints" -If your endpoints come back, you're set — see [What you can do](#what-you-can-do) for more examples. +If your endpoints come back, you're set. See [What you can do](#what-you-can-do) for more examples. ## What's included @@ -57,6 +54,8 @@ The plugin installs a router and a set of focused skills: | **flash** | Writes and deploys your own Python code to Runpod Serverless using the [runpod-flash](/flash/overview) SDK. | | **companion-clis** | Uses supporting CLIs such as Hugging Face, Docker, and the AWS CLI when a task needs them. | | **runpod-usage** | Provides conceptual knowledge about Pods, Serverless, storage, and GPU selection. | +| **runpod-templates** | Picks and configures Runpod's official Pod templates, such as ComfyUI and PyTorch, and fixes problems with a running template. | +| **runpod-build-template** | Builds a complete template project (Dockerfile and app files) for Pods or Serverless from a request, local code, or a GitHub repo. | | **runpod-migrate** | Migrates a codebase from the GraphQL API or REST v1 to REST v2. It inventories which API each call site uses, rewrites the call sites, and verifies the result. | The plugin also includes a collection of worked, end-to-end reference tasks, such as deploying a Whisper endpoint or running a model on a Pod, that guide your agent through common workflows. @@ -130,13 +129,15 @@ Gemini can install the plugin natively through the bundled `gemini-extension.jso ## Connecting the MCP server on other agents -The hosted Runpod MCP server gives your agent structured control-plane tools for managing Pods, endpoints, and other resources. Claude Code sets it up automatically during a native install. On other agents, run the guided installer, which detects your agent and configures the connection: +The hosted Runpod MCP server gives your agent structured control-plane tools for managing Pods, endpoints, and other resources. Claude Code and Codex set it up automatically when you install the plugin. On other agents, run the installer with your agent's id. It writes the config without prompts: ```bash -npx @runpod/mcp-server@latest add +npx @runpod/mcp-server@latest add --client ``` -The installer authenticates the MCP server for you. To reuse the API key you already set instead, pass it as a bearer header when you add the server. For example, in Claude Code: +Run `npx @runpod/mcp-server@latest add --list` to see the supported ids (Cursor, Windsurf, VS Code, opencode, Pi, and Claude Desktop). For other agents, see [Onboard your agent to Runpod](/agent-setup#2-register-the-mcp-server). + +You sign in with Runpod in the browser the first time the agent calls a Runpod tool. To use an API key instead, pass it as a bearer header when you add the server. For example, in Claude Code: ```bash claude mcp add --transport http runpod -s user https://mcp.runpod.io/ \ diff --git a/get-started/concepts.mdx b/get-started/concepts.mdx index d72176051..e23f00e4b 100644 --- a/get-started/concepts.mdx +++ b/get-started/concepts.mdx @@ -11,9 +11,13 @@ The web interface for managing your compute resources, account, teams, and billi A pay-as-you-go compute solution designed for dynamic autoscaling in production AI/ML apps. +## [Runpod SDK](/sdk/overview) + +The `runpod` Python package, the single SDK for building on Runpod. Define GPU functions with decorators such as `@app.queue`, `@app.task`, and `@app.api` on a `runpod.App`, deploy them as apps, call existing Serverless endpoints, build Serverless workers, and manage Pods and other resources. It includes the `rp` command-line interface. + ## [Flash](/flash/overview) -A framework for building distributed GPU applications using local Python scripts. Write functions with the `@Endpoint` decorator, and Flash automatically executes them on Runpod's infrastructure. +The decorator-based framework that is now part of the [Runpod SDK](/sdk/overview). The standalone `runpod-flash` package gets no fixes or updates after October 29, 2026. For new projects, use the Runpod SDK; to move existing code, see [Migrating from Flash](/sdk/migrate-from-flash). ## [Pod](/pods/overview) diff --git a/get-started/mcp-servers.mdx b/get-started/mcp-servers.mdx index f029dcf4e..59cf9b41f 100644 --- a/get-started/mcp-servers.mdx +++ b/get-started/mcp-servers.mdx @@ -16,7 +16,7 @@ The Runpod API MCP server gives AI tools access to the [Runpod REST API](/api-re **Hosted endpoint:** `https://mcp.runpod.io/` (Streamable HTTP transport) -If you connected earlier using `https://mcp.getrunpod.io/`, that address still works. New connections should use `https://mcp.runpod.io/`. +`https://mcp.getrunpod.io/` is the same server under its old name, and it still works. Older versions of the installer and the Runpod plugin used that address. Use `https://mcp.runpod.io/` for new connections. Register only one of them: each address has its own sign-in, so registering both gives you two separate entries. **Local package:** `@runpod/mcp-server` (via npx) @@ -28,14 +28,23 @@ You can connect in two ways: the recommended hosted server, or a local server th ### Hosted server (recommended) -The hosted server runs at `https://mcp.runpod.io/` and provides the same capabilities as the local server. OAuth-capable clients run a "Sign in with Runpod" flow on first connect: a browser opens, you log in to the Runpod console and approve the request, and the server obtains a session-scoped API key. Nothing is stored on disk. +The hosted server runs at `https://mcp.runpod.io/` and provides the same capabilities as the local server. OAuth-capable clients run a "Sign in with Runpod" flow on first connect: a browser opens, you log in to the Runpod console and approve the request, and the server obtains a session-scoped API key. Nothing is stored on disk. Sign-in always starts from your client. Opening `https://mcp.runpod.io/` directly in a browser doesn't sign you in; it returns HTTP `405`. -The guided installer is the recommended path for supported clients. It detects your installed clients — Claude Code, Claude Desktop, Cursor, Windsurf, and VS Code — and configures each one, handling client-specific config-key differences automatically: +Sign-in works in Claude, ChatGPT, Codex, and Cursor, and in any client that receives the sign-in on a local address (`http://localhost` or `http://127.0.0.1`). Other clients may fail with a `redirect_uri is not allowed` error. For those clients, [pass an API key as a header](#use-an-api-key-with-the-hosted-server) or use the [local server](#local-server-api-key). + +The guided installer is the recommended path for supported clients. It detects your installed clients (Claude Code, Claude Desktop, Cursor, Windsurf, VS Code, opencode, and Pi) and configures each one, handling client-specific config-key differences automatically: ```bash npx @runpod/mcp-server@latest add ``` +The installer needs Node.js 20.12 or later. Without arguments, the installer asks which clients to configure, so it needs an interactive terminal. To skip the questions, for example when a coding agent runs it, name the clients with `--client` (separate several with commas). `--list` shows each client's id and where its config file lives: + +```bash +npx @runpod/mcp-server@latest add --list +npx @runpod/mcp-server@latest add --client cursor +``` + To undo these changes, run `npx @runpod/mcp-server@latest remove`. #### Claude Code (hosted) @@ -44,6 +53,37 @@ To undo these changes, run `npx @runpod/mcp-server@latest remove`. claude mcp add --transport http runpod -s user https://mcp.runpod.io/ ``` +#### Cursor (hosted) + +Add the following to `.cursor/mcp.json` (project-level) or `~/.cursor/mcp.json` (global), or run `npx @runpod/mcp-server@latest add --client cursor`. Then restart Cursor and connect **runpod** under **Settings > MCP**: + +```json +{ + "mcpServers": { + "runpod": { + "url": "https://mcp.runpod.io/" + } + } +} +``` + +#### opencode (hosted) + +opencode uses the `mcp` key, not `mcpServers`, and needs `"type": "remote"`. Add this to `opencode.json`, or run `npx @runpod/mcp-server@latest add --client opencode`: + +```json +{ + "mcp": { + "runpod": { + "type": "remote", + "url": "https://mcp.runpod.io/" + } + } +} +``` + +Run `opencode mcp auth runpod` to sign in, and `opencode mcp list` to check the connection. + #### Claude Desktop (hosted) 1. Open **Settings** in Claude Desktop. @@ -73,20 +113,31 @@ For any other MCP-compatible client, use the remote URL `https://mcp.runpod.io/` Clients differ in the exact key name they use for a remote server URL. For the clients the guided installer supports, it writes the correct configuration automatically; for any other client, consult that client's MCP documentation for how to add a remote HTTP server, using the URL above. - -To skip the OAuth flow, pass your API key as a bearer header. For example, in Claude Code: +#### Use an API key with the hosted server + +To skip the OAuth flow, or if your client can't complete it, pass your API key as a bearer header. For example, in Claude Code: ```bash claude mcp add --transport http runpod -s user https://mcp.runpod.io/ \ --header "Authorization: Bearer $RUNPOD_API_KEY" ``` -Clients configured with JSON use a `headers` block instead. - +Clients configured with JSON use a `headers` block instead: + +```json +{ + "mcpServers": { + "runpod": { + "url": "https://mcp.runpod.io/", + "headers": { "Authorization": "Bearer your_api_key_here" } + } + } +} +``` ### Local server (API key) -Use this method to run the server locally with an API key stored in your client config — for example, if your client doesn't support the hosted server's sign-in flow, or you prefer to manage the credential yourself. +Use this method to run the server locally with an API key stored in your client config. For example, use it if your client doesn't support the hosted server's sign-in flow, or you prefer to manage the credential yourself. #### Supported clients diff --git a/instant-clusters/configuration.mdx b/instant-clusters/configuration.mdx index cb052c913..bf8b6d399 100644 --- a/instant-clusters/configuration.mdx +++ b/instant-clusters/configuration.mdx @@ -44,6 +44,12 @@ Instant Clusters support up to 8 high-bandwidth interfaces per node. The `PRIMAR Do not use `eth0` for inter-node communication. The 172.xxx IP addresses on `eth0` are reserved for internet connectivity only and will result in connection timeouts for distributed workloads. +### RDMA + +Where the hardware supports it, traffic between nodes uses RDMA (InfiniBand or RoCE) instead of TCP. Runpod sets up each Pod for this: it adds the `/dev/infiniband/uverbs*` devices, removes the memlock limit, and sets `NCCL_IB_HCA` and `NCCL_IB_DISABLE=0` (plus `NCCL_IB_GID_INDEX` on RoCE). NCCL then uses RDMA without any extra configuration. On hosts without RDMA, NCCL falls back to TCP over the `ens` interfaces. + +GPUDirect RDMA is also enabled on the hosts, so libraries that register GPU memory directly with the verbs API, such as KV cache transfer libraries, can work. Runpod tests NCCL, not these libraries. Before relying on one, check that it can see the `mlx5` devices and that `NCCL_IB_HCA` is set. + ## NCCL configuration [NCCL](https://developer.nvidia.com/nccl) (NVIDIA Collective Communications Library) handles GPU-to-GPU communication in distributed training. You must configure NCCL to use the internal network interfaces. diff --git a/pods/manage-pods.mdx b/pods/manage-pods.mdx index 626ade90c..d54175360 100644 --- a/pods/manage-pods.mdx +++ b/pods/manage-pods.mdx @@ -1,6 +1,6 @@ --- title: "Manage Pods" -description: "Create, start, stop, and terminate Pods using the Runpod console or CLI. Review setup, configuration, and operations guidance for Runpod Pods." +description: "Create, start, stop, restart, and terminate Pods using the Runpod console or CLI. Review setup, configuration, and operations guidance for Runpod Pods." --- import { MachineTooltip, TemplatesTooltip } from "/snippets/tooltips.jsx"; @@ -18,6 +18,7 @@ runpodctl config --apiKey RUNPOD_API_KEY | **Deploy** | [Pods page](https://www.console.runpod.io/pods) → Deploy | `runpodctl pod create --name NAME --gpu-id "GPU" --image "IMAGE"` | | **Start** | Expand Pod → Play icon | `runpodctl pod start POD_ID` | | **Stop** | Expand Pod → Stop icon | `runpodctl pod stop POD_ID` | +| **Restart** | Three-dot menu → Restart Pod | `runpodctl pod restart POD_ID` | | **Update** | Three-dot menu → Edit Pod | `runpodctl pod update POD_ID` | | **Terminate** | Expand Pod → Trash icon | `runpodctl pod delete POD_ID` | | **List** | [Pods page](https://www.console.runpod.io/pods) | `runpodctl pod list` | @@ -170,6 +171,39 @@ curl --request POST \ +## Restart a Pod + +Restarting reboots a running Pod's container on the same machine. The Pod keeps its GPU, so it's a safe way to reload a process, such as one running as PID 1. Unlike stopping and starting, restarting can't leave you with [zero GPUs](/pods/troubleshooting/zero-gpus). + + + + +1. Open the [Pods page](https://www.console.runpod.io/pods). +2. Open the Pod's three-dot menu and click **Restart Pod**. + + + + + +```sh +runpodctl pod restart $RUNPOD_POD_ID +``` + + + + + +```bash +curl --request POST \ + --url "https://api.runpod.io/v2/pods/$RUNPOD_POD_ID/action" \ + --header 'Authorization: Bearer RUNPOD_API_KEY' \ + --header 'Content-Type: application/json' \ + --data '{"action": "restart"}' +``` + + + + ## Update a Pod Modify an existing Pod's configuration, such as storage size, image, ports, or environment variables. diff --git a/pods/storage/types.mdx b/pods/storage/types.mdx index 3095337ee..c0e80b464 100644 --- a/pods/storage/types.mdx +++ b/pods/storage/types.mdx @@ -18,7 +18,7 @@ For a comparison of persistent storage across Pods, Serverless, and the Runpod S | **Performance** | Fastest (local) | Fast (local) | Variable (network) | Variable (optimized for reads) | | **Shareable** | No | No | Yes (across Pods) | Yes (across Pods in any data center) | | **Resizable** | Yes | Increase only | Yes | Automatic (grows with stored data) | -| **Cost** | \$0.10/GB/month | \$0.10/GB/month (running) | \$0.07/GB/month | \$0.09/GB/month + IOPS| +| **Cost** | \$0.10/GB/month | \$0.10/GB/month (running) | \$0.07/GB/month (first 1 TB), \$0.05/GB/month beyond | \$0.09/GB/month + IOPS| | | | \$0.20/GB/month (stopped) | | | | **Best for** | OS, temp files, cache | Models, datasets, checkpoints | Shared data, portable storage | Model weights, read-heavy shared assets | diff --git a/pods/troubleshooting/pod-migration.mdx b/pods/troubleshooting/pod-migration.mdx index 4b93c0f6f..393298ebb 100644 --- a/pods/troubleshooting/pod-migration.mdx +++ b/pods/troubleshooting/pod-migration.mdx @@ -29,9 +29,9 @@ When prompted to migrate your Pod, you have three options: 1. **Do nothing**: If you don't want to migrate your data, you can wait and try again later. The GPU will become available once another user stops their Pod on that machine. -2. **Start Pod with CPUs**: If you don't need GPU access immediately, you can start your Pod with CPUs only. This lets you access your data and manually migrate files if needed, but the Pod will have limited CPU resources and is not suitable for compute-intensive tasks. +2. **Start Pod with CPUs**: If you don't need GPU access immediately, you can start your Pod with CPUs only (click **Start** and choose **CPU Only** in the GPU count selector; with the GraphQL API, use the `podResumeZeroGpu` mutation). This lets you access your data and manually migrate files if needed, but the Pod will have limited CPU resources and is not suitable for compute-intensive tasks. -3. **Automatically migrate Pod data**: This option spins up a new Pod with the same specifications as your current one and automatically migrates your data to a machine with available GPUs. The migration process finds a new machine with your requested GPU type, provisions the instance, and transfers your network volume data from the old Pod to the new one. +3. **Automatically migrate Pod data**: This option spins up a new Pod with the same specifications as your current one and automatically migrates your data to a machine with available GPUs. The migration process finds a new machine with your requested GPU type, provisions the instance, and copies your Pod's volume disk (`/workspace`) to the new Pod. Network volumes and global volumes aren't copied, because they aren't stored on the old machine: the new Pod attaches the same volume. The container disk is never migrated, so anything outside `/workspace` is lost. ## Important considerations for migration diff --git a/pods/troubleshooting/zero-gpus.mdx b/pods/troubleshooting/zero-gpus.mdx index 7653dc7f9..006721710 100644 --- a/pods/troubleshooting/zero-gpus.mdx +++ b/pods/troubleshooting/zero-gpus.mdx @@ -18,11 +18,12 @@ When this happens, Runpod gives you the option to start the Pod with zero GPUs. ## What are my options? -If you encounter this situation, you have three choices: +If you encounter this situation, you have four choices: -1. **Start with zero GPUs for data access**: Start the Pod without a GPU to access its local storage. This is useful for retrieving files, but the Pod will have limited CPU resources and is not suitable for compute tasks. You should use this option to back up or transfer your data before terminating the Pod. +1. **Start with zero GPUs for data access**: Click **Start** and choose **CPU Only** in the GPU count selector to start the Pod without a GPU and access its local storage. With the GraphQL API, use `mutation { podResumeZeroGpu(input: { podId: "YOUR_POD_ID" }) { id desiredStatus } }`. The REST API's `start` action can't do this: it returns an error when the machine's GPU is taken. This is useful for retrieving files, but the Pod will have limited CPU resources and is not suitable for compute tasks. You should use this option to back up or transfer your data before terminating the Pod. 2. **Wait and retry**: You can wait and try to restart the Pod again later. The GPU may become available if the other user stops their Pod, but there is no guarantee of when that will happen. -3. **Terminate and redeploy**: If you need a GPU immediately, terminate the current Pod and deploy a new one with the same configuration. The new Pod will be scheduled on any machine in the Runpod network with an available GPU of your chosen type. +3. **Migrate the Pod**: Let Runpod create a new Pod on a machine with a free GPU and copy your volume disk (`/workspace`) to it. See [Pod migration](/pods/troubleshooting/pod-migration). +4. **Terminate and redeploy**: If you need a GPU immediately and don't need the data on the volume disk, terminate the current Pod and deploy a new one with the same configuration. The new Pod will be scheduled on any machine in the Runpod network with an available GPU of your chosen type. ## How do I prevent this? diff --git a/public-endpoints/models/infinitetalk.mdx b/public-endpoints/models/infinitetalk.mdx index 0670e4b88..ee4868fbb 100644 --- a/public-endpoints/models/infinitetalk.mdx +++ b/public-endpoints/models/infinitetalk.mdx @@ -77,7 +77,7 @@ response = requests.post( ) result = response.json() -print(result["output"]["video_url"]) +print(result["output"]["result"]) ``` ```javascript JavaScript @@ -102,7 +102,7 @@ const response = await fetch( ); const result = await response.json(); -console.log(result.output.video_url); +console.log(result.output.result); ``` @@ -131,7 +131,7 @@ console.log(result.output.video_url); The generation result containing the video URL and cost. - + URL of the generated video. This URL expires after 7 days. @@ -149,7 +149,7 @@ console.log(result.output.video_url); "executionTime": 45678, "workerId": "oqk7ao1uomckye", "output": { - "video_url": "https://video.runpod.ai/abc123/output.mp4", + "result": "https://video.runpod.ai/abc123/output.mp4", "cost": 0.25 } } diff --git a/public-endpoints/models/kling-v2-6-motion-control.mdx b/public-endpoints/models/kling-v2-6-motion-control.mdx index 87318ff88..cb3577adb 100644 --- a/public-endpoints/models/kling-v2-6-motion-control.mdx +++ b/public-endpoints/models/kling-v2-6-motion-control.mdx @@ -81,7 +81,7 @@ response = requests.post( ) result = response.json() -print(result["output"]["video_url"]) +print(result["output"]["result"]) ``` ```javascript JavaScript @@ -106,7 +106,7 @@ const response = await fetch( ); const result = await response.json(); -console.log(result.output.video_url); +console.log(result.output.result); ``` @@ -123,7 +123,7 @@ console.log(result.output.video_url); The generation result containing the video URL and cost. - + URL of the generated video. This URL expires after 7 days. @@ -140,7 +140,7 @@ console.log(result.output.video_url); "delayTime": 25, "executionTime": 45678, "output": { - "video_url": "https://video.runpod.ai/abc123/output.mp4", + "result": "https://video.runpod.ai/abc123/output.mp4", "cost": 0.42 } } diff --git a/public-endpoints/models/kling-video-o1-r2v.mdx b/public-endpoints/models/kling-video-o1-r2v.mdx index a7ffb4d85..b6d531e3b 100644 --- a/public-endpoints/models/kling-video-o1-r2v.mdx +++ b/public-endpoints/models/kling-video-o1-r2v.mdx @@ -99,7 +99,7 @@ response = requests.post( ) result = response.json() -print(result["output"]["video_url"]) +print(result["output"]["result"]) ``` ```javascript JavaScript @@ -127,7 +127,7 @@ const response = await fetch( ); const result = await response.json(); -console.log(result.output.video_url); +console.log(result.output.result); ``` @@ -144,7 +144,7 @@ console.log(result.output.video_url); The generation result containing the video URL and cost. - + URL of the generated video. This URL expires after 7 days. @@ -161,7 +161,7 @@ console.log(result.output.video_url); "delayTime": 30, "executionTime": 75432, "output": { - "video_url": "https://video.runpod.ai/abc123/output.mp4", + "result": "https://video.runpod.ai/abc123/output.mp4", "cost": 0.56 } } diff --git a/public-endpoints/models/minimax-speech.mdx b/public-endpoints/models/minimax-speech.mdx index dd080e557..77be3dfa7 100644 --- a/public-endpoints/models/minimax-speech.mdx +++ b/public-endpoints/models/minimax-speech.mdx @@ -93,7 +93,7 @@ response = requests.post( ) result = response.json() -print(result["output"]["audio_url"]) +print(result["output"]["result"]) ``` ```javascript JavaScript @@ -120,7 +120,7 @@ const response = await fetch( ); const result = await response.json(); -console.log(result.output.audio_url); +console.log(result.output.result); ``` @@ -149,7 +149,7 @@ console.log(result.output.audio_url); The generation result containing the audio URL and cost. - + URL of the generated audio file. This URL expires after 7 days. @@ -167,7 +167,7 @@ console.log(result.output.audio_url); "executionTime": 3456, "workerId": "oqk7ao1uomckye", "output": { - "audio_url": "https://audio.runpod.ai/abc123/output.mp3", + "result": "https://audio.runpod.ai/abc123/output.mp3", "cost": 0.0055 } } diff --git a/public-endpoints/models/nano-banana-2-edit.mdx b/public-endpoints/models/nano-banana-2-edit.mdx index 532587975..9bdb60814 100644 --- a/public-endpoints/models/nano-banana-2-edit.mdx +++ b/public-endpoints/models/nano-banana-2-edit.mdx @@ -87,7 +87,7 @@ response = requests.post( ) result = response.json() -print(result["output"]["image_url"]) +print(result["output"]["result"]) ``` ```javascript JavaScript @@ -115,7 +115,7 @@ const response = await fetch( ); const result = await response.json(); -console.log(result.output.image_url); +console.log(result.output.result); ``` @@ -140,7 +140,7 @@ console.log(result.output.image_url); The generation result containing the image URL and cost. - + URL of the edited image. This URL expires after 7 days. @@ -157,7 +157,7 @@ console.log(result.output.image_url); "delayTime": 21, "executionTime": 6234, "output": { - "image_url": "https://image.runpod.ai/abc123/output.png", + "result": "https://image.runpod.ai/abc123/output.png", "cost": 0.0875 } } diff --git a/public-endpoints/models/nano-banana-pro-edit.mdx b/public-endpoints/models/nano-banana-pro-edit.mdx index e5b716983..4b1b802b3 100644 --- a/public-endpoints/models/nano-banana-pro-edit.mdx +++ b/public-endpoints/models/nano-banana-pro-edit.mdx @@ -89,7 +89,7 @@ response = requests.post( ) result = response.json() -print(result["output"]["image_url"]) +print(result["output"]["result"]) ``` ```javascript JavaScript @@ -118,7 +118,7 @@ const response = await fetch( ); const result = await response.json(); -console.log(result.output.image_url); +console.log(result.output.result); ``` @@ -135,7 +135,7 @@ console.log(result.output.image_url); The generation result containing the image URL and cost. - + URL of the edited/combined image. This URL expires after 7 days. @@ -152,7 +152,7 @@ console.log(result.output.image_url); "delayTime": 25, "executionTime": 8432, "output": { - "image_url": "https://image.runpod.ai/abc123/output.png", + "result": "https://image.runpod.ai/abc123/output.png", "cost": 0.14 } } diff --git a/public-endpoints/models/p-image-edit.mdx b/public-endpoints/models/p-image-edit.mdx index d8c5c0ad2..fcd02b2f4 100644 --- a/public-endpoints/models/p-image-edit.mdx +++ b/public-endpoints/models/p-image-edit.mdx @@ -75,7 +75,7 @@ response = requests.post( ) result = response.json() -print(result["output"]["image_url"]) +print(result["output"]["result"]) ``` ```javascript JavaScript @@ -99,7 +99,7 @@ const response = await fetch( ); const result = await response.json(); -console.log(result.output.image_url); +console.log(result.output.result); ``` @@ -116,7 +116,7 @@ console.log(result.output.image_url); The generation result containing the image URL and cost. - + URL of the edited image. This URL expires after 7 days. @@ -133,7 +133,7 @@ console.log(result.output.image_url); "delayTime": 12, "executionTime": 2456, "output": { - "image_url": "https://image.runpod.ai/abc123/output.png", + "result": "https://image.runpod.ai/abc123/output.png", "cost": 0.01 } } diff --git a/public-endpoints/models/p-image-t2i.mdx b/public-endpoints/models/p-image-t2i.mdx index 29eba7c9e..43ef7d36b 100644 --- a/public-endpoints/models/p-image-t2i.mdx +++ b/public-endpoints/models/p-image-t2i.mdx @@ -67,7 +67,7 @@ response = requests.post( ) result = response.json() -print(result["output"]["image_url"]) +print(result["output"]["result"]) ``` ```javascript JavaScript @@ -89,7 +89,7 @@ const response = await fetch( ); const result = await response.json(); -console.log(result.output.image_url); +console.log(result.output.result); ``` @@ -106,7 +106,7 @@ console.log(result.output.image_url); The generation result containing the image URL and cost. - + URL of the generated image. This URL expires after 7 days. @@ -123,7 +123,7 @@ console.log(result.output.image_url); "delayTime": 10, "executionTime": 1245, "output": { - "image_url": "https://image.runpod.ai/abc123/output.png", + "result": "https://image.runpod.ai/abc123/output.png", "cost": 0.005 } } diff --git a/public-endpoints/models/p-video.mdx b/public-endpoints/models/p-video.mdx index 0b7381790..6d84de3d0 100644 --- a/public-endpoints/models/p-video.mdx +++ b/public-endpoints/models/p-video.mdx @@ -99,7 +99,7 @@ response = requests.post( ) result = response.json() -print(result["output"]["video_url"]) +print(result["output"]["result"]) ``` ```javascript JavaScript @@ -123,7 +123,7 @@ const response = await fetch( ); const result = await response.json(); -console.log(result.output.video_url); +console.log(result.output.result); ``` @@ -148,7 +148,7 @@ console.log(result.output.video_url); The generation result containing the video URL and cost. - + URL of the generated video. This URL expires after 7 days. @@ -165,7 +165,7 @@ console.log(result.output.video_url); "delayTime": 15, "executionTime": 8542, "output": { - "video_url": "https://video.runpod.ai/abc123/output.mp4", + "result": "https://video.runpod.ai/abc123/output.mp4", "cost": 0.10 } } diff --git a/public-endpoints/models/qwen-image-edit-2511-lora.mdx b/public-endpoints/models/qwen-image-edit-2511-lora.mdx index cd70dcd6c..6f405b2a6 100644 --- a/public-endpoints/models/qwen-image-edit-2511-lora.mdx +++ b/public-endpoints/models/qwen-image-edit-2511-lora.mdx @@ -101,7 +101,7 @@ response = requests.post( ) result = response.json() -print(result["output"]["image_url"]) +print(result["output"]["result"]) ``` ```javascript JavaScript @@ -132,7 +132,7 @@ const response = await fetch( ); const result = await response.json(); -console.log(result.output.image_url); +console.log(result.output.result); ``` @@ -149,7 +149,7 @@ console.log(result.output.image_url); The generation result containing the image URL and cost. - + URL of the edited image. This URL expires after 7 days. @@ -166,7 +166,7 @@ console.log(result.output.image_url); "delayTime": 18, "executionTime": 4521, "output": { - "image_url": "https://image.runpod.ai/abc123/output.png", + "result": "https://image.runpod.ai/abc123/output.png", "cost": 0.025 } } diff --git a/public-endpoints/models/qwen-image-edit-2511.mdx b/public-endpoints/models/qwen-image-edit-2511.mdx index 904a52456..e3f311b8f 100644 --- a/public-endpoints/models/qwen-image-edit-2511.mdx +++ b/public-endpoints/models/qwen-image-edit-2511.mdx @@ -77,7 +77,7 @@ response = requests.post( ) result = response.json() -print(result["output"]["image_url"]) +print(result["output"]["result"]) ``` ```javascript JavaScript @@ -102,7 +102,7 @@ const response = await fetch( ); const result = await response.json(); -console.log(result.output.image_url); +console.log(result.output.result); ``` @@ -119,7 +119,7 @@ console.log(result.output.image_url); The generation result containing the image URL and cost. - + URL of the edited image. This URL expires after 7 days. @@ -136,7 +136,7 @@ console.log(result.output.image_url); "delayTime": 15, "executionTime": 3856, "output": { - "image_url": "https://image.runpod.ai/abc123/output.png", + "result": "https://image.runpod.ai/abc123/output.png", "cost": 0.02 } } diff --git a/public-endpoints/models/qwen-image-edit.mdx b/public-endpoints/models/qwen-image-edit.mdx index 20c3fe9ca..104668a73 100644 --- a/public-endpoints/models/qwen-image-edit.mdx +++ b/public-endpoints/models/qwen-image-edit.mdx @@ -83,7 +83,7 @@ response = requests.post( ) result = response.json() -print(result["output"]["image_url"]) +print(result["output"]["result"]) ``` ```javascript JavaScript @@ -109,7 +109,7 @@ const response = await fetch( ); const result = await response.json(); -console.log(result.output.image_url); +console.log(result.output.result); ``` @@ -138,7 +138,7 @@ console.log(result.output.image_url); The generation result containing the image URL and cost. - + URL of the edited image. This URL expires after 7 days. @@ -156,7 +156,7 @@ console.log(result.output.image_url); "executionTime": 3875, "workerId": "oqk7ao1uomckye", "output": { - "image_url": "https://image.runpod.ai/abc123/output.png", + "result": "https://image.runpod.ai/abc123/output.png", "cost": 0.02097152 } } diff --git a/public-endpoints/models/qwen-image.mdx b/public-endpoints/models/qwen-image.mdx index 6f35c9f51..93a44ac19 100644 --- a/public-endpoints/models/qwen-image.mdx +++ b/public-endpoints/models/qwen-image.mdx @@ -77,7 +77,7 @@ response = requests.post( ) result = response.json() -print(result["output"]["image_url"]) +print(result["output"]["result"]) ``` ```javascript JavaScript @@ -102,7 +102,7 @@ const response = await fetch( ); const result = await response.json(); -console.log(result.output.image_url); +console.log(result.output.result); ``` @@ -131,7 +131,7 @@ console.log(result.output.image_url); The generation result containing the image URL and cost. - + URL of the generated image. This URL expires after 7 days. @@ -149,7 +149,7 @@ console.log(result.output.image_url); "executionTime": 3254, "workerId": "oqk7ao1uomckye", "output": { - "image_url": "https://image.runpod.ai/abc123/output.png", + "result": "https://image.runpod.ai/abc123/output.png", "cost": 0.035267 } } diff --git a/public-endpoints/models/seedream-4-edit.mdx b/public-endpoints/models/seedream-4-edit.mdx index 18777f7d0..0051a714d 100644 --- a/public-endpoints/models/seedream-4-edit.mdx +++ b/public-endpoints/models/seedream-4-edit.mdx @@ -77,7 +77,7 @@ response = requests.post( ) result = response.json() -print(result["output"]["image_url"]) +print(result["output"]["result"]) ``` ```javascript JavaScript @@ -104,7 +104,7 @@ const response = await fetch( ); const result = await response.json(); -console.log(result.output.image_url); +console.log(result.output.result); ``` @@ -133,7 +133,7 @@ console.log(result.output.image_url); The generation result containing the image URL and cost. - + URL of the edited image. This URL expires after 7 days. @@ -151,7 +151,7 @@ console.log(result.output.image_url); "executionTime": 4856, "workerId": "oqk7ao1uomckye", "output": { - "image_url": "https://image.runpod.ai/abc123/output.png", + "result": "https://image.runpod.ai/abc123/output.png", "cost": 0.028311 } } diff --git a/public-endpoints/models/seedream-4-t2i.mdx b/public-endpoints/models/seedream-4-t2i.mdx index e02094602..b6339e898 100644 --- a/public-endpoints/models/seedream-4-t2i.mdx +++ b/public-endpoints/models/seedream-4-t2i.mdx @@ -77,7 +77,7 @@ response = requests.post( ) result = response.json() -print(result["output"]["image_url"]) +print(result["output"]["result"]) ``` ```javascript JavaScript @@ -102,7 +102,7 @@ const response = await fetch( ); const result = await response.json(); -console.log(result.output.image_url); +console.log(result.output.result); ``` @@ -131,7 +131,7 @@ console.log(result.output.image_url); The generation result containing the image URL and cost. - + URL of the generated image. This URL expires after 7 days. @@ -149,7 +149,7 @@ console.log(result.output.image_url); "executionTime": 5234, "workerId": "oqk7ao1uomckye", "output": { - "image_url": "https://image.runpod.ai/abc123/output.png", + "result": "https://image.runpod.ai/abc123/output.png", "cost": 0.113246 } } diff --git a/public-endpoints/models/sora-2-pro.mdx b/public-endpoints/models/sora-2-pro.mdx index 6a8fea494..2dc003313 100644 --- a/public-endpoints/models/sora-2-pro.mdx +++ b/public-endpoints/models/sora-2-pro.mdx @@ -71,7 +71,7 @@ response = requests.post( ) result = response.json() -print(result["output"]["video_url"]) +print(result["output"]["result"]) ``` ```javascript JavaScript @@ -95,7 +95,7 @@ const response = await fetch( ); const result = await response.json(); -console.log(result.output.video_url); +console.log(result.output.result); ``` @@ -124,7 +124,7 @@ console.log(result.output.video_url); The generation result containing the video URL and cost. - + URL of the generated video. This URL expires after 7 days. @@ -142,7 +142,7 @@ console.log(result.output.video_url); "executionTime": 185432, "workerId": "oqk7ao1uomckye", "output": { - "video_url": "https://video.runpod.ai/abc123/output.mp4", + "result": "https://video.runpod.ai/abc123/output.mp4", "cost": 1.20 } } diff --git a/public-endpoints/models/sora-2.mdx b/public-endpoints/models/sora-2.mdx index 8735c8dd4..ed9694c62 100644 --- a/public-endpoints/models/sora-2.mdx +++ b/public-endpoints/models/sora-2.mdx @@ -65,7 +65,7 @@ response = requests.post( ) result = response.json() -print(result["output"]["video_url"]) +print(result["output"]["result"]) ``` ```javascript JavaScript @@ -88,7 +88,7 @@ const response = await fetch( ); const result = await response.json(); -console.log(result.output.video_url); +console.log(result.output.result); ``` @@ -117,7 +117,7 @@ console.log(result.output.video_url); The generation result containing the video URL and cost. - + URL of the generated video. This URL expires after 7 days. @@ -135,7 +135,7 @@ console.log(result.output.video_url); "executionTime": 120345, "workerId": "oqk7ao1uomckye", "output": { - "video_url": "https://video.runpod.ai/abc123/output.mp4", + "result": "https://video.runpod.ai/abc123/output.mp4", "cost": 0.40 } } diff --git a/public-endpoints/models/vidu-q3-i2v.mdx b/public-endpoints/models/vidu-q3-i2v.mdx index 9b217e2a0..0451b5530 100644 --- a/public-endpoints/models/vidu-q3-i2v.mdx +++ b/public-endpoints/models/vidu-q3-i2v.mdx @@ -95,7 +95,7 @@ response = requests.post( ) result = response.json() -print(result["output"]["video_url"]) +print(result["output"]["result"]) ``` ```javascript JavaScript @@ -123,7 +123,7 @@ const response = await fetch( ); const result = await response.json(); -console.log(result.output.video_url); +console.log(result.output.result); ``` @@ -148,7 +148,7 @@ console.log(result.output.video_url); The generation result containing the video URL and cost. - + URL of the generated video. This URL expires after 7 days. @@ -165,7 +165,7 @@ console.log(result.output.video_url); "delayTime": 32, "executionTime": 45678, "output": { - "video_url": "https://video.runpod.ai/abc123/output.mp4", + "result": "https://video.runpod.ai/abc123/output.mp4", "cost": 0.75 } } diff --git a/public-endpoints/models/vidu-q3-t2v.mdx b/public-endpoints/models/vidu-q3-t2v.mdx index aed23de6c..378370b41 100644 --- a/public-endpoints/models/vidu-q3-t2v.mdx +++ b/public-endpoints/models/vidu-q3-t2v.mdx @@ -101,7 +101,7 @@ response = requests.post( ) result = response.json() -print(result["output"]["video_url"]) +print(result["output"]["result"]) ``` ```javascript JavaScript @@ -130,7 +130,7 @@ const response = await fetch( ); const result = await response.json(); -console.log(result.output.video_url); +console.log(result.output.result); ``` @@ -155,7 +155,7 @@ console.log(result.output.video_url); The generation result containing the video URL and cost. - + URL of the generated video. This URL expires after 7 days. @@ -172,7 +172,7 @@ console.log(result.output.video_url); "delayTime": 28, "executionTime": 52341, "output": { - "video_url": "https://video.runpod.ai/abc123/output.mp4", + "result": "https://video.runpod.ai/abc123/output.mp4", "cost": 0.75 } } diff --git a/public-endpoints/models/wan-2-1-i2v.mdx b/public-endpoints/models/wan-2-1-i2v.mdx index ba75df09c..7170c2deb 100644 --- a/public-endpoints/models/wan-2-1-i2v.mdx +++ b/public-endpoints/models/wan-2-1-i2v.mdx @@ -113,7 +113,7 @@ response = requests.post( ) result = response.json() -print(result["output"]["video_url"]) +print(result["output"]["result"]) ``` ```javascript JavaScript @@ -144,7 +144,7 @@ const response = await fetch( ); const result = await response.json(); -console.log(result.output.video_url); +console.log(result.output.result); ``` @@ -173,7 +173,7 @@ console.log(result.output.video_url); The generation result containing the video URL and cost. - + URL of the generated video. This URL expires after 7 days. @@ -191,7 +191,7 @@ console.log(result.output.video_url); "executionTime": 65432, "workerId": "oqk7ao1uomckye", "output": { - "video_url": "https://video.runpod.ai/abc123/output.mp4", + "result": "https://video.runpod.ai/abc123/output.mp4", "cost": 0.30 } } diff --git a/public-endpoints/models/wan-2-2-i2v.mdx b/public-endpoints/models/wan-2-2-i2v.mdx index 0a3e9bc2d..7a167891b 100644 --- a/public-endpoints/models/wan-2-2-i2v.mdx +++ b/public-endpoints/models/wan-2-2-i2v.mdx @@ -36,8 +36,8 @@ All parameters are passed within the `input` object in the request body. Video resolution in format "width*height". - - Number of denoising steps. Range: 1-50. + + Number of denoising steps. Range: 1-50. Required: requests that omit it fail validation. `30` is a good starting value. @@ -113,7 +113,7 @@ response = requests.post( ) result = response.json() -print(result["output"]["video_url"]) +print(result["output"]["result"]) ``` ```javascript JavaScript @@ -144,7 +144,7 @@ const response = await fetch( ); const result = await response.json(); -console.log(result.output.video_url); +console.log(result.output.result); ``` @@ -173,7 +173,7 @@ console.log(result.output.video_url); The generation result containing the video URL and cost. - + URL of the generated video. This URL expires after 7 days. @@ -191,7 +191,7 @@ console.log(result.output.video_url); "executionTime": 67543, "workerId": "oqk7ao1uomckye", "output": { - "video_url": "https://video.runpod.ai/abc123/output.mp4", + "result": "https://video.runpod.ai/abc123/output.mp4", "cost": 0.30 } } diff --git a/public-endpoints/models/wan-2-5.mdx b/public-endpoints/models/wan-2-5.mdx index 4f1be04e0..6d30ec0d6 100644 --- a/public-endpoints/models/wan-2-5.mdx +++ b/public-endpoints/models/wan-2-5.mdx @@ -4,7 +4,7 @@ sidebarTitle: "WAN 2.5" description: "Image-to-video generation model with prompt expansion support. Review inputs and output formats for this model on Runpod Public Endpoints." --- -WAN 2.5 is Alibaba's image-to-video generation model that creates videos from static images. It features optional prompt expansion to automatically enhance your prompts for better results. +WAN 2.5 is Alibaba's image-to-video generation model that creates videos from static images, with synchronized audio. It features optional prompt expansion to automatically enhance your prompts for better results. Test WAN 2.5 in the Runpod Hub playground. @@ -28,16 +28,20 @@ All parameters are passed within the `input` object in the request body. URL of the source image to animate. + + URL of an audio file to guide generation. WAN 2.5 generates synchronized audio on its own when you omit this. + + Elements to exclude from the video. - - Video dimensions in format "width*height". + + Video resolution. Options: `480p`, `720p`, `1080p`. The older `size` field is still accepted as an alias. - Video duration in seconds. + Video duration in seconds (3-10). @@ -62,7 +66,7 @@ curl -X POST "https://api.runpod.ai/v2/wan-2-5/runsync" \ "prompt": "A stand-up comedian delivering a dad joke", "image": "https://example.com/comedian.png", "negative_prompt": "", - "size": "1280*720", + "resolution": "720p", "duration": 5, "seed": -1, "enable_prompt_expansion": false, @@ -85,7 +89,7 @@ response = requests.post( "prompt": "A stand-up comedian delivering a dad joke", "image": "https://example.com/comedian.png", "negative_prompt": "", - "size": "1280*720", + "resolution": "720p", "duration": 5, "seed": -1, "enable_prompt_expansion": False, @@ -95,7 +99,7 @@ response = requests.post( ) result = response.json() -print(result["output"]["video_url"]) +print(result["output"]["result"]) ``` ```javascript JavaScript @@ -112,7 +116,7 @@ const response = await fetch( prompt: "A stand-up comedian delivering a dad joke", image: "https://example.com/comedian.png", negative_prompt: "", - size: "1280*720", + resolution: "720p", duration: 5, seed: -1, enable_prompt_expansion: false, @@ -123,7 +127,7 @@ const response = await fetch( ); const result = await response.json(); -console.log(result.output.video_url); +console.log(result.output.result); ``` @@ -152,7 +156,7 @@ console.log(result.output.video_url); The generation result containing the video URL and cost. - + URL of the generated video. This URL expires after 7 days. @@ -170,7 +174,7 @@ console.log(result.output.video_url); "executionTime": 78901, "workerId": "oqk7ao1uomckye", "output": { - "video_url": "https://video.runpod.ai/abc123/output.mp4", + "result": "https://video.runpod.ai/abc123/output.mp4", "cost": 0.50 } } diff --git a/public-endpoints/models/wan-2-6-i2v.mdx b/public-endpoints/models/wan-2-6-i2v.mdx index 4b349abe4..18453c510 100644 --- a/public-endpoints/models/wan-2-6-i2v.mdx +++ b/public-endpoints/models/wan-2-6-i2v.mdx @@ -101,7 +101,7 @@ response = requests.post( ) result = response.json() -print(result["output"]["video_url"]) +print(result["output"]["result"]) ``` ```javascript JavaScript @@ -128,7 +128,7 @@ const response = await fetch( ); const result = await response.json(); -console.log(result.output.video_url); +console.log(result.output.result); ``` @@ -153,7 +153,7 @@ console.log(result.output.video_url); The generation result containing the video URL and cost. - + URL of the generated video. This URL expires after 7 days. @@ -170,7 +170,7 @@ console.log(result.output.video_url); "delayTime": 28, "executionTime": 95432, "output": { - "video_url": "https://video.runpod.ai/abc123/output.mp4", + "result": "https://video.runpod.ai/abc123/output.mp4", "cost": 0.50 } } diff --git a/public-endpoints/models/wan-2-6-t2i.mdx b/public-endpoints/models/wan-2-6-t2i.mdx index fe87e6db4..dc439017f 100644 --- a/public-endpoints/models/wan-2-6-t2i.mdx +++ b/public-endpoints/models/wan-2-6-t2i.mdx @@ -71,7 +71,7 @@ response = requests.post( ) result = response.json() -print(result["output"]["image_url"]) +print(result["output"]["result"]) ``` ```javascript JavaScript @@ -95,7 +95,7 @@ const response = await fetch( ); const result = await response.json(); -console.log(result.output.image_url); +console.log(result.output.result); ``` @@ -112,7 +112,7 @@ console.log(result.output.image_url); The generation result containing the image URL and cost. - + URL of the generated image. This URL expires after 7 days. @@ -129,7 +129,7 @@ console.log(result.output.image_url); "delayTime": 15, "executionTime": 3254, "output": { - "image_url": "https://image.runpod.ai/abc123/output.png", + "result": "https://image.runpod.ai/abc123/output.png", "cost": 0.03 } } diff --git a/public-endpoints/models/wan-2-6-t2v.mdx b/public-endpoints/models/wan-2-6-t2v.mdx index eae1916e3..53928cb53 100644 --- a/public-endpoints/models/wan-2-6-t2v.mdx +++ b/public-endpoints/models/wan-2-6-t2v.mdx @@ -89,7 +89,7 @@ response = requests.post( ) result = response.json() -print(result["output"]["video_url"]) +print(result["output"]["result"]) ``` ```javascript JavaScript @@ -114,7 +114,7 @@ const response = await fetch( ); const result = await response.json(); -console.log(result.output.video_url); +console.log(result.output.result); ``` @@ -131,7 +131,7 @@ console.log(result.output.video_url); The generation result containing the video URL and cost. - + URL of the generated video. This URL expires after 7 days. @@ -148,7 +148,7 @@ console.log(result.output.video_url); "delayTime": 28, "executionTime": 85432, "output": { - "video_url": "https://video.runpod.ai/abc123/output.mp4", + "result": "https://video.runpod.ai/abc123/output.mp4", "cost": 0.50 } } diff --git a/public-endpoints/models/z-image-turbo.mdx b/public-endpoints/models/z-image-turbo.mdx index 8f081b82c..fd4f46a3a 100644 --- a/public-endpoints/models/z-image-turbo.mdx +++ b/public-endpoints/models/z-image-turbo.mdx @@ -85,7 +85,7 @@ response = requests.post( ) result = response.json() -print(result["output"]["image_url"]) +print(result["output"]["result"]) ``` ```javascript JavaScript @@ -110,7 +110,7 @@ const response = await fetch( ); const result = await response.json(); -console.log(result.output.image_url); +console.log(result.output.result); ``` @@ -135,7 +135,7 @@ console.log(result.output.image_url); The generation result containing the image URL and cost. - + URL of the generated image. This URL expires after 7 days. @@ -152,7 +152,7 @@ console.log(result.output.image_url); "delayTime": 12, "executionTime": 1856, "output": { - "image_url": "https://image.runpod.ai/abc123/output.png", + "result": "https://image.runpod.ai/abc123/output.png", "cost": 0.005 } } diff --git a/sdks/graphql/manage-pods.mdx b/sdks/graphql/manage-pods.mdx index 6c17b38b2..091f003f8 100644 --- a/sdks/graphql/manage-pods.mdx +++ b/sdks/graphql/manage-pods.mdx @@ -14,6 +14,7 @@ For the complete schema, see the [GraphQL Spec](https://graphql-spec.runpod.io/) | Create Pod | `podFindAndDeployOnDemand` | | Start Pod | `podResume` | | Stop Pod | `podStop` | +| Terminate Pod | `podTerminate` | | List all Pods | `myself { pods { ... } }` | | Get Pod by ID | `pod(input: {podId: "..."})` | | List GPU types | `gpuTypes` | @@ -218,6 +219,31 @@ mutation { +## Terminate a Pod + +Terminating a Pod permanently deletes it, along with its container disk and volume disk. Network volumes attached to the Pod are not deleted. This can't be undone. + + + +```bash +curl --request POST \ + --header 'content-type: application/json' \ + --url 'https://api.runpod.io/graphql?api_key=${YOUR_API_KEY}' \ + --data '{"query": "mutation { podTerminate(input: {podId: \"YOUR_POD_ID\"}) }"}' +``` + + + +```graphql +mutation { + podTerminate(input: { podId: "YOUR_POD_ID" }) +} +``` + + + +`podTerminate` returns no fields, so don't add a selection set. The REST equivalent is [`DELETE /v2/pods/{podId}`](/api-reference-v2/pods/terminate-a-pod). + ## Query Pods ### List all Pods diff --git a/serverless/load-balancing/overview.mdx b/serverless/load-balancing/overview.mdx index a44c35c32..add179056 100644 --- a/serverless/load-balancing/overview.mdx +++ b/serverless/load-balancing/overview.mdx @@ -117,9 +117,11 @@ When calculating endpoint metrics, Runpod calculates the cold start time for loa | Variable | Default | Description | |----------|---------|-------------| | `PORT` | `80` | Main application server port | -| `PORT_HEALTH` | Same as `PORT` | Health check endpoint port | +| `PORT_HEALTH` | `80` | Health check endpoint port | | `HEALTH_CHECK_PATH` | `/ping` | Path the load balancer polls to check worker health | +Changing `PORT` doesn't change `PORT_HEALTH`: each defaults to `80` separately. If you set `PORT` to another port, set `PORT_HEALTH` too, usually to the same value. Otherwise the load balancer checks health on port 80, gets no answer, and the worker never becomes ready. + If using a custom port, add it to your endpoint's environment variables and expose it in container configuration (under **Expose HTTP Ports (Max 10)**). ## Timeouts and limits diff --git a/serverless/vllm/environment-variables.mdx b/serverless/vllm/environment-variables.mdx index d64839841..4ef01533e 100644 --- a/serverless/vllm/environment-variables.mdx +++ b/serverless/vllm/environment-variables.mdx @@ -10,6 +10,13 @@ Environment variables let you configure your vLLM workers without rebuilding you To set environment variables, go to your [endpoint settings](/serverless/endpoints/overview#edit-an-endpoint) and add them under **Environment Variables**. +The latest vLLM worker release, [v2.30.1](https://github.com/runpod-workers/worker-vllm/releases), is built on vLLM v0.32.0. The tables below list common settings, but you're not limited to them: + +- Any environment variable named after a `vllm serve` flag in upper snake case becomes that flag. For example, `MAX_MODEL_LEN=4096` becomes `--max-model-len 4096`. +- `VLLM_EXTRA_ARGS` passes extra flags straight to `vllm serve`. They're added last, so they override any earlier setting. Use it for flags that don't have an environment variable, for example `VLLM_EXTRA_ARGS="--tool-call-parser qwen3_coder --enable-auto-tool-choice"`. + +You can use any tool-call parser or speculative decoding method that vLLM v0.32.0 supports, such as MTP, even if the tables below don't list it. + ## LLM settings These variables control the core language model configuration. diff --git a/serverless/workers/github-integration.mdx b/serverless/workers/github-integration.mdx index 94d8a17a6..322ce4ea4 100644 --- a/serverless/workers/github-integration.mdx +++ b/serverless/workers/github-integration.mdx @@ -23,6 +23,8 @@ To deploy a worker from GitHub, you need: * A Dockerfile in your repository. See [Creating a Dockerfile](/serverless/workers/deploy#creating-a-dockerfile) for details. * A GitHub account. +You don't need a `.runpod` directory (`hub.json`, `tests.json`) to deploy from GitHub. Those files are only required to list the repo on the [Runpod Hub](/hub/publishing-guide). + For an example repository containing the minimal files necessary for deployment, see [runpod-workers/worker-basic](https://github.com/runpod-workers/worker-basic) on GitHub. diff --git a/storage/globalvolume/overview.mdx b/storage/globalvolume/overview.mdx index 57084a4f8..c0a699b79 100644 --- a/storage/globalvolume/overview.mdx +++ b/storage/globalvolume/overview.mdx @@ -80,9 +80,9 @@ Workloads that write frequently or loop over large directory listings can accumu Global volumes use object-backed storage, not a fully POSIX compliant file system. Review the following before using global volumes in production. -- **GPU only:** Global volumes can't be attached to CPU Pods or CPU Serverless endpoints. +- **CPU Pods through the API only:** The console attaches global volumes to GPU Pods only. To attach one to a CPU Pod, create the Pod with the [REST API](/api-reference-v2/pods/create-a-pod) (`mounts.global`). Global volumes can't be attached to CPU Serverless endpoints. - **Limited POSIX semantics:** Global volumes do not support file locking, atomic rename, or hard links. Applications that depend on POSIX behavior may fail or produce unexpected results. -- **No permission bits:** Global volumes cannot set file permission bits. This may produce warnings when downloading models from sources such as Hugging Face. +- **No permission bits:** Global volumes cannot set file permission bits. This may produce warnings when downloading models from sources such as Hugging Face. It also means templates that install a Python environment into `/workspace`, such as the official ComfyUI template, don't work with a global volume as their primary volume: the deploy page shows an **Unsupported volume type** warning. For those templates, keep `/workspace` on the volume disk or a network volume, and mount the global volume at a second path, such as `/workspace-global`, for model files only. - **Concurrent writes to the same file:** Multiple Pods can write to a global volume concurrently, but when two Pods write to the same file at the same time, the last write wins. For workloads that require file locking or high-concurrency writes, use a network volume instead. - **Eventual consistency:** Global volumes use object storage, which means updates made to the volume after a Pod or worker starts may not be immediately visible inside the container. Plan your workload around the state of the volume at mount time. diff --git a/storage/s3-api.mdx b/storage/s3-api.mdx index 3ebf9d892..fe370fa36 100644 --- a/storage/s3-api.mdx +++ b/storage/s3-api.mdx @@ -47,10 +47,16 @@ Create your network volume in a supported datacenter to use the S3-compatible AP Next, you'll need to generate a new key called an "S3 API key" (this is separate from your Runpod API key). 1. Go to the [Credentials page](https://console.runpod.io/user/credentials) in the Runpod console. - 2. Select the **S3 API Keys** tab and click **Create an S3 API key**. + 2. Select the **S3 API keys** tab and click **Create an S3 API key**. 3. Name your key and select **Create**. 4. Save the **access key** (e.g., `user_***...`) and **secret** (e.g., `rps_***...`) to use in the next step. + + + On a team account, the **S3 API keys** tab appears only for members with the **Admin** role and permission to view S3 API keys. Creating a key also needs permission to create S3 API keys. If you don't see the tab, ask a team owner to change your role. + + + For security, Runpod will show your API key secret only once, so you may wish to save it elsewhere (e.g., in your password manager, or in a GitHub secret). Treat your API key secret like a password and don't share it with anyone. @@ -66,7 +72,7 @@ Create your network volume in a supported datacenter to use the S3-compatible AP 1. If you haven't already, [install the AWS CLI](https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html) on your local machine. 2. Run the command `aws configure` in your terminal. 3. Provide the following when prompted: - * **AWS Access Key ID**: Enter your Runpod user ID. You can find this in the [Credentials page](https://console.runpod.io/user/credentials) under **S3 API Keys**, in the description of your S3 API key. By default, the description will look similar to: `Shared Secret for user_2f21CfO73Mm2Uq2lEGFiEF24IPw 1749176107073`. `user_2f21CfO73Mm2Uq2lEGFiEF24IPw` is the user ID (yours will be different). + * **AWS Access Key ID**: Enter your Runpod user ID. You can find this in the [Credentials page](https://console.runpod.io/user/credentials) under **S3 API keys**, in the description of your S3 API key. By default, the description will look similar to: `Shared Secret for user_2f21CfO73Mm2Uq2lEGFiEF24IPw 1749176107073`. `user_2f21CfO73Mm2Uq2lEGFiEF24IPw` is the user ID (yours will be different). * **AWS Secret Access Key**: Enter your Runpod S3 API key's secret access key. * **Default Region name**: You can leave this blank. * **Default output format**: You can leave this blank or set it to `json`.