diff --git a/docs/dev/package-development/multi-configuration.md b/docs/dev/package-development/multi-configuration.md index 422c221e4..baad1ff1d 100644 --- a/docs/dev/package-development/multi-configuration.md +++ b/docs/dev/package-development/multi-configuration.md @@ -50,6 +50,14 @@ package_variants/ The contents within each variant directory include the fields that differ from one variant to another. When building a specific variant, such as `testnet`, the data from these variant-specific files is merged with the root-level `dappnode_package.json` and `docker-compose.yml`. This ensures that only the necessary variant-specific changes are applied, while the common configuration remains consistent across all variants. +### Manifest identity and inheritance + +The root manifest holds shared fields and omits `name` and `version`. Each variant supplies its own package ENS name and registry version; the merged manifest is validated as the installable package. + +Manifest objects merge recursively, scalars are overridden, and arrays merge by index. Shorter variant arrays inherit the root’s trailing entries; an empty array does not clear a root array. For `upstream[]`, retain the same repository at each overlapping position. Do not reorder entries to select a different upstream. + +After merging, every `upstream[].arg` must exist in the compose and every service declaring it must match the upstream version. Explicit `mainService`, `chain.serviceName`, `backup[].service` and `globalEnvs[].services` values must name services in the resolved compose. + You can add more variants as needed by creating additional directories and files following this structure. ### Customize the Avatar @@ -76,7 +84,7 @@ Example of a variant-specific `dappnode_package.json`: And the corresponding `docker-compose.yml`: -```json +```yaml version: "3.5" services: test: diff --git a/docs/dev/package-development/single-configuration.md b/docs/dev/package-development/single-configuration.md index ed3508e86..505efebd3 100644 --- a/docs/dev/package-development/single-configuration.md +++ b/docs/dev/package-development/single-configuration.md @@ -55,6 +55,20 @@ In the `dappnode_package.json`, update fields such as: - `name`, `version`, `description`, and `author` to reflect your package. - Add relevant `categories`, `architectures`, and exposed services (e.g., API endpoints). +### Declare upstream software + +Use `upstream[]` to connect each upstream repository to an explicit compose build argument. The upstream version is separate from the package’s own `version`. + +```json +{ + "upstream": [ + { "repo": "ethereum/go-ethereum", "version": "v1.2.3", "arg": "UPSTREAM_VERSION" } + ] +} +``` + +Set `build.args.UPSTREAM_VERSION` to `v1.2.3` in every compose service that declares it. See the [manifest reference](../references/manifest.md#upstream) for the full contract. + ### Simplified Dockerfile Here’s a simplified version of the Dockerfile inside the `geth` directory: diff --git a/docs/dev/references/docker-compose.md b/docs/dev/references/docker-compose.md index 9a1ec49a5..0d5687ccb 100644 --- a/docs/dev/references/docker-compose.md +++ b/docs/dev/references/docker-compose.md @@ -79,28 +79,37 @@ In the complex example, you'll find: - All environment variables (`envs`) defined in the `docker-compose.yml` file will be visible in the package config post-installation. Users can modify these. - If the package has a `setup-wizard`, those environment variables not set within it will be hidden by default in the configuration. Users can view these by selecting "Show advanced config". -#### The `UPSTREAM_VERSION` Argument +#### Upstream build arguments -- The `UPSTREAM_VERSION` argument is mandatory if you wish to leverage GitHub actions that automatically create PRs when there's a new release in the upstream repository. -- In the Dockerfile, `UPSTREAM_VERSION` is usually declared and used like this: +Declare an explicit argument for each upstream, such as `UPSTREAM_VERSION`, `GETH_VERSION` or `TEKU_VERSION`. The name is not fixed; it must match `upstream[].arg` exactly. Use a mapping under `build.args`. - ```dockerfile - ARG UPSTREAM_VERSION - FROM consensys/teku:$UPSTREAM_VERSION - ``` +Manifest excerpt: -:::note -The `upstreamVersion` and `upstreamRepo` fields need to be defined in the `dappnode_package.json` file for the GitHub actions to work. Here is an example: - -``` +```json { - ... - "upstreamVersion": "23.8.0", - "upstreamRepo": "ConsenSys/teku", - ... + "upstream": [ + { "repo": "ConsenSys/teku", "version": "v4.0.7", "arg": "TEKU_VERSION" } + ] } ``` -::: + +Corresponding compose build configuration: + +```yaml +services: + beacon-chain: + build: + context: beacon-chain + args: + TEKU_VERSION: v4.0.7 +``` + +```dockerfile +ARG TEKU_VERSION +FROM consensys/teku:${TEKU_VERSION} +``` + +If several services declare `TEKU_VERSION`, every value must match the manifest version. A digest pin must contain the same version followed by `@sha256:` and 64 hexadecimal characters. The legacy top-level `upstreamRepo`, `upstreamVersion` and `upstreamArg` fields are deprecated; use the [upstream array](manifest.md#upstream). #### The `EXTRA_OPTS` Environment Variable diff --git a/docs/dev/references/manifest.md b/docs/dev/references/manifest.md index 1c02316b8..2b3a61bb2 100644 --- a/docs/dev/references/manifest.md +++ b/docs/dev/references/manifest.md @@ -7,128 +7,123 @@ llm_description: "Complete reference for dappnode_package.json manifest file pro The Dappnode Package manifest defines all the necessary information for a Dappnode to understand this package: -- IPFS of BZZ hashes to download its docker image -- Docker related data to configure and run its container -- Metadata to control how the package is shown in the admin UI. +- Package identity, version and upstream software metadata. +- Installation requirements and container configuration. +- Information displayed in the admin UI. + +The [SDK package spec](https://github.com/dappnode/DAppNodeSDK/blob/master/doc/package-spec.md) defines repository rules and the warning-to-error rollout. This reference describes manifest fields. For templates, required package identity fields belong in each variant; validate the merged result. ## Example ```json { - "name": "ipfs.dnp.dappnode.eth", - "version": "0.2.0", - "upstreamVersion": "2.6.0-beta", - "shortDescription": "Distributed file system for storing and accessing data.", - "description": "Welcome! IPFS is a distributed system for storing and accessing files, websites, applications, and data. If you’re new to IPFS, check our [introductory page](https://ipfs.io/#why) for an easy overview. \n\nWith this node you can upload and download files from IPFS using it own fancy web console at [http://ipfs.dappnode:5001/webui](http://ipfs.dappnode:5001/webui). Other Dappnode Packages and external applications can use its API at the endpoint `http://ipfs.dappnode:5001/api`. Go to the [IPFS HTTP API full reference](https://docs.ipfs.io/reference/api/http/) to check all the features of the API.", + "name": "example.public.dappnode.eth", + "version": "0.1.0", + "description": "Example package with an application service.", "type": "service", - "chain": "ethereum", - "mainService": "webserver", - "dockerTimeout": "5min", - "dependencies": { - "bitcoin.dnp.dappnode.eth": "^0.1.2", - "swarm.dnp.dappnode.eth": "latest" - }, + "license": "MIT", + "upstream": [ + { + "repo": "example/upstream", + "version": "v1.2.3", + "arg": "APP_VERSION" + } + ], + "mainService": "app", + "architectures": [ + "linux/amd64", + "linux/arm64" + ], "requirements": { "minimumDappnodeVersion": "0.2.0" }, - "globalEnvs": { - "all": "true" - }, - "architectures": ["linux/amd64", "linux/arm64"], - "backup": [ + "globalEnvs": [ { - "name": "keystore", - "path": "/root/.raiden/secret/keystore", - "service": "validator" + "envs": [ + "PUBLIC_IP" + ], + "services": [ + "app" + ] } ], - "changelog": "Brief summary of the most relevant changes that the user must known before installing", - "warnings": { - "onInstall": "You must set the PASSWORD ENV before installing the Dappnode Package in order for the setup to work correctly.", - "onUpdate": "Your VPN connection will be lost when the VPN finalizes updating. Leave 1-2 minutes after executing the update and then reconnect and refresh this site.", - "onReset": "You MUST properly close your open channels before resetting this Dappnode Package or you may lose your funds.", - "onRemove": "You MUST properly close your open channels before removing this Dappnode Package or you may lose your funds." - }, - "updateAlerts": [ + "backup": [ { - "from": "0.1.x", - "to": "0.1.x", - "message": "Major update to OpenVPN: This update breaks compatibility with the last VPN version. Please read the migration guide: https://migration020.dappnode.io" + "name": "data", + "path": "/data", + "service": "app" } ], - "disclaimer": { - "message": "This software is experimental, presented “as is” and inherently carries risks." - }, - "style": { - "featuredBackground": "linear-gradient(to right, #323131, #395353)", - "featuredColor": "white", - "featuredAvatarFilter": "invert(1)" - }, "exposable": [ { - "name": "Geth JSON RPC", - "description": "JSON RPC endpoint for Geth mainnet", - "serviceName": "beacon_chain", - "port": 80 + "name": "Application", + "serviceName": "app", + "port": 8080, + "exposeByDefault": false } ], - "author": "Dappnode Association (https://github.com/dappnode)", - "contributors": [ - "Michael First (https://github.com/developerHanlder)", - "Michael Second (https://github.com/developerHanlder)" + "upnpDisable": [ + 30303 ], - "categories": ["Developer tools", "Blockchain"], - "keywords": ["DappnodeCore"], - "links": { - "homepage": "https://github.com/dappnode/DNP_IPFS#readme", - "ui": "http://ipfs.dappnode:5001/webui", - "api": "http://ipfs.dappnode:5001/api/v0", - "gateway": "http://ipfs.dappnode:8080/ipfs" - }, "repository": { "type": "git", - "url": "https://github.com/dappnode/DNP_IPFS.git", - "directory": "packages/react-dom" - }, - "bugs": { - "url": "https://github.com/dappnode/DNP_IPFS/issues" - }, - "license": "GPL-3.0" + "url": "https://github.com/example/DAppNodePackage-example" + } } ``` +The example refers to this `docker-compose.yml`. Every explicit manifest service reference must match a service key in the resolved compose, and every service declaring `APP_VERSION` must use the manifest’s upstream version. + +```yaml +version: "3.5" +services: + app: + build: + context: . + args: + APP_VERSION: v1.2.3 + image: app.example.public.dappnode.eth:0.1.0 + volumes: + - data:/data + restart: unless-stopped +volumes: + data: {} +``` + ## Properties reference -| Property | Type | Required | -| ------------------------------------- | ---------- | ------------ | -| [name](#name) | `string` | **Required** | -| [version](#version) | `string` | **Required** | -| [upstreamVersion](#upstreamversion) | `string` | Optional | -| [shortDescription](#shortdescription) | `string` | Optional | -| [description](#description) | `string` | **Required** | -| [type](#type) | `enum` | **Required** | -| [chain](#chain) | `enum` | Optional | -| [mainService](#mainservice) | `string` | Optional | -| [dockerTimeout](#dockertimeout) | `string` | Optional | -| [dependencies](#dependencies) | `object` | Optional | -| [requirements](#requirements) | `object` | Optional | -| [globalEnvs](#globalenvs) | `object` | Optional | -| [architectures](#architectures) | `enum[]` | Optional | -| [backup](#backup) | `object[]` | Optional | -| [changelog](#changelog) | `string` | Optional | -| [warnings](#warnings) | `object` | Optional | -| [updateAlerts](#updatealerts) | `object[]` | Optional | -| [disclaimer](#disclaimer) | `object` | Optional | -| [style](#style) | `object` | Optional | -| [exposable](#exposable) | `object[]` | Optional | -| [author](#author) | `string` | Optional | -| [contributors](#contributors) | `string[]` | Optional | -| [categories](#categories) | `enum[]` | Optional | -| [keywords](#keywords) | `string[]` | Optional | -| [links](#links) | `object` | Optional | -| [repository](#repository) | `object` | Optional | -| [bugs](#bugs) | `object` | Optional | -| [license](#license) | `string` | **Required** | +| Property | Type | Required | +| ------------------------------------- | ----------------------- | ------------ | +| [name](#name) | `string` | **Required** | +| [version](#version) | `string` | **Required** | +| [upstream](#upstream) | `object[]` | Optional | +| [upstreamVersion](#upstreamversion) | `string` (legacy) | Optional | +| [shortDescription](#shortdescription) | `string` | Optional | +| [description](#description) | `string` | **Required** | +| [type](#type) | `enum` | **Required** | +| [chain](#chain) | `enum` or `object` | Optional | +| [mainService](#mainservice) | `string` | Optional | +| [dockerTimeout](#dockertimeout) | `string` | Optional | +| [dependencies](#dependencies) | `object` | Optional | +| [requirements](#requirements) | `object` | Optional | +| [globalEnvs](#globalenvs) | `object[]` or legacy `object` | Optional | +| [architectures](#architectures) | `enum[]` | Optional | +| [backup](#backup) | `object[]` | Optional | +| [changelog](#changelog) | `string` | Optional | +| [warnings](#warnings) | `object` | Optional | +| [updateAlerts](#updatealerts) | `object[]` | Optional | +| [disclaimer](#disclaimer) | `object` | Optional | +| [style](#style) | `object` | Optional | +| [exposable](#exposable) | `object[]` | Optional | +| [upnpDisable](#upnpdisable) | `number[]` or `boolean` | Optional | +| [author](#author) | `string` | Optional | +| [contributors](#contributors) | `string[]` | Optional | +| [categories](#categories) | `enum[]` | Optional | +| [keywords](#keywords) | `string[]` | Optional | +| [links](#links) | `object` | Optional | +| [repository](#repository) | `object` | Optional | +| [bugs](#bugs) | `object` | Optional | +| [license](#license) | `string` | **Required** | ### name @@ -153,35 +148,38 @@ Dappnode Package semantic version (semver). All instances must conform to this regular expression ```regex -^((([0-9]+).([0-9]+).([0-9]+)))$ +^[0-9]+\.[0-9]+\.[0-9]+$ ``` -- test example: - [0.2.0]() - Example: ```json "0.2.0" ``` -### upstreamVersion +### upstream -For Dappnode Packages that only wrap existing software (i.e. Bitcoin node, Ethereum node), the underlying software -version can be specified here. It will be shown in the admin UI alongside the field `version`. +Use an array of upstream software dependencies, including when there is only one. Each entry requires `repo`, `version` and `arg`. -- is optional -- type: `string` +| Property | Meaning | +|---|---| +| `repo` | Upstream repository, such as `ethereum/go-ethereum`. | +| `version` | Version string used by the compose build argument, preserving prefixes such as `v`. This is independent of the package’s registry `version`. | +| `arg` | Exact argument name declared under a service’s `build.args`, such as `GETH_VERSION`. | -Examples: +Every service declaring the argument must use the same version. A build argument may additionally pin that version with `@sha256:` followed by 64 hexadecimal characters. The SDK reports conflicting service values; one matching service does not make the others valid. ```json -"2.6.0-beta" +[ + { "repo": "ethereum/go-ethereum", "version": "v1.2.3", "arg": "GETH_VERSION" } +] ``` -```json -"any-text" -``` +For root/variant inheritance, see [multi-config package development](../package-development/multi-configuration.md). Provider selection and source-tag transformations are automation configuration; the shared manifest contract currently defines `repo`, `version` and `arg`. + +### upstreamVersion + +The top-level `upstreamVersion`, `upstreamRepo` and `upstreamArg` fields are legacy. Existing consumers still read them during migration. New and migrated packages use `upstream[]`; do not mix both forms. ### shortDescription @@ -240,35 +238,27 @@ Examples: ### chain -Indicate that this Dappnode Package is a blockchain node so the admin UI shows its syncing status +Optional blockchain-driver metadata used to show node synchronization status. The value is either a driver name or an object with `driver`, optional `serviceName` and optional `portNumber`. -- is optional -- type: `string` - -The value of this property **must** be equal to one of the known values below. +Supported driver names are `ethereum`, `ethereum-beacon-chain`, `ethereum2-beacon-chain-prysm`, `bitcoin` and `monero`. -| Value | Description | -| ---------- | ------------------------------------------------------------------------------------------------------------- | -| `ethereum` | Must have an Ethereum JSON RPC exposed internally at port 8545 | -| `bitcoin` | Must have a standard Bitcoin JSON API exposed at the default port (8332). Uses the NPM package `bitcoin-core` | -| `monero` | Must have a Monero API exposed internally at port 18081. Uses the NPM package `monero-rpc` | - -Examples: +For a package whose resolved compose includes an `execution` service, an explicit target looks like: ```json -"ethereum" -``` - -```json -"bitcoin" +{ + "driver": "ethereum", + "serviceName": "execution", + "portNumber": 8545 +} ``` -```json -"monero" -``` +An explicit `serviceName` must exist in the resolved compose. Omitting it preserves the runtime’s default service selection. ### mainService +An explicit value must match a service key in the resolved compose, including after variant overrides. + + For multi-service packages, indicate which service is the main one. The root ENS domain of this package will be mapped to this service IP. @@ -346,6 +336,8 @@ Specify requirements to be met before allowing users to install this Dappnode Pa | Property | Type | Required | | ------------------------ | ------ | -------- | | `minimumDappnodeVersion` | string | Optional | +| `minimumDockerVersion` | string | Optional | +| `notInstalledPackages` | string[] | Optional | #### minimumDappnodeVersion @@ -357,12 +349,9 @@ Minimum Dappnode version that includes all the features necessary to run this Da All instances must conform to this regular expression ```regex -^((([0-9]+).([0-9]+).([0-9]+)))$ +^[0-9]+\.[0-9]+\.[0-9]+$ ``` -- test example: - [0.2.0]() - Example: ```json @@ -371,28 +360,24 @@ Example: ### globalEnvs -Request the DAPPMANAGER to inject global ENVs to this package's containers - -- is optional -- type: `object` with the following properties: +Request global environment variables for selected compose services. Use short names such as `PUBLIC_IP` and `HOSTNAME`. Dappmanager adds the `_DAPPNODE_GLOBAL_` prefix during installation; including the prefix in the manifest would add it twice. -| Property | Type | Required | -| -------- | ------- | -------- | -| `all` | boolean | Optional | - -#### all - -Request the DAPPMANAGER to inject all available global ENVs +```json +[ + { "envs": ["PUBLIC_IP", "HOSTNAME"], "services": ["app"] } +] +``` -- is optional -- type: `boolean` +Both `envs` and `services` are required in each entry. Empty lists are allowed. Every service must exist in the resolved compose. Names come from the shared [GLOBAL_ENVS catalogue](https://github.com/dappnode/DNP_DAPPMANAGER/blob/master/packages/types/src/globalEnvs.ts): remove the prefix from its values, rather than using its keys. For example, the key `ACTIVE` maps to `_DAPPNODE_GLOBAL_ENVS_ACTIVE`, so the manifest name is `ENVS_ACTIVE`. -Example: +The runtime also accepts the legacy all-services form: ```json -"true" +{ "all": true } ``` +Use a JSON boolean, not the string `"true"`. The value `false` leaves services unchanged. Prefer explicit lists for new packages. The published schema currently rejects this legacy object form; the corrected shared schema and matching types must be released before SDK validation accepts it everywhere. + ### architectures Build and distribute this package in multiple architectures using @@ -418,6 +403,9 @@ Example: ### backup +When `backup[].service` is specified, it must name a service in the resolved compose. Omission retains Dappmanager’s default service selection. + + Allows users to download and restore a backup of key files of this package. If this property is non-empty array, a new view will be available in the admin UI for this Dappnode Package. The files or directories specified in the array will be bundled in a tarball. As long as the name properties stay the same, their associated paths can change in future @@ -509,69 +497,22 @@ Example: ### warnings -Very relevant information that MUST be shown to the user BEFORE executing a specific action in the Dappnode Package's -lifecycle. - -- is optional -- type: `object` with the following properties: +Optional messages shown for package operations. Use the supported update-level fields rather than a generic `onUpdate` key. -| Property | Type | Required | -| ----------- | ------ | -------- | +| Property | Type | Required | +|---|---|---| | `onInstall` | string | Optional | -| `onUpdate` | string | Optional | -| `onReset` | string | Optional | -| `onRemove` | string | Optional | - -#### onInstall - -Will be shown before installing the Dappnode Package. - -- is optional -- type: `string` - -Example: - -```json -"You must set the PASSWORD ENV before installing the Dappnode Package in order for the setup to work correctly." -``` - -#### onUpdate - -Will be shown before updating the Dappnode Package, not in the first installation. - -- is optional -- type: `string` - -Example: +| `onPatchUpdate` | string | Optional | +| `onMinorUpdate` | string | Optional | +| `onMajorUpdate` | string | Optional | +| `onReset` | string | Optional | +| `onRemove` | string | Optional | ```json -"Your VPN connection will be lost when the VPN finalizes updating. Leave 1-2 minutes after executing the update and then reconnect and refresh this site." -``` - -#### onReset - -Will be shown before resetting the Dappnode Package. - -- is optional -- type: `string` - -Example: - -```json -"You MUST properly close your open channels before resetting this Dappnode Package or you may lose your funds." -``` - -#### onRemove - -Will be shown before removing the Dappnode Package. - -- is optional -- type: `string` - -Example: - -```json -"You MUST properly close your open channels before removing this Dappnode Package or you may lose your funds." +{ + "onMajorUpdate": "Read the migration instructions before updating.", + "onReset": "Back up your data before resetting this package." +} ``` ### updateAlerts @@ -819,6 +760,29 @@ Examples: Single exposable service item +### upnpDisable {#upnpdisable} + + + +The manifest key is `upnpDisable`; `disableUpnp` is not a supported alias. + +Controls UPnP port forwarding for this package. By default, DAppNode attempts to create UPnP port mappings for all ports defined in the docker-compose file. This property can either disable UPnP completely for the package or selectively disable it for specific ports. + +- is optional +- type: `boolean` or `number[]` + +If set to `true`, UPnP is disabled for all ports. An array selects host port numbers from 1 to 65535 and applies to both TCP and UDP. `false` or an empty array keeps UPnP enabled. + +Examples: + +```json +true +``` + +```json +[8545, 30303] +``` + ### author Main author of this Dappnode Package. Must follow the structure `${name} <${email}> (${githubUserLink})`.