Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
33 changes: 33 additions & 0 deletions doc/package-spec-audit-2026-11.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
# Package-spec audit — November 2026 baseline

This is the historical baseline used when the package spec was introduced. It predates subsequent package migrations and is not a pending-work list or a current scan. The normative rules are in [package-spec.md](package-spec.md). New scans should record their date, repository revisions and scan scope separately.

Counts are from all 113 non-archived `DAppNodePackage*` and `DNP_*` repositories in November 2026; "active" means a push in the last 12 months (87 repositories). 58 active repositories currently have at least one finding, mostly SPEC-M03; 37 repositories are already clean.

| Rule | Baseline observation |
|---|---|
| SPEC-M01 | 0 active repositories. |
| SPEC-M02 | 2 inactive (Zosma-Nethermind, Zosma-OE). |
| SPEC-M03 | 44 active repositories still use them, including DNP_BIND, DNP_IPFS, Hopr, NEAR, Hermes-agent and Honcho. This is the largest migration and removes a second code path from the SDK bumper, tropibot's discovery and edit code, and the dashboards. |
| SPEC-M04 | 9 active (SafeStake-testnet, SplitPro, avalanche, graph, monero, rocketpool, rocketpool-testnet, supabase, web3signer-kiln). |
| SPEC-M05 | 4 active (GnosisVPNExitNode, Hopr-testnet, Tornado-Cash-Relayer, aztec-testnet). |
| SPEC-M06 | 0 repositories. |
| SPEC-M07 | 1 active (owncloud: `minimumDappnodeVersion`), 5 inactive with legacy `avatar`/`image`. |
| SPEC-M08 | 8 active point elsewhere: renamed repositories (DMS, anchor-generic), sibling packages (Erigon-Sepolia-Archival, lukso-erigon, NEAR-testnet), a fork (avalanche) and upstream projects (SplitPro, supabase). |
| SPEC-M09 | 2 active (op-erigon, op-geth). |
| SPEC-T01 | 5 active (juno-generic, jupyter-generic, nethermind-generic, ollama-generic, pathfinder-generic) and madara-generic. |
| SPEC-T02 | 0 repositories. |
| SPEC-T03 | 0 repositories. |
| SPEC-T04 | 1 active (erigon-generic: a stale root `prometheus-targets.json`). |
| SPEC-T05 | 0 repositories. |
| SPEC-C01 | 0 repositories use the flag. |
| SPEC-C02 | 6 active (Hermes-agent, Nextcloud, graph, jupyter-generic, ollama-generic, openclaw). |
| SPEC-C03 | 2 active (graph: `postgres`; owncloud: `webhippie/mariadb:latest`, `webhippie/redis:latest`). |
| SPEC-C04 | 2 active (alephium, supabase) and madara-generic. |
| SPEC-C05 | 3 active (DNP_NOTIFICATIONS, pi-hole, portainer). |
| SPEC-F01 | 5 active: NEAR `setup-wizard-ui.yml`, Tornado-Cash-Relayer `avatar-default.jpg`, package-harness `avatar-harness.svg`, pathfinder-generic `getting-started.yml` (must be `.md`) and reth-gnosis `notifications.yml` (must be `.yaml`). |
| SPEC-F02 | 2 active (Lido-CSM-generic avatar 123 KB, lukso-erigon avatar 245 KB). |
| SPEC-F03 | 1 active (DMS notifications). |
| SPEC-B01 | no workflow uses it. |

Other observations from the same review: 109 repositories declared Compose version, 25 omitted architectures, 6 inactive repositories used unpinned Dockerfile bases, and 20 active repositories used remote ADD without a checksum (19 through the staking-scripts pattern). Tropibot workflows covered 46 repositories and the SDK bumper covered 5.
88 changes: 55 additions & 33 deletions doc/package-spec.md

Large diffs are not rendered by default.

11 changes: 2 additions & 9 deletions src/files/compose/types.ts
Original file line number Diff line number Diff line change
@@ -1,20 +1,13 @@
import {
ComposeNetworks,
ComposeService,
ComposeVolumes
} from "@dappnode/types";
import { Compose, ComposeService } from "@dappnode/types";

export type OptionalComposeServiceProperties = Partial<
Pick<ComposeService, "image" | "build">
>;
export type ComposeServiceFlexible = Omit<ComposeService, "image" | "build"> &
OptionalComposeServiceProperties;

export interface FlexibleCompose {
version: string;
export interface FlexibleCompose extends Omit<Compose, "services"> {
services: {
[dnpName: string]: ComposeServiceFlexible;
};
networks?: ComposeNetworks;
volumes?: ComposeVolumes;
}
82 changes: 53 additions & 29 deletions src/spec/checkPackageSpec.ts
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,10 @@ import { defaultComposeFileName, defaultVariantsDirName } from "../params.js";
import { parseGithubRepoSlug } from "../files/manifest/getRepoSlugFromManifest.js";
import { shell } from "../utils/shell.js";
import { SpecFinding, SpecRuleId } from "./rules.js";
import {
checkManifestReferences,
manifestFieldIsDefined
} from "./manifestReferences.js";

const manifestFileName = "dappnode_package.json";
const packageNamePattern =
Expand Down Expand Up @@ -384,27 +388,43 @@ export function checkPackageSpec({
);
continue;
}
// "<version>@sha256:<digest>" pins the same version by digest.
const matchesVersion = (value: string) =>
value === String(upstream.version) ||
value.startsWith(`${upstream.version}@sha256:`);
if (upstream.version !== undefined && ![...values].some(matchesVersion)) {
const argDir = composeOrigin(c =>
Object.values(c.services || {}).some(
def =>
isObject(def?.build) &&
hasBuildArg(def.build, upstream.arg as string)
)
// A digest suffix pins the same version only when its SHA-256 is complete.
const matchesVersion = (value: string) => {
const prefix = `${upstream.version}@sha256:`;
return (
value === String(upstream.version) ||
(value.startsWith(prefix) &&
/^[a-f0-9]{64}$/i.test(value.slice(prefix.length)))
);
const variant = where(upstreamDir) || where(argDir);
add(
"SPEC-M05",
`Upstream ${upstream.repo}${variant} has version "${
upstream.version
}" but build arg ${upstream.arg} is "${[...values].join('", "')}".`,
manifestFileIn(upstreamDir)
);
}
};
if (upstream.version !== undefined)
for (const [service, value] of values) {
if (matchesVersion(value)) continue;
const argDir = composeOrigin(c => {
const build = c.services?.[service]?.build;
return isObject(build) && hasBuildArg(build, upstream.arg);
});
const variant = where(upstreamDir) || where(argDir);
add(
"SPEC-M05",
`Upstream ${upstream.repo}${variant} has version "${upstream.version}" but services.${service}.build.args.${upstream.arg} is "${value}"; every service using this arg must match (optionally with @sha256:<64 hex digits>).`,
composeFileIn(argDir)
);
}
}

for (const finding of checkManifestReferences(
sourceManifest as Manifest,
compose
)) {
const dir = manifestOrigin(m => manifestFieldIsDefined(m, finding.field));
add(
finding.rule,
`${finding.message}${
pkg.variant ? ` Checked for variant "${pkg.variant}".` : ""
}`,
manifestFileIn(dir)
);
}

const services = Object.keys(compose.services || {});
Expand Down Expand Up @@ -600,25 +620,29 @@ function mergeByPosition<T>(parts: T[]): T {
return parts.reduce((acc, part) => merge(acc, part) as T, {} as T);
}

/** Build arg name -> values across all services; list-style "KEY=value" args included. */
function collectBuildArgs(compose: Compose): Map<string, Set<string>> {
const args = new Map<string, Set<string>>();
const put = (key: string, value: unknown) => {
if (!args.has(key)) args.set(key, new Set());
/** Build arg name -> service -> value; list-style "KEY=value" args included. */
function collectBuildArgs(compose: Compose): Map<string, Map<string, string>> {
const args = new Map<string, Map<string, string>>();
const put = (service: string, key: string, value: unknown) => {
if (!args.has(key)) args.set(key, new Map());
args
.get(key)
?.add(value === undefined || value === null ? "" : String(value));
?.set(
service,
value === undefined || value === null ? "" : String(value)
);
};
for (const def of Object.values(compose.services || {})) {
for (const [service, def] of Object.entries(compose.services || {})) {
if (!def || typeof def.build !== "object" || !def.build.args) continue;
const buildArgs: unknown = def.build.args;
if (Array.isArray(buildArgs))
for (const item of buildArgs) {
const [key, ...rest] = String(item).split("=");
put(key, rest.join("="));
put(service, key, rest.join("="));
}
else if (isObject(buildArgs))
for (const [key, value] of Object.entries(buildArgs)) put(key, value);
for (const [key, value] of Object.entries(buildArgs))
put(service, key, value);
}
return args;
}
Expand Down
85 changes: 85 additions & 0 deletions src/spec/manifestReferences.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,85 @@
import { Compose, GLOBAL_ENVS, Manifest } from "@dappnode/types";

interface ManifestReferenceFinding {
rule: "SPEC-M10" | "SPEC-M11";
field: string;
message: string;
}

const globalEnvPrefix = "_DAPPNODE_GLOBAL_";
// The installer prefixes manifest envs before looking them up in this catalogue.
// Use values rather than keys: ACTIVE, for example, maps to ENVS_ACTIVE.
const globalEnvNames = new Set(
Object.values(GLOBAL_ENVS).map(name => name.slice(globalEnvPrefix.length))
);

/** Checks the resolved manifest against its resolved compose, after variant merging. */
export function checkManifestReferences(
manifest: Manifest,
compose: Compose
): ManifestReferenceFinding[] {
const findings: ManifestReferenceFinding[] = [];
const services = new Set(Object.keys(compose.services || {}));
const checkService = (field: string, value: unknown) => {
// Schema validation owns malformed values; spec checks must not throw first.
if (typeof value === "string" && !services.has(value))
findings.push({
rule: "SPEC-M10",
field,
message: `${field} refers to service "${value}", which is not in the compose.`
});
};

checkService("mainService", manifest.mainService);
if (manifest.chain && typeof manifest.chain === "object")
checkService("chain.serviceName", manifest.chain.serviceName);
if (Array.isArray(manifest.backup))
manifest.backup.forEach((entry, i) =>
checkService(`backup[${i}].service`, entry?.service)
);

if (Array.isArray(manifest.globalEnvs))
manifest.globalEnvs.forEach((entry, i) => {
if (Array.isArray(entry?.services))
entry.services.forEach((service, j) =>
checkService(`globalEnvs[${i}].services[${j}]`, service)
);
if (Array.isArray(entry?.envs))
entry.envs.forEach((env, j) => {
if (typeof env !== "string" || globalEnvNames.has(env)) return;
const shortName = env.startsWith(globalEnvPrefix)
? env.slice(globalEnvPrefix.length)
: undefined;
const suggestion =
shortName && globalEnvNames.has(shortName)
? ` Use "${shortName}"; DAPPMANAGER adds ${globalEnvPrefix} when installing.`
: " Use a short name from the shared GLOBAL_ENVS catalogue.";
const field = `globalEnvs[${i}].envs[${j}]`;
findings.push({
rule: "SPEC-M11",
field,
message: `${field} requests unsupported global env "${env}".${suggestion}`
});
});
});

return findings;
}

/** Used to attribute an inherited reference to the source fragment that defines it. */
export function manifestFieldIsDefined(
manifest: Partial<Manifest>,
field: string
): boolean {
let value: unknown = manifest;
for (const key of field.replace(/\[(\d+)\]/g, ".$1").split(".")) {
if (
value === null ||
typeof value !== "object" ||
!Object.hasOwn(value, key)
)
return false;
value = (value as Record<string, unknown>)[key];
}
return value !== undefined;
}
12 changes: 10 additions & 2 deletions src/spec/rules.ts
Original file line number Diff line number Diff line change
Expand Up @@ -25,11 +25,11 @@ export const specRules = {
},
"SPEC-M05": {
title: "Upstream version equals the build arg value",
why: "Manifest and compose must describe the same upstream version; tools otherwise have to translate between tag formats."
why: "Every service using an upstream arg must match its declared version, optionally pinned by a complete SHA-256 digest."
},
"SPEC-M06": {
title: "Variant upstream[] matches the root order",
why: "Root and variant manifests are merged by array position, so a reordered or shortened variant upstream[] mixes entries."
why: "Root and variant manifests are merged by array position; reordered entries mix upstreams, while shorter lists inherit the root tail."
},
"SPEC-M07": {
title: "Only known manifest fields",
Expand All @@ -43,6 +43,14 @@ export const specRules = {
title: "Exposable services exist in compose",
why: "An exposable entry for a service that is not in the compose cannot be exposed."
},
"SPEC-M10": {
title: "Manifest service references exist in compose",
why: "mainService, chain, backup and global-env service references must resolve after root and variant files are merged."
},
"SPEC-M11": {
title: "Global environment names match the runtime catalogue",
why: "DAPPMANAGER prefixes short global-env names at install; unknown or already-prefixed names cannot be injected."
},
// Templates (multi-variant packages)
"SPEC-T01": {
title: "Template root manifest has no name or version",
Expand Down
Loading
Loading