Skip to content

Commit cf3a8e7

Browse files
committed
Remove PackageCache's per-module closest package memo
Stacked on #1946. ## Summary `PackageCache` remembers the closest `package.json` for every module path it's asked about, with a reverse index from each `package.json` to the module paths under it and a set of paths known to have no package, both of which exist only to invalidate the first. `DependencyGraph` calls `invalidate` for every changed file to keep it honest. With the previous two PRs that memo no longer pays for itself. #1945 cut closest-package lookups 3.7x (64k -> 17k per build on our mid-size RN benchmark app*), and #1946 made each one a walk of single map lookups, 2.1x faster (870ns -> 406ns). This removes the memo and both indices, and looks up the closest package on every call. The parsed contents of each `package.json` are still cached per path, and `invalidate` now just drops those. The other reason to do this is incremental resolution (#1944). Any cache that sits between the resolver and the file system has to store the observations that produced each entry and replay them on a hit, or the second resolution to hit the cache can never be invalidated. That's a fair amount of machinery to add to a cache that's worth 3% of first-build resolution time. This only makes sense on top of the stack - at #1944, removing the memo costs 14% on a first pass, and nearly doubles a pass that re-resolves everything. \* Described in the test plan. ## Benchmark Replaying every `DependencyGraph.resolveDependency` call made by one iOS dev build of the benchmark app. This goes through `DependencyGraph`'s whole-resolution cache, which is cleared only by real file change events, so each row is something that actually happens in a session. 7 interleaved rounds, medians of 15 iterations. Resolver output is byte-identical. | scenario | #1946 | this diff | |---|---|---| | first build (ms) | 92.9 | 95.6 (+2.9%) | | rebuild, nothing changed (ms) | 7.58 | 7.61 | | after a source edit, re-resolving the edited module's 8 dependencies (µs) | 155 | 155 | | after a `package.json` change, re-resolving everything (ms) | 73.6 | 80.4 (+9.3%) | | heap retained after resolving (MB) | 3.0 | **1.9 (−37%)** | The tradeoffs are: - ~3% of resolution time on a first build (3ms here). - ~9% (7ms here) when a `package.json` or symlink change resets the whole graph, which is the only time everything is re-resolved against a warm `PackageCache`. Incremental resolution aims to remove those full resets. - Nothing on an ordinary edit. Metro re-resolves only the edited module's dependencies, and the difference there is within noise. ## Test plan ``` yarn jest packages/metro-file-map packages/metro-resolver packages/metro/src/node-haste packages/metro/src/DeltaBundler packages/metro/src/integration_tests yarn flow check yarn verify-api-snapshots ``` `PackageCache-test.js` is restructured around the new contract: the closest package is looked up on every call, a change of closest package is reflected with no invalidation, each `package.json` is parsed once however many modules share it, and invalidating one re-reads it. Tests of the reverse index and the no-package set go with those structures. **Benchmark app** - a synthetic mid-size RN 0.87 app: 23 common dependencies (Reanimated, React Navigation, TanStack Query, lodash, etc.) and ~1,000 generated first-party modules, 60% of them in a workspace package outside `projectRoot` with its own `node_modules`. The iOS dev bundle is 2.8k modules, from a file map of 34k files. Timings replay the `resolveDependency` calls a real build of it makes against a fresh `DependencyGraph` per process, in interleaved rounds, with source and `package.json` edits made on disk and picked up by the watcher. Changelog: Internal
1 parent 0d622a5 commit cf3a8e7

2 files changed

Lines changed: 68 additions & 281 deletions

File tree

‎packages/metro/src/node-haste/PackageCache.js‎

Lines changed: 7 additions & 78 deletions
Original file line numberDiff line numberDiff line change
@@ -12,7 +12,7 @@
1212
import type {PackageJson} from 'metro-resolver/private/types';
1313

1414
import {readFileSync} from 'node:fs';
15-
import {dirname, sep} from 'node:path';
15+
import {dirname} from 'node:path';
1616

1717
type GetClosestPackageFn = (absoluteFilePath: string) => ?{
1818
packageJsonPath: string,
@@ -40,12 +40,6 @@ export class PackageCache {
4040
packageJson: PackageJson,
4141
},
4242
>;
43-
// Single cache: module path → pre-built result object, or null (no allocation on hit)
44-
#resultByModulePath: Map<string, PackageForModule | null>;
45-
// Reverse index for invalidation: package.json path → set of module paths
46-
#modulePathsByPackagePath: Map<string, Set<string>>;
47-
// Module paths that resolved to no package.json (null), for invalidation
48-
#modulePathsWithNoPackage: Set<string>;
4943

5044
constructor(options: {
5145
getClosestPackage: GetClosestPackageFn,
@@ -55,9 +49,6 @@ export class PackageCache {
5549
this.#getClosestPackage = options.getClosestPackage;
5650
this.#readPackageJson = options.readPackageJson ?? readPackageJsonSync;
5751
this.#packageCache = new Map();
58-
this.#resultByModulePath = new Map();
59-
this.#modulePathsByPackagePath = new Map();
60-
this.#modulePathsWithNoPackage = new Set();
6152
}
6253

6354
getPackage(filePath: string): Readonly<{
@@ -75,86 +66,24 @@ export class PackageCache {
7566
return cached;
7667
}
7768

69+
/**
70+
* The closest package is looked up on every call rather than remembered per
71+
* module path. Only the parsed contents of each `package.json` are cached.
72+
*/
7873
getPackageForModule(absoluteModulePath: string): ?PackageForModule {
79-
const cached = this.#resultByModulePath.get(absoluteModulePath);
80-
81-
// Distinguish between `null` (positively no closest package) and
82-
// `undefined` (no cached result yet)
83-
// eslint-disable-next-line lint/strictly-null
84-
if (cached !== undefined) {
85-
return cached;
86-
}
87-
8874
const closest = this.#getClosestPackage(absoluteModulePath);
8975
if (closest == null) {
90-
this.#resultByModulePath.set(absoluteModulePath, null);
91-
this.#modulePathsWithNoPackage.add(absoluteModulePath);
92-
return null;
93-
}
94-
95-
const packagePath = closest.packageJsonPath;
96-
97-
// Track module→package for invalidation
98-
let modulePaths = this.#modulePathsByPackagePath.get(packagePath);
99-
if (modulePaths == null) {
100-
modulePaths = new Set();
101-
this.#modulePathsByPackagePath.set(packagePath, modulePaths);
102-
}
103-
modulePaths.add(absoluteModulePath);
104-
105-
const pkg = this.getPackage(packagePath);
106-
if (pkg == null) {
10776
return null;
10877
}
109-
110-
// Cache the pre-built result object — no allocation on future hits
111-
const result: PackageForModule = {
78+
const pkg = this.getPackage(closest.packageJsonPath);
79+
return {
11280
packageJson: pkg.packageJson,
11381
packageRelativePath: closest.packageRelativePath,
11482
rootPath: pkg.rootPath,
11583
};
116-
this.#resultByModulePath.set(absoluteModulePath, result);
117-
return result;
11884
}
11985

12086
invalidate(filePath: string) {
12187
this.#packageCache.delete(filePath);
122-
123-
// Clean up any cached result for this module path (including null).
124-
// Derive the package.json path from the cached result to clean up the
125-
// reverse index.
126-
const cachedResult = this.#resultByModulePath.get(filePath);
127-
this.#resultByModulePath.delete(filePath);
128-
this.#modulePathsWithNoPackage.delete(filePath);
129-
130-
if (cachedResult != null) {
131-
const packagePath = cachedResult.rootPath + sep + 'package.json';
132-
const modules = this.#modulePathsByPackagePath.get(packagePath);
133-
if (modules != null) {
134-
modules.delete(filePath);
135-
if (modules.size === 0) {
136-
this.#modulePathsByPackagePath.delete(packagePath);
137-
}
138-
}
139-
}
140-
141-
// If filePath is a package.json, invalidate all module lookups pointing to it
142-
const modulePaths = this.#modulePathsByPackagePath.get(filePath);
143-
if (modulePaths != null) {
144-
for (const modulePath of modulePaths) {
145-
this.#resultByModulePath.delete(modulePath);
146-
}
147-
this.#modulePathsByPackagePath.delete(filePath);
148-
}
149-
150-
// If a package.json was created, modified, or deleted, invalidate all
151-
// null-cached module results, since modules that previously had no
152-
// enclosing package.json may now resolve to this one.
153-
if (filePath.endsWith(sep + 'package.json')) {
154-
for (const modulePath of this.#modulePathsWithNoPackage) {
155-
this.#resultByModulePath.delete(modulePath);
156-
}
157-
this.#modulePathsWithNoPackage.clear();
158-
}
15988
}
16089
}

0 commit comments

Comments
 (0)