Repository navigation
Commit cf3a8e7
committed
Remove PackageCache's per-module closest package memo
Stacked on #1946.
## Summary
`PackageCache` remembers the closest `package.json` for every module path it's asked about, with a reverse index from each `package.json` to the module paths under it and a set of paths known to have no package, both of which exist only to invalidate the first. `DependencyGraph` calls `invalidate` for every changed file to keep it honest.
With the previous two PRs that memo no longer pays for itself. #1945 cut closest-package lookups 3.7x (64k -> 17k per build on our mid-size RN benchmark app*), and #1946 made each one a walk of single map lookups, 2.1x faster (870ns -> 406ns). This removes the memo and both indices, and looks up the closest package on every call. The parsed contents of each `package.json` are still cached per path, and `invalidate` now just drops those.
The other reason to do this is incremental resolution (#1944). Any cache that sits between the resolver and the file system has to store the observations that produced each entry and replay them on a hit, or the second resolution to hit the cache can never be invalidated. That's a fair amount of machinery to add to a cache that's worth 3% of first-build resolution time.
This only makes sense on top of the stack - at #1944, removing the memo costs 14% on a first pass, and nearly doubles a pass that re-resolves everything.
\* Described in the test plan.
## Benchmark
Replaying every `DependencyGraph.resolveDependency` call made by one iOS dev build of the benchmark app. This goes through `DependencyGraph`'s whole-resolution cache, which is cleared only by real file change events, so each row is something that actually happens in a session. 7 interleaved rounds, medians of 15 iterations. Resolver output is byte-identical.
| scenario | #1946 | this diff |
|---|---|---|
| first build (ms) | 92.9 | 95.6 (+2.9%) |
| rebuild, nothing changed (ms) | 7.58 | 7.61 |
| after a source edit, re-resolving the edited module's 8 dependencies (µs) | 155 | 155 |
| after a `package.json` change, re-resolving everything (ms) | 73.6 | 80.4 (+9.3%) |
| heap retained after resolving (MB) | 3.0 | **1.9 (−37%)** |
The tradeoffs are:
- ~3% of resolution time on a first build (3ms here).
- ~9% (7ms here) when a `package.json` or symlink change resets the whole graph, which is the only time everything is re-resolved against a warm `PackageCache`. Incremental resolution aims to remove those full resets.
- Nothing on an ordinary edit. Metro re-resolves only the edited module's dependencies, and the difference there is within noise.
## Test plan
```
yarn jest packages/metro-file-map packages/metro-resolver packages/metro/src/node-haste packages/metro/src/DeltaBundler packages/metro/src/integration_tests
yarn flow check
yarn verify-api-snapshots
```
`PackageCache-test.js` is restructured around the new contract: the closest package is looked up on every call, a change of closest package is reflected with no invalidation, each `package.json` is parsed once however many modules share it, and invalidating one re-reads it. Tests of the reverse index and the no-package set go with those structures.
**Benchmark app** - a synthetic mid-size RN 0.87 app: 23 common dependencies (Reanimated, React Navigation, TanStack Query, lodash, etc.) and ~1,000 generated first-party modules, 60% of them in a workspace package outside `projectRoot` with its own `node_modules`. The iOS dev bundle is 2.8k modules, from a file map of 34k files. Timings replay the `resolveDependency` calls a real build of it makes against a fresh `DependencyGraph` per process, in interleaved rounds, with source and `package.json` edits made on disk and picked up by the watcher.
Changelog: Internal1 parent 0d622a5 commit cf3a8e7
2 files changed
Lines changed: 68 additions & 281 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
12 | 12 | | |
13 | 13 | | |
14 | 14 | | |
15 | | - | |
| 15 | + | |
16 | 16 | | |
17 | 17 | | |
18 | 18 | | |
| |||
40 | 40 | | |
41 | 41 | | |
42 | 42 | | |
43 | | - | |
44 | | - | |
45 | | - | |
46 | | - | |
47 | | - | |
48 | | - | |
49 | 43 | | |
50 | 44 | | |
51 | 45 | | |
| |||
55 | 49 | | |
56 | 50 | | |
57 | 51 | | |
58 | | - | |
59 | | - | |
60 | | - | |
61 | 52 | | |
62 | 53 | | |
63 | 54 | | |
| |||
75 | 66 | | |
76 | 67 | | |
77 | 68 | | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
78 | 73 | | |
79 | | - | |
80 | | - | |
81 | | - | |
82 | | - | |
83 | | - | |
84 | | - | |
85 | | - | |
86 | | - | |
87 | | - | |
88 | 74 | | |
89 | 75 | | |
90 | | - | |
91 | | - | |
92 | | - | |
93 | | - | |
94 | | - | |
95 | | - | |
96 | | - | |
97 | | - | |
98 | | - | |
99 | | - | |
100 | | - | |
101 | | - | |
102 | | - | |
103 | | - | |
104 | | - | |
105 | | - | |
106 | | - | |
107 | 76 | | |
108 | 77 | | |
109 | | - | |
110 | | - | |
111 | | - | |
| 78 | + | |
| 79 | + | |
112 | 80 | | |
113 | 81 | | |
114 | 82 | | |
115 | 83 | | |
116 | | - | |
117 | | - | |
118 | 84 | | |
119 | 85 | | |
120 | 86 | | |
121 | 87 | | |
122 | | - | |
123 | | - | |
124 | | - | |
125 | | - | |
126 | | - | |
127 | | - | |
128 | | - | |
129 | | - | |
130 | | - | |
131 | | - | |
132 | | - | |
133 | | - | |
134 | | - | |
135 | | - | |
136 | | - | |
137 | | - | |
138 | | - | |
139 | | - | |
140 | | - | |
141 | | - | |
142 | | - | |
143 | | - | |
144 | | - | |
145 | | - | |
146 | | - | |
147 | | - | |
148 | | - | |
149 | | - | |
150 | | - | |
151 | | - | |
152 | | - | |
153 | | - | |
154 | | - | |
155 | | - | |
156 | | - | |
157 | | - | |
158 | | - | |
159 | 88 | | |
160 | 89 | | |
0 commit comments