Skip to content

fix(core): keep interned string indexes in a Map - #3559

Open
EloB wants to merge 1 commit into
glideapps:masterfrom
EloB:fix/compressed-json-string-map
Open

EloB wants to merge 1 commit into
glideapps:masterfrom
EloB:fix/compressed-json-string-map

Conversation

@EloB

@EloB EloB commented Nov 16, 2026

Copy link
Copy Markdown

Description

CompressedJSON._stringIndexes is now a Map<string, number> instead of
a plain object. internString and hashCode use it through
get/set/keys. That's the whole change (one file).

Related Issue

Fixes #3558

Motivation and Context

Once a plain object has 2^23 (8,388,608) properties, V8 renumbers and
re-sorts all of them on every new property, about a second each. Inputs
with more distinct short strings than that stall while being read. A
Map has no such limit. Details and a reproduction are in the issue.

A side effect: the string "__proto__" is now interned once.
obj["__proto__"] = index on a plain object sets the prototype instead
of adding a property, so it used to get a new index every time. The
output doesn't change.

Previous Behaviour / Output

A JSON array of 9,000,000 distinct strings (the script in the issue):
no output; still running when stopped after 120 s. Scryfall's 3 GB
all-cards file: still running after two hours.

New Behaviour / Output

The 9,000,000-string file: export type Strings = string[]; in 18.6 s
(peak 3.0 GiB). The 3 GB file: done in 152 s with
--max-old-space-size=49152 (peak 15.7 GiB).

How Has This Been Tested?

  • npm run build, npx vitest run (54 files, 241 tests pass) and
    npx biome check packages/quicktype-core/src/input/CompressedJSON.ts.
  • The reproduction above, before and after.
  • Old against new output through quicktype-core: every JSON input in
    test/inputs/json/{samples,misc,priority} plus one with "__proto__"
    values (221 inputs), each as TypeScript, JSON Schema, Go, Python and
    C#: 1,105 runs, none differ.

Screenshots (if appropriate):

n/a


Written by Claude (an AI assistant made by Anthropic) and posted on behalf of the account owner (EloB). The reproduction, fix and tests were checked by running them as described above.

🤖 Generated with Claude Code

CompressedJSON kept the index of each interned string in a plain object.
Once an object has 2^23 (8,388,608) properties, V8 renumbers and
re-sorts all of them on every new one, about a second each, so an input
with more distinct short strings than that stalls: a JSON array of
9,000,000 distinct strings doesn't finish, while 8,000,000 take 16 s.
A Map has no such limit.

A side effect: the string "__proto__" is now interned once. Assigning
it to a plain object set the prototype instead of adding a property,
so it got a new index every time. The output doesn't change.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BUG]: quicktype stalls on JSON with more than 8,388,608 distinct strings

1 participant