Store each thumbnail as a single 400×225 PNG and stop using thumbnail_encoding #6351

Overview

contentnode.thumbnail_encoding holds about 355 GB of base64 thumbnails in TOAST. 98.7% of them are a plain resize of the node's thumbnail File. About 98K are user crops, and only 107 nodes have been re-cropped in a later session since November 2022. Each thumbnail should instead be stored once, as a 400×225 PNG File with transparent letterboxing, and thumbnail_encoding should no longer be used.

Complexity: High
Target branch: hotfixes

Context

• In a 0.1% production sample of 7,644 nodes with an encoding: 98.7% are {"points": [], "zoom": 0} with a resize of the node's thumbnail File, 1.2% are Croppa crops, and 0.04% are crops from the older points/zoom cropper.

• Change history since November 2022. A re-crop is two crop changes on the same node with different crop settings and no thumbnail upload in between:

  Measure	Count
  Updates that set a thumbnail encoding	44,033 (42,628 with crop settings)
  Nodes cropped by a user	39,271
  Thumbnail uploads attached to those nodes	39,178
  Nodes re-cropped without a new upload	707 (1.8%)
  …re-cropped in a later session	107 (0.27%)

• About 0.1% of nodes with an encoding have no thumbnail File, so their base64 is the only copy.

• Thumbnail Files: 10,853,488 rows referencing 289,745 distinct images, 56.9 GB in storage.

• thumbnail_encoding is written by:

  • the cropper: ContentNodeThumbnail.vue:467
  • ricecooker uploads: utils/nodes.py:103
  • publish: utils/publish.py:596-636 builds a resized copy from it, or regenerates and saves it when empty, then exports the copy as a new unattached File.

• Display reads encoding.base64 before the File URL: Thumbnail.vue:156, StudioThumbnail.vue:106, ImageOnlyThumbnail.vue:117.

The Change

• Every thumbnail File should be a 400×225 PNG: the image scaled down to fit, or left at its own size if smaller, and centered on a transparent background.
• Thumbnails added in the editor should be converted in the browser before upload. A user's crop is what gets uploaded, and the original image is not kept.
• A management command, run from make deploy-migrate (per Makefile:32-40), should:
  • convert each distinct thumbnail image once, and update every File row that references it in place, keeping its id;
  • for nodes with a user crop, or whose base64 is the only copy, convert that base64 into the node's thumbnail File.
• Publish should export the node's thumbnail File without processing the image, except for channels whose ricecooker_version is below 0.9.0, whose thumbnails publish should still convert.
• No code should read or write thumbnail_encoding. Display should use the thumbnail File's URL.

Out of Scope

• Dropping the thumbnail_encoding column, which will be a follow-up issue.
• Channel thumbnails (Channel.thumbnail_encoding).
• Converting thumbnails in ricecooker.
• Deleting the original thumbnail images from storage (56.9 GB, about $18 a year).

Acceptance Criteria

General

• Adding a thumbnail in the editor, with or without a crop, uploads only a 400×225 PNG with transparent letterboxing, and it becomes the node's thumbnail File
• After the command runs on hotfixes, every thumbnail File references a 400×225 PNG and keeps its id
• For a sample of nodes with a user crop, the converted thumbnail shows the same crop as the previous base64
• Every node whose base64 was the only copy has a thumbnail File after the command
• Publishing a channel with no ricecooker_version, or 0.9.0 or later, creates no new thumbnail File rows
• Publishing a channel with ricecooker_version below 0.9.0 exports 400×225 PNG thumbnails
• Apart from the model field and migrations, no code reads or writes thumbnail_encoding

Testing

• Unit tests cover converting landscape, portrait, and smaller-than-400×225 images, and check that the letterbox is transparent
• Running the command again after it completes changes nothing

AI usage

Drafted with Claude Code, which measured thumbnail encodings, thumbnail File counts and re-crop history on production through read-only queries in the disposable debugging job, and traced the code paths that read and write thumbnail_encoding. I chose the design (one stored 400×225 thumbnail, no originals kept), the letterboxing, the ricecooker cutoff, and the scope.