Skip to content

Add system table for cache policies and enable local disk paging - #6443

Open
joshua-spacetime wants to merge 3 commits into
kim/tiered-storage-iofrom
joshua/cache-policies
Open

joshua-spacetime wants to merge 3 commits into
kim/tiered-storage-iofrom
joshua/cache-policies

Conversation

@joshua-spacetime

Copy link
Copy Markdown
Collaborator

Description of Changes

Add st_table_cache_policy to persist each table’s caching policy and enable the local disk paging implemented in #6352. Tables default to remaining in memory, preserving existing behavior. Changes include:

  • Add st_table_cache_policy to store each table’s policy. Tables without an entry default to remaining in memory.
  • Apply policies when creating tables and loading a database. System tables and blobs remain in memory.

API and ABI breaking changes

N/A

Rollback safety impact

N/A

Expected complexity level and risk

3

Testing

TODO

Comment thread crates/engine/src/persistence.rs Resolved
Comment thread crates/table/src/page.rs Resolved
/// Allow tables to evict pages according to their cache policy.
pub evict_pages: bool,
/// Allow blob eviction. Disable this when tables need their blobs to stay in
/// memory: blobs can be shared, and their references do not track table policies.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about this: we make ResidentObjects::reserve_blob take an eviction policy, derived from the policy of the blob's insertion table, and store it in the BlobFrame directly.

pub local_store: BackingStore,
pub remote_store: Option<BackingStore>,
/// Allow tables to evict pages according to their cache policy.
pub evict_pages: bool,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's a bit unclear what this is for.
In particular, the combination evict_pages: true, evict_blobs: false does not seem useful, except maybe for testing.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is just here because I wanted to keep the scope of this change small and so I punted on blob eviction.

@kim
kim removed this pull request from stack #6360 November 3, 2026 07:03

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants