diff --git a/CHANGELOG.md b/CHANGELOG.md index c23f0b3..ee4a640 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -20,6 +20,40 @@ pre-1.0, so a minor version may contain compatibility changes. integration tests in addition to compiling every package. The required check keeps its existing `windows-compile` name. This is not a full Desktop GUI or power-loss qualification. +- Codex-compressed rollouts are folded too. Codex now compresses every rollout + idle for seven days to a sibling `rollout-*.jsonl.zst` (zstd level 3) and + deletes the `.jsonl`, which pulled sessions out of the store one by one: on + a real machine 1,288 managed sessions left this way, the saved figure fell + from 69% to 38%, and 3,623 `.zst` files held 24.2 GB that CodexFold never + touched. The service now folds each settled `.zst` in place: it proves that + libzstd 1.5.7 (the version Codex links, built in through + `github.com/DataDog/zstd`) re-encodes the decoded content to the file's exact + bytes, folds the content into the store, writes a marker attribute and + releases the file's data blocks with a hole punch. Path, inode, size, mode, + times and other attributes stay. The mount re-encodes the original bytes on + read, refuses writes, hides the marker, and fails with EIO rather than serve + the empty blocks if it cannot. All 3,623 real files reproduced byte for byte. + On the same 3,280 sessions the store needs 12.5 GB where Codex's per-file + zstd needs 18.2 GB, because forks and subagents repeat long prefixes. + A dry run prices every file first, and a file is folded only when that + costs under 80% of its `.zst`: a rollout that shares nothing with the store + folds larger than Codex's own zstd (one real 205 MB rollout: 83.5 MB folded + against a 68.6 MB `.zst`), so it stays as it is. Folding follows the + automatic-folding policy and never runs alongside an enrollment cycle. Manifests whose file Codex deleted are released only after + two scans fifteen minutes apart; a file that lost its marker is found again by + inode and re-marked instead. +- `fs native-zst status` reports folded and unfolded files. `fs native-zst + restore --all --apply` writes the original bytes back; run it, with + automatic folding disabled, before installing a CodexFold build without this + support, which would otherwise serve the empty blocks. + +### Changed + +- The storage figures (`storage.json`, the saved percentage) now count the + native session tree as well as the store: logical bytes are all session + content, decoded, and physical bytes are everything on disk. Before, native + `.jsonl` and `.zst` files were left out, which hid how much space Codex's + sessions really used. ### Fixed @@ -65,6 +99,63 @@ pre-1.0, so a minor version may contain compatibility changes. repaired is logged and served unchanged instead of aborting startup. - `fs validate-native --repair` applies the same repair by hand while the filesystem service is stopped. Without the flag the command stays read-only. +- Deleting sessions now shrinks the store. Maintenance only rebuilt the pack + when a top-level manifest was newer than it, so a session Codex or the user + deleted kept its objects in the pack indefinitely. A manifest the published + pack still carries but the store no longer has now triggers a compacting + build, and nested manifests count as new. +- Folding works on a nearly full disk. Every operation must leave the 5 GiB + free-space reserve at its peak, and space it frees only counts afterwards, + so pessimistic peaks blocked the very work that frees space: + - A Codex `.zst` is folded straight from its decoder instead of staging its + decoded content (up to 2.7 GB for one real rollout). When space is short a + dry run prices the fold exactly, so a file whose content the store already + holds costs almost nothing. Such a fold may run below the reserve, down to + 1 GiB free, when its peak is under 64 MiB and it frees at least four times + that; files that do not fit yet are retried, not given up. + - A pack build no longer copies the whole pack when objects were dropped. + It rewrites only the old pack files whose dead and raw blocks make it + worthwhile, best payoff per copied byte first and only as many as fit the + free space, and hard-links the rest. A file with no live block is dropped + without copying anything. Dead pack bytes trigger another build once there + is room to make progress. + - Session enrollment priced every rollout at three times its raw size, so + on a tight disk a large session was never enrolled even when it was a + fork whose content the store already held. When that price does not fit, + the planner now asks for the exact price: a dry-run fold that writes + nothing, cached by file identity and pack generation, charged twice (loose + objects and the next pack). `codexfold fold --apply` measures the same way + when space is short, and pack builds price loose objects by their + compressed size on disk instead of their raw size. +- Loose objects are compressed at the pack builder's level through one shared + encoder instead of a new encoder, with its own worker goroutines and + window, per object. +- Pack builds entropy-code small blocks. `SpeedDefault` gave up on blocks whose + only redundancy is entropy (base64 payloads, encrypted reasoning) and a 2:1 + cut-off kept other small blocks raw: 7.07 GB of a real 14.83 GB pack. Builds + use `SpeedBetterCompression`, keep a block raw only when compression does not + shrink it, and recompress raw blocks during compaction; about 2.1 GB of that + pack goes away. Decoding cost is unchanged. +- Mount reads no longer fail with "Too many open files" while several folded + `.zst` files are read at once. The FSKit extension ran at launchd's + 256-descriptor limit, and a folded file used about fourteen of them because + it was handed a shared read window that primed nine prefetch channels on + open. Folded files now get no shared window. The extension raises its + descriptor limit at start (1,200 files held open through a real mount put + 1,438 descriptors on it without an error) and caps prefetch channels at 48 + across all open files. The extension change needs the App build 117 install, + not just a daemon live update. +- A folded `.zst` from which APFS kept a few extents allocated was re-proved + and re-punched every thirty seconds. It is now left alone until it changes, + and only bytes actually released are counted as freed. +- A managed session's native snapshot left behind after deletion no longer + blocks readiness. The mount hides that path on purpose, but the canonical + readiness probe waited for it. As a result, automatic enrollment reported + `mount-warming` for every session (1,626 on a real machine) and enrolled + nothing, and `fs service start` failed its health gate. The probe now skips + the routes of deletion tombstones. A failed native FSKit start also left the + mount attached with no backend, so every read in it failed; it now reclaims + the mount the way an explicit stop does. ## [0.4.0-beta.2] - 2026-10-01 diff --git a/go.mod b/go.mod index 90b117e..19f503a 100644 --- a/go.mod +++ b/go.mod @@ -3,6 +3,7 @@ module github.com/samekind/codexfold go 1.26 require ( + github.com/DataDog/zstd v1.5.7 github.com/Microsoft/go-winio v0.6.2 github.com/jchv/go-webview2 v0.0.0-20260205173254-56598839c808 github.com/klauspost/compress v1.19.0 diff --git a/go.sum b/go.sum index ad88398..17d7268 100644 --- a/go.sum +++ b/go.sum @@ -1,3 +1,5 @@ +github.com/DataDog/zstd v1.5.7 h1:ybO8RBeh29qrxIhCA9E8gKY6xfONU9T6G6aP9DTKfLE= +github.com/DataDog/zstd v1.5.7/go.mod h1:g4AWEaM3yOg3HYfnJ3YIawPnVdXJh9QME85blwSAmyw= github.com/Microsoft/go-winio v0.6.2 h1:F2VQgta7ecxGYO8k3ZZz3RS8fVIXVxONVUPlNERoyfY= github.com/Microsoft/go-winio v0.6.2/go.mod h1:yd8OoFMLzJbo9gZq8j5qaps8bJ9aShtEA8Ipt1oGCvU= github.com/cpuguy83/go-md2man/v2 v2.0.6/go.mod h1:oOW0eioCTA6cOiMLiUPZOpcVxMig6NIQQ7OS05n1F4g= diff --git a/internal/cli/fold.go b/internal/cli/fold.go index 84f77c4..e9b47b5 100644 --- a/internal/cli/fold.go +++ b/internal/cli/fold.go @@ -54,6 +54,17 @@ func newFoldCommand() *cobra.Command { defer recordIndex.Close() options.RecordIndex = recordIndex } + // On a nearly full disk the default price, every source byte new, + // rejects folds that add little. Measure what this one adds first. + if options.Apply && options.NewBytesEstimate == 0 { + if info, err := os.Stat(session.RolloutPath); err == nil && foldSpaceIsTight(options.StoreDir, info.Size()) { + newBytes, err := dryRunFoldBytes(command.Context(), options.StoreDir, toFoldSession(session), options) + if err != nil { + return err + } + options.NewBytesEstimate = max(newBytes, 1) + } + } result, err := fold.Fold(command.Context(), toFoldSession(session), options) if err != nil { return err diff --git a/internal/cli/fs.go b/internal/cli/fs.go index ac276c2..919146b 100644 --- a/internal/cli/fs.go +++ b/internal/cli/fs.go @@ -21,6 +21,7 @@ import ( "github.com/samekind/codexfold/internal/cdc" "github.com/samekind/codexfold/internal/codex" + "github.com/samekind/codexfold/internal/codexzstd" "github.com/samekind/codexfold/internal/compat" "github.com/samekind/codexfold/internal/enroll" "github.com/samekind/codexfold/internal/fold" @@ -28,6 +29,7 @@ import ( "github.com/samekind/codexfold/internal/fskitproto" "github.com/samekind/codexfold/internal/fskitstatus" "github.com/samekind/codexfold/internal/mountfs" + "github.com/samekind/codexfold/internal/nativezst" "github.com/samekind/codexfold/internal/pack" "github.com/samekind/codexfold/internal/scan" "github.com/samekind/codexfold/internal/service" @@ -121,6 +123,7 @@ func newFSCommand() *cobra.Command { command.AddCommand(newFSStatusCommand()) command.AddCommand(newFSDoctorCommand()) command.AddCommand(newFSValidateNativeCommand()) + command.AddCommand(newFSNativeZstCommand()) command.AddCommand(newFSCompatibilityCommand()) command.AddCommand(newFSCompatibilityImportCommand()) command.AddCommand(newFSBenchmarkCommand()) @@ -718,9 +721,15 @@ func newFSServeCommand() *cobra.Command { } } filesystem := mountfs.New() + var nativeZstSource *nativezst.Source if canonicalNamespace { filesystem = mountfs.NewCanonical() filesystem.SetNativeRoot(nativeRoot) + // Serve Codex .zst files folded into the store; a build that + // cannot re-encode them reports EIO, never their empty blocks. + nativeZstSource = nativezst.NewSource(store, storePacks{}) + defer nativeZstSource.Close() + filesystem.SetNativeZstSource(nativeZstSource) if err := filesystem.SetManagedStore(store); err != nil { return fmt.Errorf("recover managed file names: %w", err) } @@ -1328,6 +1337,10 @@ func newFSServeCommand() *cobra.Command { if runtime.GOOS == "windows" { storageStatusPath = filepath.Join(store, "fs", "status", "storage.json") } + storageScan := storage.Scan + if canonicalNamespace { + storageScan = withNativeFootprint(nativeRoot, storage.Scan) + } storageStatusDone = startStorageStatusReporter( ctx, command.ErrOrStderr(), @@ -1336,7 +1349,7 @@ func newFSServeCommand() *cobra.Command { 10*time.Minute, storageMaintenanceDone, storageStatusRefresh, - storage.Scan, + storageScan, fskitstatus.Write, ) activityCounter = &mountfs.IOActivityCounter{} @@ -1345,6 +1358,17 @@ func newFSServeCommand() *cobra.Command { namespaceMaintenanceDone := startNamespaceRetirement(ctx, filesystem, store, func() bool { return namespaceFoldingEnabled(flags, enrollmentInterval, runtime.GOOS == "windows") }, command.ErrOrStderr()) + var nativeZstDone <-chan struct{} + if nativeZstSource != nil && codexzstd.Supported() { + nativeZstDone = startNativeZstFolding(ctx, filesystem, nativeZstSource, store, func() bool { + return namespaceFoldingEnabled(flags, enrollmentInterval, false) + }, func() { + select { + case storageStatusRefresh <- struct{}{}: + default: + } + }, command.ErrOrStderr()) + } watcherDone := make(chan struct{}) go func() { defer close(watcherDone) @@ -1380,6 +1404,9 @@ func newFSServeCommand() *cobra.Command { } <-runtimeMemoryMaintenanceDone <-namespaceMaintenanceDone + if nativeZstDone != nil { + <-nativeZstDone + } <-enrollmentDone var nativeWatcherErr error if nativeWatcherDone != nil { diff --git a/internal/cli/fs_enroll.go b/internal/cli/fs_enroll.go index 28ff665..f09804b 100644 --- a/internal/cli/fs_enroll.go +++ b/internal/cli/fs_enroll.go @@ -584,6 +584,9 @@ func runPeriodicEnrollment(ctx context.Context, flags enrollmentFlags, interval } } }() + // Folding Codex .zst files adds manifests; one appearing between this + // cycle's pack build and its loose retirement fails the pack-only proof. + storeMaintenanceMu.Lock() result, err := runServiceEnrollmentCycle(cycleCtx, cycleFlags, enrollmentApplyHooks{ onManagedCount: func(count int) { progressMu.Lock() @@ -607,6 +610,7 @@ func runPeriodicEnrollment(ctx context.Context, flags enrollmentFlags, interval return !resolveEnrollmentControl(flags, interval).Enabled }, }) + storeMaintenanceMu.Unlock() cycleCancel() <-watchDone if report != nil { @@ -987,7 +991,7 @@ func buildEnrollmentPlan(ctx context.Context, flags enrollmentFlags) (enroll.Pla mountHealthy := mountHealthProbe(mount) == nil readiness := canonicalNamespaceReadiness{} if flags.canonicalNamespace && mountHealthy { - readiness = enrollmentCanonicalNamespaceReadinessProbe(home, mount, nativeRoot) + readiness = enrollmentCanonicalNamespaceReadinessProbe(home, store, mount, nativeRoot) } guard, err := storage.DefaultGuard(store) if err != nil { @@ -1015,6 +1019,7 @@ func buildEnrollmentPlan(ctx context.Context, flags enrollmentFlags) (enroll.Pla // occurs between those decisions. Reuse one inventory for this plan; // fold/pack/apply retain their fresh per-operation budget checks. Budget: &storage.OnceInventoryGuard{Guard: guard}, + Price: enrollmentDryRunPricer(store), } plan, err := enroll.Build(ctx, input) if err != nil || len(plan.Selected) == 0 { @@ -1310,25 +1315,80 @@ func enrollmentHasManifestsNewerThanPack(store string) (bool, error) { return false, err } packPresent := err == nil - entries, err := os.ReadDir(filepath.Join(store, "manifests")) - if errors.Is(err, os.ErrNotExist) { - return false, nil - } - if err != nil { - return false, err - } - for _, entry := range entries { + // Manifests below manifests/ (folded Codex .zst files) are live too. + pending := false + live := make(map[string]struct{}) + err = filepath.WalkDir(filepath.Join(store, "manifests"), func(path string, entry fs.DirEntry, walkErr error) error { + if walkErr != nil { + return walkErr + } if entry.IsDir() || filepath.Ext(entry.Name()) != ".json" { - continue + return nil } + relative, err := filepath.Rel(store, path) + if err != nil { + return err + } + live[filepath.ToSlash(relative)] = struct{}{} if !packPresent { - return true, nil + pending = true + return nil } info, err := entry.Info() if err != nil { - return false, err + return err } if info.ModTime().After(current.ModTime()) { + pending = true + } + return nil + }) + if errors.Is(err, os.ErrNotExist) { + return false, nil + } + if err != nil || pending || !packPresent { + return pending, err + } + // A manifest the pack still carries but the store no longer has (a + // deleted session, a released .zst) leaves its objects in the pack until + // a build compacts them away. Without this nothing ever repacks after + // deletions alone. + if dropped, err := packHasDroppedManifests(store, live); err != nil || dropped { + return dropped, err + } + return packWasteWorthCompacting(store), nil +} + +// packWasteWorthCompacting reports dead pack bytes that a build could now +// free. On a nearly full disk a build rewrites only what fits, so the rest +// is picked up by later cycles once space allows, not retried in vain. +func packWasteWorthCompacting(store string) bool { + files, wasted, err := pack.GenerationWaste(store) + if err != nil || wasted < max(int64(512<<20), files/10) { + return false + } + limits, err := storage.LoadLimits(store) + if err != nil { + return false + } + available, err := storage.AvailableBytes(store) + return err == nil && available-limits.FreeSpaceReserveBytes >= 1<<30 +} + +func packHasDroppedManifests(store string, live map[string]struct{}) (bool, error) { + generation, err := pack.CurrentGeneration(store) + if err != nil { + return false, nil + } + catalog, err := pack.ReadRecoveryCatalog(filepath.Join(store, "packs", generation)) + if err != nil { + return false, nil + } + for _, file := range catalog.Files { + if !strings.HasPrefix(file.Path, "manifests/") { + continue + } + if _, ok := live[file.Path]; !ok { return true, nil } } diff --git a/internal/cli/fs_enroll_price.go b/internal/cli/fs_enroll_price.go new file mode 100644 index 0000000..4e638d0 --- /dev/null +++ b/internal/cli/fs_enroll_price.go @@ -0,0 +1,113 @@ +package cli + +import ( + "context" + "encoding/json" + "errors" + "os" + "path/filepath" + "sync" + + "github.com/samekind/codexfold/internal/codex" + "github.com/samekind/codexfold/internal/fold" + "github.com/samekind/codexfold/internal/pack" + "github.com/samekind/codexfold/internal/storage" +) + +// A dry-run fold prices a session exactly: it reads and compresses every +// object but writes nothing. Prices are cached by file identity and pack +// generation, since only those change what a fold would add. + +type enrollmentPrice struct { + Path string `json:"path"` + Size int64 `json:"size"` + ModTimeUnixNano int64 `json:"mod_time_unix_nano"` + Generation string `json:"generation"` + NewBytes int64 `json:"new_bytes"` +} + +var enrollmentPriceMu sync.Mutex + +func enrollmentPricePath(store string) string { + return filepath.Join(filepath.Clean(store), "enrollment", "prices.json") +} + +func enrollmentDryRunPricer(store string) func(context.Context, codex.Session, os.FileInfo) (int64, error) { + return func(ctx context.Context, session codex.Session, info os.FileInfo) (int64, error) { + generation, _ := pack.CurrentGeneration(store) + enrollmentPriceMu.Lock() + defer enrollmentPriceMu.Unlock() + prices := make(map[string]enrollmentPrice) + if data, err := os.ReadFile(enrollmentPricePath(store)); err == nil { + _ = json.Unmarshal(data, &prices) + } + if cached, ok := prices[session.ID]; ok && cached.Path == session.RolloutPath && cached.Size == info.Size() && + cached.ModTimeUnixNano == info.ModTime().UnixNano() && cached.Generation == generation { + return cached.NewBytes, nil + } + newBytes, err := dryRunFoldBytes(ctx, store, toFoldSession(session), fold.FoldOptions{}) + if err != nil { + return 0, err + } + prices[session.ID] = enrollmentPrice{ + Path: session.RolloutPath, Size: info.Size(), ModTimeUnixNano: info.ModTime().UnixNano(), + Generation: generation, NewBytes: newBytes, + } + data, err := json.Marshal(prices) + if err == nil { + _ = writeFileAtomically(enrollmentPricePath(store), data) + } + return newBytes, nil + } +} + +// dryRunFoldBytes reports what folding session would add to store. +func dryRunFoldBytes(ctx context.Context, store string, session fold.Session, options fold.FoldOptions) (int64, error) { + options.StoreDir = store + options.Apply, options.Overwrite, options.RemoveSource = false, false, false + if options.ExistingReader == nil { + if resolver, err := pack.Open(store, pack.OpenOptions{CacheBytes: -1}); err == nil { + defer resolver.Close() + options.ExistingReader = resolver + } + } + result, err := fold.Fold(ctx, session, options) + if err != nil { + return 0, err + } + return result.NewStoredBytes, nil +} + +// foldSpaceIsTight reports whether pricing a fold at its full source size +// would leave less than a comfortable margin above the free-space reserve. +var foldSpaceIsTight = func(store string, sourceBytes int64) bool { + limits, err := storage.LoadLimits(store) + if err != nil { + return false + } + available, err := storage.AvailableBytes(store) + if err != nil { + return false + } + return available < limits.FreeSpaceReserveBytes+sourceBytes+sourceBytes/16+(2<<30) +} + +func writeFileAtomically(path string, data []byte) error { + if err := os.MkdirAll(filepath.Dir(path), 0o700); err != nil { + return err + } + temporary, err := os.CreateTemp(filepath.Dir(path), ".prices-*.tmp") + if err != nil { + return err + } + name := temporary.Name() + defer os.Remove(name) + if _, err := temporary.Write(data); err != nil { + _ = temporary.Close() + return err + } + if err := temporary.Close(); err != nil { + return err + } + return errors.Join(os.Rename(name, path)) +} diff --git a/internal/cli/fs_enroll_price_test.go b/internal/cli/fs_enroll_price_test.go new file mode 100644 index 0000000..456db67 --- /dev/null +++ b/internal/cli/fs_enroll_price_test.go @@ -0,0 +1,67 @@ +package cli + +import ( + "bytes" + "context" + "encoding/json" + "os" + "path/filepath" + "testing" + "time" + + "github.com/samekind/codexfold/internal/codex" + "github.com/samekind/codexfold/internal/fold" +) + +func TestEnrollmentDryRunPricerMeasuresAndCaches(t *testing.T) { + store := t.TempDir() + content := bytes.Repeat([]byte("{\"type\":\"response_item\",\"text\":\"a long shared prefix that every fork repeats\"}\n"), 20000) + foldFixtureManifest(t, store, "original", "", content) + fork := filepath.Join(t.TempDir(), "rollout-fork.jsonl") + if err := os.WriteFile(fork, content, 0o600); err != nil { + t.Fatal(err) + } + info, err := os.Stat(fork) + if err != nil { + t.Fatal(err) + } + price := enrollmentDryRunPricer(store) + session := codex.Session{ID: "fork", RolloutPath: fork, Archived: true} + newBytes, err := price(context.Background(), session, info) + if err != nil { + t.Fatal(err) + } + if newBytes != 0 { + t.Fatalf("fork of stored content priced at %d new bytes", newBytes) + } + if _, err := os.Stat(fold.ManifestPath(store, "fork")); !os.IsNotExist(err) { + t.Fatalf("pricing wrote a manifest: %v", err) + } + // A cached price is reused for the same file identity. + prices := map[string]enrollmentPrice{} + data, err := os.ReadFile(enrollmentPricePath(store)) + if err != nil { + t.Fatal(err) + } + if err := json.Unmarshal(data, &prices); err != nil { + t.Fatal(err) + } + cached := prices["fork"] + cached.NewBytes = 12345 + prices["fork"] = cached + data, _ = json.Marshal(prices) + if err := os.WriteFile(enrollmentPricePath(store), data, 0o600); err != nil { + t.Fatal(err) + } + if again, err := price(context.Background(), session, info); err != nil || again != 12345 { + t.Fatalf("cached price = %d, %v", again, err) + } + later := time.Now().Add(time.Minute) + if err := os.Chtimes(fork, later, later); err != nil { + t.Fatal(err) + } + info, _ = os.Stat(fork) + if again, err := price(context.Background(), session, info); err != nil || again != 0 { + t.Fatalf("changed file price = %d, %v", again, err) + } +} diff --git a/internal/cli/fs_namespace_readiness.go b/internal/cli/fs_namespace_readiness.go index e414f81..14c38b3 100644 --- a/internal/cli/fs_namespace_readiness.go +++ b/internal/cli/fs_namespace_readiness.go @@ -7,9 +7,11 @@ import ( "io/fs" "os" "path/filepath" + "strings" "time" "github.com/samekind/codexfold/internal/sessionns" + "github.com/samekind/codexfold/internal/vfs" ) type canonicalNamespaceReadiness struct { @@ -24,26 +26,51 @@ var enrollmentCanonicalNamespaceReadinessProbe = probeEnrollmentCanonicalNamespa var waitForCanonicalNamespaceActivation = waitForCanonicalNamespacePassthrough -func probeEnrollmentCanonicalNamespaceReadiness(home string, mount string, nativeRoot string) canonicalNamespaceReadiness { +func probeEnrollmentCanonicalNamespaceReadiness(home string, store string, mount string, nativeRoot string) canonicalNamespaceReadiness { status, err := sessionns.Inspect(sessionns.Options{Home: home, Mount: mount, NativeRoot: nativeRoot, MountProbe: mountHealthProbe}) if err != nil || !status.Active { return canonicalNamespaceReadiness{} } - if err := probeCanonicalNativePassthrough(mount, nativeRoot); err != nil { + hidden := deletedSessionRoutes(store) + if err := probeCanonicalNativePassthrough(mount, nativeRoot, hidden); err != nil { return canonicalNamespaceReadiness{Active: true} } - if err := probeCanonicalNativePassthrough(home, nativeRoot); err != nil { + if err := probeCanonicalNativePassthrough(home, nativeRoot, hidden); err != nil { return canonicalNamespaceReadiness{Active: true} } return canonicalNamespaceReadiness{Active: true, Ready: true} } +// deletedSessionRoutes returns the namespace paths of managed sessions that +// were deleted. A deleted session's native snapshot can stay behind at its +// route, and the mount deliberately hides it, so readiness must not wait for +// it to appear. Without this, one such leftover kept automatic enrollment at +// "mount warming" forever and made every service start roll back. +func deletedSessionRoutes(store string) map[string]struct{} { + if store == "" { + return nil + } + deletions, err := vfs.DiscoverSessionDeletions(store) + if err != nil || len(deletions) == 0 { + return nil + } + routes := make(map[string]struct{}, len(deletions)) + for _, deletion := range deletions { + route := strings.TrimPrefix(filepath.ToSlash(filepath.Clean(deletion.Route)), "/") + if route != "" && route != "." { + routes[route] = struct{}{} + } + } + return routes +} + // probeCanonicalNativePassthrough verifies that every unmanaged native entry // currently has a corresponding path in the mounted namespace. It checks // visibility and kind only, not content, so active appends cannot turn a -// readiness check into a false mismatch. -func probeCanonicalNativePassthrough(mount string, nativeRoot string) error { - return probeCanonicalNativePassthroughMetadata(mount, nativeRoot, false) +// readiness check into a false mismatch. Paths in hidden, relative to the +// namespace root, are skipped. +func probeCanonicalNativePassthrough(mount string, nativeRoot string, hidden map[string]struct{}) error { + return probeCanonicalNativePassthroughMetadata(mount, nativeRoot, false, hidden) } func waitForCanonicalNativePassthrough(ctx context.Context, mount string, nativeRoot string, timeout time.Duration) error { @@ -57,9 +84,9 @@ func waitForCanonicalNamespacePassthrough(ctx context.Context, home string, moun deadline := time.Now().Add(timeout) var lastErr error for { - lastErr = probeCanonicalNativePassthroughMetadata(mount, nativeRoot, true) + lastErr = probeCanonicalNativePassthroughMetadata(mount, nativeRoot, true, nil) if lastErr == nil && home != "" { - lastErr = probeCanonicalNativePassthroughMetadata(home, nativeRoot, true) + lastErr = probeCanonicalNativePassthroughMetadata(home, nativeRoot, true, nil) } if lastErr == nil { return nil @@ -75,7 +102,7 @@ func waitForCanonicalNamespacePassthrough(ctx context.Context, home string, moun } } -func probeCanonicalNativePassthroughMetadata(mount string, nativeRoot string, requireSize bool) error { +func probeCanonicalNativePassthroughMetadata(mount string, nativeRoot string, requireSize bool, hidden map[string]struct{}) error { mount = filepath.Clean(mount) nativeRoot = filepath.Clean(nativeRoot) for _, namespace := range []string{"sessions", "archived_sessions"} { @@ -105,6 +132,9 @@ func probeCanonicalNativePassthroughMetadata(mount string, nativeRoot string, re if !info.IsDir() && !info.Mode().IsRegular() { return nil } + if _, skip := hidden[namespace+"/"+filepath.ToSlash(relative)]; skip && !info.IsDir() { + return nil + } mountedPath := filepath.Join(mountedBase, relative) mounted, err := os.Lstat(mountedPath) if err != nil { diff --git a/internal/cli/fs_native_zst.go b/internal/cli/fs_native_zst.go new file mode 100644 index 0000000..58d1c1a --- /dev/null +++ b/internal/cli/fs_native_zst.go @@ -0,0 +1,384 @@ +package cli + +import ( + "context" + "errors" + "fmt" + "io" + "os" + "path/filepath" + "sort" + "sync" + "time" + + "github.com/samekind/codexfold/internal/codex" + "github.com/samekind/codexfold/internal/codexzstd" + "github.com/samekind/codexfold/internal/enroll" + "github.com/samekind/codexfold/internal/mountfs" + "github.com/samekind/codexfold/internal/nativezst" + "github.com/samekind/codexfold/internal/pack" + "github.com/samekind/codexfold/internal/storage" + "github.com/spf13/cobra" +) + +// storeMaintenanceMu serializes the service enrollment cycle with folding of +// Codex .zst files: a manifest that appears between the cycle's pack build +// and its loose retirement would fail the pack-only proof. +var storeMaintenanceMu sync.Mutex + +const ( + nativeZstPassInterval = 30 * time.Second + nativeZstPassBudget = 2 * time.Minute + // Codex renames a finished .zst into place; leave it alone for a while + // after its last metadata change. + nativeZstSettle = 10 * time.Minute + nativeZstMinimumBytes = 16 << 10 + nativeZstSweepEvery = 10 * time.Minute +) + +type nativeZstFolder struct { + filesystem *mountfs.Filesystem + source *nativezst.Source + store string + diagnostics io.Writer + refresh func() + skipped map[string]nativeZstIdentity + lastSweep time.Time + lastError string +} + +type nativeZstIdentity struct { + size int64 + modTime time.Time +} + +func startNativeZstFolding(ctx context.Context, filesystem *mountfs.Filesystem, source *nativezst.Source, store string, enabled func() bool, refresh func(), diagnostics io.Writer) <-chan struct{} { + done := make(chan struct{}) + folder := &nativeZstFolder{ + filesystem: filesystem, source: source, store: store, diagnostics: diagnostics, refresh: refresh, + skipped: make(map[string]nativeZstIdentity), + } + go func() { + defer close(done) + ticker := time.NewTicker(nativeZstPassInterval) + defer ticker.Stop() + for { + select { + case <-ctx.Done(): + return + case <-ticker.C: + } + if !enabled() || !storeMaintenanceMu.TryLock() { + continue + } + err := folder.pass(ctx) + storeMaintenanceMu.Unlock() + folder.report(err) + } + }() + return done +} + +func (z *nativeZstFolder) report(err error) { + if err == nil || errors.Is(err, context.Canceled) || errors.Is(err, storage.ErrOperationLockHeld) { + z.lastError = "" + return + } + if err.Error() != z.lastError { + _, _ = fmt.Fprintf(z.diagnostics, "fold Codex .zst files: %v\n", err) + z.lastError = err.Error() + } +} + +func (z *nativeZstFolder) pass(ctx context.Context) error { + scan, err := z.filesystem.ScanNativeZst(ctx) + if err != nil { + return err + } + maintenance, err := storage.BeginMaintenance(ctx, z.store) + if err != nil { + return err + } + defer maintenance.Close() + work := maintenance.Context() + + var converted, finished, waiting int + var freed, stored int64 + for _, file := range scan.Folded { + identity := nativeZstIdentity{size: file.Info.Size(), modTime: file.Info.ModTime()} + if nativezst.DataReleased(file.Info) || z.skipped[file.Path] == identity { + continue + } + err := z.filesystem.FinishNativeZst(work, z.source, file) + if err == nil { + // APFS may keep some extents of a hole-punched file allocated, for + // one real 56 MB file 580 KiB, and every later pass then re-proved + // and re-punched it. Count only what was freed, and leave a file + // whose blocks did not move alone until it changes. + after, statErr := os.Lstat(file.Path) + if statErr != nil { + continue + } + released := nativezst.AllocatedBytes(file.Info) - nativezst.AllocatedBytes(after) + if released > 0 { + finished++ + freed += released + } + if !nativezst.DataReleased(after) { + z.skipped[file.Path] = nativeZstIdentity{size: after.Size(), modTime: after.ModTime()} + } + continue + } + if work.Err() != nil { + return maintenance.YieldError(work.Err()) + } + if !errors.Is(err, mountfs.ErrNativeZstBusy) { + z.skipped[file.Path] = identity + _, _ = fmt.Fprintf(z.diagnostics, "finish folding Codex .zst %s: %v\n", file.Path, err) + } + } + candidates := make([]nativezst.NativeFile, 0, len(scan.Candidates)) + settledBefore := time.Now().Add(-nativeZstSettle) + for _, file := range scan.Candidates { + identity := nativeZstIdentity{size: file.Info.Size(), modTime: file.Info.ModTime()} + if skipped, ok := z.skipped[file.Path]; ok && skipped == identity { + continue + } + if file.Info.Size() < nativeZstMinimumBytes || nativezst.ChangedAfter(file.Info, settledBefore) { + continue + } + candidates = append(candidates, file) + } + // The largest files free the most space per pass. + sort.Slice(candidates, func(i, j int) bool { return candidates[i].Info.Size() > candidates[j].Info.Size() }) + deadline := time.Now().Add(nativeZstPassBudget) + var passErr error + for _, file := range candidates { + if time.Now().After(deadline) { + break + } + prepared, err := nativezst.Prepare(work, z.source, file.Path) + if err != nil { + if work.Err() != nil { + passErr = maintenance.YieldError(work.Err()) + break + } + if errors.Is(err, storage.ErrOperationLockHeld) { + break + } + if errors.Is(err, nativezst.ErrChanged) || errors.Is(err, os.ErrNotExist) { + continue + } + // A later, cheaper file may still fit; this one is retried once + // folding has freed some space. + if errors.Is(err, storage.ErrBudgetExceeded) { + waiting++ + continue + } + z.skipped[file.Path] = nativeZstIdentity{size: file.Info.Size(), modTime: file.Info.ModTime()} + if !errors.Is(err, codexzstd.ErrNotReproducible) && !errors.Is(err, nativezst.ErrNotWorthwhile) { + _, _ = fmt.Fprintf(z.diagnostics, "fold Codex .zst %s: %v\n", file.Path, err) + } + continue + } + if err := z.filesystem.ConvertNativeZst(z.store, prepared); err != nil { + if !errors.Is(err, nativezst.ErrMarkerKept) { + nativezst.Discard(prepared) + } + if !errors.Is(err, mountfs.ErrNativeZstBusy) && !errors.Is(err, nativezst.ErrChanged) { + _, _ = fmt.Fprintf(z.diagnostics, "convert Codex .zst %s: %v\n", file.Path, err) + } + continue + } + converted++ + freed += nativezst.AllocatedBytes(file.Info) + stored += prepared.NewStoredBytes + } + if converted+finished > 0 || waiting > 0 { + _, _ = fmt.Fprintf(z.diagnostics, "%s folded %d Codex .zst file(s): freed %s, stored %s new; %d waiting for disk space\n", + time.Now().UTC().Format(time.RFC3339Nano), converted+finished, formatBytes(freed), formatBytes(stored), waiting) + } + if converted+finished > 0 { + if z.refresh != nil { + z.refresh() + } + } + if passErr == nil && time.Since(z.lastSweep) >= nativeZstSweepEvery { + result, err := nativezst.Sweep(work, z.store, scan, time.Now()) + if err != nil { + return errors.Join(passErr, err) + } + z.lastSweep = time.Now() + if result.Released+result.Repaired > 0 { + _, _ = fmt.Fprintf(z.diagnostics, "%s Codex .zst sweep: released %d manifest(s), repaired %d marker(s)\n", + time.Now().UTC().Format(time.RFC3339Nano), result.Released, result.Repaired) + } + } + return passErr +} + +// withNativeFootprint adds the native session tree to the storage figures, so +// the saved percentage compares all session content with everything on disk. +func withNativeFootprint(nativeRoot string, scan func(context.Context, storage.Options) (storage.Inventory, error)) func(context.Context, storage.Options) (storage.Inventory, error) { + return func(ctx context.Context, options storage.Options) (storage.Inventory, error) { + inventory, err := scan(ctx, options) + if err != nil || nativeRoot == "" { + return inventory, err + } + native, nativeErr := nativezst.ScanNative(ctx, nativeRoot) + if nativeErr != nil { + return inventory, nil + } + inventory.LogicalSessionBytes += native.Footprint.LogicalBytes + inventory.TotalPhysicalBytes += native.Footprint.PhysicalBytes + return inventory, nil + } +} + +type FSNativeZstStatus struct { + NativeRoot string `json:"native_root"` + Supported bool `json:"supported"` + Footprint nativezst.Footprint `json:"footprint"` + Manifests int `json:"manifests"` + Restored int `json:"restored,omitempty"` + Failures []string `json:"failures,omitempty"` + Sweep *nativezst.SweepResult `json:"sweep,omitempty"` +} + +func newFSNativeZstCommand() *cobra.Command { + command := &cobra.Command{ + Use: "native-zst", + Short: "Inspect or restore Codex-compressed rollouts folded into the store", + } + command.AddCommand(newFSNativeZstStatusCommand(), newFSNativeZstRestoreCommand()) + return command +} + +func resolveNativeZstPaths(codexHome string, storeDir string, nativeRoot string) (string, string, error) { + home, err := codex.ResolveHome(codexHome) + if err != nil { + return "", "", err + } + if nativeRoot == "" { + nativeRoot = filepath.Join(home, "fold-native") + } + return resolveFoldStore(home, storeDir), filepath.Clean(nativeRoot), nil +} + +func newFSNativeZstStatusCommand() *cobra.Command { + var codexHome, storeDir, nativeRoot string + var jsonOutput bool + command := &cobra.Command{ + Use: "status", + Short: "Report folded and unfolded Codex .zst rollouts", + Args: cobra.NoArgs, + RunE: func(command *cobra.Command, _ []string) error { + store, root, err := resolveNativeZstPaths(codexHome, storeDir, nativeRoot) + if err != nil { + return err + } + scan, err := nativezst.ScanNative(command.Context(), root) + if err != nil { + return err + } + entries, _ := os.ReadDir(nativezst.ManifestRoot(store)) + status := FSNativeZstStatus{NativeRoot: root, Supported: codexzstd.Supported(), Footprint: scan.Footprint, Manifests: len(entries)} + if jsonOutput { + return writeJSON(command, status) + } + footprint := status.Footprint + _, err = fmt.Fprintf(command.OutOrStdout(), "supported=%t folded=%d (%s encoded, %s decoded) unfolded=%d (%s) manifests=%d native_on_disk=%s\n", + status.Supported, footprint.FoldedFiles, formatBytes(footprint.FoldedEncodedBytes), formatBytes(footprint.FoldedLogicalBytes), + footprint.CompressedFiles, formatBytes(footprint.CompressedBytes), status.Manifests, formatBytes(footprint.PhysicalBytes)) + return err + }, + } + command.Flags().StringVar(&codexHome, "codex-home", "", "Codex home directory; defaults to CODEX_HOME or ~/.codex") + command.Flags().StringVar(&storeDir, "store", "", "Fold store directory; defaults to /fold-store") + command.Flags().StringVar(&nativeRoot, "native-root", "", "Native rollout root; defaults to /fold-native") + command.Flags().BoolVar(&jsonOutput, "json", false, "Emit JSON output") + return command +} + +func newFSNativeZstRestoreCommand() *cobra.Command { + var codexHome, storeDir, nativeRoot string + var all, apply, jsonOutput bool + command := &cobra.Command{ + Use: "restore [path...]", + Short: "Write folded Codex .zst files back as real files; required before downgrading CodexFold", + Long: "Restore re-encodes each folded .zst from the store, writes the exact original bytes back in place and removes " + + "its marker. Disable automatic folding first, or the service folds the files again. Dry-run unless --apply.", + RunE: func(command *cobra.Command, args []string) error { + if all == (len(args) != 0) { + return errors.New("name folded files or pass --all") + } + store, root, err := resolveNativeZstPaths(codexHome, storeDir, nativeRoot) + if err != nil { + return err + } + // The service would release the blocks of a file being restored, + // taking it for a conversion interrupted after its marker. + if control, err := enroll.LoadControl(enroll.ControlPath(store)); err != nil { + return err + } else if control.Enabled && apply { + return errors.New("automatic folding is enabled; turn it off before restoring folded .zst files") + } + paths := args + if all { + scan, err := nativezst.ScanNative(command.Context(), root) + if err != nil { + return err + } + paths = paths[:0] + for _, file := range scan.Folded { + paths = append(paths, file.Path) + } + } + status := FSNativeZstStatus{NativeRoot: root, Supported: codexzstd.Supported()} + if apply { + source := nativezst.NewSource(store, storePacks{}) + defer source.Close() + for _, path := range paths { + if err := nativezst.Restore(command.Context(), source, path); err != nil { + status.Failures = append(status.Failures, fmt.Sprintf("%s: %v", path, err)) + continue + } + status.Restored++ + } + } + if jsonOutput { + if err := writeJSON(command, status); err != nil { + return err + } + } else if _, err := fmt.Fprintf(command.OutOrStdout(), "candidates=%d restored=%d failed=%d dry_run=%t\n", len(paths), status.Restored, len(status.Failures), !apply); err != nil { + return err + } + if len(status.Failures) != 0 { + return fmt.Errorf("%d folded file(s) could not be restored", len(status.Failures)) + } + return nil + }, + } + command.Flags().StringVar(&codexHome, "codex-home", "", "Codex home directory; defaults to CODEX_HOME or ~/.codex") + command.Flags().StringVar(&storeDir, "store", "", "Fold store directory; defaults to /fold-store") + command.Flags().StringVar(&nativeRoot, "native-root", "", "Native rollout root; defaults to /fold-native") + command.Flags().BoolVar(&all, "all", false, "Restore every folded file under the native root") + command.Flags().BoolVar(&apply, "apply", false, "Write the files; omitted means dry-run") + command.Flags().BoolVar(&jsonOutput, "json", false, "Emit JSON output") + return command +} + +// storePacks gives the native .zst source the store's published packs. +type storePacks struct{} + +func (storePacks) CurrentGeneration(store string) (string, error) { + return pack.CurrentGeneration(store) +} + +func (storePacks) Open(store string) (nativezst.Pack, error) { + resolver, err := pack.Open(store, pack.OpenOptions{}) + if err != nil { + return nil, err + } + return resolver, nil +} diff --git a/internal/cli/fs_native_zst_test.go b/internal/cli/fs_native_zst_test.go new file mode 100644 index 0000000..9741f9b --- /dev/null +++ b/internal/cli/fs_native_zst_test.go @@ -0,0 +1,99 @@ +package cli + +import ( + "bytes" + "context" + "io" + "os" + "path/filepath" + "strings" + "testing" + "time" + + "github.com/samekind/codexfold/internal/enroll" + "github.com/samekind/codexfold/internal/fold" + "github.com/samekind/codexfold/internal/pack" +) + +func foldFixtureManifest(t *testing.T, store string, id string, manifestPath string, content []byte) { + t.Helper() + source := filepath.Join(t.TempDir(), id+".jsonl") + if err := os.WriteFile(source, content, 0o600); err != nil { + t.Fatal(err) + } + options := fold.FoldOptions{StoreDir: store, Apply: true} + if manifestPath != "" { + if err := os.MkdirAll(filepath.Dir(manifestPath), 0o755); err != nil { + t.Fatal(err) + } + options.ManifestPathOverride = manifestPath + } + if _, err := fold.Fold(context.Background(), fold.Session{ID: id, RolloutPath: source, Archived: true}, options); err != nil { + t.Fatal(err) + } +} + +func TestPackPendingSeesNestedAndDroppedManifests(t *testing.T) { + store := t.TempDir() + foldFixtureManifest(t, store, "session-a", "", bytes.Repeat([]byte("{\"a\":\"one\"}\n"), 400)) + foldFixtureManifest(t, store, "session-b", "", bytes.Repeat([]byte("{\"b\":\"two\"}\n"), 400)) + if _, err := pack.Build(context.Background(), store, pack.BuildOptions{}); err != nil { + t.Fatal(err) + } + past := time.Now().Add(-time.Hour) + for _, id := range []string{"session-a", "session-b"} { + if err := os.Chtimes(fold.ManifestPath(store, id), past, past); err != nil { + t.Fatal(err) + } + } + if pending, err := enrollmentHasManifestsNewerThanPack(store); err != nil || pending { + t.Fatalf("fresh pack pending=%t err=%v", pending, err) + } + + nested := filepath.Join(store, "manifests", "native-zst", "zst-000000000000000000000000-1.json") + foldFixtureManifest(t, store, "zst-000000000000000000000000-1", nested, bytes.Repeat([]byte("{\"c\":\"three\"}\n"), 400)) + if pending, err := enrollmentHasManifestsNewerThanPack(store); err != nil || !pending { + t.Fatalf("nested manifest pending=%t err=%v", pending, err) + } + if _, err := pack.Build(context.Background(), store, pack.BuildOptions{}); err != nil { + t.Fatal(err) + } + if err := os.Chtimes(nested, past, past); err != nil { + t.Fatal(err) + } + if pending, err := enrollmentHasManifestsNewerThanPack(store); err != nil || pending { + t.Fatalf("after rebuild pending=%t err=%v", pending, err) + } + + // A deleted session leaves its objects in the pack until a compaction. + if err := os.Remove(fold.ManifestPath(store, "session-b")); err != nil { + t.Fatal(err) + } + if pending, err := enrollmentHasManifestsNewerThanPack(store); err != nil || !pending { + t.Fatalf("dropped manifest pending=%t err=%v", pending, err) + } +} + +func TestNativeZstRestoreRefusesWhileFoldingIsEnabled(t *testing.T) { + home := t.TempDir() + store := filepath.Join(home, "fold-store") + if err := enroll.SaveControl(enroll.ControlPath(store), enroll.Control{Enabled: true, Interval: 30 * time.Minute, StableFor: time.Hour}); err != nil { + t.Fatal(err) + } + command := newFSNativeZstRestoreCommand() + command.SetArgs([]string{"--codex-home", home, "--all", "--apply"}) + command.SetOut(io.Discard) + command.SetErr(io.Discard) + if err := command.Execute(); err == nil || !strings.Contains(err.Error(), "automatic folding is enabled") { + t.Fatalf("restore with folding enabled = %v", err) + } + if err := enroll.SaveControl(enroll.ControlPath(store), enroll.Control{Enabled: false, Interval: 30 * time.Minute, StableFor: time.Hour}); err != nil { + t.Fatal(err) + } + command = newFSNativeZstRestoreCommand() + command.SetArgs([]string{"--codex-home", home, "--all", "--apply"}) + command.SetOut(io.Discard) + if err := command.Execute(); err != nil { + t.Fatalf("restore with folding disabled = %v", err) + } +} diff --git a/internal/cli/fs_service.go b/internal/cli/fs_service.go index a4c719a..54f8b32 100644 --- a/internal/cli/fs_service.go +++ b/internal/cli/fs_service.go @@ -908,15 +908,14 @@ func startPlatformService(ctx context.Context, platform service.Platform, defini _ = manager.Bootout(ctx, definitionPath) return err } + // By now the supervisor has mounted. Stopping the jobs alone leaves + // that mount behind with no backend, and every read in it fails; + // a failed start reclaims it exactly as an explicit stop does. if err := waitLaunchdNativeFSKitHealthy(ctx, manager, label, definitionPath, mountPoint, nativeFSKitStartupTimeout); err != nil { - _ = manager.Bootout(ctx, supervisorDefinition) - _ = manager.Bootout(ctx, definitionPath) - return err + return errors.Join(err, stopPlatformService(ctx, service.PlatformLaunchd, definitionPath)) } if err := verifyServiceBuild(service.PlatformLaunchd, definitionPath, mountPoint); err != nil { - _ = manager.Bootout(ctx, supervisorDefinition) - _ = manager.Bootout(ctx, definitionPath) - return err + return errors.Join(err, stopPlatformService(ctx, service.PlatformLaunchd, definitionPath)) } return nil } @@ -1138,6 +1137,10 @@ func waitLaunchdNativeFSKitHealthy(ctx context.Context, manager service.Manager, if err != nil { return err } + var hidden map[string]struct{} + if store, err := service.DefinitionStore(service.PlatformLaunchd, definitionPath); err == nil { + hidden = deletedSessionRoutes(store) + } deadline := time.NewTimer(timeout) defer deadline.Stop() ticker := time.NewTicker(50 * time.Millisecond) @@ -1149,7 +1152,7 @@ func waitLaunchdNativeFSKitHealthy(ctx context.Context, manager service.Manager, daemon = validateLaunchdChildProcess(manager.Status(ctx, label, mountPoint), lockPaths.daemon, "daemon") supervisor = validateLaunchdChildProcess(manager.Status(ctx, supervisorLabel, mountPoint), lockPaths.supervisor, "supervisor") if daemon.DaemonRunning && supervisor.DaemonRunning && daemon.MountHealthy { - passthroughErr = probeCanonicalNativePassthrough(mountPoint, nativeRoot) + passthroughErr = probeCanonicalNativePassthrough(mountPoint, nativeRoot, hidden) if passthroughErr == nil { return nil } diff --git a/internal/cli/fs_test.go b/internal/cli/fs_test.go index 721e42d..197a980 100644 --- a/internal/cli/fs_test.go +++ b/internal/cli/fs_test.go @@ -5073,7 +5073,7 @@ func allowFixtureMount(t *testing.T) { func allowEnrollmentNamespaceReadiness(t *testing.T) { t.Helper() previous := enrollmentCanonicalNamespaceReadinessProbe - enrollmentCanonicalNamespaceReadinessProbe = func(string, string, string) canonicalNamespaceReadiness { + enrollmentCanonicalNamespaceReadinessProbe = func(string, string, string, string) canonicalNamespaceReadiness { return canonicalNamespaceReadiness{Active: true, Ready: true} } t.Cleanup(func() { enrollmentCanonicalNamespaceReadinessProbe = previous }) @@ -5094,18 +5094,65 @@ func TestCanonicalNativePassthroughProbeRequiresEveryNativeRoute(t *testing.T) { if err := os.WriteFile(nativeFile, []byte("native\n"), 0o600); err != nil { t.Fatal(err) } - if err := probeCanonicalNativePassthrough(mount, nativeRoot); err == nil { + if err := probeCanonicalNativePassthrough(mount, nativeRoot, nil); err == nil { t.Fatal("missing mounted native route passed readiness") } mountedFile := filepath.Join(mount, "sessions", "2026", "07", "unmanaged.jsonl") if err := os.WriteFile(mountedFile, []byte("mounted\n"), 0o600); err != nil { t.Fatal(err) } - if err := probeCanonicalNativePassthrough(mount, nativeRoot); err != nil { + if err := probeCanonicalNativePassthrough(mount, nativeRoot, nil); err != nil { t.Fatalf("complete native passthrough failed readiness: %v", err) } } +// A deleted managed session's native snapshot can stay at its route while the +// mount hides it. Readiness must not wait for that path, or enrollment stays +// at "mount warming" and every service start rolls back. +func TestCanonicalNativePassthroughProbeSkipsDeletedSessionRoutes(t *testing.T) { + fixture := interruptedCanonicalMigrationFixture(t) + if err := fixture.resolver.Close(); err != nil { + t.Fatal(err) + } + route := "/archived_sessions/rollout-session.jsonl" + if _, err := vfs.PublishSessionDeletion(fixture.store, fixture.managed.State(), route); err != nil { + t.Fatal(err) + } + hidden := deletedSessionRoutes(fixture.store) + if _, ok := hidden["archived_sessions/rollout-session.jsonl"]; !ok || len(hidden) != 1 { + t.Fatalf("deleted routes = %#v", hidden) + } + + root := t.TempDir() + mount := filepath.Join(root, "mount") + nativeRoot := filepath.Join(root, "native") + for _, base := range []string{mount, nativeRoot} { + for _, namespace := range []string{"sessions", "archived_sessions"} { + if err := os.MkdirAll(filepath.Join(base, namespace), 0o700); err != nil { + t.Fatal(err) + } + } + } + if err := os.WriteFile(filepath.Join(nativeRoot, "archived_sessions", "rollout-session.jsonl"), []byte("leftover\n"), 0o600); err != nil { + t.Fatal(err) + } + if err := probeCanonicalNativePassthrough(mount, nativeRoot, hidden); err != nil { + t.Fatalf("hidden deleted route failed readiness: %v", err) + } + if err := probeCanonicalNativePassthroughMetadata(mount, nativeRoot, true, hidden); err != nil { + t.Fatalf("hidden deleted route failed sized readiness: %v", err) + } + if err := os.WriteFile(filepath.Join(nativeRoot, "sessions", "other.jsonl"), []byte("visible\n"), 0o600); err != nil { + t.Fatal(err) + } + if err := probeCanonicalNativePassthrough(mount, nativeRoot, hidden); err == nil { + t.Fatal("a missing route that was never deleted passed readiness") + } + if got := deletedSessionRoutes(""); got != nil { + t.Fatalf("empty store routes = %#v", got) + } +} + func TestRetainCanonicalSnapshotReusesAnIdenticalStagedSnapshot(t *testing.T) { storeDir := t.TempDir() nativePath := filepath.Join(t.TempDir(), "native.jsonl") diff --git a/internal/codexzstd/codexzstd.go b/internal/codexzstd/codexzstd.go new file mode 100644 index 0000000..9ecae47 --- /dev/null +++ b/internal/codexzstd/codexzstd.go @@ -0,0 +1,201 @@ +// Package codexzstd reproduces the zstd frames Codex writes when it compresses +// an idle rollout, byte for byte. +// +// Codex (codex-rs/rollout/src/compression.rs) streams the rollout through +// libzstd 1.5.7 at level 3 with the source size pledged and no checksum. The +// same library and parameters produce the same bytes, so a folded rollout can +// be served back as its original .zst without keeping the compressed copy. +// Go's own zstd encoders produce different frames and cannot be used here. +package codexzstd + +import ( + "bytes" + "context" + "crypto/sha256" + "encoding/hex" + "errors" + "fmt" + "hash" + "io" + "os" + + "github.com/klauspost/compress/zstd" +) + +// Profile names the exact encoder recipe. A folded .zst records it, and is +// only ever served by a build whose Profile matches. +const Profile = "libzstd-1.5.7-level3-pledged-nochecksum" + +var ErrUnsupported = errors.New("exact Codex zstd encoding is unavailable in this build") + +// ErrNotReproducible means the file is a valid zstd stream that this encoder +// does not reproduce exactly, so it must stay as it is. +var ErrNotReproducible = errors.New("zstd file is not reproducible by the Codex encoder profile") + +// Proof describes a .zst whose bytes were reproduced from its decoded content. +type Proof struct { + Profile string `json:"profile"` + EncodedBytes int64 `json:"encoded_bytes"` + EncodedSHA256 string `json:"encoded_sha256"` + ContentBytes int64 `json:"content_bytes"` + ContentSHA256 string `json:"content_sha256"` +} + +// Encode writes the Codex frame for content, which must be exactly size bytes. +func Encode(ctx context.Context, w io.Writer, content io.Reader, size int64) error { + encoder, err := newEncoder(w, size) + if err != nil { + return err + } + buffer := make([]byte, 128<<10) + var copied int64 + for { + if err := ctx.Err(); err != nil { + encoder.abort() + return err + } + n, readErr := content.Read(buffer) + if n > 0 { + copied += int64(n) + if copied > size { + encoder.abort() + return fmt.Errorf("zstd content exceeds pledged size %d", size) + } + if _, err := encoder.Write(buffer[:n]); err != nil { + encoder.abort() + return err + } + } + if errors.Is(readErr, io.EOF) { + break + } + if readErr != nil { + encoder.abort() + return readErr + } + } + if copied != size { + encoder.abort() + return fmt.Errorf("zstd content is %d bytes, pledged %d", copied, size) + } + return encoder.Close() +} + +// NewReader streams the Codex frame for content. Closing it stops the encoder. +func NewReader(ctx context.Context, content io.Reader, size int64) io.ReadCloser { + reader, writer := io.Pipe() + ctx, cancel := context.WithCancel(ctx) + go func() { + writer.CloseWithError(Encode(ctx, writer, content, size)) + }() + return &pipeReader{PipeReader: reader, cancel: cancel} +} + +type pipeReader struct { + *io.PipeReader + cancel context.CancelFunc +} + +func (r *pipeReader) Close() error { + r.cancel() + return r.PipeReader.Close() +} + +// ContentSize reads the decoded size a single-frame zstd file declares. +func ContentSize(file io.ReaderAt) (int64, error) { + header := make([]byte, zstd.HeaderMaxSize) + n, err := file.ReadAt(header, 0) + if err != nil && !errors.Is(err, io.EOF) { + return 0, err + } + var parsed zstd.Header + if err := parsed.Decode(header[:n]); err != nil { + return 0, fmt.Errorf("%w: %v", ErrNotReproducible, err) + } + if parsed.Skippable || !parsed.HasFCS || parsed.HasCheckSum || parsed.DictionaryID != 0 { + return 0, ErrNotReproducible + } + if parsed.FrameContentSize > uint64(1<<62) { + return 0, ErrNotReproducible + } + return int64(parsed.FrameContentSize), nil +} + +// Verify proves that path is exactly the Codex frame of its own content. It +// decodes the file, re-encodes the content and compares every byte, without +// holding either copy in memory. +func Verify(ctx context.Context, path string) (Proof, error) { + if !Supported() { + return Proof{}, ErrUnsupported + } + file, err := os.Open(path) + if err != nil { + return Proof{}, err + } + defer file.Close() + info, err := file.Stat() + if err != nil { + return Proof{}, err + } + if !info.Mode().IsRegular() { + return Proof{}, fmt.Errorf("%s is not a regular file", path) + } + size, err := ContentSize(file) + if err != nil { + return Proof{}, err + } + decoder, err := zstd.NewReader(io.NewSectionReader(file, 0, info.Size()), zstd.WithDecoderConcurrency(1), zstd.WithDecoderLowmem(true), zstd.WithDecoderMaxWindow(1<<31)) + if err != nil { + return Proof{}, err + } + defer decoder.Close() + contentHash := sha256.New() + original := &matchingWriter{source: io.NewSectionReader(file, 0, info.Size()), hash: sha256.New()} + if err := Encode(ctx, original, io.TeeReader(decoder, contentHash), size); err != nil { + if errors.Is(err, errMismatch) { + return Proof{}, ErrNotReproducible + } + return Proof{}, fmt.Errorf("re-encode %s: %w", path, err) + } + if original.offset != info.Size() { + return Proof{}, ErrNotReproducible + } + after, err := os.Stat(path) + if err != nil { + return Proof{}, err + } + if after.Size() != info.Size() || !after.ModTime().Equal(info.ModTime()) { + return Proof{}, fmt.Errorf("%s changed during verification", path) + } + return Proof{ + Profile: Profile, EncodedBytes: info.Size(), EncodedSHA256: hex.EncodeToString(original.hash.Sum(nil)), + ContentBytes: size, ContentSHA256: hex.EncodeToString(contentHash.Sum(nil)), + }, nil +} + +var errMismatch = errors.New("re-encoded zstd bytes differ") + +// matchingWriter compares what the encoder emits with the original file. +type matchingWriter struct { + source *io.SectionReader + hash hash.Hash + offset int64 + buffer []byte +} + +func (w *matchingWriter) Write(p []byte) (int, error) { + if cap(w.buffer) < len(p) { + w.buffer = make([]byte, len(p)) + } + original := w.buffer[:len(p)] + n, err := w.source.ReadAt(original, w.offset) + if n != len(p) || !bytes.Equal(original, p) { + if err != nil && !errors.Is(err, io.EOF) { + return 0, err + } + return 0, errMismatch + } + _, _ = w.hash.Write(p) + w.offset += int64(n) + return len(p), nil +} diff --git a/internal/codexzstd/codexzstd_test.go b/internal/codexzstd/codexzstd_test.go new file mode 100644 index 0000000..7a47876 --- /dev/null +++ b/internal/codexzstd/codexzstd_test.go @@ -0,0 +1,208 @@ +package codexzstd + +import ( + "bytes" + "context" + "crypto/sha256" + "encoding/hex" + "errors" + "fmt" + "io" + "math/rand" + "os" + "os/exec" + "path/filepath" + "strings" + "testing" + + "github.com/klauspost/compress/zstd" +) + +// goldenSHA256 is `zstd -3 --single-thread --no-check` (CLI v1.5.7) of +// rolloutFixture(). A different libzstd version or parameter changes it, and +// a build that changes it can no longer serve folded Codex .zst files. +const goldenSHA256 = "22984b02b80a7949f1d916063dbfd16303449a8d218c07a0e14250840ff23dec" + +func rolloutFixture() []byte { + random := rand.New(rand.NewSource(20261102)) + var out bytes.Buffer + for line := 0; out.Len() < 900<<10; line++ { + payload := make([]byte, 40+random.Intn(400)) + for i := range payload { + payload[i] = "abcdefghijklmnopqrstuvwxyz0123456789 "[random.Intn(37)] + } + fmt.Fprintf(&out, `{"timestamp":"2026-10-%02dT08:%02d:%02d.000Z","type":"response_item","payload":{"type":"message","text":%q}}`+"\n", + 1+line%28, line%60, (line*7)%60, payload) + } + return out.Bytes() +} + +func encodeBytes(t *testing.T, content []byte) []byte { + t.Helper() + var out bytes.Buffer + if err := Encode(context.Background(), &out, bytes.NewReader(content), int64(len(content))); err != nil { + t.Fatal(err) + } + return out.Bytes() +} + +func TestEncodeMatchesPinnedCodexFrame(t *testing.T) { + if !Supported() { + t.Skip("exact encoder needs cgo") + } + content := rolloutFixture() + encoded := encodeBytes(t, content) + digest := sha256.Sum256(encoded) + if got := hex.EncodeToString(digest[:]); got != goldenSHA256 { + t.Fatalf("encoded SHA-256 = %s, want %s", got, goldenSHA256) + } + decoded, err := zstd.NewReader(nil) + if err != nil { + t.Fatal(err) + } + defer decoded.Close() + roundTrip, err := decoded.DecodeAll(encoded, nil) + if err != nil || !bytes.Equal(roundTrip, content) { + t.Fatalf("round trip failed: %v", err) + } + if size, err := ContentSize(bytes.NewReader(encoded)); err != nil || size != int64(len(content)) { + t.Fatalf("ContentSize = %d, %v", size, err) + } + if cli, err := exec.LookPath("zstd"); err == nil { + if !strings.HasPrefix(strings.TrimSpace(mustVersion(t, cli)), "*** Zstandard CLI (64-bit) v1.5.7") { + t.Skip("installed zstd CLI is not 1.5.7") + } + // The CLI only pledges the size when it reads a regular file. + input := filepath.Join(t.TempDir(), "rollout.jsonl") + if err := os.WriteFile(input, content, 0o600); err != nil { + t.Fatal(err) + } + output, err := exec.Command(cli, "-3", "--single-thread", "--no-check", "-q", "-c", input).Output() + if err != nil { + t.Fatal(err) + } + if !bytes.Equal(output, encoded) { + t.Fatal("encoder output differs from the zstd 1.5.7 CLI") + } + } +} + +func mustVersion(t *testing.T, cli string) string { + t.Helper() + output, err := exec.Command(cli, "--version").Output() + if err != nil { + t.Fatal(err) + } + return string(output) +} + +func TestReaderStreamsTheSameFrame(t *testing.T) { + if !Supported() { + t.Skip("exact encoder needs cgo") + } + content := rolloutFixture() + reader := NewReader(context.Background(), bytes.NewReader(content), int64(len(content))) + streamed, err := io.ReadAll(reader) + if err != nil { + t.Fatal(err) + } + _ = reader.Close() + if !bytes.Equal(streamed, encodeBytes(t, content)) { + t.Fatal("streamed frame differs") + } + early := NewReader(context.Background(), bytes.NewReader(content), int64(len(content))) + if _, err := io.ReadFull(early, make([]byte, 16)); err != nil { + t.Fatal(err) + } + if err := early.Close(); err != nil { + t.Fatal(err) + } +} + +func TestEncodeRejectsWrongContentSize(t *testing.T) { + if !Supported() { + t.Skip("exact encoder needs cgo") + } + content := []byte("{\"a\":1}\n") + for _, size := range []int64{int64(len(content)) - 1, int64(len(content)) + 1} { + if err := Encode(context.Background(), io.Discard, bytes.NewReader(content), size); err == nil { + t.Fatalf("size %d accepted for %d content bytes", size, len(content)) + } + } +} + +func TestVerify(t *testing.T) { + if !Supported() { + t.Skip("exact encoder needs cgo") + } + content := rolloutFixture() + dir := t.TempDir() + exact := filepath.Join(dir, "exact.jsonl.zst") + if err := os.WriteFile(exact, encodeBytes(t, content), 0o600); err != nil { + t.Fatal(err) + } + proof, err := Verify(context.Background(), exact) + if err != nil { + t.Fatal(err) + } + contentDigest := sha256.Sum256(content) + if proof.ContentBytes != int64(len(content)) || proof.ContentSHA256 != hex.EncodeToString(contentDigest[:]) || proof.Profile != Profile { + t.Fatalf("proof = %#v", proof) + } + + other, err := zstd.NewWriter(nil, zstd.WithEncoderLevel(zstd.SpeedDefault)) + if err != nil { + t.Fatal(err) + } + foreign := filepath.Join(dir, "foreign.jsonl.zst") + if err := os.WriteFile(foreign, other.EncodeAll(content, nil), 0o600); err != nil { + t.Fatal(err) + } + if _, err := Verify(context.Background(), foreign); !errors.Is(err, ErrNotReproducible) { + t.Fatalf("foreign encoder verify error = %v", err) + } + truncated := filepath.Join(dir, "truncated.jsonl.zst") + data := encodeBytes(t, content) + if err := os.WriteFile(truncated, data[:len(data)/2], 0o600); err != nil { + t.Fatal(err) + } + if _, err := Verify(context.Background(), truncated); err == nil { + t.Fatal("truncated frame verified") + } + trailing := filepath.Join(dir, "trailing.jsonl.zst") + if err := os.WriteFile(trailing, append(append([]byte(nil), data...), data...), 0o600); err != nil { + t.Fatal(err) + } + if _, err := Verify(context.Background(), trailing); err == nil { + t.Fatal("two concatenated frames verified") + } +} + +// TestVerifyExternalCodexFiles checks real Codex output. Point +// CODEXFOLD_CODEX_ZST_ROOT at a directory such as ~/.codex/fold-native. +func TestVerifyExternalCodexFiles(t *testing.T) { + root := os.Getenv("CODEXFOLD_CODEX_ZST_ROOT") + if root == "" { + t.Skip("set CODEXFOLD_CODEX_ZST_ROOT to verify real Codex .zst files") + } + var verified, failed int + var bytesVerified int64 + err := filepath.WalkDir(root, func(path string, entry os.DirEntry, err error) error { + if err != nil || entry.IsDir() || !strings.HasSuffix(entry.Name(), ".jsonl.zst") || strings.HasPrefix(entry.Name(), "._") { + return err + } + proof, verifyErr := Verify(context.Background(), path) + if verifyErr != nil { + failed++ + t.Errorf("%s: %v", path, verifyErr) + return nil + } + verified++ + bytesVerified += proof.EncodedBytes + return nil + }) + if err != nil { + t.Fatal(err) + } + t.Logf("verified %d Codex .zst files (%d bytes), %d failed", verified, bytesVerified, failed) +} diff --git a/internal/codexzstd/encoder_cgo.go b/internal/codexzstd/encoder_cgo.go new file mode 100644 index 0000000..d248341 --- /dev/null +++ b/internal/codexzstd/encoder_cgo.go @@ -0,0 +1,157 @@ +//go:build cgo && (darwin || linux) + +package codexzstd + +/* +#include + +// libzstd's public API, resolved against the copy that +// github.com/DataDog/zstd v1.5.7 compiles into the binary. +typedef struct ZSTD_CCtx_s ZSTD_CCtx; +typedef struct { const void* src; size_t size; size_t pos; } ZSTD_inBuffer; +typedef struct { void* dst; size_t size; size_t pos; } ZSTD_outBuffer; +extern ZSTD_CCtx* ZSTD_createCCtx(void); +extern size_t ZSTD_freeCCtx(ZSTD_CCtx* cctx); +extern size_t ZSTD_CCtx_setParameter(ZSTD_CCtx* cctx, int param, int value); +extern size_t ZSTD_CCtx_setPledgedSrcSize(ZSTD_CCtx* cctx, unsigned long long pledgedSrcSize); +extern size_t ZSTD_compressStream2(ZSTD_CCtx* cctx, ZSTD_outBuffer* output, ZSTD_inBuffer* input, int endOp); +extern unsigned ZSTD_isError(size_t code); +extern const char* ZSTD_getErrorName(size_t code); +extern unsigned ZSTD_versionNumber(void); +extern size_t ZSTD_CStreamOutSize(void); + +static size_t codexfold_stream(ZSTD_CCtx* cctx, void* dst, size_t dst_size, size_t* dst_pos, + const void* src, size_t src_size, size_t* src_pos, int end_op) { + ZSTD_outBuffer output = { dst, dst_size, *dst_pos }; + ZSTD_inBuffer input = { src, src_size, *src_pos }; + size_t result = ZSTD_compressStream2(cctx, &output, &input, end_op); + *dst_pos = output.pos; + *src_pos = input.pos; + return result; +} +*/ +import "C" + +import ( + "errors" + "fmt" + "io" + "unsafe" + + _ "github.com/DataDog/zstd" +) + +const ( + zstdVersion = 10507 + zstdCompressionLevel = 100 + zstdContentSizeFlag = 200 + zstdChecksumFlag = 201 + zstdNbWorkers = 400 + zstdEndDirectiveCont = 0 + zstdEndDirectiveEnd = 2 + codexCompressionLevel = 3 +) + +func Supported() bool { return C.ZSTD_versionNumber() == zstdVersion } + +type encoder struct { + cctx *C.ZSTD_CCtx + w io.Writer + output []byte +} + +func zstdError(code C.size_t) error { + if C.ZSTD_isError(code) == 0 { + return nil + } + return fmt.Errorf("zstd: %s", C.GoString(C.ZSTD_getErrorName(code))) +} + +func newEncoder(w io.Writer, size int64) (*encoder, error) { + if !Supported() { + return nil, ErrUnsupported + } + if size < 0 { + return nil, errors.New("negative zstd content size") + } + cctx := C.ZSTD_createCCtx() + if cctx == nil { + return nil, errors.New("zstd: cannot allocate compression context") + } + e := &encoder{cctx: cctx, w: w, output: make([]byte, int(C.ZSTD_CStreamOutSize()))} + for _, setting := range [][2]C.int{ + {zstdCompressionLevel, codexCompressionLevel}, + {zstdContentSizeFlag, 1}, + {zstdChecksumFlag, 0}, + {zstdNbWorkers, 0}, + } { + if err := zstdError(C.ZSTD_CCtx_setParameter(cctx, setting[0], setting[1])); err != nil { + e.abort() + return nil, err + } + } + if err := zstdError(C.ZSTD_CCtx_setPledgedSrcSize(cctx, C.ulonglong(size))); err != nil { + e.abort() + return nil, err + } + return e, nil +} + +// stream feeds input (nil at the end) and writes every byte libzstd emits. +func (e *encoder) stream(input []byte, end bool) error { + directive := C.int(zstdEndDirectiveCont) + if end { + directive = zstdEndDirectiveEnd + } + var src unsafe.Pointer + if len(input) > 0 { + src = unsafe.Pointer(&input[0]) + } + var srcPos C.size_t + for { + var dstPos C.size_t + remaining := C.codexfold_stream(e.cctx, unsafe.Pointer(&e.output[0]), C.size_t(len(e.output)), &dstPos, + src, C.size_t(len(input)), &srcPos, directive) + if err := zstdError(remaining); err != nil { + return err + } + if dstPos > 0 { + if _, err := e.w.Write(e.output[:dstPos]); err != nil { + return err + } + } + if end { + if remaining == 0 { + return nil + } + } else if int(srcPos) == len(input) { + return nil + } + } +} + +func (e *encoder) Write(p []byte) (int, error) { + if e.cctx == nil { + return 0, errors.New("zstd encoder is closed") + } + if err := e.stream(p, false); err != nil { + return 0, err + } + return len(p), nil +} + +func (e *encoder) Close() error { + if e.cctx == nil { + return nil + } + err := e.stream(nil, true) + e.abort() + return err +} + +func (e *encoder) abort() { + if e.cctx != nil { + C.ZSTD_freeCCtx(e.cctx) + e.cctx = nil + } +} diff --git a/internal/codexzstd/encoder_other.go b/internal/codexzstd/encoder_other.go new file mode 100644 index 0000000..e79e174 --- /dev/null +++ b/internal/codexzstd/encoder_other.go @@ -0,0 +1,17 @@ +//go:build !cgo || !(darwin || linux) + +package codexzstd + +import "io" + +func Supported() bool { return false } + +type encoder struct{} + +func newEncoder(io.Writer, int64) (*encoder, error) { return nil, ErrUnsupported } + +func (*encoder) Write([]byte) (int, error) { return 0, ErrUnsupported } + +func (*encoder) Close() error { return ErrUnsupported } + +func (*encoder) abort() {} diff --git a/internal/enroll/planner.go b/internal/enroll/planner.go index d0edbd2..cb840a5 100644 --- a/internal/enroll/planner.go +++ b/internal/enroll/planner.go @@ -74,6 +74,11 @@ type Input struct { Gates Gates WriterActive WriterProbe Budget storage.Checker + // Price, when set, returns the exact bytes folding a session would add + // to the store. It is asked only when the conservative price does not + // fit, so a nearly full disk can still take sessions whose content the + // store mostly holds already. + Price func(context.Context, codex.Session, os.FileInfo) (int64, error) } type Decision struct { @@ -228,7 +233,15 @@ func Build(ctx context.Context, input Input) (Plan, error) { if !errors.Is(err, storage.ErrBudgetExceeded) { return Plan{}, err } - decision.Reasons = append(decision.Reasons, ReasonInsufficientBudget) + exact, priced, err := exactEnrollmentPrice(ctx, input, session, info, selectedPersistentBytes) + if err != nil { + return Plan{}, err + } + if priced { + cumulative = exact + } else { + decision.Reasons = append(decision.Reasons, ReasonInsufficientBudget) + } } } if len(decision.Reasons) == 0 { @@ -276,6 +289,43 @@ func safeSessionID(sessionID string) bool { return sessionID != "" && sessionID != "." && sessionID != ".." && !strings.ContainsAny(sessionID, "/\\\x00") } +// exactEnrollmentPrice retries a session that the conservative price rejected +// with the bytes a fold would really add. Enrollment writes those bytes twice +// at its peak, as loose objects and in the next pack, before retiring the +// native file and the loose copies. It returns the new cumulative batch cost. +func exactEnrollmentPrice(ctx context.Context, input Input, session codex.Session, info os.FileInfo, selected int64) (int64, bool, error) { + if input.Price == nil { + return 0, false, nil + } + newBytes, err := input.Price(ctx, session, info) + if err != nil { + return 0, false, err + } + if newBytes < 0 || newBytes > info.Size() { + newBytes = info.Size() + } + single, err := enrollmentPersistentBytes(newBytes) + if err != nil { + return 0, false, err + } + // enrollmentPersistentBytes prices three copies of its input; the exact + // path needs two. + single = single / 3 * 2 + cumulative, overflow := addBytes(selected, single) + if overflow { + return 0, false, errors.New("enrollment batch byte estimate overflow") + } + if _, err := input.Budget.Check(ctx, storage.Projection{ + Operation: "enroll:" + session.ID, AdditionalPersistentBytes: cumulative, TemporaryBytes: min(info.Size(), 64<<20), + }); err != nil { + if errors.Is(err, storage.ErrBudgetExceeded) { + return 0, false, nil + } + return 0, false, err + } + return cumulative, true, nil +} + func enrollmentPersistentBytes(rawBytes int64) (int64, error) { if rawBytes < 0 { return 0, errors.New("enrollment byte estimate cannot be negative") diff --git a/internal/enroll/planner_test.go b/internal/enroll/planner_test.go index 8ec0911..752d55d 100644 --- a/internal/enroll/planner_test.go +++ b/internal/enroll/planner_test.go @@ -1,6 +1,7 @@ package enroll import ( + "bytes" "context" "fmt" "os" @@ -427,3 +428,52 @@ func TestPlannerDoesNotPriceSessionsBeyondTheBatchLimit(t *testing.T) { t.Fatalf("waiting = %d, want %d: sessions past the limit are still waiting", WaitingCount(plan), len(sessions)) } } + +type thresholdBudget struct{ limit int64 } + +func (b thresholdBudget) Check(_ context.Context, projection storage.Projection) (storage.Assessment, error) { + if projection.AdditionalPersistentBytes > b.limit { + return storage.Assessment{}, storage.ErrBudgetExceeded + } + return storage.Assessment{}, nil +} + +func TestPlannerUsesTheExactPriceWhenTheConservativeOneDoesNotFit(t *testing.T) { + root := t.TempDir() + now := time.Now() + path := filepath.Join(root, "rollout-fork.jsonl") + if err := os.WriteFile(path, bytes.Repeat([]byte("{\"fork\":\"of an enrolled session\"}\n"), 120000), 0o600); err != nil { + t.Fatal(err) + } + info, err := os.Stat(path) + if err != nil { + t.Fatal(err) + } + old := now.Add(-2 * time.Hour) + if err := os.Chtimes(path, old, old); err != nil { + t.Fatal(err) + } + info, _ = os.Stat(path) + session := codex.Session{ID: "fork", RolloutPath: path, Archived: true} + plan := func(price func(context.Context, codex.Session, os.FileInfo) (int64, error)) Plan { + t.Helper() + result, err := Build(context.Background(), Input{ + Sessions: []codex.Session{session}, + Previous: Observations{"fork": {Path: path, Size: info.Size(), ModTimeUnixNano: info.ModTime().UnixNano(), StableSinceUnixNano: now.Add(-2 * time.Hour).UnixNano()}}, + Now: now, Policy: Policy{StableFor: time.Hour, BatchSize: 1, ArchivedOnly: true}, + Gates: Gates{DoctorHealthy: true, MountHealthy: true, CanonicalNamespace: true, NamespaceActive: true, NamespaceReady: true, EnrollmentAllowed: true}, + WriterActive: func(context.Context, codex.Session) (bool, error) { return false, nil }, + Budget: thresholdBudget{limit: 8 << 20}, + Price: price, + }) + if err != nil { + t.Fatal(err) + } + return result + } + assertDecisionReason(t, plan(nil), "fork", ReasonInsufficientBudget) + if selected := plan(func(context.Context, codex.Session, os.FileInfo) (int64, error) { return 64 << 10, nil }).Selected; len(selected) != 1 { + t.Fatalf("exactly priced fork not selected: %#v", selected) + } + assertDecisionReason(t, plan(func(context.Context, codex.Session, os.FileInfo) (int64, error) { return info.Size(), nil }), "fork", ReasonInsufficientBudget) +} diff --git a/internal/fold/fold.go b/internal/fold/fold.go index fa2ba81..5efb870 100644 --- a/internal/fold/fold.go +++ b/internal/fold/fold.go @@ -36,7 +36,17 @@ type FoldOptions struct { ExistingReader ObjectReader RecordIndex DuplicateRecordIndex RecordThreshold int64 - beforeCommit func() error + // Content, when set, is folded instead of session.RolloutPath and must be + // exactly ContentBytes long. The caller proves its identity; RemoveSource + // is not allowed. + Content io.Reader + ContentBytes int64 + // NewBytesEstimate, when positive, prices the fold instead of assuming + // every source byte is new. A dry run's NewStoredBytes is exact for the + // same store, so a fold whose content the store already holds can run + // on a nearly full disk. + NewBytesEstimate int64 + beforeCommit func() error } type Session struct { @@ -117,19 +127,27 @@ func Fold(ctx context.Context, session Session, options FoldOptions) (FoldResult } } - file, err := os.Open(session.RolloutPath) - if err != nil { - return FoldResult{}, fmt.Errorf("open rollout: %w", err) - } - fileClosed := false - defer func() { - if !fileClosed { - _ = file.Close() + var file *os.File + var before os.FileInfo + var content io.Reader + contentBytes := options.ContentBytes + if options.Content != nil { + if options.RemoveSource || contentBytes < 0 { + return FoldResult{}, errors.New("a streamed fold needs an exact size and keeps its source") + } + content = options.Content + } else { + var err error + file, err = os.Open(session.RolloutPath) + if err != nil { + return FoldResult{}, fmt.Errorf("open rollout: %w", err) } - }() - before, err := file.Stat() - if err != nil { - return FoldResult{}, fmt.Errorf("stat rollout: %w", err) + defer file.Close() + before, err = file.Stat() + if err != nil { + return FoldResult{}, fmt.Errorf("stat rollout: %w", err) + } + content, contentBytes = file, before.Size() } var storageAssessment storage.Assessment if options.Apply { @@ -141,18 +159,21 @@ func Fold(ctx context.Context, session Session, options FoldOptions) (FoldResult } budget = guard } - persistentBytes, err := estimateFoldStorageBytes(before.Size()) + persistentBytes, err := estimateFoldStorageBytes(contentBytes) + if options.NewBytesEstimate > 0 { + persistentBytes, err = estimateFoldStorageBytes(options.NewBytesEstimate) + } if err != nil { return FoldResult{}, err } - maximumObjectBytes := min(before.Size(), max(options.CDC.MaxBytes, options.MaxJSONLineBytes)) + maximumObjectBytes := min(contentBytes, max(options.CDC.MaxBytes, options.MaxJSONLineBytes)) temporaryBytes, err := estimateFoldStorageBytes(maximumObjectBytes) if err != nil { return FoldResult{}, err } reclaimableBytes := int64(0) if options.RemoveSource { - reclaimableBytes = before.Size() + reclaimableBytes = contentBytes } storageAssessment, err = budget.Check(ctx, storage.Projection{ Operation: "fold", AdditionalPersistentBytes: persistentBytes, @@ -313,7 +334,7 @@ func Fold(ctx context.Context, session Session, options FoldOptions) (FoldResult return chunker.Write(line[cursor:]) } - reader := bufio.NewReaderSize(io.LimitReader(file, before.Size()), 1024*1024) + reader := bufio.NewReaderSize(io.LimitReader(content, contentBytes), 1024*1024) var line bytes.Buffer oversized := options.Opaque for { @@ -369,10 +390,13 @@ func Fold(ctx context.Context, session Session, options FoldOptions) (FoldResult } complete: - if err := file.Close(); err != nil { - return FoldResult{}, fmt.Errorf("close rollout after fold: %w", err) + if file != nil { + if err := file.Close(); err != nil { + return FoldResult{}, fmt.Errorf("close rollout after fold: %w", err) + } + } else if result.SourceBytes != contentBytes { + return FoldResult{}, fmt.Errorf("streamed fold content is %d bytes, expected %d", result.SourceBytes, contentBytes) } - fileClosed = true result.SourceSHA256 = hex.EncodeToString(sourceHash.Sum(nil)) manifest.Source = ManifestSource{Bytes: result.SourceBytes, SHA256: result.SourceSHA256} result.PartCount = len(manifest.Parts) @@ -386,15 +410,17 @@ complete: return FoldResult{}, fmt.Errorf("before fold commit: %w", err) } } - after, err := os.Stat(session.RolloutPath) - if err != nil { - return FoldResult{}, fmt.Errorf("stat rollout after fold: %w", err) - } - if before.Size() != after.Size() || before.ModTime().UnixNano() != after.ModTime().UnixNano() || !os.SameFile(before, after) { - return FoldResult{}, errors.New("rollout changed during fold; manifest was not committed") - } - if err := verifyCurrentSource(session.RolloutPath, before, manifest.Source); err != nil { - return FoldResult{}, err + if file != nil { + after, err := os.Stat(session.RolloutPath) + if err != nil { + return FoldResult{}, fmt.Errorf("stat rollout after fold: %w", err) + } + if before.Size() != after.Size() || before.ModTime().UnixNano() != after.ModTime().UnixNano() || !os.SameFile(before, after) { + return FoldResult{}, errors.New("rollout changed during fold; manifest was not committed") + } + if err := verifyCurrentSource(session.RolloutPath, before, manifest.Source); err != nil { + return FoldResult{}, err + } } if !options.Apply { return result, nil diff --git a/internal/fold/store.go b/internal/fold/store.go index 342f01f..6e49439 100644 --- a/internal/fold/store.go +++ b/internal/fold/store.go @@ -1,7 +1,6 @@ package fold import ( - "bytes" "context" "crypto/sha256" "encoding/hex" @@ -294,18 +293,18 @@ func (s *ObjectStore) Read(ref ObjectRef) ([]byte, error) { return data, nil } +// objectEncoder is shared: EncodeAll is safe for concurrent use, while a new +// encoder per object started worker goroutines and allocated its window for +// every chunk. SpeedDefault left entropy-only text (base64, encrypted +// reasoning) nearly uncompressed; SpeedBetter matches the pack builder. +var objectEncoder = sync.OnceValues(func() (*zstd.Encoder, error) { + return zstd.NewWriter(nil, zstd.WithEncoderLevel(zstd.SpeedBetterCompression), zstd.WithEncoderConcurrency(1)) +}) + func compressObject(data []byte) ([]byte, error) { - var buffer bytes.Buffer - encoder, err := zstd.NewWriter(&buffer, zstd.WithEncoderLevel(zstd.SpeedDefault)) + encoder, err := objectEncoder() if err != nil { return nil, fmt.Errorf("create zstd encoder: %w", err) } - if _, err := encoder.Write(data); err != nil { - encoder.Close() - return nil, fmt.Errorf("compress object: %w", err) - } - if err := encoder.Close(); err != nil { - return nil, fmt.Errorf("close zstd encoder: %w", err) - } - return buffer.Bytes(), nil + return encoder.EncodeAll(data, make([]byte, 0, len(data)/2+64)), nil } diff --git a/internal/mountfs/filesystem.go b/internal/mountfs/filesystem.go index b3858bb..c528304 100644 --- a/internal/mountfs/filesystem.go +++ b/internal/mountfs/filesystem.go @@ -12,6 +12,7 @@ import ( "os" "path" "path/filepath" + "slices" "sort" "strings" "sync" @@ -21,6 +22,7 @@ import ( "unicode/utf8" "github.com/samekind/codexfold/internal/fskitproto" + "github.com/samekind/codexfold/internal/nativezst" "github.com/samekind/codexfold/internal/storage" "github.com/samekind/codexfold/internal/vfs" ) @@ -131,6 +133,8 @@ type fileHandle struct { nativeAppend *nativeAppendState read *vfs.ReadHandle write *vfs.WriteHandle + // folded serves a Codex .zst whose data blocks were released. + folded *nativezst.Reader append bool appendStream bool appendFloor int64 @@ -189,6 +193,7 @@ type Filesystem struct { pendingManagedUnlinks map[string]string publishingManagedDeletions map[string]struct{} deletedManagedSessions map[string]struct{} + nativeZst *nativezst.Source } func New() *Filesystem { @@ -1057,6 +1062,9 @@ func (f *Filesystem) SetAttributes(name string, request SetAttrRequest) syscall. } func (f *Filesystem) GetXattr(name string, attribute string) ([]byte, syscall.Errno) { + if attribute == nativezst.MarkerAttribute { + return nil, xattrMissingErrno() + } xattrPath, managed, errno := f.xattrPath(name, false) if errno != 0 { return nil, errno @@ -1075,6 +1083,9 @@ func (f *Filesystem) SetXattr(name string, attribute string, value []byte, polic if attribute == "" || strings.ContainsRune(attribute, '\x00') { return syscall.EINVAL } + if attribute == nativezst.MarkerAttribute { + return syscall.EPERM + } createCarrier := policy != fskitproto.XattrDelete xattrPath, managed, errno := f.xattrPath(name, createCarrier) if errno != 0 { @@ -1105,6 +1116,7 @@ func (f *Filesystem) ListXattrs(name string) ([]string, syscall.Errno) { } return nil, errnoFor(err) } + attributes = slices.DeleteFunc(attributes, func(attribute string) bool { return attribute == nativezst.MarkerAttribute }) sort.Strings(attributes) return attributes, 0 } @@ -1152,6 +1164,17 @@ func (f *Filesystem) Open(name string, flags int) (uint64, syscall.Errno) { if !ok { return 0, errno } + if reader, foldedErrno, folded := f.openFoldedNative(nativePath, flags); folded { + if foldedErrno != 0 { + return 0, foldedErrno + } + f.mu.Lock() + handleID := f.next + f.next++ + f.handles[handleID] = &fileHandle{path: cleanPath(name), nativePath: nativePath, folded: reader} + f.mu.Unlock() + return handleID, 0 + } // FUSE may split or retry one append syscall as positional writes. Keep // append and truncate semantics in the transaction layer instead of the // backing descriptor. @@ -1237,6 +1260,13 @@ func (f *Filesystem) Read(handleID uint64, destination []byte, offset int64) (in } handle.mu.Lock() defer handle.mu.Unlock() + if handle.folded != nil { + n, err := handle.folded.ReadAt(destination, offset) + if err != nil && !errors.Is(err, io.EOF) { + return n, syscall.EIO + } + return n, 0 + } if handle.native != nil { n, err := handle.nativeAppend.ReadAt(handle.native, destination, offset) if err != nil && !errors.Is(err, io.EOF) { @@ -1377,7 +1407,9 @@ func (f *Filesystem) StreamBufferedRead(handleID uint64, offset int64, length in defer handle.mu.Unlock() var size int64 - if handle.native != nil { + if handle.folded != nil { + size = handle.folded.Size() + } else if handle.native != nil { if handle.nativeAppend == nil { return 0, syscall.EBADF } @@ -1409,7 +1441,9 @@ func (f *Filesystem) StreamBufferedRead(handleID uint64, offset int64, length in need := min(len(buffer), total-n) var count int var readErr error - if handle.native != nil { + if handle.folded != nil { + count, readErr = handle.folded.ReadAt(buffer[:need], offset+int64(n)) + } else if handle.native != nil { count, readErr = handle.nativeAppend.ReadAt(handle.native, buffer[:need], offset+int64(n)) } else { count, readErr = handle.read.ReadAt(context.Background(), buffer[:need], offset+int64(n)) @@ -1578,6 +1612,9 @@ func (f *Filesystem) TruncatePath(name string, size int64) syscall.Errno { if errno != 0 { if f.canonical { if nativePath, ok := f.nativePath(cleanPath(name)); ok { + if _, folded, _ := nativezst.ReadMarker(nativePath); folded { + return syscall.EPERM + } state, err := f.loadNativeAppendState(nativePath) if err != nil { return errnoFor(err) @@ -1691,6 +1728,9 @@ func (f *Filesystem) Release(handleID uint64) syscall.Errno { } handle.mu.Lock() defer handle.mu.Unlock() + if handle.folded != nil { + return errnoFor(handle.folded.Close()) + } if handle.native != nil { var commitErr error if handle.append { diff --git a/internal/mountfs/native_fskit_fd_darwin_test.go b/internal/mountfs/native_fskit_fd_darwin_test.go index a7a0c0a..20f47a4 100644 --- a/internal/mountfs/native_fskit_fd_darwin_test.go +++ b/internal/mountfs/native_fskit_fd_darwin_test.go @@ -927,3 +927,42 @@ func assertMappedNativeFDTestBytes(t *testing.T, fd int, want []byte, maximumByt t.Fatal(err) } } + +func TestNativeFSKitServerGivesFoldedZstNoSharedWindow(t *testing.T) { + filesystem, source, store, route, original := foldedZstFixture(t) + convertFixture(t, filesystem, source, store, route) + filesystem.SetNativeZstSource(source) + root := filepath.Dir(filesystem.nativeZstRoot()) + client, stop := startNativeFSKitTestServer(t, filesystem, root) + _ = client.Close() + defer stop() + descriptor := readNativeFDTestDescriptor(t, root) + connection := dialNativeFDTestSocket(t, descriptor) + defer connection.Close() + callNativeFDTestHelloCapabilities(t, connection, descriptor, fskitproto.CapabilitySharedWindow|fskitproto.CapabilitySharedFileWindow) + + openPayload := fskitproto.NewEncoder(128) + openPayload.String(route) + openPayload.Uint32(uint32(os.O_RDONLY)) + openResponse := callNativeFDTestFrame(t, connection, descriptor, 2, fskitproto.OpOpen, openPayload.Data()) + // Without a window the extension does not prime its prefetch channels. + if openResponse.Flags&(fskitproto.FlagSharedWindow|fskitproto.FlagSharedFileWindow|fskitproto.FlagNativeReadFD) != 0 { + t.Fatalf("folded open flags = %#x, want no shared window or native descriptor", openResponse.Flags) + } + decoder := fskitproto.NewDecoder(openResponse.Payload) + handle, err := decoder.Uint64() + if err != nil { + t.Fatal(err) + } + response := callNativeFDTestRead(t, connection, descriptor, 3, handle, 0, 4096) + if response.Flags&fskitproto.FlagSharedReadFD != 0 { + receiveFDTestMarker(t, connection, fskitproto.SharedReadFDMarker) + t.Skip("shared read descriptors are covered elsewhere") + } + payload := fskitproto.NewDecoder(response.Payload) + count, err := payload.Uint32() + if err != nil || count != 4096 || len(response.Payload) != 4+4096 || !bytes.Equal(response.Payload[4:], original[:4096]) { + t.Fatalf("folded buffered read count=%d payload=%d err=%v", count, len(response.Payload), err) + } + callNativeFDTestHandle(t, connection, descriptor, 4, fskitproto.OpRelease, handle) +} diff --git a/internal/mountfs/native_fskit_server.go b/internal/mountfs/native_fskit_server.go index ba465cf..fecca87 100644 --- a/internal/mountfs/native_fskit_server.go +++ b/internal/mountfs/native_fskit_server.go @@ -939,6 +939,14 @@ func (c *nativeFSKitConnection) writeOpenResponseWithOptimizedFD(response fskitp if !exists || handleState.health || handleState.flags&(os.O_WRONLY|os.O_RDWR) != 0 { return false, false, nil } + // A shared window makes the extension prime up to nine prefetch channels + // on open, each with its own socket and window descriptor. A folded .zst + // is read at the encoder's pace anyway, and priming it cost the + // extension, capped at 256 descriptors, about fourteen per open file: + // a handful of concurrent readers made every mount read fail with EMFILE. + if c.server.filesystem.IsFoldedHandle(handleState.coreHandle) { + return false, false, nil + } windowBytes := min(nativeFSKitSharedWindowBytes, int(c.server.maxPayload)-4) var window *nativeSharedReadWindow var windowFlag uint32 diff --git a/internal/mountfs/native_zst.go b/internal/mountfs/native_zst.go new file mode 100644 index 0000000..fb44b1d --- /dev/null +++ b/internal/mountfs/native_zst.go @@ -0,0 +1,132 @@ +package mountfs + +import ( + "context" + "errors" + "os" + "path/filepath" + "strings" + "syscall" + + "github.com/samekind/codexfold/internal/nativezst" +) + +// ErrNativeZstBusy means a handle is open on the file; it is retried later. +var ErrNativeZstBusy = errors.New("native .zst is open") + +// SetNativeZstSource lets the mount serve Codex .zst files whose data blocks +// were folded into store. Without it a folded file fails with EIO instead of +// returning its released, all-zero blocks. +func (f *Filesystem) SetNativeZstSource(source *nativezst.Source) { + f.mu.Lock() + f.nativeZst = source + f.mu.Unlock() +} + +// openFoldedNative opens a folded .zst. folded is false for every other file, +// which then takes the ordinary native path. +func (f *Filesystem) openFoldedNative(nativePath string, flags int) (*nativezst.Reader, syscall.Errno, bool) { + if !strings.HasSuffix(nativePath, nativezst.Suffix) { + return nil, 0, false + } + marker, folded, err := nativezst.ReadMarker(nativePath) + if errors.Is(err, os.ErrNotExist) { + return nil, 0, false + } + if err != nil { + return nil, syscall.EIO, true + } + if !folded { + return nil, 0, false + } + // Codex only reads, renames and deletes a compressed rollout. + if access := flags & (os.O_WRONLY | os.O_RDWR); access != os.O_RDONLY || flags&os.O_TRUNC != 0 { + return nil, syscall.EPERM, true + } + f.mu.RLock() + source := f.nativeZst + f.mu.RUnlock() + if source == nil { + return nil, syscall.EIO, true + } + reader, err := source.Open(marker) + if err != nil { + return nil, syscall.EIO, true + } + return reader, 0, true +} + +func (f *Filesystem) nativeZstRoot() string { + f.mu.RLock() + defer f.mu.RUnlock() + return f.nativeRoot +} + +// ScanNativeZst walks the native tree with renames held off, so a folded file +// cannot move between directories while its marker is being counted. +func (f *Filesystem) ScanNativeZst(ctx context.Context) (nativezst.Scan, error) { + root := f.nativeZstRoot() + if root == "" { + return nativezst.Scan{}, errors.New("native root is not configured") + } + f.renameMu.RLock() + defer f.renameMu.RUnlock() + return nativezst.ScanNative(ctx, root) +} + +func (f *Filesystem) nativeZstName(path string) (string, error) { + root := f.nativeZstRoot() + relative, err := filepath.Rel(root, path) + if err != nil || relative == "." || strings.HasPrefix(relative, "..") { + return "", errors.New("native .zst lies outside the native root") + } + return "/" + filepath.ToSlash(relative), nil +} + +// withNativeZstExclusive runs mutate while no open, rename or unlink can +// reach path, and only if no handle is open on it. +func (f *Filesystem) withNativeZstExclusive(path string, mutate func() error) error { + name, err := f.nativeZstName(path) + if err != nil { + return err + } + f.renameMu.Lock() + defer f.renameMu.Unlock() + f.mu.RLock() + busy := f.nativePathBusyLocked(name) + f.mu.RUnlock() + if busy { + return ErrNativeZstBusy + } + f.markNativeInternalMutation(path) + return mutate() +} + +// ConvertNativeZst releases the blocks of a prepared file. +func (f *Filesystem) ConvertNativeZst(store string, prepared nativezst.Prepared) error { + return f.withNativeZstExclusive(prepared.Path, func() error { + return nativezst.Convert(store, prepared) + }) +} + +// FinishNativeZst completes a conversion that stopped after its marker. The +// served bytes are proven before the lock is taken. +func (f *Filesystem) FinishNativeZst(ctx context.Context, source *nativezst.Source, file nativezst.NativeFile) error { + if err := nativezst.Prove(ctx, source, file.Marker); err != nil { + return err + } + return f.withNativeZstExclusive(file.Path, func() error { + return nativezst.FinishRelease(file.Path, file.Info) + }) +} + +// IsFoldedHandle reports whether an open handle serves a folded .zst. +func (f *Filesystem) IsFoldedHandle(handleID uint64) bool { + handle, errno := f.handle(handleID) + if errno != 0 { + return false + } + handle.mu.Lock() + defer handle.mu.Unlock() + return handle.folded != nil +} diff --git a/internal/mountfs/native_zst_test.go b/internal/mountfs/native_zst_test.go new file mode 100644 index 0000000..835b39c --- /dev/null +++ b/internal/mountfs/native_zst_test.go @@ -0,0 +1,209 @@ +package mountfs + +import ( + "bytes" + "context" + "crypto/sha256" + "errors" + "fmt" + "os" + "path/filepath" + "slices" + "syscall" + "testing" + "time" + + "github.com/samekind/codexfold/internal/codexzstd" + "github.com/samekind/codexfold/internal/nativezst" +) + +func foldedZstFixture(t *testing.T) (*Filesystem, *nativezst.Source, string, string, []byte) { + t.Helper() + if !codexzstd.Supported() { + t.Skip("folded .zst files need cgo") + } + root := t.TempDir() + store := filepath.Join(root, "store") + nativeRoot := filepath.Join(root, "native") + route := "/archived_sessions/rollout-2026-09-01T10-00-00-0199aaaa-0000-7000-8000-00000000000a.jsonl.zst" + nativePath := nativePathFromRoot(nativeRoot, route) + if err := os.MkdirAll(filepath.Dir(nativePath), 0o700); err != nil { + t.Fatal(err) + } + if err := os.MkdirAll(store, 0o700); err != nil { + t.Fatal(err) + } + // A unique history replayed beyond zstd's window, as a fork does: the + // store folds it for less than Codex's own .zst. + var history bytes.Buffer + for line := 0; history.Len() < 2400<<10; line++ { + fmt.Fprintf(&history, `{"ordinal":%d,"text":"line %d of a compressed Codex rollout %x"}`+"\n", line, line*7919, sha256.Sum256([]byte(fmt.Sprint(line)))) + } + var content bytes.Buffer + content.Write(history.Bytes()) + content.Write(history.Bytes()) + var encoded bytes.Buffer + if err := codexzstd.Encode(context.Background(), &encoded, bytes.NewReader(content.Bytes()), int64(content.Len())); err != nil { + t.Fatal(err) + } + if err := os.WriteFile(nativePath, encoded.Bytes(), 0o600); err != nil { + t.Fatal(err) + } + modTime := time.Now().Add(-10 * 24 * time.Hour).Truncate(time.Second) + if err := os.Chtimes(nativePath, modTime, modTime); err != nil { + t.Fatal(err) + } + filesystem := NewCanonical() + filesystem.SetNativeRoot(nativeRoot) + source := nativezst.NewSource(store, nil) + t.Cleanup(func() { _ = source.Close() }) + return filesystem, source, store, route, encoded.Bytes() +} + +func convertFixture(t *testing.T, filesystem *Filesystem, source *nativezst.Source, store string, route string) { + t.Helper() + nativePath := nativePathFromRoot(filesystem.nativeZstRoot(), route) + prepared, err := nativezst.Prepare(context.Background(), source, nativePath) + if err != nil { + t.Fatal(err) + } + if err := filesystem.ConvertNativeZst(store, prepared); err != nil { + t.Fatal(err) + } +} + +func TestFoldedNativeZstServesOriginalBytes(t *testing.T) { + filesystem, source, store, route, original := foldedZstFixture(t) + before, errno := filesystem.Getattr(route) + if errno != 0 { + t.Fatal(errno) + } + convertFixture(t, filesystem, source, store, route) + + // Without a source the mount must refuse rather than serve zeros. + if _, errno := filesystem.Open(route, os.O_RDONLY); errno != syscall.EIO { + t.Fatalf("open without source = %v, want EIO", errno) + } + filesystem.SetNativeZstSource(source) + + after, errno := filesystem.Getattr(route) + if errno != 0 { + t.Fatal(errno) + } + if after.Size != before.Size || !after.ModTime.Equal(before.ModTime) || after.Mode != before.Mode || after.ObjectID != before.ObjectID { + t.Fatalf("attributes changed: before=%+v after=%+v", before, after) + } + handle, errno := filesystem.Open(route, os.O_RDONLY) + if errno != 0 { + t.Fatal(errno) + } + var got bytes.Buffer + buffer := make([]byte, 4096) + for offset := int64(0); ; { + n, errno := filesystem.Read(handle, buffer, offset) + if errno != 0 { + t.Fatal(errno) + } + if n == 0 { + break + } + got.Write(buffer[:n]) + offset += int64(n) + } + if !bytes.Equal(got.Bytes(), original) { + t.Fatal("mount served different bytes") + } + var streamed bytes.Buffer + if _, err := filesystem.StreamBufferedRead(handle, 0, len(original)+10, 1000, func(total int, chunk []byte) error { + if total != len(original) { + return fmt.Errorf("total = %d", total) + } + streamed.Write(chunk) + return nil + }); err != nil { + t.Fatal(err) + } + if !bytes.Equal(streamed.Bytes(), original) { + t.Fatal("buffered stream served different bytes") + } + if handled, _, _ := filesystem.StreamNativeRead(handle, 0, 10, func(*os.File, int64, int) (int, error) { return 0, errors.New("unexpected") }); handled { + t.Fatal("folded file exposed its native descriptor") + } + if handled, _ := filesystem.WithNativeReadFD(handle, func(*os.File) error { return errors.New("unexpected") }); handled { + t.Fatal("folded file transferred its native descriptor") + } + + // A conversion never runs under an open handle. + nativePath := nativePathFromRoot(filesystem.nativeZstRoot(), route) + marker, _, err := nativezst.ReadMarker(nativePath) + if err != nil { + t.Fatal(err) + } + info, err := os.Lstat(nativePath) + if err != nil { + t.Fatal(err) + } + if err := filesystem.FinishNativeZst(context.Background(), source, nativezst.NativeFile{Path: nativePath, Info: info, Marker: marker, Folded: true}); !errors.Is(err, ErrNativeZstBusy) { + t.Fatalf("finish under an open handle = %v", err) + } + if errno := filesystem.Release(handle); errno != 0 { + t.Fatal(errno) + } +} + +func TestFoldedNativeZstIsReadOnlyAndHidesItsMarker(t *testing.T) { + filesystem, source, store, route, _ := foldedZstFixture(t) + convertFixture(t, filesystem, source, store, route) + filesystem.SetNativeZstSource(source) + for _, flags := range []int{os.O_WRONLY, os.O_RDWR, os.O_RDONLY | os.O_TRUNC} { + if _, errno := filesystem.Open(route, flags); errno != syscall.EPERM { + t.Fatalf("open flags %#x = %v, want EPERM", flags, errno) + } + } + if errno := filesystem.TruncatePath(route, 0); errno != syscall.EPERM { + t.Fatalf("truncate = %v, want EPERM", errno) + } + if _, errno := filesystem.GetXattr(route, nativezst.MarkerAttribute); errno != xattrMissingErrno() { + t.Fatalf("marker read through the mount = %v", errno) + } + attributes, errno := filesystem.ListXattrs(route) + if errno != 0 || slices.Contains(attributes, nativezst.MarkerAttribute) { + t.Fatalf("listxattr = %v %v", attributes, errno) + } + if errno := filesystem.SetXattr(route, nativezst.MarkerAttribute, []byte("{}"), 0); errno != syscall.EPERM { + t.Fatalf("marker overwrite = %v, want EPERM", errno) + } +} + +func TestConvertNativeZstRefusesAnOpenFile(t *testing.T) { + filesystem, source, store, route, _ := foldedZstFixture(t) + filesystem.SetNativeZstSource(source) + handle, errno := filesystem.Open(route, os.O_RDONLY) + if errno != 0 { + t.Fatal(errno) + } + nativePath := nativePathFromRoot(filesystem.nativeZstRoot(), route) + prepared, err := nativezst.Prepare(context.Background(), source, nativePath) + if err != nil { + t.Fatal(err) + } + if err := filesystem.ConvertNativeZst(store, prepared); !errors.Is(err, ErrNativeZstBusy) { + t.Fatalf("convert under an open handle = %v", err) + } + if _, folded, _ := nativezst.ReadMarker(nativePath); folded { + t.Fatal("busy file was converted") + } + filesystem.Release(handle) +} + +func TestScanNativeZstFindsFoldedFiles(t *testing.T) { + filesystem, source, store, route, _ := foldedZstFixture(t) + convertFixture(t, filesystem, source, store, route) + scan, err := filesystem.ScanNativeZst(context.Background()) + if err != nil { + t.Fatal(err) + } + if len(scan.Folded) != 1 || len(scan.Candidates) != 0 { + t.Fatalf("scan folded=%d candidates=%d", len(scan.Folded), len(scan.Candidates)) + } +} diff --git a/internal/nativezst/fold.go b/internal/nativezst/fold.go new file mode 100644 index 0000000..f605bb1 --- /dev/null +++ b/internal/nativezst/fold.go @@ -0,0 +1,442 @@ +package nativezst + +import ( + "context" + "crypto/sha256" + "encoding/hex" + "encoding/json" + "errors" + "fmt" + "io" + "os" + "path/filepath" + "strconv" + "time" + + "github.com/klauspost/compress/zstd" + "github.com/samekind/codexfold/internal/codexzstd" + "github.com/samekind/codexfold/internal/fold" + "github.com/samekind/codexfold/internal/storage" +) + +// Prepared is a .zst whose content is folded and proven, not yet converted. +type Prepared struct { + Path string + Info os.FileInfo + Marker Marker + ManifestPath string + // NewStoredBytes is what the fold added to the store; content already in + // the store (a managed copy of the same session) costs nothing. + NewStoredBytes int64 +} + +// Prepare folds the decoded content of a Codex .zst into the store and proves +// that the store re-encodes it to the file's exact bytes. The file itself is +// left untouched; Discard undoes a Prepare that is not converted. +// +// Prepare is built for a nearly full disk: the content is folded straight +// from the decoder, never staged, and when free space is short a dry run +// prices the fold exactly, so a file whose content the store already holds +// costs almost nothing. +func Prepare(ctx context.Context, source *Source, path string) (Prepared, error) { + if !supported() { + return Prepared{}, ErrUnsupported + } + info, err := os.Lstat(path) + if err != nil { + return Prepared{}, err + } + if !info.Mode().IsRegular() || info.Size() == 0 { + return Prepared{}, fmt.Errorf("%s is not a non-empty regular file", path) + } + if _, folded, err := ReadMarker(path); err != nil || folded { + return Prepared{}, errors.Join(err, errors.New("native .zst is already folded")) + } + proof, err := codexzstd.Verify(ctx, path) + if err != nil { + return Prepared{}, err + } + store := source.Store() + guard, err := storage.DefaultGuard(store) + if err != nil { + return Prepared{}, err + } + existing, release, err := source.ExistingObjects() + if err != nil { + return Prepared{}, err + } + defer release() + options := fold.FoldOptions{StoreDir: store, ExistingReader: existing} + // The dry run writes nothing. It prices the fold exactly for the + // free-space check, and keeps a small rollout the store shares nothing + // with as Codex's .zst: its manifest and index would outweigh the bytes + // freed. + dry, err := foldDecoded(ctx, path, proof, key(proof), options) + if err != nil { + return Prepared{}, err + } + if foldCost(dry) >= info.Size()*worthwhileShare/100 { + return Prepared{}, ErrNotWorthwhile + } + options.NewBytesEstimate = max(dry.NewStoredBytes, 1) + + manifestKey := key(proof) + if err := os.MkdirAll(ManifestRoot(store), 0o755); err != nil { + return Prepared{}, err + } + manifestPath := filepath.Join(ManifestRoot(store), manifestKey+".json") + options.Apply = true + options.ManifestPathOverride = manifestPath + // A full store scan per file cost more than the fold itself; free space + // is still probed for every file. + options.Budget = reclaimingGuard{inner: storage.SharedInventoryGuard{Guard: guard}, reclaim: info.Size()} + result, err := foldDecoded(ctx, path, proof, manifestKey, options) + if err != nil { + _ = os.Remove(manifestPath) + return Prepared{}, err + } + prepared := Prepared{ + Path: path, Info: info, ManifestPath: manifestPath, NewStoredBytes: result.NewStoredBytes, + Marker: Marker{ + Version: MarkerVersion, Kind: MarkerKind, Profile: proof.Profile, Manifest: manifestRelative(manifestKey), + EncodedBytes: proof.EncodedBytes, EncodedSHA256: proof.EncodedSHA256, + ContentBytes: proof.ContentBytes, ContentSHA256: proof.ContentSHA256, + }, + } + if result.SourceBytes != proof.ContentBytes || result.SourceSHA256 != proof.ContentSHA256 { + Discard(prepared) + return Prepared{}, errors.New("folded content does not match the decoded .zst") + } + if err := proveServed(ctx, source, prepared.Marker); err != nil { + Discard(prepared) + return Prepared{}, err + } + if after, err := os.Lstat(path); err != nil || !sameVersion(info, after) { + Discard(prepared) + return Prepared{}, errors.Join(err, ErrChanged) + } + return prepared, nil +} + +func key(proof codexzstd.Proof) string { + return "zst-" + proof.EncodedSHA256[:24] + "-" + strconv.FormatInt(time.Now().UnixNano(), 36) +} + +// foldDecoded folds the file's decoded content as it streams out of the +// decoder. Prepare compares the result with the proof, so a file changed +// after verification is never folded silently. +func foldDecoded(ctx context.Context, path string, proof codexzstd.Proof, id string, options fold.FoldOptions) (fold.FoldResult, error) { + input, err := os.Open(path) + if err != nil { + return fold.FoldResult{}, err + } + defer input.Close() + decoder, err := zstd.NewReader(input, zstd.WithDecoderConcurrency(1), zstd.WithDecoderLowmem(true), zstd.WithDecoderMaxWindow(1<<31)) + if err != nil { + return fold.FoldResult{}, err + } + defer decoder.Close() + options.Content = &contextReader{ctx: ctx, reader: decoder} + options.ContentBytes = proof.ContentBytes + return fold.Fold(ctx, fold.Session{ID: id, RolloutPath: path, Archived: true}, options) +} + +// ErrNotWorthwhile means folding the file would not free space: the store +// shares too little of its content to beat Codex's own compression. +var ErrNotWorthwhile = errors.New("folding this .zst would not save space") + +// A fold must cost less than this share of the file it frees. +const worthwhileShare = 80 + +// foldCost bounds what a fold occupies once packed: its new objects, plus a +// manifest part and pack index records for each, with the manifest stored +// again in the pack's recovery archive. +func foldCost(dry fold.FoldResult) int64 { + const perPart, perObject, fixed = 3 * 160, 256, 8 << 10 + return dry.NewStoredBytes + int64(dry.PartCount)*perPart + int64(dry.UniqueObjects)*perObject + fixed +} + +const ( + // A fold this small may run below the free-space reserve when its + // conversion frees at least reclaimRatio times as much, down to + // emergencyFloor: that is how the store digs itself out of a full disk. + emergencyPeakBytes = 64 << 20 + reclaimRatio = 4 + emergencyFloor = 1 << 30 +) + +type reclaimingGuard struct { + inner storage.Checker + reclaim int64 +} + +func (g reclaimingGuard) Check(ctx context.Context, projection storage.Projection) (storage.Assessment, error) { + assessment, err := g.inner.Check(ctx, projection) + if err == nil || !errors.Is(err, storage.ErrBudgetExceeded) { + return assessment, err + } + report := assessment.Budget + if len(report.Rejections) != 1 || report.Rejections[0] != storage.RejectionFreeSpaceReserve { + return assessment, err + } + peak := projection.AdditionalPersistentBytes + projection.TemporaryBytes - projection.TemporaryPersistentOverlapBytes + if peak > emergencyPeakBytes || g.reclaim < reclaimRatio*peak || report.FreeSpaceAfterPeak < emergencyFloor { + return assessment, err + } + assessment.Budget.Allowed = true + assessment.Budget.Rejections = nil + return assessment, nil +} + +// Discard removes the manifest of a Prepare that will not be converted. Its +// objects become unreferenced and leave with the next pack and GC. +func Discard(prepared Prepared) { + if prepared.ManifestPath != "" { + _ = os.Remove(prepared.ManifestPath) + } +} + +type contextReader struct { + ctx context.Context + reader io.Reader +} + +func (r *contextReader) Read(p []byte) (int, error) { + if err := r.ctx.Err(); err != nil { + return 0, err + } + return r.reader.Read(p) +} + +// proveServed re-encodes the stored content through the encoder the mount +// uses and checks every byte against the marker. +func proveServed(ctx context.Context, source *Source, marker Marker) error { + stream, err := source.Stream(ctx, marker) + if err != nil { + return err + } + defer stream.Close() + hasher := sha256.New() + n, err := io.Copy(hasher, &contextReader{ctx: ctx, reader: stream}) + if err != nil { + return fmt.Errorf("re-encode folded .zst: %w", err) + } + if n != marker.EncodedBytes || hex.EncodeToString(hasher.Sum(nil)) != marker.EncodedSHA256 { + return errors.New("the store does not reproduce the original .zst bytes") + } + return nil +} + +func sameVersion(before os.FileInfo, after os.FileInfo) bool { + return os.SameFile(before, after) && before.Size() == after.Size() && before.ModTime().Equal(after.ModTime()) && + changeTime(before).Equal(changeTime(after)) +} + +type journalRecord struct { + Version int `json:"version"` + Key string `json:"key"` + Path string `json:"path"` + Inode uint64 `json:"inode"` + EncodedBytes int64 `json:"encoded_bytes"` + EncodedSHA256 string `json:"encoded_sha256"` + FoldedAt string `json:"folded_at"` +} + +func journalPath(store string, key string) string { + return filepath.Join(StateRoot(store), "journal", key+".json") +} + +// Convert binds the file to its manifest and frees its data blocks in place. +// The caller must hold the mount's namespace lock and know that no handle is +// open on the file. Size, inode, mode and times are kept. +func Convert(store string, prepared Prepared) error { + key, err := ManifestKey(prepared.Marker.Manifest) + if err != nil { + return err + } + file, err := os.OpenFile(prepared.Path, os.O_RDWR, 0) + if err != nil { + return err + } + defer file.Close() + current, err := file.Stat() + if err != nil { + return err + } + if !sameVersion(prepared.Info, current) { + return ErrChanged + } + // The journal outlives a marker lost to a crash: the sweep finds the + // inode again and restores the marker instead of releasing the content. + record := journalRecord{ + Version: 1, Key: key, Path: prepared.Path, Inode: inode(current), + EncodedBytes: prepared.Marker.EncodedBytes, EncodedSHA256: prepared.Marker.EncodedSHA256, + FoldedAt: time.Now().UTC().Format(time.RFC3339Nano), + } + if err := writeDurableJSON(journalPath(store, key), record); err != nil { + return err + } + accessTime, modifyTime := fileTimes(current) + if err := writeMarker(file, prepared.Marker); err != nil { + _ = os.Remove(journalPath(store, key)) + return err + } + if err := durableSync(file); err != nil { + if removeMarker(file) == nil { + _ = os.Remove(journalPath(store, key)) + return err + } + return errors.Join(err, ErrMarkerKept) + } + // From here the file is bound to its manifest: whatever happens, the + // manifest must stay. + if err := releaseBlocks(file, current.Size()); err != nil { + return errors.Join(err, ErrMarkerKept) + } + if err := durableSync(file); err != nil { + return errors.Join(err, ErrMarkerKept) + } + if err := os.Chtimes(prepared.Path, accessTime, modifyTime); err != nil { + return errors.Join(err, ErrMarkerKept) + } + return nil +} + +// Prove re-encodes the stored content exactly as the mount serves it and +// checks it against the marker. +func Prove(ctx context.Context, source *Source, marker Marker) error { + return proveServed(ctx, source, marker) +} + +// FinishRelease completes a conversion interrupted after its marker was +// written. Call Prove first, outside the caller's namespace lock. +func FinishRelease(path string, info os.FileInfo) error { + file, err := os.OpenFile(path, os.O_RDWR, 0) + if err != nil { + return err + } + defer file.Close() + current, err := file.Stat() + if err != nil { + return err + } + if !sameVersion(info, current) { + return ErrChanged + } + if _, folded, err := ReadMarker(path); err != nil || !folded { + return errors.Join(err, ErrChanged) + } + accessTime, modifyTime := fileTimes(current) + if err := releaseBlocks(file, current.Size()); err != nil { + return err + } + if err := durableSync(file); err != nil { + return err + } + return os.Chtimes(path, accessTime, modifyTime) +} + +// Restore writes the original .zst bytes back into a folded file and removes +// its marker. Until the marker is gone the mount keeps serving from the +// store, so an interrupted restore is still read correctly and can be rerun. +func Restore(ctx context.Context, source *Source, path string) error { + marker, folded, err := ReadMarker(path) + if err != nil { + return err + } + if !folded { + return fmt.Errorf("%s is not a folded .zst", path) + } + stream, err := source.Stream(ctx, marker) + if err != nil { + return err + } + defer stream.Close() + file, err := os.OpenFile(path, os.O_RDWR, 0) + if err != nil { + return err + } + defer file.Close() + info, err := file.Stat() + if err != nil { + return err + } + if info.Size() != marker.EncodedBytes { + return fmt.Errorf("%s is %d bytes, its marker records %d", path, info.Size(), marker.EncodedBytes) + } + accessTime, modifyTime := fileTimes(info) + hasher := sha256.New() + buffer := make([]byte, 1<<20) + var offset int64 + for { + if err := ctx.Err(); err != nil { + return err + } + n, readErr := stream.Read(buffer) + if n > 0 { + if offset+int64(n) > marker.EncodedBytes { + return errors.New("re-encoded .zst is longer than its marker records") + } + if _, err := file.WriteAt(buffer[:n], offset); err != nil { + return err + } + _, _ = hasher.Write(buffer[:n]) + offset += int64(n) + } + if errors.Is(readErr, io.EOF) { + break + } + if readErr != nil { + return readErr + } + } + if offset != marker.EncodedBytes || hex.EncodeToString(hasher.Sum(nil)) != marker.EncodedSHA256 { + return errors.New("restored bytes do not match the folded .zst") + } + if err := durableSync(file); err != nil { + return err + } + if err := removeMarker(file); err != nil { + return err + } + if err := durableSync(file); err != nil { + return err + } + return os.Chtimes(path, accessTime, modifyTime) +} + +func writeDurableJSON(path string, value any) error { + data, err := json.MarshalIndent(value, "", " ") + if err != nil { + return err + } + if err := os.MkdirAll(filepath.Dir(path), 0o700); err != nil { + return err + } + temporary, err := os.CreateTemp(filepath.Dir(path), ".tmp-*.json") + if err != nil { + return err + } + name := temporary.Name() + defer os.Remove(name) + if _, err := temporary.Write(data); err != nil { + _ = temporary.Close() + return err + } + if err := durableSync(temporary); err != nil { + _ = temporary.Close() + return err + } + if err := temporary.Close(); err != nil { + return err + } + if err := os.Rename(name, path); err != nil { + return err + } + directory, err := os.Open(filepath.Dir(path)) + if err != nil { + return err + } + syncErr := directory.Sync() + return errors.Join(syncErr, directory.Close()) +} diff --git a/internal/nativezst/marker.go b/internal/nativezst/marker.go new file mode 100644 index 0000000..81b4d66 --- /dev/null +++ b/internal/nativezst/marker.go @@ -0,0 +1,148 @@ +// Package nativezst folds the rollouts Codex compresses itself. +// +// Codex compresses an idle rollout to a sibling rollout-*.jsonl.zst and +// deletes the .jsonl. Those files are immutable: Codex only reads, renames and +// deletes them. A folded .zst keeps its path, inode, size, mode and times, but +// its data blocks are released: a marker extended attribute names the fold +// manifest that holds the decoded content, and the mount re-encodes the +// original bytes on read with the exact libzstd profile Codex used. +// +// The marker is written and flushed before any block is released, and every +// state in between still serves correct bytes through the mount: either the +// file still holds its own data, or the marker points at content proven to +// re-encode to it. +package nativezst + +import ( + "encoding/hex" + "encoding/json" + "errors" + "fmt" + "os" + "path" + "path/filepath" + "strings" + "time" + + "github.com/samekind/codexfold/internal/codexzstd" +) + +const ( + // MarkerAttribute is hidden from clients of the mount. + MarkerAttribute = "user.codexfold.native-zst" + MarkerKind = "codexfold-native-zst" + MarkerVersion = 1 + Suffix = ".jsonl.zst" + manifestDir = "native-zst" + stateDir = "native-zst" + maxMarkerBytes = 4 << 10 +) + +var ( + ErrUnsupported = errors.New("folded Codex .zst files are unsupported on this platform") + ErrChanged = errors.New("native .zst changed before it could be folded") + // ErrMarkerKept means a conversion failed after the file was bound to its + // manifest; the manifest must not be discarded. + ErrMarkerKept = errors.New("native .zst kept its marker") +) + +// AllocatedBytes is the disk space a file's data blocks occupy. +func AllocatedBytes(info os.FileInfo) int64 { return allocatedBytes(info) } + +// DataReleased reports whether a hole-punched file holds no data blocks. +// Allocation alone cannot say so on every filesystem: ext4 counts a block +// that holds extended attributes, among them the fold marker itself, so a +// fully punched file still reports one block there. +func DataReleased(info os.FileInfo) bool { return dataReleased(info) } + +func dataReleased(info os.FileInfo) bool { + return allocatedBytes(info) <= metadataAllocation(info) +} + +// ChangedAfter reports whether the file's metadata changed after t. +func ChangedAfter(info os.FileInfo, t time.Time) bool { return changeTime(info).After(t) } + +// Marker is stored on a folded file and binds it to its fold manifest. +type Marker struct { + Version int `json:"version"` + Kind string `json:"kind"` + Profile string `json:"profile"` + Manifest string `json:"manifest"` + EncodedBytes int64 `json:"encoded_bytes"` + EncodedSHA256 string `json:"encoded_sha256"` + ContentBytes int64 `json:"content_bytes"` + ContentSHA256 string `json:"content_sha256"` +} + +func (m Marker) Validate() error { + if m.Version != MarkerVersion || m.Kind != MarkerKind { + return fmt.Errorf("unsupported native .zst marker %s v%d", m.Kind, m.Version) + } + if m.Profile != codexzstd.Profile { + return fmt.Errorf("native .zst was folded with encoder profile %q, this build has %q", m.Profile, codexzstd.Profile) + } + if _, err := ManifestKey(m.Manifest); err != nil { + return err + } + if m.EncodedBytes <= 0 || m.ContentBytes < 0 || !validDigest(m.EncodedSHA256) || !validDigest(m.ContentSHA256) { + return errors.New("native .zst marker has invalid sizes or digests") + } + return nil +} + +func (m Marker) encode() ([]byte, error) { + if err := m.Validate(); err != nil { + return nil, err + } + return json.Marshal(m) +} + +func decodeMarker(data []byte) (Marker, error) { + var marker Marker + if err := json.Unmarshal(data, &marker); err != nil { + return Marker{}, fmt.Errorf("decode native .zst marker: %w", err) + } + return marker, marker.Validate() +} + +// ManifestKey returns the key of a marker manifest path such as +// manifests/native-zst/.json. +func ManifestKey(manifest string) (string, error) { + if path.Dir(manifest) != "manifests/"+manifestDir || path.Ext(manifest) != ".json" { + return "", fmt.Errorf("unsafe native .zst manifest %q", manifest) + } + key := strings.TrimSuffix(path.Base(manifest), ".json") + if !strings.HasPrefix(key, "zst-") || strings.ContainsAny(key, `/\.`+"\x00") { + return "", fmt.Errorf("unsafe native .zst manifest %q", manifest) + } + return key, nil +} + +// ManifestRoot holds one manifest per folded file. Every manifest walker in +// the store (pack references, recovery, GC, doctor) descends into it, so its +// objects stay referenced exactly as long as the manifest exists. +func ManifestRoot(store string) string { + return filepath.Join(store, "manifests", manifestDir) +} + +func manifestRelative(key string) string { return "manifests/" + manifestDir + "/" + key + ".json" } + +func ManifestPath(store string, marker Marker) (string, error) { + key, err := ManifestKey(marker.Manifest) + if err != nil { + return "", err + } + return filepath.Join(ManifestRoot(store), key+".json"), nil +} + +func StateRoot(store string) string { return filepath.Join(store, stateDir) } + +// IsCandidateName reports whether a file name is a Codex-compressed rollout. +func IsCandidateName(name string) bool { + return strings.HasPrefix(name, "rollout-") && strings.HasSuffix(name, Suffix) && !strings.HasPrefix(name, "._") +} + +func validDigest(value string) bool { + decoded, err := hex.DecodeString(value) + return err == nil && len(decoded) == 32 +} diff --git a/internal/nativezst/nativezst_test.go b/internal/nativezst/nativezst_test.go new file mode 100644 index 0000000..358f971 --- /dev/null +++ b/internal/nativezst/nativezst_test.go @@ -0,0 +1,539 @@ +package nativezst + +import ( + "bytes" + "context" + "errors" + "fmt" + "io" + "math/rand" + "os" + "path/filepath" + "sync" + "testing" + "time" + + "github.com/samekind/codexfold/internal/codexzstd" + "github.com/samekind/codexfold/internal/pack" + "github.com/samekind/codexfold/internal/storage" +) + +func requireSupport(t *testing.T) { + t.Helper() + if !supported() || !codexzstd.Supported() { + t.Skip("folded .zst files need cgo on darwin or linux") + } +} + +// rolloutContent looks like a forked rollout: a long unique history replayed +// further back than zstd's 2 MiB window at level 3, which per-file zstd +// cannot share and content-defined chunking can. +func rolloutContent(seed int64, lines int) []byte { + random := rand.New(rand.NewSource(seed)) + var history bytes.Buffer + for line := 0; line < lines || history.Len() < 2400<<10; line++ { + payload := make([]byte, 100+random.Intn(3000)) + for i := range payload { + payload[i] = "abcdefghij klmnop"[random.Intn(17)] + } + fmt.Fprintf(&history, `{"ordinal":%d,"text":%q}`+"\n", line, payload) + } + return append(append([]byte(nil), history.Bytes()...), history.Bytes()...) +} + +// writeCodexZst writes what Codex's compression worker leaves behind. +func writeCodexZst(t *testing.T, path string, content []byte, modTime time.Time) []byte { + t.Helper() + var encoded bytes.Buffer + if err := codexzstd.Encode(context.Background(), &encoded, bytes.NewReader(content), int64(len(content))); err != nil { + t.Fatal(err) + } + if err := os.MkdirAll(filepath.Dir(path), 0o755); err != nil { + t.Fatal(err) + } + if err := os.WriteFile(path, encoded.Bytes(), 0o640); err != nil { + t.Fatal(err) + } + if err := os.Chtimes(path, modTime, modTime); err != nil { + t.Fatal(err) + } + return encoded.Bytes() +} + +func readAllAt(t *testing.T, reader *Reader, chunk int) []byte { + t.Helper() + var out bytes.Buffer + buffer := make([]byte, chunk) + for offset := int64(0); ; { + n, err := reader.ReadAt(buffer, offset) + out.Write(buffer[:n]) + offset += int64(n) + if err == io.EOF || n == 0 { + break + } + if err != nil { + t.Fatal(err) + } + } + return out.Bytes() +} + +func fixture(t *testing.T) (string, string) { + t.Helper() + root := t.TempDir() + store := filepath.Join(root, "fold-store") + native := filepath.Join(root, "fold-native") + if err := os.MkdirAll(store, 0o700); err != nil { + t.Fatal(err) + } + return store, native +} + +func TestFoldConvertServeAndRestore(t *testing.T) { + requireSupport(t) + store, native := fixture(t) + path := filepath.Join(native, "sessions", "2026", "09", "01", "rollout-2026-09-01T10-00-00-0199aaaa-0000-7000-8000-000000000001.jsonl.zst") + modTime := time.Date(2026, 9, 1, 10, 0, 0, 0, time.UTC) + content := rolloutContent(1, 300) + original := writeCodexZst(t, path, content, modTime) + if err := setTestXattr(path, "user.example.kept", []byte("value")); err != nil { + t.Fatal(err) + } + + source := NewSource(store, testPacks{}) + defer source.Close() + prepared, err := Prepare(context.Background(), source, path) + if err != nil { + t.Fatal(err) + } + before, _ := os.Lstat(path) + if err := Convert(store, prepared); err != nil { + t.Fatal(err) + } + after, err := os.Lstat(path) + if err != nil { + t.Fatal(err) + } + if !os.SameFile(before, after) || after.Size() != int64(len(original)) || !after.ModTime().Equal(modTime) || after.Mode().Perm() != 0o640 { + t.Fatalf("conversion changed identity: size=%d mtime=%v mode=%v", after.Size(), after.ModTime(), after.Mode()) + } + if !dataReleased(after) { + t.Fatalf("converted file still allocates %d bytes", allocatedBytes(after)) + } + if value := make([]byte, 16); true { + n, err := getTestXattr(path, "user.example.kept", value) + if err != nil || string(value[:n]) != "value" { + t.Fatalf("unrelated xattr lost: %q %v", value[:n], err) + } + } + marker, folded, err := ReadMarker(path) + if err != nil || !folded || marker != prepared.Marker { + t.Fatalf("marker = %#v %t %v", marker, folded, err) + } + + reader, err := source.Open(marker) + if err != nil { + t.Fatal(err) + } + if got := readAllAt(t, reader, 64<<10); !bytes.Equal(got, original) { + t.Fatal("served bytes differ from the original .zst") + } + // Reads behind and ahead of the encoder cursor. + for _, offset := range []int64{int64(len(original)) - 100, 10, int64(len(original)) / 2, 0} { + buffer := make([]byte, 100) + n, err := reader.ReadAt(buffer, offset) + if err != nil && err != io.EOF { + t.Fatal(err) + } + if !bytes.Equal(buffer[:n], original[offset:offset+int64(n)]) { + t.Fatalf("read at %d differs", offset) + } + } + _ = reader.Close() + + if err := Restore(context.Background(), source, path); err != nil { + t.Fatal(err) + } + restored, err := os.ReadFile(path) + if err != nil || !bytes.Equal(restored, original) { + t.Fatalf("restore failed: %v", err) + } + if _, folded, _ := ReadMarker(path); folded { + t.Fatal("restore kept the marker") + } + if info, _ := os.Lstat(path); !info.ModTime().Equal(modTime) { + t.Fatalf("restore changed mtime to %v", info.ModTime()) + } +} + +func TestServeFromPackAfterLooseRetirement(t *testing.T) { + requireSupport(t) + store, native := fixture(t) + path := filepath.Join(native, "archived_sessions", "rollout-2026-09-02T10-00-00-0199aaaa-0000-7000-8000-000000000002.jsonl.zst") + original := writeCodexZst(t, path, rolloutContent(2, 200), time.Now().Add(-48*time.Hour)) + source := NewSource(store, testPacks{}) + defer source.Close() + prepared, err := Prepare(context.Background(), source, path) + if err != nil { + t.Fatal(err) + } + if err := Convert(store, prepared); err != nil { + t.Fatal(err) + } + if _, err := pack.Build(context.Background(), store, pack.BuildOptions{}); err != nil { + t.Fatal(err) + } + if _, err := pack.RetireLoose(context.Background(), store, pack.RetireLooseOptions{Apply: true}); err != nil { + t.Fatal(err) + } + reader, err := source.Open(prepared.Marker) + if err != nil { + t.Fatal(err) + } + defer reader.Close() + if got := readAllAt(t, reader, 1<<20); !bytes.Equal(got, original) { + t.Fatal("bytes served from the pack differ") + } +} + +func TestSecondCopyReusesStoredContent(t *testing.T) { + requireSupport(t) + store, native := fixture(t) + content := rolloutContent(3, 200) + first := filepath.Join(native, "sessions", "rollout-a-0199aaaa-0000-7000-8000-000000000003.jsonl.zst") + second := filepath.Join(native, "sessions", "rollout-b-0199aaaa-0000-7000-8000-000000000004.jsonl.zst") + writeCodexZst(t, first, content, time.Now().Add(-time.Hour)) + writeCodexZst(t, second, content, time.Now().Add(-time.Hour)) + source := NewSource(store, testPacks{}) + defer source.Close() + one, err := Prepare(context.Background(), source, first) + if err != nil { + t.Fatal(err) + } + two, err := Prepare(context.Background(), source, second) + if err != nil { + t.Fatal(err) + } + if one.NewStoredBytes == 0 || two.NewStoredBytes != 0 { + t.Fatalf("new stored bytes = %d then %d, want >0 then 0", one.NewStoredBytes, two.NewStoredBytes) + } + if one.Marker.Manifest == two.Marker.Manifest { + t.Fatal("two files share one manifest") + } +} + +func TestPrepareRejectsForeignZstd(t *testing.T) { + requireSupport(t) + store, native := fixture(t) + path := filepath.Join(native, "sessions", "rollout-c-0199aaaa-0000-7000-8000-000000000005.jsonl.zst") + if err := os.MkdirAll(filepath.Dir(path), 0o755); err != nil { + t.Fatal(err) + } + // A checksummed frame is valid zstd that Codex never writes. + if err := os.WriteFile(path, []byte{0x28, 0xb5, 0x2f, 0xfd, 0x24, 0x02, 0x11, 0x00, 0x00, 0x7b, 0x7d, 0x00, 0x00, 0x00, 0x00}, 0o644); err != nil { + t.Fatal(err) + } + source := NewSource(store, testPacks{}) + defer source.Close() + if _, err := Prepare(context.Background(), source, path); err == nil { + t.Fatal("foreign zstd file was prepared") + } + if entries, _ := os.ReadDir(ManifestRoot(store)); len(entries) != 0 { + t.Fatalf("rejected file left %d manifests", len(entries)) + } +} + +func TestSweepReleasesOnlyAfterTwoMissedScans(t *testing.T) { + requireSupport(t) + store, native := fixture(t) + keep := filepath.Join(native, "sessions", "rollout-keep-0199aaaa-0000-7000-8000-000000000006.jsonl.zst") + gone := filepath.Join(native, "sessions", "rollout-gone-0199aaaa-0000-7000-8000-000000000007.jsonl.zst") + writeCodexZst(t, keep, rolloutContent(4, 100), time.Now().Add(-time.Hour)) + writeCodexZst(t, gone, rolloutContent(5, 100), time.Now().Add(-time.Hour)) + source := NewSource(store, testPacks{}) + defer source.Close() + var prepared []Prepared + for _, path := range []string{keep, gone} { + p, err := Prepare(context.Background(), source, path) + if err != nil { + t.Fatal(err) + } + if err := Convert(store, p); err != nil { + t.Fatal(err) + } + prepared = append(prepared, p) + } + if err := os.Remove(gone); err != nil { + t.Fatal(err) + } + now := time.Now() + sweep := func(at time.Time) SweepResult { + t.Helper() + scan, err := ScanNative(context.Background(), native) + if err != nil { + t.Fatal(err) + } + result, err := Sweep(context.Background(), store, scan, at) + if err != nil { + t.Fatal(err) + } + return result + } + if result := sweep(now); result.Released != 0 || result.Pending != 1 || result.Referenced != 1 { + t.Fatalf("young manifest sweep = %#v", result) + } + later := now.Add(sweepGrace + time.Minute) + if result := sweep(later); result.Released != 0 { + t.Fatalf("first missed scan released: %#v", result) + } + if result := sweep(later.Add(sweepGrace / 2)); result.Released != 0 { + t.Fatalf("second scan inside the grace released: %#v", result) + } + if result := sweep(later.Add(sweepGrace + time.Minute)); result.Released != 1 || result.Referenced != 1 { + t.Fatalf("second missed scan = %#v", result) + } + if _, err := os.Stat(prepared[1].ManifestPath); !os.IsNotExist(err) { + t.Fatalf("released manifest still present: %v", err) + } + if _, err := os.Stat(prepared[0].ManifestPath); err != nil { + t.Fatalf("referenced manifest removed: %v", err) + } +} + +func TestSweepRepairsALostMarker(t *testing.T) { + requireSupport(t) + store, native := fixture(t) + path := filepath.Join(native, "sessions", "rollout-lost-0199aaaa-0000-7000-8000-000000000008.jsonl.zst") + original := writeCodexZst(t, path, rolloutContent(6, 100), time.Now().Add(-time.Hour)) + source := NewSource(store, testPacks{}) + defer source.Close() + prepared, err := Prepare(context.Background(), source, path) + if err != nil { + t.Fatal(err) + } + if err := Convert(store, prepared); err != nil { + t.Fatal(err) + } + if err := removeTestXattr(path, MarkerAttribute); err != nil { + t.Fatal(err) + } + scan, err := ScanNative(context.Background(), native) + if err != nil { + t.Fatal(err) + } + result, err := Sweep(context.Background(), store, scan, time.Now().Add(2*sweepGrace)) + if err != nil || result.Repaired != 1 || result.Released != 0 { + t.Fatalf("repair sweep = %#v, %v", result, err) + } + marker, folded, err := ReadMarker(path) + if err != nil || !folded || marker != prepared.Marker { + t.Fatalf("repaired marker = %#v %t %v", marker, folded, err) + } + reader, err := source.Open(marker) + if err != nil { + t.Fatal(err) + } + defer reader.Close() + if !bytes.Equal(readAllAt(t, reader, 1<<20), original) { + t.Fatal("repaired file serves different bytes") + } +} + +func TestScanFootprint(t *testing.T) { + requireSupport(t) + store, native := fixture(t) + content := rolloutContent(7, 100) + folded := filepath.Join(native, "sessions", "rollout-f-0199aaaa-0000-7000-8000-000000000009.jsonl.zst") + plain := filepath.Join(native, "sessions", "rollout-p-0199aaaa-0000-7000-8000-000000000010.jsonl.zst") + jsonl := filepath.Join(native, "sessions", "rollout-j-0199aaaa-0000-7000-8000-000000000011.jsonl") + encoded := writeCodexZst(t, folded, content, time.Now().Add(-time.Hour)) + writeCodexZst(t, plain, content, time.Now().Add(-time.Hour)) + if err := os.WriteFile(jsonl, content, 0o644); err != nil { + t.Fatal(err) + } + source := NewSource(store, testPacks{}) + defer source.Close() + prepared, err := Prepare(context.Background(), source, folded) + if err != nil { + t.Fatal(err) + } + if err := Convert(store, prepared); err != nil { + t.Fatal(err) + } + scan, err := ScanNative(context.Background(), native) + if err != nil { + t.Fatal(err) + } + footprint := scan.Footprint + if footprint.Files != 3 || footprint.FoldedFiles != 1 || footprint.CompressedFiles != 1 || len(scan.Candidates) != 1 { + t.Fatalf("footprint = %#v", footprint) + } + if footprint.LogicalBytes != 3*int64(len(content)) || footprint.CompressedBytes != int64(len(encoded)) { + t.Fatalf("logical=%d compressed=%d", footprint.LogicalBytes, footprint.CompressedBytes) + } +} + +type testPacks struct{} + +func (testPacks) CurrentGeneration(store string) (string, error) { + return pack.CurrentGeneration(store) +} + +func (testPacks) Open(store string) (Pack, error) { return pack.Open(store, pack.OpenOptions{}) } + +func TestPrepareStreamsAndPricesTheFoldExactly(t *testing.T) { + requireSupport(t) + store, native := fixture(t) + content := rolloutContent(8, 200) + first := filepath.Join(native, "sessions", "rollout-t1-0199aaaa-0000-7000-8000-000000000012.jsonl.zst") + second := filepath.Join(native, "sessions", "rollout-t2-0199aaaa-0000-7000-8000-000000000013.jsonl.zst") + original := writeCodexZst(t, first, content, time.Now().Add(-time.Hour)) + writeCodexZst(t, second, content, time.Now().Add(-time.Hour)) + source := NewSource(store, testPacks{}) + defer source.Close() + one, err := Prepare(context.Background(), source, first) + if err != nil { + t.Fatal(err) + } + two, err := Prepare(context.Background(), source, second) + if err != nil { + t.Fatal(err) + } + if one.NewStoredBytes == 0 || two.NewStoredBytes != 0 { + t.Fatalf("new stored bytes = %d then %d", one.NewStoredBytes, two.NewStoredBytes) + } + if entries, _ := os.ReadDir(filepath.Join(StateRoot(store), "staging")); len(entries) != 0 { + t.Fatalf("prepare staged %d files", len(entries)) + } + reader, err := source.Open(two.Marker) + if err != nil { + t.Fatal(err) + } + defer reader.Close() + if !bytes.Equal(readAllAt(t, reader, 1<<20), original) { + t.Fatal("dry-run priced fold serves different bytes") + } +} + +type fixedChecker struct { + report storage.BudgetReport + err error +} + +func (c fixedChecker) Check(context.Context, storage.Projection) (storage.Assessment, error) { + return storage.Assessment{Budget: c.report}, c.err +} + +func TestReclaimingGuardDigsOutOnlyWhenItFreesMore(t *testing.T) { + reserve := fmt.Errorf("%w: fold rejected", storage.ErrBudgetExceeded) + small := storage.Projection{AdditionalPersistentBytes: 2 << 20, TemporaryBytes: 8 << 20, TemporaryPersistentOverlapBytes: 2 << 20} + cases := []struct { + name string + report storage.BudgetReport + reclaim int64 + allowed bool + }{ + {"frees more above the floor", storage.BudgetReport{Rejections: []storage.RejectionReason{storage.RejectionFreeSpaceReserve}, FreeSpaceAfterPeak: 2 << 30}, 1 << 30, true}, + {"frees too little", storage.BudgetReport{Rejections: []storage.RejectionReason{storage.RejectionFreeSpaceReserve}, FreeSpaceAfterPeak: 2 << 30}, 16 << 20, false}, + {"below the floor", storage.BudgetReport{Rejections: []storage.RejectionReason{storage.RejectionFreeSpaceReserve}, FreeSpaceAfterPeak: 512 << 20}, 1 << 30, false}, + {"another limit", storage.BudgetReport{Rejections: []storage.RejectionReason{storage.RejectionFreeSpaceReserve, storage.RejectionPhysicalBudget}, FreeSpaceAfterPeak: 2 << 30}, 1 << 30, false}, + } + for _, tc := range cases { + guard := reclaimingGuard{inner: fixedChecker{report: tc.report, err: reserve}, reclaim: tc.reclaim} + _, err := guard.Check(context.Background(), small) + if (err == nil) != tc.allowed { + t.Errorf("%s: err = %v, allowed %t", tc.name, err, tc.allowed) + } + } + large := storage.Projection{AdditionalPersistentBytes: 1 << 30} + guard := reclaimingGuard{inner: fixedChecker{report: cases[0].report, err: reserve}, reclaim: 64 << 30} + if _, err := guard.Check(context.Background(), large); err == nil { + t.Error("a large fold ran below the reserve") + } +} + +func TestPrepareKeepsAZstThatFoldingWouldNotShrink(t *testing.T) { + requireSupport(t) + store, native := fixture(t) + random := rand.New(rand.NewSource(9)) + var content bytes.Buffer + for line := 0; line < 300; line++ { + payload := make([]byte, 200+random.Intn(800)) + for i := range payload { + payload[i] = "abcdefghij klmnop"[random.Intn(17)] + } + fmt.Fprintf(&content, `{"ordinal":%d,"text":%q}`+"\n", line, payload) + } + path := filepath.Join(native, "sessions", "rollout-u-0199aaaa-0000-7000-8000-000000000014.jsonl.zst") + original := writeCodexZst(t, path, content.Bytes(), time.Now().Add(-time.Hour)) + source := NewSource(store, testPacks{}) + defer source.Close() + if _, err := Prepare(context.Background(), source, path); !errors.Is(err, ErrNotWorthwhile) { + t.Fatalf("unique rollout prepare = %v, want ErrNotWorthwhile", err) + } + if entries, _ := os.ReadDir(ManifestRoot(store)); len(entries) != 0 { + t.Fatalf("declined fold left %d manifests", len(entries)) + } + if got, err := os.ReadFile(path); err != nil || !bytes.Equal(got, original) { + t.Fatalf("declined fold touched the file: %v", err) + } +} + +func TestConcurrentOutOfOrderHandlesShareOneEncoder(t *testing.T) { + requireSupport(t) + store, native := fixture(t) + path := filepath.Join(native, "sessions", "rollout-c-0199aaaa-0000-7000-8000-000000000015.jsonl.zst") + original := writeCodexZst(t, path, rolloutContent(15, 300), time.Now().Add(-time.Hour)) + source := NewSource(store, testPacks{}) + defer source.Close() + prepared, err := Prepare(context.Background(), source, path) + if err != nil { + t.Fatal(err) + } + if err := Convert(store, prepared); err != nil { + t.Fatal(err) + } + before := streamsStarted.Load() + // The FSKit extension prefetches through several handles at once, each + // taking whichever block is next, so every handle sees jumps both ways. + const block = 64 << 10 + blocks := (len(original) + block - 1) / block + order := rand.New(rand.NewSource(3)).Perm(blocks) + got := make([]byte, len(original)) + var group sync.WaitGroup + errs := make(chan error, 4) + for worker := 0; worker < 4; worker++ { + reader, err := source.Open(prepared.Marker) + if err != nil { + t.Fatal(err) + } + group.Add(1) + go func(reader *Reader, worker int) { + defer group.Done() + defer reader.Close() + for index := worker; index < len(order); index += 4 { + offset := int64(order[index]) * block + n, err := reader.ReadAt(got[offset:min(offset+block, int64(len(got)))], offset) + if err != nil && err != io.EOF { + errs <- err + return + } + if n == 0 { + errs <- fmt.Errorf("empty read at %d", offset) + return + } + } + }(reader, worker) + } + group.Wait() + close(errs) + for err := range errs { + t.Fatal(err) + } + if !bytes.Equal(got, original) { + t.Fatal("concurrent out-of-order reads returned different bytes") + } + if started := streamsStarted.Load() - before; started != 1 { + t.Fatalf("encoders started = %d, want 1", started) + } +} diff --git a/internal/nativezst/platform_darwin.go b/internal/nativezst/platform_darwin.go new file mode 100644 index 0000000..5a9c728 --- /dev/null +++ b/internal/nativezst/platform_darwin.go @@ -0,0 +1,58 @@ +//go:build darwin + +package nativezst + +import ( + "os" + "syscall" + "time" + "unsafe" + + "golang.org/x/sys/unix" +) + +var noAttribute = unix.ENOATTR + +type fpunchhole struct { + flags uint32 + reserved uint32 + offset int64 + length int64 +} + +// releaseBlocks frees every data block while keeping the size, so reads +// outside the mount see zeros and the inode keeps its identity. +func releaseBlocks(file *os.File, size int64) error { + arguments := fpunchhole{length: (size + 4095) &^ 4095} + if _, _, errno := unix.Syscall(unix.SYS_FCNTL, file.Fd(), uintptr(unix.F_PUNCHHOLE), uintptr(unsafe.Pointer(&arguments))); errno != 0 { + return &os.PathError{Op: "fcntl F_PUNCHHOLE", Path: file.Name(), Err: errno} + } + return nil +} + +// durableSync reaches the disk itself: plain fsync on macOS stops at the +// drive cache, and the marker must survive power loss before data is freed. +func durableSync(file *os.File) error { + if _, err := unix.FcntlInt(file.Fd(), unix.F_FULLFSYNC, 0); err != nil { + return file.Sync() + } + return nil +} + +func fileTimes(info os.FileInfo) (time.Time, time.Time) { + if stat, ok := info.Sys().(*syscall.Stat_t); ok { + return time.Unix(stat.Atimespec.Unix()), info.ModTime() + } + return info.ModTime(), info.ModTime() +} + +func changeTime(info os.FileInfo) time.Time { + if stat, ok := info.Sys().(*syscall.Stat_t); ok { + return time.Unix(stat.Ctimespec.Unix()) + } + return time.Time{} +} + +// metadataAllocation is zero: APFS does not count extended attributes in a +// file's allocated blocks. +func metadataAllocation(os.FileInfo) int64 { return 0 } diff --git a/internal/nativezst/platform_linux.go b/internal/nativezst/platform_linux.go new file mode 100644 index 0000000..de87dfa --- /dev/null +++ b/internal/nativezst/platform_linux.go @@ -0,0 +1,45 @@ +//go:build linux + +package nativezst + +import ( + "os" + "syscall" + "time" + + "golang.org/x/sys/unix" +) + +var noAttribute = unix.ENODATA + +func releaseBlocks(file *os.File, size int64) error { + if err := unix.Fallocate(int(file.Fd()), unix.FALLOC_FL_PUNCH_HOLE|unix.FALLOC_FL_KEEP_SIZE, 0, size); err != nil { + return &os.PathError{Op: "fallocate punch hole", Path: file.Name(), Err: err} + } + return nil +} + +func durableSync(file *os.File) error { return file.Sync() } + +func fileTimes(info os.FileInfo) (time.Time, time.Time) { + if stat, ok := info.Sys().(*syscall.Stat_t); ok { + return time.Unix(stat.Atim.Unix()), info.ModTime() + } + return info.ModTime(), info.ModTime() +} + +func changeTime(info os.FileInfo) time.Time { + if stat, ok := info.Sys().(*syscall.Stat_t); ok { + return time.Unix(stat.Ctim.Unix()) + } + return time.Time{} +} + +// metadataAllocation is what a file's allocation may include beyond its data: +// ext4 and XFS count a block of extended attributes in st_blocks. +func metadataAllocation(info os.FileInfo) int64 { + if stat, ok := info.Sys().(*syscall.Stat_t); ok && stat.Blksize > 0 { + return int64(stat.Blksize) + } + return 4096 +} diff --git a/internal/nativezst/platform_other.go b/internal/nativezst/platform_other.go new file mode 100644 index 0000000..9f79810 --- /dev/null +++ b/internal/nativezst/platform_other.go @@ -0,0 +1,30 @@ +//go:build !darwin && !linux + +package nativezst + +import ( + "os" + "time" +) + +func ReadMarker(string) (Marker, bool, error) { return Marker{}, false, nil } + +func writeMarker(*os.File, Marker) error { return ErrUnsupported } + +func removeMarker(*os.File) error { return ErrUnsupported } + +func releaseBlocks(*os.File, int64) error { return ErrUnsupported } + +func durableSync(file *os.File) error { return file.Sync() } + +func allocatedBytes(info os.FileInfo) int64 { return info.Size() } + +func metadataAllocation(os.FileInfo) int64 { return 0 } + +func inode(os.FileInfo) uint64 { return 0 } + +func fileTimes(info os.FileInfo) (time.Time, time.Time) { return info.ModTime(), info.ModTime() } + +func changeTime(os.FileInfo) time.Time { return time.Time{} } + +func supported() bool { return false } diff --git a/internal/nativezst/platform_unix.go b/internal/nativezst/platform_unix.go new file mode 100644 index 0000000..764cc8e --- /dev/null +++ b/internal/nativezst/platform_unix.go @@ -0,0 +1,64 @@ +//go:build darwin || linux + +package nativezst + +import ( + "errors" + "os" + "syscall" + + "golang.org/x/sys/unix" +) + +// ReadMarker reports the marker on path, without following symlinks. +func ReadMarker(path string) (Marker, bool, error) { + buffer := make([]byte, maxMarkerBytes) + n, err := unix.Lgetxattr(path, MarkerAttribute, buffer) + if err != nil { + if missingAttribute(err) { + return Marker{}, false, nil + } + return Marker{}, false, &os.PathError{Op: "getxattr", Path: path, Err: err} + } + marker, err := decodeMarker(buffer[:n]) + return marker, true, err +} + +func writeMarker(file *os.File, marker Marker) error { + data, err := marker.encode() + if err != nil { + return err + } + if err := unix.Fsetxattr(int(file.Fd()), MarkerAttribute, data, 0); err != nil { + return &os.PathError{Op: "setxattr", Path: file.Name(), Err: err} + } + return nil +} + +func removeMarker(file *os.File) error { + if err := unix.Fremovexattr(int(file.Fd()), MarkerAttribute); err != nil && !missingAttribute(err) { + return &os.PathError{Op: "removexattr", Path: file.Name(), Err: err} + } + return nil +} + +func missingAttribute(err error) bool { + return errors.Is(err, noAttribute) || errors.Is(err, unix.ENOTSUP) || errors.Is(err, unix.EOPNOTSUPP) +} + +// allocatedBytes is the space a file's data blocks occupy. +func allocatedBytes(info os.FileInfo) int64 { + if stat, ok := info.Sys().(*syscall.Stat_t); ok { + return int64(stat.Blocks) * 512 + } + return info.Size() +} + +func inode(info os.FileInfo) uint64 { + if stat, ok := info.Sys().(*syscall.Stat_t); ok { + return uint64(stat.Ino) + } + return 0 +} + +func supported() bool { return true } diff --git a/internal/nativezst/source.go b/internal/nativezst/source.go new file mode 100644 index 0000000..297e0e4 --- /dev/null +++ b/internal/nativezst/source.go @@ -0,0 +1,560 @@ +package nativezst + +import ( + "context" + "errors" + "fmt" + "io" + "os" + "path/filepath" + "sync" + "sync/atomic" + "time" + + "github.com/samekind/codexfold/internal/codexzstd" + "github.com/samekind/codexfold/internal/fold" +) + +// Pack is one opened, published pack generation. Opening it must hold the +// generation's lease until Close. +type Pack interface { + fold.ObjectReader + Generation() string + Close() error +} + +// PackOpener reaches the store's packs. It is injected so the mount core does +// not depend on the pack builder and its database. +type PackOpener interface { + // CurrentGeneration fails with os.ErrNotExist before the first pack. + CurrentGeneration(store string) (string, error) + Open(store string) (Pack, error) +} + +// Source serves folded .zst content from the store. It shares one pack +// resolver per published generation; a resolver holds its generation's lease, +// so a reader that opened against an old generation keeps it alive. +type Source struct { + store string + packs PackOpener + loose *fold.ObjectStore + mu sync.Mutex + current *sharedResolver + streams map[string]*sharedStream + closed bool +} + +type sharedResolver struct { + resolver Pack + generation string + references int + retired bool +} + +// NewSource serves from loose objects only when packs is nil. +func NewSource(store string, packs PackOpener) *Source { + return &Source{store: store, packs: packs, loose: fold.NewObjectStore(store)} +} + +func (s *Source) Store() string { return s.store } + +// acquire returns the resolver for the current generation, or nil before the +// first pack exists. Callers release it exactly once. +func (s *Source) acquire() (*sharedResolver, error) { + if s.packs == nil { + return nil, nil + } + generation, err := s.packs.CurrentGeneration(s.store) + if errors.Is(err, os.ErrNotExist) { + return nil, nil + } + if err != nil { + return nil, err + } + s.mu.Lock() + defer s.mu.Unlock() + if s.closed { + return nil, errors.New("native .zst source is closed") + } + if s.current != nil && s.current.generation == generation { + s.current.references++ + return s.current, nil + } + resolver, err := s.packs.Open(s.store) + if err != nil { + return nil, err + } + if s.current != nil { + s.current.retired = true + if s.current.references == 0 { + _ = s.current.resolver.Close() + } + } + s.current = &sharedResolver{resolver: resolver, generation: resolver.Generation(), references: 1} + return s.current, nil +} + +func (s *Source) release(shared *sharedResolver) { + if shared == nil { + return + } + s.mu.Lock() + defer s.mu.Unlock() + shared.references-- + if shared.references == 0 && (shared.retired || s.closed) { + _ = shared.resolver.Close() + if s.current == shared { + s.current = nil + } + } +} + +func (s *Source) Close() error { + s.mu.Lock() + defer s.mu.Unlock() + s.closed = true + for key, stream := range s.streams { + if stream.expiry != nil { + stream.expiry.Stop() + } + delete(s.streams, key) + go stream.shutdown() + } + if s.current != nil && s.current.references == 0 { + err := s.current.resolver.Close() + s.current = nil + return err + } + return nil +} + +// ExistingObjects lets a fold reuse objects the current pack already holds. +func (s *Source) ExistingObjects() (fold.ObjectReader, func(), error) { + shared, err := s.acquire() + if err != nil { + return nil, nil, err + } + if shared == nil { + return nil, func() {}, nil + } + return shared.resolver, func() { s.release(shared) }, nil +} + +// openObject prefers loose objects, then the shared resolver, and retries +// once against a newer generation: a loose object may have been packed and +// retired after this reader's resolver was opened. +func (s *Source) openObject(ctx context.Context, ref fold.ObjectRef, shared **sharedResolver) (io.ReadCloser, error) { + if s.loose.HasObject(ref) { + if stream, err := s.loose.OpenObject(ctx, ref); err == nil { + return stream, nil + } + } + var lastErr error + for attempt := 0; attempt < 2; attempt++ { + if *shared != nil { + stream, err := (*shared).resolver.OpenObject(ctx, ref) + if err == nil { + return stream, nil + } + lastErr = err + } + if attempt == 0 { + next, err := s.acquire() + if err != nil { + return nil, err + } + if next == *shared { + s.release(next) + break + } + s.release(*shared) + *shared = next + if s.loose.HasObject(ref) { + if stream, err := s.loose.OpenObject(ctx, ref); err == nil { + return stream, nil + } + } + } + } + if lastErr == nil { + lastErr = os.ErrNotExist + } + return nil, fmt.Errorf("open folded object %s: %w", ref.SHA256, lastErr) +} + +func (s *Source) loadParts(marker Marker) ([]fold.Part, error) { + if err := marker.Validate(); err != nil { + return nil, err + } + if !codexzstd.Supported() { + return nil, codexzstd.ErrUnsupported + } + manifestPath, err := ManifestPath(s.store, marker) + if err != nil { + return nil, err + } + manifest, err := fold.LoadManifestPath(manifestPath) + if err != nil { + return nil, err + } + if manifest.Source.Bytes != marker.ContentBytes || manifest.Source.SHA256 != marker.ContentSHA256 { + return nil, fmt.Errorf("native .zst manifest %s does not match its marker", marker.Manifest) + } + return manifest.Parts, nil +} + +// Stream re-encodes the file once from the start, for a caller that reads it +// sequentially (a proof, a restore). It spills nothing to disk. +func (s *Source) Stream(ctx context.Context, marker Marker) (io.ReadCloser, error) { + parts, err := s.loadParts(marker) + if err != nil { + return nil, err + } + return s.encode(ctx, marker, parts), nil +} + +func (s *Source) encode(ctx context.Context, marker Marker, parts []fold.Part) io.ReadCloser { + ctx, cancel := context.WithCancel(ctx) + output, input := io.Pipe() + go func() { + content := &contentReader{ctx: ctx, source: s, parts: parts} + err := codexzstd.Encode(ctx, input, content, marker.ContentBytes) + content.close() + input.CloseWithError(err) + }() + return &pipeStream{PipeReader: output, cancel: cancel} +} + +type pipeStream struct { + *io.PipeReader + cancel context.CancelFunc +} + +func (p *pipeStream) Close() error { + p.cancel() + return p.PipeReader.Close() +} + +// Open returns a random-access reader of the exact original .zst bytes. All +// readers of one file share one encoder: zstd output can only be produced +// from the start of the frame, and the FSKit extension reads a file through +// several handles at once, out of order, so a stream per handle re-encoded +// the file from the beginning over and over. +func (s *Source) Open(marker Marker) (*Reader, error) { + key, err := ManifestKey(marker.Manifest) + if err != nil { + return nil, err + } + s.mu.Lock() + if s.streams == nil { + s.streams = make(map[string]*sharedStream) + } + stream := s.streams[key] + if stream != nil && stream.marker == marker && !stream.failed() { + stream.references++ + if stream.expiry != nil { + stream.expiry.Stop() + stream.expiry = nil + } + s.mu.Unlock() + return &Reader{stream: stream, marker: marker}, nil + } + s.mu.Unlock() + parts, err := s.loadParts(marker) + if err != nil { + return nil, err + } + stream, err = s.newSharedStream(key, marker, parts) + if err != nil { + return nil, err + } + s.mu.Lock() + if current := s.streams[key]; current != nil && current.marker == marker && !current.failed() { + current.references++ + if current.expiry != nil { + current.expiry.Stop() + current.expiry = nil + } + s.mu.Unlock() + stream.shutdown() + return &Reader{stream: current, marker: marker}, nil + } + if previous := s.streams[key]; previous != nil && previous.references == 0 { + previous.shutdown() + } + stream.references = 1 + s.streams[key] = stream + s.mu.Unlock() + return &Reader{stream: stream, marker: marker}, nil +} + +const ( + // A read asks the encoder for this much beyond what it needs, so the + // extension's next prefetch block is usually ready. + streamReadAhead = 8 << 20 + streamChunk = 256 << 10 + // An unused stream that got this far is kept a while for a reopen; one + // that only produced a file's head is cheaper to redo than to keep. + streamKeepBytes = 4 << 20 + streamKeepFor = 2 * time.Minute +) + +// streamsStarted counts shared encoders, for tests. +var streamsStarted atomic.Int64 + +// sharedStream is one encoder for one folded file, whose output is spilled +// to an unlinked temporary file as it is produced, so every byte is encoded +// once whatever order and handles the reads come through. +type sharedStream struct { + source *Source + key string + marker Marker + spill *os.File + encoded io.ReadCloser + mu sync.Mutex + advanced *sync.Cond + frontier int64 + target int64 + err error + closed bool + references int + expiry *time.Timer +} + +func (s *Source) newSharedStream(key string, marker Marker, parts []fold.Part) (*sharedStream, error) { + directory := filepath.Join(StateRoot(s.store), "spill") + if err := os.MkdirAll(directory, 0o700); err != nil { + return nil, err + } + spill, err := os.CreateTemp(directory, ".spill-*.tmp") + if err != nil { + return nil, err + } + // Unlinked at once: the space returns when the stream closes or the + // daemon exits, and nothing is left behind for a crash to strand. + _ = os.Remove(spill.Name()) + streamsStarted.Add(1) + stream := &sharedStream{source: s, key: key, marker: marker, spill: spill, encoded: s.encode(context.Background(), marker, parts)} + stream.advanced = sync.NewCond(&stream.mu) + go stream.produce() + return stream, nil +} + +func (t *sharedStream) failed() bool { + t.mu.Lock() + defer t.mu.Unlock() + return t.err != nil || t.closed +} + +func (t *sharedStream) produce() { + buffer := make([]byte, streamChunk) + for { + t.mu.Lock() + for !t.closed && t.err == nil && t.frontier >= t.target && t.frontier < t.marker.EncodedBytes { + t.advanced.Wait() + } + if t.closed || t.err != nil || t.frontier >= t.marker.EncodedBytes { + t.mu.Unlock() + return + } + offset := t.frontier + t.mu.Unlock() + + n, readErr := t.encoded.Read(buffer) + var writeErr error + if n > 0 { + _, writeErr = t.spill.WriteAt(buffer[:n], offset) + } + t.mu.Lock() + if writeErr == nil { + t.frontier += int64(n) + } + switch { + case writeErr != nil: + t.err = writeErr + case errors.Is(readErr, io.EOF): + if t.frontier != t.marker.EncodedBytes { + t.err = fmt.Errorf("folded .zst ended at %d of %d bytes", t.frontier, t.marker.EncodedBytes) + } + case readErr != nil: + t.err = readErr + } + t.advanced.Broadcast() + t.mu.Unlock() + } +} + +// await returns once [0, end) is in the spill file. +func (t *sharedStream) await(end int64) error { + t.mu.Lock() + defer t.mu.Unlock() + if want := min(end+streamReadAhead, t.marker.EncodedBytes); want > t.target { + t.target = want + t.advanced.Broadcast() + } + for t.frontier < end && t.err == nil && !t.closed { + t.advanced.Wait() + } + if t.frontier >= end { + return nil + } + if t.err != nil { + return t.err + } + return os.ErrClosed +} + +func (t *sharedStream) shutdown() { + t.mu.Lock() + if t.closed { + t.mu.Unlock() + return + } + t.closed = true + t.advanced.Broadcast() + t.mu.Unlock() + _ = t.encoded.Close() + _ = t.spill.Close() +} + +// release drops one reader. A stream that produced little is closed at +// once; a large one stays briefly for a reopen. +func (t *sharedStream) release() { + s := t.source + s.mu.Lock() + defer s.mu.Unlock() + t.references-- + if t.references > 0 { + return + } + t.mu.Lock() + frontier := t.frontier + t.mu.Unlock() + if frontier < streamKeepBytes || s.closed || t.failed() { + if s.streams[t.key] == t { + delete(s.streams, t.key) + } + go t.shutdown() + return + } + t.expiry = time.AfterFunc(streamKeepFor, func() { + s.mu.Lock() + if t.references != 0 { + s.mu.Unlock() + return + } + if s.streams[t.key] == t { + delete(s.streams, t.key) + } + s.mu.Unlock() + t.shutdown() + }) +} + +// Reader serves one open handle of a folded file. +type Reader struct { + stream *sharedStream + marker Marker + mu sync.Mutex + closed bool +} + +func (r *Reader) Size() int64 { return r.marker.EncodedBytes } + +func (r *Reader) Marker() Marker { return r.marker } + +func (r *Reader) ReadAt(destination []byte, offset int64) (int, error) { + r.mu.Lock() + closed := r.closed + r.mu.Unlock() + if closed { + return 0, os.ErrClosed + } + if offset < 0 { + return 0, errors.New("negative read offset") + } + size := r.marker.EncodedBytes + if offset >= size { + return 0, io.EOF + } + end := min(offset+int64(len(destination)), size) + if err := r.stream.await(end); err != nil { + return 0, err + } + n, err := r.stream.spill.ReadAt(destination[:end-offset], offset) + if err != nil && !errors.Is(err, io.EOF) { + return n, err + } + if n < len(destination) { + return n, io.EOF + } + return n, nil +} + +func (r *Reader) Close() error { + r.mu.Lock() + defer r.mu.Unlock() + if r.closed { + return nil + } + r.closed = true + r.stream.release() + return nil +} + +// contentReader concatenates a manifest's parts: decoded rollout content. It +// runs on the encoder goroutine and owns its resolver reference. +type contentReader struct { + ctx context.Context + source *Source + parts []fold.Part + shared *sharedResolver + opened bool + index int + current io.ReadCloser +} + +func (c *contentReader) close() { + if c.current != nil { + _ = c.current.Close() + c.current = nil + } + c.source.release(c.shared) + c.shared = nil +} + +func (c *contentReader) Read(destination []byte) (int, error) { + for { + if c.current == nil { + if c.index >= len(c.parts) { + return 0, io.EOF + } + if !c.opened { + shared, err := c.source.acquire() + if err != nil { + return 0, err + } + c.shared, c.opened = shared, true + } + stream, err := c.source.openObject(c.ctx, c.parts[c.index].Object, &c.shared) + if err != nil { + return 0, err + } + c.current = stream + c.index++ + } + n, err := c.current.Read(destination) + if errors.Is(err, io.EOF) { + closeErr := c.current.Close() + c.current = nil + if closeErr != nil { + return n, closeErr + } + if n > 0 { + return n, nil + } + continue + } + return n, err + } +} diff --git a/internal/nativezst/sweep.go b/internal/nativezst/sweep.go new file mode 100644 index 0000000..9db8a49 --- /dev/null +++ b/internal/nativezst/sweep.go @@ -0,0 +1,257 @@ +package nativezst + +import ( + "context" + "encoding/json" + "errors" + "io/fs" + "os" + "path/filepath" + "strings" + "time" + + "github.com/samekind/codexfold/internal/codexzstd" + "github.com/samekind/codexfold/internal/fold" +) + +// NativeFile is one folded or foldable file found under the native root. +type NativeFile struct { + Path string + Info os.FileInfo + Marker Marker + Folded bool +} + +// Scan is a walk of the native session tree. +type Scan struct { + Candidates []NativeFile + Folded []NativeFile + // References maps a manifest key to the folded files that use it. + References map[string][]string + // Unmarked indexes unfolded .zst files by inode, so the sweep can find a + // file whose marker was lost. + Unmarked map[uint64]NativeFile + Footprint Footprint +} + +// Footprint is what the native tree holds, decoded and on disk. +type Footprint struct { + Files int `json:"files"` + LogicalBytes int64 `json:"logical_bytes"` + PhysicalBytes int64 `json:"physical_bytes"` + CompressedFiles int `json:"compressed_files"` + CompressedBytes int64 `json:"compressed_bytes"` + FoldedFiles int `json:"folded_files"` + FoldedLogicalBytes int64 `json:"folded_logical_bytes"` + FoldedEncodedBytes int64 `json:"folded_encoded_bytes"` +} + +// ScanNative walks sessions/ and archived_sessions/ under root. A marker that +// cannot be read fails the scan: a sweep must never act on a partial view. +func ScanNative(ctx context.Context, root string) (Scan, error) { + scan := Scan{References: make(map[string][]string), Unmarked: make(map[uint64]NativeFile)} + for _, directory := range []string{"sessions", "archived_sessions"} { + err := filepath.WalkDir(filepath.Join(root, directory), func(path string, entry fs.DirEntry, walkErr error) error { + if walkErr != nil { + if errors.Is(walkErr, fs.ErrNotExist) { + return nil + } + return walkErr + } + if err := ctx.Err(); err != nil { + return err + } + if !entry.Type().IsRegular() || strings.HasPrefix(entry.Name(), "._") { + return nil + } + info, err := entry.Info() + if errors.Is(err, fs.ErrNotExist) { + return nil + } + if err != nil { + return err + } + scan.Footprint.Files++ + scan.Footprint.PhysicalBytes += allocatedBytes(info) + if !IsCandidateName(entry.Name()) { + scan.Footprint.LogicalBytes += info.Size() + return nil + } + marker, folded, err := ReadMarker(path) + if errors.Is(err, fs.ErrNotExist) { + return nil + } + if err != nil { + return err + } + file := NativeFile{Path: path, Info: info, Marker: marker, Folded: folded} + if folded { + key, _ := ManifestKey(marker.Manifest) + scan.References[key] = append(scan.References[key], path) + scan.Folded = append(scan.Folded, file) + scan.Footprint.FoldedFiles++ + scan.Footprint.FoldedLogicalBytes += marker.ContentBytes + scan.Footprint.FoldedEncodedBytes += marker.EncodedBytes + scan.Footprint.LogicalBytes += marker.ContentBytes + return nil + } + scan.Candidates = append(scan.Candidates, file) + scan.Unmarked[inode(info)] = file + scan.Footprint.CompressedFiles++ + scan.Footprint.CompressedBytes += info.Size() + scan.Footprint.LogicalBytes += decodedSize(path, info) + return nil + }) + if err != nil { + return Scan{}, err + } + } + return scan, nil +} + +func decodedSize(path string, info os.FileInfo) int64 { + file, err := os.Open(path) + if err != nil { + return info.Size() + } + defer file.Close() + if size, err := codexzstd.ContentSize(file); err == nil { + return size + } + return info.Size() +} + +// Manifests younger than this may belong to a fold that has not converted +// its file yet; a key must also be missing from two scans this far apart +// before its content is released. +const sweepGrace = 15 * time.Minute + +type sweepState struct { + Version int `json:"version"` + Missing map[string]int64 `json:"missing"` +} + +type SweepResult struct { + Manifests int `json:"manifests"` + Referenced int `json:"referenced"` + Pending int `json:"pending"` + Repaired int `json:"repaired"` + Released int `json:"released"` +} + +// Sweep releases manifests no folded file refers to any more: Codex deleted +// the file, or a restore made it real again. The released content leaves the +// store with the next pack build and GC. +func Sweep(ctx context.Context, store string, scan Scan, now time.Time) (SweepResult, error) { + var result SweepResult + entries, err := os.ReadDir(ManifestRoot(store)) + if errors.Is(err, fs.ErrNotExist) { + return result, nil + } + if err != nil { + return result, err + } + statePath := filepath.Join(StateRoot(store), "sweep.json") + state := sweepState{Version: 1, Missing: make(map[string]int64)} + if data, err := os.ReadFile(statePath); err == nil { + _ = json.Unmarshal(data, &state) + if state.Missing == nil { + state.Missing = make(map[string]int64) + } + } + seen := make(map[string]bool) + for _, entry := range entries { + if err := ctx.Err(); err != nil { + return result, err + } + if entry.IsDir() || filepath.Ext(entry.Name()) != ".json" { + continue + } + key := strings.TrimSuffix(entry.Name(), ".json") + if _, err := ManifestKey(manifestRelative(key)); err != nil { + continue + } + seen[key] = true + result.Manifests++ + if len(scan.References[key]) != 0 { + result.Referenced++ + delete(state.Missing, key) + continue + } + info, err := entry.Info() + if err != nil { + continue + } + if now.Sub(info.ModTime()) < sweepGrace { + result.Pending++ + continue + } + if repaired, err := repairMarker(store, key, scan); err != nil { + return result, err + } else if repaired { + result.Repaired++ + delete(state.Missing, key) + continue + } + first, missing := state.Missing[key] + if !missing { + state.Missing[key] = now.Unix() + result.Pending++ + continue + } + if now.Sub(time.Unix(first, 0)) < sweepGrace { + result.Pending++ + continue + } + if err := os.Remove(filepath.Join(ManifestRoot(store), entry.Name())); err != nil && !errors.Is(err, fs.ErrNotExist) { + return result, err + } + _ = os.Remove(journalPath(store, key)) + delete(state.Missing, key) + result.Released++ + } + for key := range state.Missing { + if !seen[key] { + delete(state.Missing, key) + } + } + return result, writeDurableJSON(statePath, state) +} + +// repairMarker restores the marker of a file that was converted but lost its +// extended attribute: its inode is still in the tree, empty and the right size. +func repairMarker(store string, key string, scan Scan) (bool, error) { + data, err := os.ReadFile(journalPath(store, key)) + if errors.Is(err, fs.ErrNotExist) { + return false, nil + } + if err != nil { + return false, err + } + var record journalRecord + if err := json.Unmarshal(data, &record); err != nil || record.Key != key { + return false, nil + } + file, found := scan.Unmarked[record.Inode] + if !found || file.Info.Size() != record.EncodedBytes || !dataReleased(file.Info) { + return false, nil + } + manifest, err := fold.LoadManifestPath(filepath.Join(ManifestRoot(store), key+".json")) + if err != nil { + return false, err + } + marker := Marker{ + Version: MarkerVersion, Kind: MarkerKind, Profile: codexzstd.Profile, Manifest: manifestRelative(key), + EncodedBytes: record.EncodedBytes, EncodedSHA256: record.EncodedSHA256, + ContentBytes: manifest.Source.Bytes, ContentSHA256: manifest.Source.SHA256, + } + handle, err := os.OpenFile(file.Path, os.O_RDWR, 0) + if err != nil { + return false, err + } + defer handle.Close() + if err := writeMarker(handle, marker); err != nil { + return false, err + } + return true, durableSync(handle) +} diff --git a/internal/nativezst/xattr_other_test.go b/internal/nativezst/xattr_other_test.go new file mode 100644 index 0000000..037f4ca --- /dev/null +++ b/internal/nativezst/xattr_other_test.go @@ -0,0 +1,9 @@ +//go:build !darwin && !linux + +package nativezst + +func setTestXattr(string, string, []byte) error { return ErrUnsupported } + +func getTestXattr(string, string, []byte) (int, error) { return 0, ErrUnsupported } + +func removeTestXattr(string, string) error { return ErrUnsupported } diff --git a/internal/nativezst/xattr_unix_test.go b/internal/nativezst/xattr_unix_test.go new file mode 100644 index 0000000..11b56e3 --- /dev/null +++ b/internal/nativezst/xattr_unix_test.go @@ -0,0 +1,17 @@ +//go:build darwin || linux + +package nativezst + +import "golang.org/x/sys/unix" + +func setTestXattr(path string, name string, value []byte) error { + return unix.Setxattr(path, name, value, 0) +} + +func getTestXattr(path string, name string, value []byte) (int, error) { + return unix.Getxattr(path, name, value) +} + +func removeTestXattr(path string, name string) error { + return unix.Removexattr(path, name) +} diff --git a/internal/pack/build.go b/internal/pack/build.go index 5aa8fb2..1d59e06 100644 --- a/internal/pack/build.go +++ b/internal/pack/build.go @@ -84,6 +84,7 @@ func Build(ctx context.Context, storeDir string, options BuildOptions) (BuildRes } defer references.Close() var packedSource *Resolver + var rewrite rewritePlan if previousGeneration != "" { packedSource, err = openGeneration(filepath.Join(storeDir, "packs", previousGeneration), -1, false) if err != nil { @@ -91,19 +92,11 @@ func Build(ctx context.Context, storeDir string, options BuildOptions) (BuildRes } defer packedSource.Close() if options.Incremental { - dropped, err := hasDroppedPackedObjects(ctx, references, packedSource) - if err != nil { - return BuildResult{}, err - } blockBytes := packedSource.index.BlockBytes if packedSource.v3 != nil { blockBytes = packedSource.v3.meta.BlockBytes } - packLimit, limitErr := compactPackFileLimit(packedSource, options.PackBytes) - if limitErr != nil { - return BuildResult{}, limitErr - } - if dropped || len(packedSource.packs) >= packLimit || options.BlockBytes < blockBytes { + if options.BlockBytes < blockBytes { options.Incremental = false } } @@ -130,7 +123,26 @@ func Build(ctx context.Context, storeDir string, options BuildOptions) (BuildRes if err != nil { return BuildResult{}, err } - estimatedBytes = min(estimatedBytes, incrementalBytes) + // Rewrite only the old pack files worth it that fit the free space: + // dropped objects no longer force copying the whole pack at once. + usage, usageErr := packFileUsage(ctx, storeDir, references, packedSource, options.BlockBytes) + if usageErr != nil { + return BuildResult{}, usageErr + } + packLimit, limitErr := compactPackFileLimit(packedSource, options.PackBytes) + if limitErr != nil { + return BuildResult{}, limitErr + } + allowance, allowanceErr := packRewriteAllowance(storeDir, incrementalBytes) + if allowanceErr != nil { + return BuildResult{}, allowanceErr + } + rewrite = choosePackRewrite(usage, allowance, packLimit, options.PackBytes, incrementalBytes) + if rewrite.full { + options.Incremental = false + } else { + estimatedBytes = min(estimatedBytes, incrementalBytes+rewrite.copyBytes) + } } storageAssessment, err := budget.Check(ctx, storage.Projection{ Operation: "pack-build", @@ -164,8 +176,12 @@ func Build(ctx context.Context, storeDir string, options BuildOptions) (BuildRes } defer objectsFile.Close() defer blocksFile.Close() + // SpeedDefault gives up on blocks whose only redundancy is entropy (base64 + // payloads, encrypted reasoning text) and leaves them raw. SpeedBetter + // entropy-codes them: on a real 15 GB pack it shrank the raw blocks by 30% + // for a slower background build and no change to decode cost. encoder, err := zstd.NewWriter(nil, - zstd.WithEncoderLevel(zstd.SpeedDefault), + zstd.WithEncoderLevel(zstd.SpeedBetterCompression), zstd.WithEncoderConcurrency(1), zstd.WithLowerEncoderMem(true), ) @@ -220,7 +236,9 @@ func Build(ctx context.Context, storeDir string, options BuildOptions) (BuildRes } var name string var offset int64 - if options.Incremental { + storedBytes := block.StoredBytes + rawEncoding := block.Encoding == EncodingRaw + if options.Incremental && !rewrite.rewrite[block.Pack] { name = imported[block.Pack] if name == "" { key := sha256.Sum256([]byte(previousGeneration + "/" + block.Pack)) @@ -236,6 +254,16 @@ func Build(ctx context.Context, storeDir string, options BuildOptions) (BuildRes if _, err := file.ReadAt(data, block.PackOffset); err != nil { return BuildResult{}, err } + // Older generations kept compressible small blocks raw. + // Compaction rewrites every block anyway, so store + // those compressed now. + if rawEncoding { + if compressed := encoder.EncodeAll(data, nil); !keepPackBlockRaw(data, compressed) { + data = compressed + rawEncoding = false + } + } + storedBytes = int64(len(data)) var err error name, offset, err = writer.write(data) if err != nil { @@ -253,15 +281,15 @@ func Build(ctx context.Context, storeDir string, options BuildOptions) (BuildRes return BuildResult{}, err } encoding := byte(0) - if block.Encoding == EncodingRaw { + if rawEncoding { encoding = 1 } - if _, err := blocksFile.Write(encodeBlockV3(blockV3Record{PackIndex: index, PackOffset: offset, StoredBytes: block.StoredBytes, RawOffset: block.RawOffset, RawBytes: block.RawBytes, Digest: digest, Encoding: encoding})); err != nil { + if _, err := blocksFile.Write(encodeBlockV3(blockV3Record{PackIndex: index, PackOffset: offset, StoredBytes: storedBytes, RawOffset: block.RawOffset, RawBytes: block.RawBytes, Digest: digest, Encoding: encoding})); err != nil { return BuildResult{}, err } result.BlockCount++ result.RawBytes += block.RawBytes - result.StoredBytes += block.StoredBytes + result.StoredBytes += storedBytes } digest, err := decodeDigest(ref.SHA256) if err != nil { @@ -296,7 +324,7 @@ func Build(ctx context.Context, storeDir string, options BuildOptions) (BuildRes compressed := encoder.EncodeAll(raw, nil) stored := compressed encoding := EncodingZstd - if preferRawBlock(raw, compressed) { + if keepPackBlockRaw(raw, compressed) { stored = raw encoding = EncodingRaw } @@ -451,51 +479,14 @@ func compactPackFileLimit(previous *Resolver, packBytes int64) (int, error) { return max(16, int(minimum)+8), nil } -func hasDroppedPackedObjects(ctx context.Context, references *sql.DB, previous *Resolver) (bool, error) { - rows, err := references.QueryContext(ctx, `select digest from pack_references order by digest`) - if err != nil { - return false, err - } - defer rows.Close() - var position int64 - var previousDigest string - for rows.Next() { - if err := ctx.Err(); err != nil { - return false, err - } - var digest string - if err := rows.Scan(&digest); err != nil { - return false, err - } - if position >= previous.ObjectCount() { - continue - } - if previousDigest == "" { - object, err := previous.objectAt(position) - if err != nil { - return false, err - } - previousDigest = object.SHA256 - } - if previousDigest < digest { - return true, nil - } - if previousDigest == digest { - position++ - previousDigest = "" - } - } - if err := rows.Err(); err != nil { - return false, err - } - return position < previous.ObjectCount(), nil -} +// keepPackBlockRaw is preferRawBlock outside tests, which swap in the old +// cut-off to build a legacy generation. +var keepPackBlockRaw = preferRawBlock +// preferRawBlock keeps a block raw only when compression does not shrink it. +// The old 2:1 cut-off for small blocks left 7 GB of a real 15 GB pack raw. func preferRawBlock(raw []byte, compressed []byte) bool { - if len(compressed) >= len(raw) { - return true - } - return len(raw) <= 16<<10 && len(compressed)*2 >= len(raw) + return len(compressed) >= len(raw) } func estimatedGenerationBytes(rawBytes int64) (int64, error) { diff --git a/internal/pack/incremental_budget.go b/internal/pack/incremental_budget.go index adcf2a3..5cd9a87 100644 --- a/internal/pack/incremental_budget.go +++ b/internal/pack/incremental_budget.go @@ -33,11 +33,19 @@ func estimateCompactedGeneration(ctx context.Context, store string, refs *sql.DB if err := rows.Scan(&digest, &size); err != nil { return 0, err } - _, looseErr := os.Lstat(objects.ObjectPath(digest)) + loose, looseErr := os.Lstat(objects.ObjectPath(digest)) if looseErr != nil && !errors.Is(looseErr, os.ErrNotExist) { return 0, looseErr } - if looseErr == nil || previous == nil { + if looseErr == nil { + encoded := looseEncodedEstimate(size, loose) + if copiedEncoded > math.MaxInt64-encoded { + return 0, errors.New("compacted generation projection overflow") + } + copiedEncoded += encoded + continue + } + if previous == nil { if size < 0 || reencodedRaw > math.MaxInt64-size { return 0, errors.New("compacted generation projection overflow") } @@ -112,11 +120,15 @@ func estimateIncrementalGeneration(ctx context.Context, store string, refs *sql. if err != nil { return 0, err } - _, looseErr := os.Lstat(objects.ObjectPath(digest)) + loose, looseErr := os.Lstat(objects.ObjectPath(digest)) if looseErr != nil && !errors.Is(looseErr, os.ErrNotExist) { return 0, looseErr } - if !found || object.RawBytes != size || looseErr == nil { + if looseErr == nil { + if err := add(looseEncodedEstimate(size, loose)); err != nil { + return 0, err + } + } else if !found || object.RawBytes != size { if err := add(size); err != nil { return 0, err } @@ -176,3 +188,19 @@ func estimateGenerationMetadata(ctx context.Context, store string, count, rawByt }) return added, err } + +// looseEncodedEstimate prices a loose object's packed blocks from its +// compressed size on disk rather than its raw size, which overstated a build +// of new session content about threefold and kept it from running on a +// nearly full disk. Pack blocks use a stronger level than loose objects, so +// a quarter more plus per-object slack is a safe bound; raw stays the cap. +func looseEncodedEstimate(rawBytes int64, loose os.FileInfo) int64 { + if rawBytes < 0 { + return 0 + } + stored := loose.Size() + if stored < 0 || stored > (math.MaxInt64-4096)/5*4 { + return rawBytes + } + return min(rawBytes, stored*5/4+4096) +} diff --git a/internal/pack/pack_test.go b/internal/pack/pack_test.go index 99b5728..89ebcf9 100644 --- a/internal/pack/pack_test.go +++ b/internal/pack/pack_test.go @@ -5,6 +5,7 @@ import ( "context" cryptorand "crypto/rand" "crypto/sha256" + "encoding/base64" "encoding/hex" "encoding/json" "errors" @@ -2413,3 +2414,66 @@ func (c *rejectingChecker) Check(context.Context, storage.Projection) (storage.A c.Calls++ return storage.Assessment{}, storage.ErrBudgetExceeded } + +func TestCompactionCompressesLegacyRawBlocks(t *testing.T) { + root := t.TempDir() + random := make([]byte, 6<<10) + if _, err := cryptorand.Read(random); err != nil { + t.Fatal(err) + } + // Base64 text shrinks by roughly a quarter: below the old 2:1 cut-off, + // still worth compressing. + moderate := []byte(base64.StdEncoding.EncodeToString(random)) + dropped := make([]byte, 4<<10) + if _, err := cryptorand.Read(dropped); err != nil { + t.Fatal(err) + } + refs := putObjects(t, root, moderate, dropped) + writeManifest(t, root, "keep", refs[:1]) + writeManifest(t, root, "drop", refs[1:]) + + keepPackBlockRaw = func(raw []byte, compressed []byte) bool { + return len(compressed) >= len(raw) || (len(raw) <= 16<<10 && len(compressed)*2 >= len(raw)) + } + _, err := Build(context.Background(), root, BuildOptions{}) + keepPackBlockRaw = preferRawBlock + if err != nil { + t.Fatal(err) + } + if got := blockEncodings(t, root)[refs[0].SHA256]; got != EncodingRaw { + t.Fatalf("legacy build stored the moderate block as %q, want raw", got) + } + if _, err := RetireLoose(context.Background(), root, RetireLooseOptions{Apply: true}); err != nil { + t.Fatal(err) + } + if err := os.Remove(fold.ManifestPath(root, "drop")); err != nil { + t.Fatal(err) + } + if _, err := Build(context.Background(), root, BuildOptions{Incremental: true}); err != nil { + t.Fatal(err) + } + if got := blockEncodings(t, root)[refs[0].SHA256]; got != EncodingZstd { + t.Fatalf("compaction kept the moderate block as %q, want zstd", got) + } + resolver, err := Open(root, OpenOptions{}) + if err != nil { + t.Fatal(err) + } + defer resolver.Close() + got := make([]byte, len(moderate)) + if n, err := resolver.ReadAt(context.Background(), refs[0], got, 0); n != len(moderate) || err != nil || !bytes.Equal(got, moderate) { + t.Fatalf("ReadAt after recompression = %d, %v", n, err) + } +} + +func blockEncodings(t *testing.T, root string) map[string]string { + t.Helper() + encodings := make(map[string]string) + for _, object := range loadCurrentIndex(t, root).Objects { + if len(object.Blocks) != 1 { + t.Fatalf("object %s blocks = %d, want 1", object.SHA256, len(object.Blocks)) + } + encodings[object.SHA256] = object.Blocks[0].Encoding + } + return encodings +} diff --git a/internal/pack/selective.go b/internal/pack/selective.go new file mode 100644 index 0000000..75904f5 --- /dev/null +++ b/internal/pack/selective.go @@ -0,0 +1,213 @@ +package pack + +import ( + "bufio" + "context" + "database/sql" + "encoding/binary" + "errors" + "io" + "math" + "os" + "path/filepath" + "sort" + + "github.com/samekind/codexfold/internal/fold" + "github.com/samekind/codexfold/internal/storage" +) + +// A build links every previous pack file into the new generation, except the +// files it rewrites. Rewriting copies a file's live blocks, so it costs their +// size up front and frees the file's dead blocks once the old generation is +// collected. Pricing files one at a time lets a nearly full disk compact a +// few files per build instead of needing room for the whole pack at once. + +type packUsage struct { + size int64 + live int64 + // raw is live bytes stored raw, which a rewrite recompresses. + raw int64 +} + +type rewritePlan struct { + rewrite map[string]bool + copyBytes int64 + full bool +} + +const ( + // A rewrite is worth it when it frees at least this share of the file. + rewriteBenefitShare = 10 + // Share of raw block bytes a rewrite is expected to save by compressing + // them; measured at 30% on a real pack. + rawRecompressionShare = 30 + rewriteSafetyMargin = int64(512 << 20) +) + +// packFileUsage attributes the live blocks of the next generation to the +// previous pack files that hold them. +func packFileUsage(ctx context.Context, store string, refs *sql.DB, previous *Resolver, blockBytes int64) (map[string]*packUsage, error) { + sizes, err := resolverPackSizes(previous.packs) + if err != nil { + return nil, err + } + usage := make(map[string]*packUsage, len(sizes)) + for name, size := range sizes { + usage[name] = &packUsage{size: size} + } + rows, err := refs.QueryContext(ctx, `select digest,raw_bytes from pack_references order by digest`) + if err != nil { + return nil, err + } + defer rows.Close() + objects := fold.NewObjectStore(store) + for rows.Next() { + if err := ctx.Err(); err != nil { + return nil, err + } + var digest string + var size int64 + if err := rows.Scan(&digest, &size); err != nil { + return nil, err + } + // Build re-encodes a loose object instead of reusing its packed copy. + if _, err := os.Lstat(objects.ObjectPath(digest)); err == nil { + continue + } else if !errors.Is(err, os.ErrNotExist) { + return nil, err + } + object, found, err := previous.lookupObject(digest) + if err != nil { + return nil, err + } + if !found || object.RawBytes != size { + continue + } + for _, block := range object.Blocks { + if block.RawBytes > blockBytes { + continue + } + file := usage[block.Pack] + if file == nil { + continue + } + file.live += block.StoredBytes + if block.Encoding == EncodingRaw { + file.raw += block.StoredBytes + } + } + } + return usage, rows.Err() +} + +// choosePackRewrite picks the files whose rewrite frees the most per byte +// copied, while the copies fit in allowance. Once the generation would hold +// more files than limit, the smallest kept files are merged too. +func choosePackRewrite(usage map[string]*packUsage, allowance int64, limit int, packBytes int64, newBytes int64) rewritePlan { + plan := rewritePlan{rewrite: make(map[string]bool)} + type candidate struct { + name string + benefit int64 + live int64 + } + var candidates []candidate + for name, file := range usage { + dead := max(0, file.size-file.live) + benefit := dead + file.raw*rawRecompressionShare/100 + if benefit*100 >= file.size*rewriteBenefitShare && benefit > 0 { + candidates = append(candidates, candidate{name: name, benefit: benefit, live: file.live}) + } + } + // Highest benefit per copied byte first; ties keep a stable order. + sort.Slice(candidates, func(i, j int) bool { + left := float64(candidates[i].benefit) / float64(max(candidates[i].live, 1)) + right := float64(candidates[j].benefit) / float64(max(candidates[j].live, 1)) + if left != right { + return left > right + } + return candidates[i].name < candidates[j].name + }) + for _, candidate := range candidates { + if plan.copyBytes > math.MaxInt64-candidate.live || plan.copyBytes+candidate.live > allowance { + continue + } + plan.rewrite[candidate.name] = true + plan.copyBytes += candidate.live + } + kept := make([]string, 0, len(usage)) + for name := range usage { + if !plan.rewrite[name] { + kept = append(kept, name) + } + } + sort.Slice(kept, func(i, j int) bool { + if usage[kept[i]].size != usage[kept[j]].size { + return usage[kept[i]].size < usage[kept[j]].size + } + return kept[i] < kept[j] + }) + files := func() int { + written := plan.copyBytes + newBytes + return len(kept) + int((written+packBytes-1)/max(packBytes, 1)) + } + for len(kept) > 0 && files() > limit { + smallest := usage[kept[0]] + if plan.copyBytes+smallest.live > allowance { + break + } + plan.rewrite[kept[0]] = true + plan.copyBytes += smallest.live + kept = kept[1:] + } + plan.full = len(kept) == 0 + return plan +} + +// packRewriteAllowance is rewriteAllowance outside tests. +var packRewriteAllowance = rewriteAllowance + +// rewriteAllowance is how many bytes a build may copy out of old pack files +// on top of the new data, keeping the store's free-space reserve. +func rewriteAllowance(store string, newBytes int64) (int64, error) { + limits, err := storage.LoadLimits(store) + if err != nil { + return 0, err + } + available, err := storage.AvailableBytes(store) + if err != nil { + return 0, err + } + return max(0, available-limits.FreeSpaceReserveBytes-newBytes-rewriteSafetyMargin), nil +} + +// GenerationWaste reports the bytes of the current generation's pack files +// and how many of them no live block uses. +func GenerationWaste(store string) (fileBytes int64, wastedBytes int64, err error) { + generation, err := CurrentGeneration(store) + if err != nil { + return 0, 0, err + } + directory := filepath.Join(store, "packs", generation) + index, err := openIndexV3(directory) + if err != nil { + return 0, 0, err + } + defer index.close() + var live int64 + reader := bufio.NewReaderSize(io.NewSectionReader(index.blocks, 0, index.meta.BlockCount*blockV3RecordBytes), 1<<20) + var record [blockV3RecordBytes]byte + for position := int64(0); position < index.meta.BlockCount; position++ { + if _, err := io.ReadFull(reader, record[:]); err != nil { + return 0, 0, err + } + live += int64(binary.BigEndian.Uint64(record[12:20])) + } + for _, name := range index.meta.Packs { + info, err := os.Stat(filepath.Join(directory, name)) + if err != nil { + return 0, 0, err + } + fileBytes += info.Size() + } + return fileBytes, max(0, fileBytes-live), nil +} diff --git a/internal/pack/selective_test.go b/internal/pack/selective_test.go new file mode 100644 index 0000000..2af316d --- /dev/null +++ b/internal/pack/selective_test.go @@ -0,0 +1,145 @@ +package pack + +import ( + "bytes" + "context" + cryptorand "crypto/rand" + "os" + "path/filepath" + "strings" + "testing" + + "github.com/samekind/codexfold/internal/fold" +) + +func TestChoosePackRewrite(t *testing.T) { + usage := map[string]*packUsage{ + "clean": {size: 1000, live: 990}, + "half-dead": {size: 1000, live: 500}, + "all-dead": {size: 1000, live: 0}, + "raw-heavy": {size: 1000, live: 1000, raw: 800}, + } + plan := choosePackRewrite(usage, 1<<40, 100, 1000, 0) + if !plan.rewrite["half-dead"] || !plan.rewrite["all-dead"] || !plan.rewrite["raw-heavy"] || plan.rewrite["clean"] || plan.full { + t.Fatalf("ample space plan = %#v", plan) + } + plan = choosePackRewrite(usage, 600, 100, 1000, 0) + if !plan.rewrite["all-dead"] || !plan.rewrite["half-dead"] || plan.rewrite["raw-heavy"] || plan.copyBytes != 500 { + t.Fatalf("tight plan = %#v", plan) + } + plan = choosePackRewrite(usage, 0, 100, 1000, 0) + if len(plan.rewrite) != 1 || !plan.rewrite["all-dead"] || plan.copyBytes != 0 { + t.Fatalf("no-room plan = %#v", plan) + } + small := map[string]*packUsage{"a": {size: 10, live: 10}, "b": {size: 20, live: 20}, "c": {size: 1000, live: 1000}} + plan = choosePackRewrite(small, 1<<40, 2, 1000, 0) + if !plan.rewrite["a"] || !plan.rewrite["b"] || plan.rewrite["c"] { + t.Fatalf("file-count plan = %#v", plan) + } +} + +func TestBuildRewritesOnlyTheFilesThatFit(t *testing.T) { + root := t.TempDir() + values := make([][]byte, 3) + for index := range values { + values[index] = make([]byte, 64<<10) + if _, err := cryptorand.Read(values[index]); err != nil { + t.Fatal(err) + } + } + refs := putObjects(t, root, values...) + for index, ref := range refs { + writeManifest(t, root, []string{"keep-a", "drop", "keep-c"}[index], []fold.ObjectRef{ref}) + } + // One object per pack file. + if _, err := Build(context.Background(), root, BuildOptions{PackBytes: 64 << 10}); err != nil { + t.Fatal(err) + } + if _, err := RetireLoose(context.Background(), root, RetireLooseOptions{Apply: true}); err != nil { + t.Fatal(err) + } + if err := os.Remove(fold.ManifestPath(root, "drop")); err != nil { + t.Fatal(err) + } + previous := packRewriteAllowance + packRewriteAllowance = func(string, int64) (int64, error) { return 0, nil } + t.Cleanup(func() { packRewriteAllowance = previous }) + if _, err := Build(context.Background(), root, BuildOptions{Incremental: true, PackBytes: 64 << 10}); err != nil { + t.Fatal(err) + } + current, err := CurrentGeneration(root) + if err != nil { + t.Fatal(err) + } + entries, err := os.ReadDir(filepath.Join(root, "packs", current)) + if err != nil { + t.Fatal(err) + } + var reused, total int + for _, entry := range entries { + if strings.HasSuffix(entry.Name(), ".pack") { + total++ + if strings.HasPrefix(entry.Name(), "pack-reused-") { + reused++ + } + } + } + // The dead file is dropped for free; the two live files are linked. + if reused != 2 || total != 2 { + t.Fatalf("pack files reused=%d total=%d, want 2 linked and nothing copied", reused, total) + } + resolver, err := Open(root, OpenOptions{}) + if err != nil { + t.Fatal(err) + } + defer resolver.Close() + for _, index := range []int{0, 2} { + got := make([]byte, len(values[index])) + if n, err := resolver.ReadAt(context.Background(), refs[index], got, 0); n != len(got) || err != nil || !bytes.Equal(got, values[index]) { + t.Fatalf("object %d after selective build: %d %v", index, n, err) + } + } + if resolver.HasObject(refs[1]) { + t.Fatal("dropped object is still indexed") + } +} + +func TestGenerationWaste(t *testing.T) { + root := t.TempDir() + value := make([]byte, 32<<10) + if _, err := cryptorand.Read(value); err != nil { + t.Fatal(err) + } + refs := putObjects(t, root, value) + writeManifest(t, root, "only", refs) + if _, err := Build(context.Background(), root, BuildOptions{}); err != nil { + t.Fatal(err) + } + files, wasted, err := GenerationWaste(root) + if err != nil || files <= 0 || wasted != 0 { + t.Fatalf("waste = %d/%d, %v", wasted, files, err) + } +} + +func TestBuildEstimatesPriceLooseObjectsByStoredSize(t *testing.T) { + root := t.TempDir() + compressible := bytes.Repeat([]byte(`{"type":"response_item","text":"the same assistant text again"}`+"\n"), 300000) + refs := putObjects(t, root, compressible) + writeManifest(t, root, "session", refs) + references, count, rawBytes, err := buildReferenceIndex(context.Background(), root, filepath.Join(t.TempDir(), "refs.sqlite")) + if err != nil { + t.Fatal(err) + } + defer references.Close() + compacted, err := estimateCompactedGeneration(context.Background(), root, references, nil, count, rawBytes, defaultBlockBytes) + if err != nil { + t.Fatal(err) + } + metadataOnly, err := estimateGenerationMetadata(context.Background(), root, count, rawBytes, defaultBlockBytes, 0) + if err != nil { + t.Fatal(err) + } + if payload := compacted - metadataOnly; payload <= 0 || payload > int64(len(compressible))/10 { + t.Fatalf("loose payload priced at %d bytes for %d raw bytes", payload, len(compressible)) + } +} diff --git a/platform/darwin/fskit/CodexFoldFSKit.xcodeproj/project.pbxproj b/platform/darwin/fskit/CodexFoldFSKit.xcodeproj/project.pbxproj index 2677594..0415858 100644 --- a/platform/darwin/fskit/CodexFoldFSKit.xcodeproj/project.pbxproj +++ b/platform/darwin/fskit/CodexFoldFSKit.xcodeproj/project.pbxproj @@ -510,7 +510,7 @@ CLANG_WARN__DUPLICATE_METHOD_MATCH = YES; CODE_SIGN_STYLE = Automatic; COPY_PHASE_STRIP = NO; - CURRENT_PROJECT_VERSION = 116; + CURRENT_PROJECT_VERSION = 117; DEBUG_INFORMATION_FORMAT = dwarf; DEVELOPMENT_TEAM = Y987FUR837; ENABLE_STRICT_OBJC_MSGSEND = YES; @@ -642,7 +642,7 @@ CLANG_WARN__DUPLICATE_METHOD_MATCH = YES; CODE_SIGN_STYLE = Automatic; COPY_PHASE_STRIP = NO; - CURRENT_PROJECT_VERSION = 116; + CURRENT_PROJECT_VERSION = 117; DEBUG_INFORMATION_FORMAT = "dwarf-with-dsym"; DEVELOPMENT_TEAM = Y987FUR837; ENABLE_NS_ASSERTIONS = NO; diff --git a/platform/darwin/fskit/Extension/ProfileModule.swift b/platform/darwin/fskit/Extension/ProfileModule.swift index 5fdfcb5..7bcb8d1 100644 --- a/platform/darwin/fskit/Extension/ProfileModule.swift +++ b/platform/darwin/fskit/Extension/ProfileModule.swift @@ -8,11 +8,59 @@ import OSLog @main struct CodexFoldFSKitModule: UnaryFileSystemExtension { + init() { + CodexFoldDescriptorLimit.raise() + } + var fileSystem: FSUnaryFileSystem & FSUnaryFileSystemOperations { CodexFoldFileSystem() } } +/// The extension starts with launchd's 256-descriptor soft limit. An open +/// file can hold a daemon socket, a native or shared-window descriptor and +/// up to nine prefetch channels with their own sockets, so a burst of +/// concurrent readers exhausted the limit and every mount read failed with +/// EMFILE. Raise the soft limit within the hard limit once at start. +enum CodexFoldDescriptorLimit { + static let target: rlim_t = 16384 + + static func raise() { + var limit = rlimit() + guard getrlimit(RLIMIT_NOFILE, &limit) == 0 else { return } + for wanted in [min(limit.rlim_max, target), min(limit.rlim_max, rlim_t(OPEN_MAX))] where limit.rlim_cur < wanted { + var raised = limit + raised.rlim_cur = wanted + if setrlimit(RLIMIT_NOFILE, &raised) == 0 { + return + } + } + } +} + +/// Bounds prefetch channels across every open file, so descriptor use stays +/// fixed however many files are read at once. A file that cannot get a +/// channel is still read through its own handle, only without prefetch. +enum CodexFoldPrefetchBudget { + static let limit = 48 + private static let lock = NSLock() + private static var used = 0 + + static func reserve() -> Bool { + lock.lock() + defer { lock.unlock() } + guard used < limit else { return false } + used += 1 + return true + } + + static func release() { + lock.lock() + used = max(0, used - 1) + lock.unlock() + } +} + final class CodexFoldFileSystem: FSUnaryFileSystem, FSUnaryFileSystemOperations { private let logger = Logger( subsystem: "vip.jstar.codexfold.fskitprofileprobe.module", @@ -148,6 +196,7 @@ private final class CodexFoldPrefetchChannel { } try? client.handleOperation(.release, handle: handle, connection: connection) connection.close() + CodexFoldPrefetchBudget.release() } deinit { @@ -155,6 +204,9 @@ private final class CodexFoldPrefetchChannel { Darwin.close(nativeReadFD) } connection.close() + if !closed { + CodexFoldPrefetchBudget.release() + } } } @@ -404,7 +456,7 @@ private final class CodexFoldIO { prefetchPoolLock.unlock() return channel } - guard prefetchChannelCount < maxCachedBlocks else { + guard prefetchChannelCount < maxCachedBlocks, CodexFoldPrefetchBudget.reserve() else { prefetchPoolLock.unlock() return nil } @@ -416,6 +468,7 @@ private final class CodexFoldIO { do { binding = try client.openBinding(path, flags: O_RDONLY) } catch { + CodexFoldPrefetchBudget.release() releasePrefetchChannelReservation() throw error } diff --git a/platform/darwin/fskit/project.yml b/platform/darwin/fskit/project.yml index be6bd4b..81d4d15 100644 --- a/platform/darwin/fskit/project.yml +++ b/platform/darwin/fskit/project.yml @@ -8,7 +8,7 @@ settings: CODE_SIGN_STYLE: Automatic SWIFT_VERSION: "5.0" MARKETING_VERSION: "0.4.0" - CURRENT_PROJECT_VERSION: "116" + CURRENT_PROJECT_VERSION: "117" targets: CodexFoldFSKit: type: application