Skip to content

Incorrect process memory reported in htop 3.5.3 on macOS (#2132) - #2144

Open
hugojosealmeida wants to merge 5 commits into
htop-dev:mainfrom
hugojosealmeida:sdlc/issue-2132-incorrect-process-memory-reported-in-hto
Open

hugojosealmeida wants to merge 5 commits into
htop-dev:mainfrom
hugojosealmeida:sdlc/issue-2132-incorrect-process-memory-reported-in-hto

Conversation

@hugojosealmeida

@hugojosealmeida hugojosealmeida commented Oct 28, 2026 •

Copy link
Copy Markdown

Summary

The mismatch in issue #2132 is not a measurement bug: htop's RES column reports proc_taskinfo.pti_resident_size (resident set size — only pages currently in physical RAM), while Activity Monitor's "Memory" column reports the kernel's phys_footprint ledger (ri_phys_footprint from proc_pid_rusage(2)), which also counts dirty memory that has been compressed or swapped out. For a process whose memory is largely swapped/compressed (as in the report: 16.77 GB footprint but ~36 MB resident), the two metrics legitimately diverge by orders of magnitude.

The fix gives macOS users the metric they were comparing against: a new opt-in M_FOOTPRINT process column that reports phys_footprint — the same ledger Activity Monitor's "Memory" column and the footprint(1) tool display. The value is read from the rusage_info_v2 struct that htop already fetches per process for the I/O columns, so no additional syscall is introduced when I/O columns are enabled, and only one extra proc_pid_rusage call per scan when only FOOTPRINT is enabled. The column shows N/A (and sorts last) where proc_pid_rusage fails (e.g. other users' processes without privileges), reusing the repo's existing ULLONG_MAX/SPACESHIP_NUMBER_NA conventions. RES semantics are deliberately unchanged, with the man page and column description documenting the difference.

Changes

  • darwin/ProcessField.h: new platform field M_FOOTPRINT = 41 (numbered adjacent to M_RESIDENT).
  • darwin/DarwinProcess.h: new footprint member (KiB, ULLONG_MAX when unavailable) and PROCESS_FLAG_FOOTPRINT data-collection flag.
  • darwin/DarwinProcess.c: field metadata (FOOTPRINT column with description referencing Activity Monitor), value writer via Row_printKBytes, descending sort comparator with N-A-last handling via SPACESHIP_NUMBER_NA, and initialization.
  • darwin/DarwinProcessTable.c: store ri.ri_phys_footprint / ONE_K in the existing proc_pid_rusage(RUSAGE_INFO_V2) reader (failure path sets ULLONG_MAX); the rusage scan now runs when either the I/O or FOOTPRINT columns are active.
  • htop.1.in: document M_FOOTPRINT next to M_RESIDENT.
  • .github/workflows/ci.yml, .github/workflows/htoprc-darwin: macOS CI job now runs automated smoke tests — ./htop --sort-key=help | grep -q M_FOOTPRINT (fails without this change) plus ./htop -n 5 / ./htop -n 5 -t using a darwin config with M_FOOTPRINT as a visible column and sort key, exercising the flag computation, rusage read, rendering, and comparator.

Verification

  • make -j8 -> clean build and link (exit 0); the two touched .c files also compile warning-free under the project's full warning set with -Werror.
  • ./htop --sort-key=help | grep M_FOOTPRINT -> lists the new column with its description (exit 0).
  • Ground-truth comparison with a crafted process (257616 KB clean file-backed RSS, 1280 KB footprint): htop rendered PID 91273 ... RES 251M, FOOTPRINT 1280 — RES matches ps -o rss and FOOTPRINT matches proc_pid_rusage/footprint(1) (Footprint: 201 MB, phys_footprint_KB=1280) exactly.
  • Issue scenario reproduced: llama-server shows RES 8960 vs FOOTPRINT 3534M — the Activity-Monitor-scale value htop previously had no way to display.
  • pty runs with .github/workflows/htoprc-darwin (interactive + --plain-less CI-style): exit 0, sorted column correct, N/A for processes where rusage is denied, kernel-task rows sort last; I/O columns (RBYTES) still populate when FOOTPRINT and IO are enabled together (no regression to the I/O gating).
  • HTOPRC=.github/workflows/htoprc-darwin ./htop -n 5 and -n 5 -t (the new CI steps, run locally) -> exit 0; checked-in config remains pristine after runs.

Risks / Follow-ups

  • The FOOTPRINT column is opt-in and not added to the default Main screen, so existing users' configs are untouched; users hitting the issue should add it via F2 → Columns (or use it to explain that RES is RSS, not Activity Monitor's "Memory").
  • Enabling FOOTPRINT without I/O columns adds one proc_pid_rusage call per process per scan on macOS — the same cost the existing I/O columns already pay, and zero cost when the column is off.
  • PERCENT_MEM intentionally remains RSS-based (unchanged semantics); a footprint-based percentage would be a separate discussion.

This is AI Assisted
Closes #2132

@coderabbitai

coderabbitai Bot commented Oct 28, 2026 •

Copy link
Copy Markdown

Review in Change Stack →

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review

No actionable comments were generated in the recent review.

Recent review info
Run configuration
  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Repository UI
  • Review profile: ASSERTIVE
  • Plan: Advanced
  • Run ID: 6ce487fb-dfd0-42fd-8bb4-268f090df481
Commits

Reviewing files that changed from the base of the PR and between aa6f68a and b8e5eca.

Files selected for processing: 2
  • .github/workflows/ci.yml
  • darwin/DarwinProcess.c

Included review availability: This review used your included allowance. Your plan provides up to 8 included reviews per hour; 4 remain after this review.

Walkthrough

Darwin process data now includes physical memory footprint from proc_pid_rusage. The new M_FOOTPRINT column displays and sorts this value, and its documentation distinguishes it from resident memory. The macOS CI workflow checks for the column and runs htop with a configuration that uses it.

Assessment against linked issues

Objective Addressed Explanation
Report a macOS process memory value comparable to Activity Monitor [#2132] ✅

Possibly related issues

  • Issue 2132 — Adds an opt-in M_FOOTPRINT column that reports the metric identified in the issue.

Suggested reviewers: samm-git, natoscott

Priority: Normal

Change: Feature

Merge Risk: Minimal · up to b8e5e

Footprint sorting and collection are consistent at the reviewed head, with no remaining merge-blocking concern.

- [ ] Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Footprint values join the view,
Kernel ledgers make them new.
Sorted columns show their place,
CI checks the macOS case.
Resident memory stays distinct,
Measured bytes leave their imprint.

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3


Review info
Run configuration
  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Repository UI
  • Review profile: ASSERTIVE
  • Plan: Advanced
  • Run ID: 9797afe7-61c2-4a5d-99af-d09967ce93c3
Commits

Reviewing files that changed from the base of the PR and between 394b4a3 and 24ac527.

Files selected for processing: 7
  • .github/workflows/ci.yml
  • .github/workflows/htoprc-darwin
  • darwin/DarwinProcess.c
  • darwin/DarwinProcess.h
  • darwin/DarwinProcessTable.c
  • darwin/ProcessField.h
  • htop.1.in

Included review availability: This review used your included allowance. Your plan provides up to 8 included reviews per hour; 7 remain after this review.

Comment thread darwin/DarwinProcess.c Outdated
Comment thread darwin/DarwinProcess.c Outdated
Comment thread darwin/DarwinProcessTable.c Outdated
DarwinProcess_setFromLibprocPidinfo(proc, dpt, time_interval_ns);

if (host->settings->ss->flags & PROCESS_FLAG_IO) {
if (host->settings->ss->flags & (PROCESS_FLAG_IO | PROCESS_FLAG_FOOTPRINT)) {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Major · Functional Correctness · Quick win

Supported by static analysis

Script executed:

rg -n 'sortKey|->flags' CommandLine.c Settings.c ScreenSettings.c Settings.h | head -80
sed -n '150,175p' darwin/DarwinProcessTable.c

Repository: htop-dev/htop

Length of output: 3409


Script executed:

set -o pipefail
printf '%s\n' '--- Settings flag recomputation and sort setter ---'
nl -ba Settings.c | sed -n '250,345p;1145,1180p'
printf '%s\n' '--- Command-line sort setup ---'
nl -ba CommandLine.c | sed -n '440,480p'
printf '%s\n' '--- Process-field declarations and flag definitions ---'
rg -n -F --glob '*.{c,h}' -e 'PROCESS_FLAG_IO' -e 'PROCESS_FLAG_FOOTPRINT' -e 'M_FOOTPRINT' -e 'IO_RATE' -e 'Process_fields' .
printf '%s\n' '--- Screen flag recomputation callers ---'
rg -n -F --glob '*.{c,h}' -e 'ss->flags' -e 'ScreenSettings_setSortKey' -e 'Settings_set' .

Repository: htop-dev/htop

Length of output: 21565


Script executed:

set -o pipefail
printf '%s\n' '--- Darwin rusage collection and value handling ---'
nl -ba darwin/DarwinProcessTable.c | sed -n '130,175p;250,315p'
printf '%s\n' '--- Darwin process footprint fields and sorting ---'
nl -ba darwin/DarwinProcess.c | sed -n '1,75p;100,170p;330,430p'
printf '%s\n' '--- Active sort-key accessors and screen state ---'
nl -ba Settings.h | sed -n '35,165p'
rg -n -F --glob '*.{c,h}' -e 'treeSortKey' -e 'sortKey' -e 'Object_compare' -e 'Process_compare' -e 'Row_compare' .
printf '%s\n' '--- Darwin default screen ---'
nl -ba darwin/Platform.c | sed -n '55,85p'

Repository: htop-dev/htop

Length of output: 34243


Script executed:

set -o pipefail
printf '%s\n' '--- Footprint reader and assignments ---'
rg -n -C 12 -F --glob 'darwin/*.{c,h}' -e 'DarwinProcessTable_readProcRusage' -e 'footprint' .
printf '%s\n' '--- Process initialization around DarwinProcess_new ---'
nl -ba darwin/DarwinProcess.c | sed -n '68,110p'
printf '%s\n' '--- Process sorting call and active key use ---'
nl -ba Process.c | sed -n '925,975p'
rg -n -F --glob '*.{c,h}' -e 'ScreenSettings_getActiveSortKey' -e 'getActiveSortKey' .

Repository: htop-dev/htop

Length of output: 21217


Collect data required by the active sort key.

htop -s M_FOOTPRINT changes the sort key but does not add the footprint column. ScreenSettings_setSortKey does not update flags, so the Darwin scan skips DarwinProcessTable_readProcRusage. New processes keep footprint = 0, and the PID tie-breaker determines the sort order.

Include the active sort key's flags in the collection condition.

Suggested fix
--- "a/darwin/DarwinProcessTable.c"
+++ "b/darwin/DarwinProcessTable.c"
@@ -158,9 +158,11 @@
       DarwinProcess_setFromKInfoProc(&proc->super, &ps[i], preExisting);
       DarwinProcess_setFromLibprocPidinfo(proc, dpt, time_interval_ns);
 
-      if (host->settings->ss->flags & (PROCESS_FLAG_IO | PROCESS_FLAG_FOOTPRINT)) {
+      if ((host->settings->ss->flags |
+           Process_fields[ScreenSettings_getActiveSortKey(host->settings->ss)].flags) &
+          (PROCESS_FLAG_IO | PROCESS_FLAG_FOOTPRINT)) {
          DarwinProcessTable_readProcRusage(proc);
       }
 
       // Deduce further process states not covered in the libproc call above
       if (ps[i].kp_proc.p_stat == SZOMB) {
Committable suggestion

IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
if (host->settings->ss->flags & (PROCESS_FLAG_IO | PROCESS_FLAG_FOOTPRINT)) {
if ((host->settings->ss->flags |
Process_fields[ScreenSettings_getActiveSortKey(host->settings->ss)].flags) &
(PROCESS_FLAG_IO | PROCESS_FLAG_FOOTPRINT)) {

Addresses review feedback and CI failures.
@hugojosealmeida

hugojosealmeida commented Oct 28, 2026 •

Copy link
Copy Markdown
Author

Addresses the three review points on the previous push:

  1. Header/value width mismatch — Row_printKBytes emits a fixed 6-character field (and Process.h requires titles to match their value width), but the title was " FOOTPRINT " (11 chars), shifting all following headers/values and breaking mouse column hit-testing. The title is now " FOOT " (6 chars), with the man page updated to M_FOOTPRINT (FOOT).
  2. N/A sort direction — SPACESHIP_NUMBER_NA sorts unavailable values lowest, so pressing the ascending key put N/A footprints first despite the PR's "sorts last" goal. The comparator now accounts for the active sort direction (ScreenSettings_getActiveDirection), keeping N/A visually last in both ascending and descending order.
  3. htop -s M_FOOTPRINT collected nothing — ScreenSettings_setSortKey does not update ss->flags (which is derived only from visible columns), so the rusage scan was skipped and every footprint stayed 0, leaving the sort entirely to the PID tie-breaker. The scan gate now ORs the active sort key's field flags into the visible-column flags before checking PROCESS_FLAG_IO | PROCESS_FLAG_FOOTPRINT.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1


Review info
Run configuration
  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Repository UI
  • Review profile: ASSERTIVE
  • Plan: Advanced
  • Run ID: 7154a86e-b02a-4c75-bf8c-d0d2e71aefb5
Commits

Reviewing files that changed from the base of the PR and between 24ac527 and e8fc39c.

Files selected for processing: 5
  • .github/workflows/check-htop-first-row.py
  • .github/workflows/ci.yml
  • darwin/DarwinProcess.c
  • darwin/DarwinProcessTable.c
  • htop.1.in

Included review availability: This review used your included allowance. Your plan provides up to 8 included reviews per hour; 7 remain after this review.

Comment thread .github/workflows/ci.yml Outdated
Addresses review feedback and CI failures.
@hugojosealmeida

Copy link
Copy Markdown
Author

Addressed the remaining open CodeRabbit comment: the macOS CI check for invisible-M_FOOTPRINT-sort-key data collection spawned its "big" process with bash -c 'x=$(head -c 400000000 /dev/zero); sleep 60', but bash drops NUL bytes in command substitution, so the process retained ~2 MB instead of 400 MB — the footprint check could pass without actually validating collection. The big process now allocates via python3 -c 'x = bytearray(400000000); x[::4096] = ...' (the reviewer's suggested fix), which verifiably holds ~388 MB phys_footprint; a short RSS-based readiness wait was added so the process is present in htop's immediately-rendered first frame (the previous spawn-then-run-immediately sequence could flakily miss it). The other three actionable comments (FOOT six-char title, direction-aware N/A-last comparator, and collecting data for the active sort key via rusageFlags) were already implemented in the branch's prior fix commit and re-verified against the live code. No C sources changed in this round.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1


Review info
Run configuration
  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Repository UI
  • Review profile: ASSERTIVE
  • Plan: Advanced
  • Run ID: cff4cc0c-bc43-4eb1-8684-eb0f91f4edaa
Commits

Reviewing files that changed from the base of the PR and between e8fc39c and 51dd985.

Files selected for processing: 1
  • .github/workflows/ci.yml

Included review availability: This review used your included allowance. Your plan provides up to 8 included reviews per hour; 7 remain after this review.

Comment thread .github/workflows/ci.yml
Addresses review feedback and CI failures.
@hugojosealmeida

hugojosealmeida commented Oct 28, 2026 •

Copy link
Copy Markdown
Author

CodeRabbit's latest review posted one remaining actionable item: the macOS CI step's readiness-polling loop could exhaust its 30 iterations and run htop anyway, so a never-allocating test process would be reported by check-htop-first-row.py as a footprint-collection failure instead of a readiness failure. This commit makes the step fail fast with an explicit readiness error when the large process doesn't become resident in time. The four earlier inline findings (6-char FOOT title, direction-aware N/A-last comparator, rusage collection for an invisible sort key, and the python3 bytearray allocator) are already implemented on the branch.

@hugojosealmeida

Copy link
Copy Markdown
Author

@coderabbitai full review

@coderabbitai

coderabbitai Bot commented Oct 28, 2026 •

Copy link
Copy Markdown
✅ Action performed

Full review finished.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2


Review info
Run configuration
  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Repository UI
  • Review profile: ASSERTIVE
  • Plan: Advanced
  • Run ID: d1b43f5a-f720-40b0-8f01-ddc1ad41569f
Commits

Reviewing files that changed from the base of the PR and between 394b4a3 and aa6f68a.

Files selected for processing: 8
  • .github/workflows/check-htop-first-row.py
  • .github/workflows/ci.yml
  • .github/workflows/htoprc-darwin
  • darwin/DarwinProcess.c
  • darwin/DarwinProcess.h
  • darwin/DarwinProcessTable.c
  • darwin/ProcessField.h
  • htop.1.in

Included review availability: This review used your included allowance. Your plan provides up to 8 included reviews per hour; 5 remain after this review.

Comment thread .github/workflows/ci.yml
Comment thread darwin/DarwinProcess.c
Addresses review feedback and CI failures.
@hugojosealmeida

Copy link
Copy Markdown
Author

Round-2 review feedback on this PR (two remaining actionable comments, both Minor) is addressed; no other files change relative to the PR's existing tip.

  • Thread rows showed a footprint of 0 (darwin/DarwinProcess.c): DarwinProcess_scanThreads created thread rows via DarwinProcess_new and never assigned footprint, so every thread displayed and sorted as zero memory even when its task had a valid (or unavailable/N/A) footprint. Memory on macOS is accounted per task, not per thread, so the thread row now inherits the parent's value — including ULLONG_MAX, which keeps the existing N/A-sorts-last behaviour. This is the reviewer's suggested first option, applied before the hide_userlandThreads early-continue so hidden rows stay consistent too. The pre-existing DarwinProcess* tdproc cast inside the HAVE_THREAD_EXTENDED_INFO_DATA_T block was hoisted to the top of the loop iteration rather than shadowed, since the build runs with -Wshadow.
  • The FOOTPRINT CI check could pass while htop itself failed (.github/workflows/ci.yml): the invisible-sort-key check pipes htop into check-htop-first-row.py, so the step's exit status was the checker's. The step now starts with set -o pipefail, so an htop failure after a rendered frame fails the job (the reviewer's second suggested alternative; shell: bash alone would not fix it, as macOS runners already default to bash -e).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Incorrect process memory reported in htop 3.5.3 on macOS

1 participant