[tanishqbaweja]

Problem

FileSystem in read mode refuses a file over MAX_READ_SIZE (10 MB) and says what to do instead:

Error: File too large (11,110,000 bytes). Maximum is 10,485,760 bytes. Use offset/limit parameters or the Shell tool for large files.

read_file() checks the size before it looks at offset and limit, though, so a read that passes them gets the same error. The remedy the message offers can never work: two lines of an 11 MB log are refused just like the whole file, and an agent that follows the advice gets the same answer again and has to fall back to the PowerShell tool. The message also points to a Shell tool; the tool is registered as PowerShell.

For files under the limit, the ranged branch reads the whole file with readlines() to return a few lines of it.

Reproduced at a736a72 on Windows 11 with Python 3.14, with a log of 110,000 numbered lines (11,110,000 bytes) in a temp folder:

read_file(p, offset=100, limit=2)  # the "File too large" error above
read_file(p, limit=3)              # the same error
read_file(p, offset=109_999)       # the same error

• The registered FileSystem handler gives the same error for FileSystem(mode="read", path=p, offset=100, limit=2).
• Returning lines 100-101 of a 9.1 MB file, which is under the limit, peaks at 13.7 MB of Python memory (measured with tracemalloc).

Fix

In read_file():

• Only a read without offset and limit is refused for a file over the limit. The message is unchanged apart from naming the PowerShell tool.
• A ranged read streams the file with readline() instead of loading it with readlines(). It still counts every line, so the Lines a-b of N: header is the same as before, and it keeps only the requested lines.
• The text a ranged read returns is capped at MAX_READ_SIZE (10,485,760) characters, the same number that limits whole reads. Past that it answers Error: The requested lines add up to more than 10,485,760 characters. Use a smaller limit or the PowerShell tool.
• readline() gets MAX_READ_SIZE as its size argument, so a file that is one very long line, such as minified JSON or a binary file read by mistake, is read in pieces instead of being loaded whole. A single line longer than the cap gets the error above.
• A negative limit is now refused with Error: limit cannot be negative. It used to slice from the end of the list of lines: on a 10-line file, limit=-2 returned lines 1-8 under the header Lines 1--2 of 10, and offset=3, limit=-1 returned no lines under Lines 3-1 of 10. Streaming cannot reproduce that slicing, and neither result makes sense as a number of lines, so I made a negative limit an explicit error. limit=0 still reads to the end of the file, and an offset below 1 still starts at line 1.

With the change:

• On the 11.1 MB log, read_file(p, offset=100, limit=2) returns the header Lines 100-101 of 110000: and those two lines, in about 35 ms here. offset=109_999 without a limit returns the last two lines. A read without offset and limit still gets the "File too large" error, which now names the PowerShell tool. The FileSystem handler returns the same.
• Reading two or three lines peaks at about 0.2 MB of Python memory for the 9.1 MB file, the 11.1 MB log and a 303 MB log of 3 million lines. On the 303 MB log such a read takes about 1.3 s, because it still reads to the end to count the lines for the header.
• The line loop costs time where readlines() was fast: on a file made of very short lines (10 MiB of empty lines) a ranged read takes about 2.3 s instead of 0.8 s, while its peak memory drops from 89 MB to 0.2 MB. Binary content read by mistake is now scanned at about 45 MB/s instead of being refused, so a multi-GB binary file takes tens of seconds. Counting the lines after the requested range in 1 MiB chunks would make those cases much faster; I can add that if you prefer.
• On a 60 MB file that is one line followed by a line last, offset=2, limit=1 returns Lines 2-2 of 2: and last with a peak of about 32 MB, and offset=1, limit=1 gets the "add up to more than" error.
• On files under the limit, ranged reads return what they returned before. I compared the old and the new read_file() in 2,696 calls: small files with LF, CRLF, CR and mixed line endings, without a final newline, empty, with invalid UTF-8 and with UTF-8 and UTF-16 BOMs, plus 2.7 to 6.1 MB files with lines of up to 20,000 characters; offsets from negative to past the end of the file, limits from none and 0 up, and the utf-8, latin-1 and cp1252 encodings. Every result was identical. The Python loop is slower than readlines(): lines 100-101 of the 9.1 MB file take about 27 ms instead of 17 ms.

This only touches read_file(). #537 changes write_file() in the same file and adds tests to TestWriteFile; the two branches merge without conflicts, and TestReadFile and TestWriteFile pass on the merge.

Tests

• tests/test_filesystem_service.py: 9 cases in TestReadFile, with a fixture that writes the 110,000-line, 11,110,000-byte log to tmp_path:

  • the first 3 lines, lines 100-101, the last lines with limit=10, and offset=109999 without a limit come back with the right header and lines;
  • a whole read is still refused, and the message names the PowerShell tool;
  • offset=1, limit=110000 and offset=1 alone get the "add up to more than" error;
  • in a file whose first line is longer than MAX_READ_SIZE, offset=2, limit=1 returns Lines 2-2 of 2: and the second line, and reading the first line gets the error;
  • a negative limit gets Error: limit cannot be negative.

  All 9 fail on main and pass with this change. The other TestReadFile cases pass on both, including the BOM case from fix(filesystem): decode BOM-marked UTF-16 and UTF-8 files in read mode #516 that reads with offset=2, limit=1. The new cases write about 90 MB of temp files in all and take about 2 to 3 s here.

• tests/test_filesystem_service.py (45 passed on main, 54 with the new tests), tests/test_tool_boolean_flags.py (3 passed), tests/test_filesystem_overlap.py (8 passed) and tests/test_filesystem_views.py (18 passed): same results on main and on this branch. I ran these files one at a time, not the whole suite. The 14 symlink and junction tests in test_filesystem_service.py and the 2 case-only rename tests in test_filesystem_overlap.py fail here on main and on this branch alike, because the drive I ran them on is exFAT, which has no reparse points and keeps the old name on a case-only rename. They need NTFS, as on the CI runner.

• ruff check is clean on the changed files, and ruff format --check is clean on the test file. For service.py, ruff format --check reports formatting throughout the file, mostly single quotes it would turn into double quotes, the same as on main. The changed lines use single quotes like the rest of the file, and I did not reformat the other lines.

Not tested: a full MCP client session (I called read_file() and the registered FileSystem handler directly, the way tests/test_tool_boolean_flags.py does), files larger than 303 MB, and encodings other than UTF-8, UTF-16, latin-1 and cp1252.