Skip to content

fix(tools): accept JSON-string arrays for Snapshot/Screenshot display and App window_loc/window_size - #545

Merged
Jeomon merged 1 commit into
CursorTouch:mainfrom
tanishqbaweja:fix/json-string-list-params
Nov 4, 2026
Merged

Jeomon merged 1 commit into
CursorTouch:mainfrom
tanishqbaweja:fix/json-string-list-params

Conversation

@tanishqbaweja

Copy link
Copy Markdown
Contributor

Problem

Some MCP clients strip the anyOf schema of optional parameters, so the model sends a list argument as its JSON text: display="[0, 1]" instead of display=[0, 1]. #272 described this for loc and noted that the same fix applies to "any other optional list / dict params". #277 then made loc, locs and labels accept a string, and region (#388) and App's args (#335) were later made to accept one as well. Four list parameters still do not: display on Snapshot and Screenshot, and window_loc and window_size on App. They are declared list[int] | None, so their schema has no string branch, unlike region next to them:

Snapshot.display: {"anyOf": [{"items": {"type": "integer"}, "type": "array"}, {"type": "null"}], "default": null}
Snapshot.region:  {"anyOf": [{"items": {"type": "integer"}, "type": "array"}, {"type": "string"}, {"type": "null"}], "default": null}

FastMCP rejects a call that sends one of them as text before the tool runs. For a client that sends lists as text, display cannot pick monitors, and App's resize mode cannot be given a position or a size. region sent as text is accepted, but a call that also sends display as text fails.

Reproduced at 92f5e8c on Windows 11 with Python 3.14.9, fastmcp 4.1.3 and pydantic 2.13.5. I registered the App, Snapshot/Screenshot and input tools on a FastMCP instance and called them in-process with await mcp.call_tool(...), so the arguments went through FastMCP's own validation. The desktop was a fake that only records calls, and capture_desktop_state and build_snapshot_response in tools.snapshot were replaced by a recorder and a stub, so nothing was captured, moved or resized:

  • Scroll with loc="[10, 20]": OK, the desktop gets [10, 20] (_as_loc).
  • Snapshot with display=[0]: OK.
  • Snapshot with display="[0]" fails, and display="[0, 1]" fails the same way:
    1 validation error for call[_state_tool]
    display
      Input should be a valid list [type=list_type, input_value='[0]', input_type=str]
    
  • Screenshot with display="[1]": the same list_type error for call[_screenshot_tool].
  • App with mode="resize", window_loc=[0, 0] and window_size=[800, 600]: OK.
  • The same call with window_loc="[0, 0]" and window_size="[800, 600]": 2 validation errors for call[app_tool], list_type for both.
  • Snapshot and Screenshot with region="[0, 0, 100, 100]": OK, the capture gets [0, 0, 100, 100]. With display="[0]" added to the same call, it fails on display.

Fix

  • The four parameters are declared list[int] | str | None, like loc and region.
  • A new as_int_list(value, name) in desktop/utils.py, next to as_bool(), parses the string. tools/snapshot.py and tools/app.py import it as _as_int_list, the way other tools import as_bool. None and lists are returned unchanged (FastMCP has already validated a list against list[int]). A string has to hold a JSON array of integers; anything else, such as "[0,", "0", '["0"]' or "[true]", raises ValueError("<name> must be a JSON array of integers"). A malformed value is reported rather than dropped, and the string form cannot hand resize anything the list form would reject: '["a", "b"]' would otherwise only fail inside resize_app(), after the window lookup.
  • Snapshot and Screenshot parse display where they already parse region, inside their try, so a bad value comes back through each tool's existing message (Error capturing desktop state: display must be a JSON array of integers. Please try again. from Snapshot, Error capturing screenshot: ... from Screenshot). App parses after the launch_executable branch, which still rejects window_loc and window_size as before.
  • Desktop.parse_display_selection() is unchanged and still rejects raw strings (test_rejects_invalid_display_values in tests/test_snapshot_display_filter.py).

With the change, in the same setup, the four schemas include {"type": "string"}. display="[0]", "[0, 1]" and (Screenshot) "[1]" reach the capture as [0], [0, 1] and [1], App gets [0, 0] and [800, 600], and the call with both region and display as text works. Lists pass unchanged. display="[0," returns the message above without capturing, and window_loc="[0," or '["a", "b"]' gives Error calling tool 'App': window_loc must be a JSON array of integers without reaching the desktop.

Tests

  • New tests/test_tool_list_arguments.py, a sibling of tests/test_tool_boolean_flags.py. It calls the tools through a real FastMCP with Client(...).call_tool(..., raise_on_error=False), as tests/test_app_switch_foreground.py does, because the failure is in FastMCP's argument validation, which the FakeMCP in tests/test_app_tool.py skips. The desktop is a FakeDesktop that records app() calls, and the fixtures monkeypatch capture_desktop_state in tools.snapshot with a recorder and build_snapshot_response with a stub. register() rebinds snapshot.state_tool and snapshot.screenshot_tool, which windows_mcp.__main__ hands to tests/test_snapshot_display_filter.py, so the fixture puts them back afterwards.

    • test_display_accepts_json_text, for Snapshot and Screenshot: "[0]" and "[0, 1]" reach the capture as lists, and [1] and None pass unchanged.
    • test_display_text_that_is_not_an_array_of_integers_is_reported: "[0,", "0", '["0"]' and "[true]" return the error and capture nothing.
    • test_app_resize_accepts_json_text: "[10, 20]"/"[800, 600]" and the list form both reach desktop.app() as [10, 20] and [800, 600].
    • test_app_text_that_is_not_an_array_of_integers_is_reported: "[10,", "10" and '["10", "20"]', for each of the two parameters, are MCP errors that name the parameter, and desktop.app() is not called.

    19 of the 24 cases fail on main with the list_type error, and all 24 pass with this change. The 5 that pass on both are the list and None cases.

  • tests/test_app_tool.py (16 passed), tests/test_snapshot_display_filter.py (37 passed), tests/test_app_switch_foreground.py (9 passed) and tests/test_desktop_utils.py (9 passed): the same on main and on this branch. I ran these files one at a time, not the whole suite.

  • The branch is rebased on main at 92f5e8c, which includes feat(snapshot): add save_path to write the screenshot to a PNG file #533 (save_path, which also edits tools/snapshot.py), and merges without conflicts with the open fix(clipboard): retry OpenClipboard while another process holds the clipboard #527-feat(safety): focus the confirmation dialog, catch the kill-switch hotkey in the input hook, Esc answers No #531 (feat(safety): check launches against the resolved executable, including .lnk targets #529-feat(safety): focus the confirmation dialog, catch the kill-switch hotkey in the input hook, Esc answers No #531 also edit tools/app.py).

  • ruff check is clean on the four files, and ruff format --check is clean on tools/app.py and the new test file. For desktop/utils.py and tools/snapshot.py, ruff format --check reports the existing formatting (single quotes, except (AttributeError, Exception):, blank lines), the same as on main. It does not change the new lines, and I did not reformat the others.

Not tested: a live session with a client that strips the schema. The tests send the string form that #272 and #342 recorded for loc ('[1000, 600]', '[500, 500]'). I did not capture the screen or move or resize real windows.

… and App window_loc/window_size

MCP clients that strip the anyOf schema of optional parameters make the
model send a list argument as its JSON text, display="[0, 1]" instead of
display=[0, 1] (CursorTouch#272). CursorTouch#277 made loc, locs and labels accept that form,
and region (CursorTouch#388) and App's args (CursorTouch#335) later took it as well, but
display on Snapshot and Screenshot and window_loc and window_size on
App were still declared list-only. FastMCP rejected any call that sent one of them as text with
"Input should be a valid list" before the tool ran.

Declare the four parameters as array | string like the others, and parse
a string with a new as_int_list() next to as_bool() in desktop.utils. A
string must hold a JSON array of integers; anything else is rejected with
an error that names the parameter. Lists pass through unchanged, and
Desktop.parse_display_selection() still rejects raw strings.
@Jeomon
Jeomon merged commit 9ab92bc into CursorTouch:main Nov 4, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants