Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions .agents/skills/vecscope/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -110,6 +110,10 @@ Inputs: one vector per row in `.npy/.npz/.parquet/.jsonl/.json/.csv`.
- The response lists `inconclusive_findings` (hubness, antihubs and connectivity verdicts
other than `fail`, outlier verdicts other than `ok`): never report those as healthy; say
they need `--sample 0`.
- Check `evidence_scope` first: `sampled` means every number is an estimate from a subset,
and `needs_full_audit` names the findings (inconclusive or unstable) whose verdict cannot
stand for the corpus. Report those as unsettled and re-run with `--sample 0` (MCP:
`sample=0`) before acting on them or passing the JSON on.
- anisotropy, effective_rank, rogue_dimensions, norms, metric_mismatch and
distance_concentration are stable estimates.
- `value_range` is the spread between repeated samples, not an interval that contains the
Expand Down
21 changes: 20 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,10 +4,20 @@ All notable changes to this project are documented here. The format follows
[Keep a Changelog](https://keepachangelog.com/en/1.1.0/) and the project uses
[Semantic Versioning](https://semver.org/).

## [Unreleased]
## [0.12.0] - 2026-11-06

### Added

- `evidence_scope` (`full` or `sampled`) and `needs_full_audit` (the inconclusive and unstable
findings of a sampled audit) at the top level of the evidence JSON, and the MCP summary line
starts with the scope, so a sampled estimate is never read as a full audit.
- MCP server options `--max-rows` and `--max-dim` (`VECSCOPE_MAX_ROWS`, `VECSCOPE_MAX_DIM`)
refuse a matrix over the limit whatever its byte size; a store stops being read at
`--max-rows`. A `MemoryError` during a call is returned to the agent as a tool error that
names the ways out.
- `tests/test_mcp_security.py` pins the server's guards: paths outside the allowed
directories, the read, row and column limits, network stores without opt-in, secrets kept
out of errors, the timeout option, the `MemoryError` path and the scope of a sampled response.
- Documentation site at <https://076923.github.io/vecscope/>, built with Zensical from the README,
`docs/` and the Korean translations, and deployed by `.github/workflows/docs.yml`.
- `CODE_OF_CONDUCT.md` (Contributor Covenant 2.1), `CITATION.cff`, a logo and social preview
Expand All @@ -16,13 +26,22 @@ All notable changes to this project are documented here. The format follows

### Changed

- `docs/verification.md` is organized into five areas (algorithm exactness, thresholds and
retrieval quality, sampled audits, ANN recall and index sweeps, MCP security and operations),
each ending with what it does not cover.
- Workflows run with no default token permissions, pin actions to commit SHAs, and do not keep
the checkout credentials. CI also builds the package, checks it with `twine`, and lints the
workflows with zizmor. The release workflow checks that the built version matches the tag.
- The code is formatted with `ruff format`, and CI and pre-commit check it.
- The Korean README and the agent cases page link to repository files by URL, so the links work
on the documentation site as well as on GitHub.

### Fixed

- The `timeout=` option of a network store rejects `inf`, `nan`, non-numeric and missing
values and caps the wait at 3600 s; before, `timeout=inf` passed the positive check and
disabled the timeout.

## [0.11.0] - 2026-11-01

### Changed
Expand Down
13 changes: 12 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -497,7 +497,8 @@ Options go in the query string:
- `pk=` (Milvus primary key)
- `api_key_env=NAME` (the key is read from that environment variable; keys in the URI itself
are not used)
- `timeout=` seconds per request to a network store (default 60)
- `timeout=` seconds per request to a network store (default 60; a finite positive number,
at most 3600)

Reading is read-only, except that a Milvus collection that is not loaded is loaded for the
read and released again.
Expand Down Expand Up @@ -646,8 +647,14 @@ One request cannot exhaust the server's memory:
- A file larger than `--max-read-mb` (default 4096; `VECSCOPE_MAX_READ_MB`; 0 for no limit)
is refused before it is read.
- A store stops being read once its vectors pass the same size.
- `--max-rows` and `--max-dim` (`VECSCOPE_MAX_ROWS`, `VECSCOPE_MAX_DIM`; default 0 for no
limit) refuse a matrix with more rows or columns than that, whatever its byte size; the
queries file is checked the same way, and a store stops being read at `--max-rows`.
- Files are parsed straight from the checked handle, never copied into memory first. On a
500,000 x 384 `.npy` file (0.72 GiB) an MCP read peaks at 0.79 GiB, as the CLI does.
- A `MemoryError` while reading or auditing is returned to the agent as a tool error that
names the ways out (a sample, a smaller input, the limits above) instead of ending the
call silently.

Calls in parallel, or with expensive settings, are bounded too:

Expand Down Expand Up @@ -970,6 +977,10 @@ about the sample. Which findings need every row:

A sampled audit therefore:

- sets `evidence_scope` to `sampled` (`full` when every row was audited) and lists in
`needs_full_audit` the findings whose verdict cannot stand for the corpus: the inconclusive
ones and, with `--sample-repeats`, the unstable ones. Anything consuming the JSON can gate
on these two keys without reading the per-finding caveats
- lists `inconclusive_findings` (the verdicts the table marks inconclusive)
- flags them `INCONCLUSIVE (sampled)` in the text output and names them in the gate line
- takes the collapse verdict from `details.estimated_full_share` (see below)
Expand Down
12 changes: 11 additions & 1 deletion docs/ko/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -486,7 +486,8 @@ vecscope audit "weaviate://localhost:8080/Docs?grpc_port=50051"
- `https=1`
- `pk=`(Milvus 기본 키)
- `api_key_env=NAME`(키는 그 환경 변수에서 읽고, URI에 직접 적은 키는 쓰지 않음)
- `timeout=`(네트워크 저장소 요청 하나의 제한 시간, 초 단위, 기본값 60)
- `timeout=`(네트워크 저장소 요청 하나의 제한 시간, 초 단위, 기본값 60. 유한한 양수여야 하고 최대
3600)

읽기만 하며, 로드되지 않은 Milvus 컬렉션만 읽는 동안 로드했다가 다시 해제합니다.

Expand Down Expand Up @@ -629,8 +630,13 @@ args = ["--allow-dir", "/data/embeddings"]
- `--max-read-mb`(기본값 4096, `VECSCOPE_MAX_READ_MB`, 0이면 제한 없음)보다 큰 파일은 읽기 전에
거부합니다.
- 저장소도 벡터가 같은 크기를 넘으면 읽기를 멈춥니다.
- `--max-rows`와 `--max-dim`(`VECSCOPE_MAX_ROWS`, `VECSCOPE_MAX_DIM`, 기본값 0이면 제한 없음)은
바이트 크기와 상관없이 행이나 열이 그보다 많은 행렬을 거부합니다. 질의 파일도 같은 기준으로
확인하고, 저장소는 `--max-rows`에서 읽기를 멈춥니다.
- 파일은 확인한 핸들에서 바로 파싱하고, 먼저 메모리에 통째로 복사하지 않습니다. 500,000 x 384
`.npy` 파일(0.72 GiB)을 MCP로 읽을 때 최대 메모리는 CLI와 같은 0.79 GiB입니다.
- 읽기나 감사 중에 `MemoryError`가 나면 호출을 조용히 끝내지 않고, 빠져나갈 방법(표본, 더 작은
입력, 위의 상한)을 적은 도구 오류로 에이전트에게 돌려줍니다.

병렬 호출과 비용이 큰 설정에도 한도가 있습니다.

Expand Down Expand Up @@ -950,6 +956,10 @@ zipf)을 받고 실행 환경 정보를 JSON에 함께 기록합니다.

그래서 표본 감사는 다음과 같이 처리합니다.

- `evidence_scope`를 `sampled`로 둡니다(모든 행을 감사했으면 `full`). 코퍼스 전체를 대신할 수
없는 판정, 즉 불확정(inconclusive) 항목과 `--sample-repeats`를 썼을 때의 불안정 항목을
`needs_full_audit`에 나열합니다. JSON을 받는 쪽은 항목별 주의 문구를 읽지 않아도 이 두 키로
게이트를 걸 수 있습니다.
- 표에서 inconclusive로 표시한 판정들이 `inconclusive_findings` 목록으로 들어갑니다.
- 그 판정들은 텍스트 출력에서는 `INCONCLUSIVE (sampled)`로, gate 줄에서는 이름으로
표시됩니다.
Expand Down
Loading
Loading