Skip to content

Merge small chunks before computing windowed statistics - #1366

Open
Mohit-Ak wants to merge 1 commit into
sgkit-dev:mainfrom
Mohit-Ak:fix/window-longer-than-chunk
Open

Mohit-Ak wants to merge 1 commit into
sgkit-dev:mainfrom
Mohit-Ak:fix/window-longer-than-chunk

Conversation

@Mohit-Ak

Copy link
Copy Markdown

Problem

window_statistic only merged the last chunk when it was shorter than the longest window. With window_by_position a window can easily span more variants than a middle chunk. map_overlap then rechunks the input silently, the windows_per_chunk passed as output chunks no longer matches, and divergence/Fst fail with ValueError: Dimension 0 has N blocks, adjust_chunks specified with M blocks.

Change

Before calling map_overlap, merge consecutive chunks so that each one is at least as long as the longest window. The old last-chunk special case was one instance of this.

Testing

  • New test_window_statistic__window_longer_than_chunk (3 chunk layouts): fails on main with the error from the issue and passes with the fix.
  • The issue's pipeline (window_by_position → divergence → Fst) on chunked simulated data now matches the unchunked result.
  • pytest sgkit/tests: 1337 passed. black/isort/flake8 at the pre-commit versions are clean.

Fixes #1343

window_statistic only merged the last chunk when it was shorter than the
longest window, so a long window elsewhere (common with window_by_position)
made map_overlap rechunk and the windows-per-chunk no longer matched.

Fixes sgkit-dev#1343
@Mohit-Ak
Mohit-Ak marked this pull request as ready for review November 3, 2026 03:40

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Error trying to get Fst for window_by_position

1 participant