Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/source/faq.md
Original file line number Diff line number Diff line change
Expand Up @@ -132,7 +132,7 @@ Notes:

- Flanks only see the variants returned for the query region. Flank bases that extend past the region use the reference, not variants outside the region.
- `flank_source="haplotype"` cannot be combined with `with_settings(unphased_union=True)`, because a union row mixes both haplotypes and has no single haplotype context. The combination raises a `ValueError`.
- Haplotype flanks cost no extra throughput. The kernel skips the reference window read, and in benchmarks it is 15–58% faster than reference flanks.
- Haplotype flanks cost about the same as reference flanks. In kernel benchmarks they take 0.94–1.7x the time of reference flanks; the extra work is building a short per-row haplotype buffer.

## Can I build a dataset in parallel shards and merge them?

Expand Down
4 changes: 2 additions & 2 deletions skills/genvarloader/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -320,9 +320,9 @@ fw.alt # bare alt allele tokens (no flanks); fw.alt_window is
fw.ref_window.shape # the window buffer's own shape: (b, p, ~v, ~len)
```

**Ride-along `FlatVariants.flank_tokens`** (`with_seqs("variants")` + `with_settings(flank_length=L, token_alphabet=..., unknown_token=...)`): appends a `FlatRagged` of shape `(b, p, ~v, 2L)` to the returned `FlatVariants`. Per variant the buffer holds `[flank5 | flank3]` reference-context tokens (each `L` long). Coordinate rule: `flank5 = [start-L, start)`, `flank3 = [end, end+L)` where `end = start - min(ilen, 0) + 1`. `token_alphabet` (`str`, `bytes`, or `seqpro.NucleotideAlphabet` — e.g. `sp.alphabets.DNA` / `sp.DNA.alphabet`; normalized to `bytes` at the `with_settings`/`build_token_lut` boundary) and `unknown_token` (int) together build a 256-entry byte→token LUT (seqpro-style): each alphabet byte → its 0-based index; every other byte (including `N` and out-of-bounds padding) → `unknown_token`. `flank_length=0`/`None` disables; both `token_alphabet` and `unknown_token` must be set together. Token dtype is `uint8` when max token id ≤ 255, else `int32`; offsets are `int64`. When `with_settings(dummy_variant=...)` is set, each empty `(region, sample, ploid)` group's `flank_tokens` row is a `2L`-long run of `unknown_token`.
**Ride-along `FlatVariants.flank_tokens`** (`with_seqs("variants")` + `with_settings(flank_length=L, token_alphabet=..., unknown_token=...)`): appends a `FlatRagged` of shape `(b, p, ~v, 2L)` to the returned `FlatVariants`. Per variant the buffer holds `[flank5 | flank3]` reference-context tokens (default; see `flank_source` below) (each `L` long). Coordinate rule: `flank5 = [start-L, start)`, `flank3 = [end, end+L)` where `end = start - min(ilen, 0) + 1`. `token_alphabet` (`str`, `bytes`, or `seqpro.NucleotideAlphabet` — e.g. `sp.alphabets.DNA` / `sp.DNA.alphabet`; normalized to `bytes` at the `with_settings`/`build_token_lut` boundary) and `unknown_token` (int) together build a 256-entry byte→token LUT (seqpro-style): each alphabet byte → its 0-based index; every other byte (including `N` and out-of-bounds padding) → `unknown_token`. `flank_length=0`/`None` disables; both `token_alphabet` and `unknown_token` must be set together. Token dtype is `uint8` when max token id ≤ 255, else `int32`; offsets are `int64`. When `with_settings(dummy_variant=...)` is set, each empty `(region, sample, ploid)` group's `flank_tokens` row is a `2L`-long run of `unknown_token`.

**Haplotype ("personalized") flanks: `flank_source="haplotype"`** (on `with_settings(...)` for the ride-along, or on `VarWindowOpt(...)` for windows; default `"reference"`). Flanks come from the haplotype that carries each variant instead of the reference, so neighbouring ALTs on the same haplotype appear in them. `flank5` is the `L` haplotype bases before the variant and `flank3` the `L` after its ALT. For windows, `alt_window` = the haplotype around the variant (`flank5 · ALT · flank3`) and `ref_window` = `flank5 · REF · flank3` (same haplotype context, focal variant reverted). It follows reconstruction semantics: greedy left-to-right, and a variant overlapping an earlier applied one is not on the haplotype; its flanks are the haplotype around its reference position. Flanks only see the variants in the query region; bases past the region come from the reference. Incompatible with `unphased_union=True` (raises `ValueError`). It costs no extra throughput: it skips the reference window read and benchmarks 0.42–0.86× the time of reference flanks.
**Haplotype ("personalized") flanks: `flank_source="haplotype"`** (on `with_settings(...)` for the ride-along, or on `VarWindowOpt(...)` for windows; default `"reference"`). Flanks come from the haplotype that carries each variant instead of the reference, so neighbouring ALTs on the same haplotype appear in them. `flank5` is the `L` haplotype bases before the variant and `flank3` the `L` after its ALT. For windows, `alt_window` = the haplotype around the variant (`flank5 · ALT · flank3`) and `ref_window` = `flank5 · REF · flank3` (same haplotype context, focal variant reverted). It follows reconstruction semantics: greedy left-to-right, and a variant overlapping an earlier applied one is not on the haplotype; its flanks are the haplotype around its reference position. Flanks only see the variants in the query region; bases past the region come from the reference. Incompatible with `unphased_union=True` (raises `ValueError`). Cost is close to reference flanks: kernel benchmarks put it at 0.94–1.7× their time (the extra work is a short per-row haplotype buffer).

**`VarWindowOpt` / `FlatVariantWindows`** (`with_seqs("variant-windows", opt)`): each variant gets a fixed-length token buffer in two modes selected independently for ref and alt via `VarWindowOpt.ref` / `VarWindowOpt.alt` ∈ `{"window", "allele"}`:
- `"window"`: flanked + tokenized — ref-window = tokenized `[start-L, end+L)` reference read; alt-window = tokenized `flank5 · alt-allele · flank3` assembly.
Expand Down
Loading
Loading