Skip to content

[WIP] Experimenting with late interaction - #478

Draft
ahuber21 wants to merge 68 commits into
mainfrom
dev/ahuber/late-interaction
Draft

ahuber21 wants to merge 68 commits into
mainfrom
dev/ahuber/late-interaction

Conversation

@ahuber21

Copy link
Copy Markdown
Member

No description provided.

ahuber21 and others added 30 commits October 30, 2026 02:59
… vectors

Distances for multi-vector retrieval in the style of GEM (arXiv:2603.20336)
over a document's token set stored as one fixed-width vector of centroid
ids ([n, ids..., padding]):

- QEmd: exact earth mover's distance over a centroid-pair cost table, solved
  with POT's network simplex (vendored unmodified in third-party/pot).
- QChamfer: quantized Chamfer distance with both sides quantized.
- QChamferExactQuery: raw float query tokens against document centroids,
  with the per-query dot table built in fix_argument.

The kernels are instantiated per ISA in multi-arch/x86 like L2 and IP.
pre-commit skips the vendored headers so they stay byte-identical.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
DefaultBuildAdaptor default-constructs its distance instead of copying the
one passed to the builder, which does not compile for QEmd and QChamfer
(they hold their centroid table), and PruneStrategy has no primary
template. Override both for these functors.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A vector set is a variable-cardinality set of token vectors, as used by
late-interaction retrieval. The concept comes first and the only model is a
non-owning view over caller-held tokens and offsets; an owning container can
follow behind the same concept. The view also models ImmutableMemoryDataset,
so existing index code can traverse it.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Sum over query tokens of the maximum inner product over document tokens,
built on the existing dispatched inner product so every supported element
type pair works without new SIMD code. A tile-blocked kernel can replace the
body of maxsim_batch behind the same signature.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Pins QEmd, QChamfer, QChamferExactQuery, CentroidTable and the Vamana
build-adaptor overloads with hand-computed values before the row layout
changes. Also registers the vector-set view and MaxSim tests.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
WithVectorSets pairs the dataset the graph traverses with the vector sets
it was derived from. The single_search overload for it runs the ordinary
graph search, then scores every candidate in the search buffer with MaxSim
against the query set and returns them best-first. Static indexes only: an
index with id translation is rejected at compile time.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
QEmd, QChamfer and QChamferExactQuery accept a ContiguousVectorSet next to
the padded float row: a set of unsigned centroid ids with one id per token,
and a float query set. Both layouts share the per-ISA cores, and the padded
path keeps its arithmetic. The view gains the extent member VamanaIndex
reads, so a graph can be built and searched over it directly.

The id helpers called from the per-ISA structs become members of those
structs, so each ISA keeps its own copy of them.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
First compile of the branch. The re-rank overload names its unused index
parameter, the re-rank test supplies the prune strategy its local distance
needs, and the MaxSim asymmetry test uses sets whose two directions differ.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Moving merge_duplicates into the per-ISA struct made it private and broke an
external timing harness that measures it separately. The two overloads are
public again; behaviour is unchanged.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
VectorSetData<T, Extent, Alloc> owns the same CSR layout
ConstVectorSetView reads: a SimpleData token matrix plus a
std::vector<uint64_t> of offsets, so SVS's allocators, alignment, and
binary file format carry over unchanged. Construction takes both by
value and moves them in, so a caller handing over existing storage
pays no copy; copy_from() covers the case where a caller only has
borrowed memory. A Blocked allocator is rejected at compile time since
it would tear a document's tokens across blocks.

Save/load follows SimpleData's protocol: the token matrix nests its
own save() under a sub-table, the offsets go to a standalone binary
file referenced by metadata. VectorSetDataLoader mirrors
VectorDataLoader so auto_assemble can reload a saved index over this
dataset; auto_assemble never calls verify_and_set_default_index_parameters
(that only runs from the build-parameters constructor), so a custom
distance such as QChamfer hits no obstacle there. Validation is
factored out of ConstVectorSetView's constructor into a shared
detail::validate_vector_set_layout so the view and the owning
container cannot drift apart.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Add save()/load() to WithVectorSets, nesting each member under its
own sub-table. Both are constrained by a requires-clause checking
Primary::save/load and Sets::save/load exist, so a WithVectorSets
over a non-owning Sets (e.g. ConstVectorSetView, which has neither)
still compiles as a dataset member; the methods simply don't
participate for that instantiation.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
QEmd, QChamfer and QChamferExactQuery accepted two token-set layouts: the
old fixed-width padded float row and a vector set. Only the vector-set
layout survives; the padded-row span overloads, their gem_detail decoders
(set_size/set_ids, vector_set_size/vector_set_tokens), and the per-ISA
float-pointer overloads they fed are deleted. QEmdImpl::merge_duplicates
is private again now that its only caller is the vector-set path; the
external gem_time harness that needed it public will be adapted
separately, outside this repository.

Vector-set callers are unaffected: their arithmetic and dispatch are
unchanged. Tests are rewritten to pass the same hand-computed cases as
vector sets instead of padded spans, dropping only the padding-is-ignored
and layout-comparison cases that tested the removed layout; no expected
value changed. A static_assert pins down that a bare std::span can no
longer satisfy QChamfer's compute() constraint.

BREAKING CHANGE: removes QEmdImpl::compute(span,span),
QChamferImpl::compute(span,span), QChamferImpl::decode_ids,
QEmdImpl::merge_duplicates(const float*, ...) (now private for the
uint32_t overload too), QEmd::compute(span,span), QChamfer::compute(span,
span), QChamferExactQuery::fix_argument(span) and
QChamferExactQuery::compute(span,span). gem_detail::set_size, set_ids,
vector_set_size and vector_set_tokens are also removed.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
QChamferExactQuery::fix_argument built its (K x n_query) dot table with a
per-centroid-row loop limited to 2 independent FMA accumulators, which left
it latency-bound well short of the AVX-512 FMA ceiling. Factor that build
into a standalone inner_product_matrix kernel (generic/AVX2/AVX512, same
dispatch mechanism as the other GEM distances) that blocks several A-rows
per step so the FMA dependency chain is spread across more accumulators;
GEM's build_table is now a direct call into it. The reduction half
(compute_decoded) is unchanged in this commit; its cross-TU weak-symbol
duplication is fixed separately next.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
gem_detail::qchamfer_exact_reduce was a free inline function called from
QChamferExactImpl::compute_decoded in the generic header and from both
avx2.cpp and avx512.cpp. Each TU compiled it under different -m flags but
emitted the same weak inline symbol, so the linker could keep an arbitrary
one; a host lacking AVX-512 could end up executing code built for it.

Move the reduction into QChamferExactImpl's primary template body (one
source text) and drop its AVX2/AVX512 specializations entirely: the
explicit instantiation already in avx2.cpp/avx512.cpp now compiles that
same body under each TU's own flags, giving each ISA a distinctly named,
non-weak symbol, matching how QEmdImpl/QChamferImpl already do this.

Verified via `nm -C` on both object files: no QEmdImpl/QChamferImpl/
QChamferExactImpl/InnerProductMatrixImpl symbol is shared between them
anymore. MergeScratch's compiler-generated destructor still is, but it
holds no SIMD-sensitive logic, so it is not an ODR hazard.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Extends the inner-product-matrix kernel family with a shared PackedOperand
(query packed once, reused across documents) and a column-max output form
that keeps the lane-max in registers instead of materializing the m x n
table, with a Float16 overload that converts one row-block at a time.
MaxSim now follows the fix_argument protocol (like QChamferExactQuery) so
the Vamana re-rank extension packs the query once per search instead of
once per candidate.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
…rProductMatrix

MaxSim called InnerProductMatrixImpl<AVX512|AVX2|NONE>::column_max directly
through its own three-way switch on InnerProductMatrix::select_isa(),
duplicating the kernel's own dispatch and leaking the Impl struct and the
ISA enum into a caller. InnerProductMatrix now exposes column_max(float)
and column_max(Float16) wrappers that share one private dispatch() helper
with compute(); select_isa() is private again. MaxSim calls only the
wrapper and keeps a member maxima_ buffer (sized in fix_argument) instead
of a thread_local one, since implicit_broadcast=false already gives each
search thread its own MaxSim copy.

Also tightens the Float16 generic-vs-ISA tolerance in
inner_product_matrix.cpp from an unmeasured epsilon(5e-4)/margin(5e-5) to
epsilon(2e-4)/margin(1e-4), about 2x the largest summation-order
difference (4.7e-5 abs / 3.1e-5 rel) measured by a throwaway probe over
the test's own shapes.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
…ch pair once

from_centroids now tiles the K x K ground-cost table over the upper
triangle of blocks (including diagonal blocks), computing each tile with
InnerProductMatrix::compute and mirroring the transformed value into both
T[i][j] and T[j][i] instead of calling it twice. Tiles are scheduled with
a DynamicPartition to balance the uneven tile count per row block; the
table's storage, load, and operator()(i, j) are unchanged.

Symmetry is now guaranteed by construction (one computed float stored at
both cells) rather than by two independently computed dot products, so
results can differ from the old code in the last bits.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
…o InnerProductMatrix

Adds svs::BFloat16 overloads of both InnerProductMatrixImpl output forms
(build_packed/table and column_max) at every ISA level (generic, AVX2,
AVX512), sharing the Float16 overloads' row-block-conversion shape via a
templated helper per ISA file; also fills in the previously-missing
Float16 overload of the table form since the shared helper makes it
free. BFloat16's widening to float32 is an exact zero-extend (bfloat16
is float32's high 16 bits), unlike Float16's lossy half-to-single
conversion.

PackedOperand gains pack_rounded_to_bf16, rounding B to bfloat16
(round to nearest, ties to even) and storing the result as float32:
rounding B is the caller's choice of precision, not a hidden kernel
behaviour, so it lives in how B is packed rather than in the kernel
itself. svs::BFloat16's own constructor truncates instead of rounding;
svs::round_to_bfloat16 (bfloat16.h) adds the correctly-rounded
conversion both this and the bf16 A overloads above need to agree on.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
…float16

FineCentroids now holds its K x dim matrix as svs::BFloat16 (round to
nearest, ties to even) instead of float32: one representation, no
float32 copy kept after construction/load, so every ISA level computes
on identical values and the upcoming AMX level can consume the same
layout directly. The constructor and load() round in parallel (a
thread pool each already had access to); load() additionally reads
and rounds in row-block chunks so peak memory never holds the full
float32 input alongside the full bf16 output.

QChamferExactQuery::fix_argument_decoded now packs the query rounded
to bf16 (PackedOperand::pack_rounded_to_bf16) and builds the per-query
table with InnerProductMatrix::compute_packed against the (already
bf16) centroids, so the table is <bf16(centroid), bf16(query)> on
every ISA level. CentroidTable::from_centroids rounds both the row and
column block of each tile to bf16 before computing, so its K x K table
agrees with FineCentroids on the same rounded values; the public
signature (float32 centroids in) is unchanged, since rounding now
happens per tile instead of requiring a second, already-bf16 input.

Test fixtures: all pre-existing hand-computed expectations use only
values exactly representable in bfloat16 (1, 0, -1), so none changed.
Added: a bf16-rounding section for FineCentroids (incl. a load()
round-trip through the unchanged float32 file format), a dedicated
non-representable-value case for both CentroidTable::from_centroids and
QChamferExactQuery against a naive bf16-rounded reference, and rounded
the from_centroids tiling test's random input before building its
reference (bf16 rounding error otherwise swamped the existing 1e-4
summation-order margin).

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Extends AVX_AVAILABILITY with AMX = 10, leaving the ordinals 3-9 free
for a parallel split of AVX512 into AVX512F/AVX512VNNI: nothing in
this codebase reads the enum's ordinal or order, only switches and
templates on the enumerator itself, so a gap is free. Detection in
avx_detection.h follows the existing CPUID-probe style and adds the
one-time, thread-safe OS permission request AMX tile use needs
(ARCH_REQ_XCOMP_PERM), cached in a function-local static so the
per-call cost stays one branch. Build integration in
cmake/multi-arch.cmake adds amx.cpp only when the compiler accepts
-march=sapphirerapids, gated through a build-wide
SVS_AMX_TILE_BF16_SUPPORTED definition so a build without that
support still produces a working library with the level compiled
out.

No existing kernel's dispatch changes: InnerProductMatrix::select_isa
and its exhaustive switch gain an AMX branch guarded on
SVS_AMX_COMPILED_IN, falling back to the AVX512 case when the level
was never compiled in.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Adds the AMX tile-dot-product kernel for InnerProductMatrixImpl's
table (build_packed) and column-max forms, selected only when AMX is
available, A is BFloat16, and B was packed with
PackedOperand::pack_rounded_to_bf16 (which now also fills a
pair-interleaved transposed bf16 tile layout, built with ordinary
scalar code so it needs no -mamx flags of its own, gated on the
build-wide SVS_AMX_COMPILED_IN and the runtime availability check).
Every other case (float32/Float16 A, or an unrounded B) forwards to
the AVX-512 implementation, keeping one source of truth.

The kernel processes A 16 rows and B 32 lanes (two n-tiles) at a
time, with two independent fp32 accumulator tiles so their dpbf16ps
chains overlap; A's final row-block (m not a multiple of 16) gets a
second, one-off tile reconfigure, and A's final k-chunk (d not a
multiple of 32) is copied through a zero-padded stack buffer so the
kernel never reads past the caller's d columns. Tile numbers are
literal integers throughout (0-4): gcc's tile intrinsics stringify
their register argument, so a named constant would not assemble.

_tile_loadd/_tile_stored carry no memory operand or clobber, so the
compiler is otherwise free to reorder a preceding store (the tail
copy) or a following read (emit's use of the stored tiles) across
them; both call sites get an explicit compiler memory barrier.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Sweeps InnerProductMatrixImpl<AMX>'s build_packed and column_max
against a double-precision reference and against AVX-512, over m/n/d
shapes crossing the 16-row, 16-lane, and 32-element (16-pair) tile
boundaries; checks float32/Float16 A and an unrounded B forward to
AVX-512 bit for bit; checks the availability probe agrees across
concurrent threads; and checks a thread that never itself requested
AMX permission can still run the kernel after another thread already
has. Skips when amx_tile_bf16_available() is false, so this is a
no-op on hardware or a build without the level.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
…ernel

Templates MaxSim on its document element type (default float, so the
non-template spelling keeps today's behaviour) instead of branching on it
inside fix_argument. A document of the wrong element type now fails to
compile via compute's requires-clause, and fix_argument packs the query
rounded to bf16 only for MaxSim<svs::BFloat16>, so float/Float16 callers
still pack and score unrounded, bit for bit as before.

The Vamana re-rank extension now instantiates MaxSim<Sets::const_value_type
::element_type>, so a bfloat16 vector-set store automatically gets the bf16
(and AMX, where available) InnerProductMatrix::column_max path.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
…ails

Adds a mmap/mprotect-based test that places A so it ends exactly at a
PROT_NONE page (catches an overrun), places `out` the same way, and places
A at the start of a mapping with a leading PROT_NONE page (catches an
underrun), then runs both the table and column-max forms over every tail
shape. Covers the AMX bfloat16 tile kernel plus, where available, the
AVX-512 and AVX2 float/Float16/bfloat16 kernels. POSIX-only, guarded so the
file still compiles on a non-POSIX target; no-ops when the relevant ISA is
unavailable.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
gcc 11's _tile_loadd/_tile_stored macros expand to inline asm with no
memory operand and no clobber, so the optimizer believes they neither
read nor write memory: a plain C++ store just before a load, or a
plain C++ read just after a store, through the same address had no
ordering guarantee and could be reordered around the tile
instruction. Two bare `__asm__ volatile("" ::: "memory")` barriers
papered over this; a third spot (configure_tiles' struct writes
before _tile_loadconfig) had no barrier at all and was correct only
by accident of how this compiler happens to schedule it today.

Replaces all three with named wrappers, one per literal tile number
actually used (0-4), built from extended inline asm that names the
memory each operation touches: a real "m"/"=m" operand covering the
full extent where it is a compile-time constant (the 64-byte tile
config; B's always-16x64-byte tile blocks; the local zero-padded tail
buffer; the 16x16 fp32 accumulator stores), or an input operand plus
a "memory" clobber where the extent is runtime-only (A's direct
strided load, whose row pitch depends on the caller's `d`).

Verified by a negative control: compiling this file without the
wrappers (bare intrinsics only, the reviewer's stripped copy) at
-O2/-O3 with and without -flto reproduces the hazard (garbage column
maxima); the same scenario against this commit's amx.cpp, built the
same four ways, matches a double-precision reference at all four.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
…hirerapids

-march=sapphirerapids also enables AVX512-FP16, AVX512-BF16, AVX-VNNI
and other extensions the runtime probe never tests; the compiler was
free to use them auto-vectorizing ordinary code in that file, and a
hypervisor can expose AMX without them. Extends the SVS_X86 tuple
format with an optional 4th+ field of extra compiler options (every
existing tuple unchanged: no 4th field, same -march/-mtune as
before), and uses it to compile amx.cpp with exactly
-march=cascadelake -mtune=cascadelake -mamx-tile -mamx-bf16, matching
the avx512 tuple's instruction set plus the two AMX extensions. The
object target name now keys off the tuple's name field when extra
options are present, since amx and avx512 now share the same `arch`
and would otherwise collide on `cascadelake_obj`. The compiler check
tests the two AMX flags directly rather than the (now unused)
-march=sapphirerapids.

amx_tile_bf16_available() now also requires
avx_runtime_flags.is_avx512f_supported(): the AMX level's code is
compiled at that instruction set and its non-tile paths forward to
the AVX-512 kernels directly, so AMX is only meaningful where AVX-512
already is.

Verified with objdump that the amx object's instructions are plain
AVX-512F/scalar: no vaddph/vcvtph2psx/vpdpbusd or other AVX512-FP16/
AVX-VNNI-only mnemonics appear.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Expands amx_tile_bf16_available()'s comment into a documentation
block (exempt from the two-line internal-comment limit) stating two
facts no header or source on this machine confirms: where the
ARCH_REQ_XCOMP_PERM feature number 18 comes from (the kernel's
xstate ABI doc, not any installed header) and that requesting it was
confirmed by experiment on this Linux/x86-64 machine, not assumed
from documentation; and that a seccomp policy killing on an
unlisted syscall terminates the host process at this call. Also
states, citing kernel documentation as unverified here, that the
request is process-wide (tile state becomes part of every thread's
extended processor state) and that the kernel may refuse it if an
installed alternate signal stack is too small.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Shared k-means building blocks (centroid assignment/adjustment/split,
GEMM-backed distance computation, training-set sampling) now live in
svs::kmeans under include/svs/core/kmeans_training.h, so a future
multi-vector codebook trainer can reuse them. Function bodies are moved
verbatim; the IVF-specific kmeans_training(IVFBuildParameters, ...)
overload now forwards to the shared one via a small
KMeansTrainingParameters struct. include/svs/index/ivf/common.h
re-exports every moved name with using-declarations so existing IVF
call sites, tests, bindings and benchmarks compile unchanged.

Named the shared header kmeans_training.h rather than kmeans.h: an
unrelated legacy generic k-means utility (svs::find_nearest,
svs::train, KMeansParameters) already owns include/svs/core/kmeans.h
and its always-built test tests/svs/core/kmeans.cpp; reusing that path
would have required threading an MKL dependency into the default,
IVF/multi-vector-disabled build.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Covers the L2 centroid update formula (initial centroid plus assigned
points, divided by count+1 - not a plain mean), unit-norm centroids
under IP distance, and centroid_assignment agreeing with a brute-force
nearest-centroid oracle for both distances. Registered in
tests/CMakeLists.txt under "IVF or multi-vector", matching the new
header's MKL dependency.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
SVS_EXPERIMENTAL_ENABLE_MULTI_VECTOR (default OFF) joins
SVS_EXPERIMENTAL_ENABLE_IVF as a trigger for pulling in Intel(R) MKL,
since both features train a codebook with the shared k-means code.
IVF-only gates (tests, benchmark, bindings) are untouched.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
ahuber21 and others added 30 commits November 17, 2026 05:47
Adds unreachable_nodes (BFS from a root set) and repair_connectivity, which
gives every unreachable node one inbound edge from a reachable candidate: a
plain append when the candidate has room, else a re-prune, else a forced
replacement of one of the candidate's neighbors. A re-prune or replacement
is taken only when no neighbor it drops loses its last reachable in-edge;
otherwise the repair falls back to an unsafe replacement and counts it. A
round that does not reduce the unreachable count stops the repair.

Needed because a build whose insert order puts a few nodes first (GEM's
representatives) can leave part of the graph unreachable from the roots.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ves first

Adds gem_insert_order (every valid router representative first, deduplicated
and ascending, then every other document ascending) and build_gem_graph,
which runs VamanaIndex's own two-pass build sequence over that order when
the distance carries a router, repairs connectivity from the entry point
plus every representative, and reports what it did.

Also adds the QEmd overload of extensions::build_entry_points: when the
distance carries a router, a node's insert routes through its own centroid-id
set the same way GemRoutedInitializer routes a query, merged with the fixed
entry points (minus the node itself).

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
QuantizeTimings and all std::chrono use leave codebook.h. train_codebook
and quantize are now thin compositions of public phase functions
(sample_training_tokens, train_centroids, assign_tokens,
build_ground_structures, coarse_clusters); train_centroids's k-means loop
takes a compile-time observer (default a no-op) called once per
iteration, with zero cost when unused.

codebook_timed.h is new and opt-in: it defines QuantizeTimings and
quantize_timed/train_codebook_timed, composing the public phases with
timers and an observer that records iteration seconds. Nothing in the
library includes it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…header

Review on PR 478 pointed out an inconsistency: avx2.cpp/avx512.cpp/amx.cpp
previously held only macro-driven explicit instantiations of templates
defined in their headers, but this PR added the InnerProductMatrixImpl
AVX2/AVX512/AMX kernel bodies directly in those .cpp files instead.

Moves every kernel body (and its anonymous-namespace helpers, now named
detail namespaces per ISA: inner_product_matrix_{avx2,avx512,amx}_detail)
into inner_product_matrix.h, next to the specialization each belongs to,
guarded by the same SVS_AVX512_F / SVS_AVX2 / SVS_AMX_TILE_BF16 conditions
that already guarded the specialization declarations. Each .cpp now holds
only includes and explicit instantiation lines, matching euclidean.h's and
gem.h's QEmdImpl/QChamferExactImpl pattern. No kernel logic changed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Replace the scattered #if SVS_ENABLE_STATS blocks in vamana_build.h and
maxsim.h with NullBuildStats/CountingBuildStats (and the maxsim
equivalent) collector classes. Both collectors always compile, so the
stats path is type-checked in every build; only a std::conditional_t
alias on SVS_ENABLE_STATS picks the default, and VamanaBuilder takes
the collector as a defaulted template parameter so tests can force
CountingBuildStats regardless of the build flag.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Second pass. Shares the GEM centroid/corpus fixtures and adds a generic
brute-force top-k oracle plus a degree-bound checker (tests/svs/index/vamana/
gem_test_utils.h, used by all three GEM files and by maxsim_rerank.cpp);
drops gem_routing.cpp's "no router" / "every cluster unrepresented" cases
(covered by gem_build.cpp's build_entry_points); folds connectivity.cpp's
five repair_connectivity cases into one table-driven TEST_CASE; builds one
index per TEST_CASE in gem_vector_set.cpp and maxsim_rerank.cpp and runs
every behavior as a CATCH_SECTION on it; and removes the build-time
routed-vs-unrouted recall comparison from gem_build.cpp's build_gem_graph,
keeping only the query-time one in gem_routing.cpp's "Routed search is not
worse than unrouted" (the layer that owns search-time routing).

Also drops maxsim_rerank.cpp's capacity=40 variant of the MaxSim pool-order
check: a tighter search window only re-exercises the generic search
buffer's eviction path, which search_buffer.cpp already owns, not anything
in the MaxSim re-rank extension under test.

Before -> after (lines, after clang-format):
  build_stats.cpp        139 -> 139 (unchanged, already minimal)
  connectivity.cpp       274 -> 219
  gem_build.cpp          297 -> 170
  gem_routing.cpp        394 -> 245
  gem_vector_set.cpp     313 -> 202
  maxsim_rerank.cpp      687 -> 548
  gem_test_utils.h          0 -> 150 (new, shared across all 6 files)
  Total (6 files + helper, counted together per the <=1050 target) -> 1673,
  against a 2104-line baseline for the 6 files alone.

This still falls short of the <=1050 target. Code-only lines (stripping
every blank line and every comment from all 7 files) sum to over 1300, so
the remaining gap cannot close through formatting or comment trimming
alone: closing it means dropping tested branches, not duplication. Two
branches already lost coverage in this pass, noted above; the handback
report to the coordinator lists further branches that could be cut with
their line cost, for the coordinator to choose from rather than have cut
silently.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Replace QEmdImpl's vendored-POT solve with a clean-room primal network
simplex (include/svs/core/distance/transport_simplex.h): an artificial
root node with big-M arcs gives a strongly feasible initial tree
(Ahuja, Magnanti & Orlin, "Network Flows", 1993, ch. 11), each pivot's
leaving arc is the *last* blocking arc seen walking the cycle from its
apex in the entering arc's own direction (Cunningham, "A network
simplex method", 1976), which keeps the tree strongly feasible and so
guarantees termination independent of how the entering arc is chosen.
That lets the entering rule optimize for speed instead: a block-cyclic
search over the dense cost matrix (block ~sqrt(ua*ub) arcs, rounded to
whole padded rows), picking the most negative reduced cost in the
first block that has one and resuming from where the previous search
left off, with an AVX-512/AVX2/scalar-templated row scan. No code is
reused or paraphrased from POT/LEMON; the algorithm is textbook
network simplex, so there is nothing to attribute beyond the existing
citation of Bonneel et al.'s EMD problem shape already in the new
header's top comment.

An earlier version of this solver used northwest-corner initialization
and Bland's rule; both are replaced here because northwest-corner is
not automatically strongly feasible and Bland's rule (lowest-index
entering arc) is too slow on this hot path's highly degenerate,
asymmetric problem shapes (up to ~180x32 and 180x180), where it tends
to stall rescanning the same low-index rows.

QEmdImpl<Avx>::compute_decoded keeps its signature, the ua==ub==1
shortcut, and throw-on-failure behavior; gem_node_id_type and the POT
include/pragma/undef block are gone since nothing else used them.
Also fixes a stale comment in inner_product_matrix.h that described
amx.cpp's build gate as -march=sapphirerapids support when it is
actually -mamx-tile/-mamx-bf16 support at -march=cascadelake.

Adds a random cross-check test (tests/svs/core/distances/gem.cpp):
the NONE path against an independent successive-shortest-paths
reference, and every compiled ISA path against NONE, over random,
degenerate (equal-cost, single-node-side, wide weight range), and
highly degenerate (uniform weights, tiny integer cost range, up to
64x64) problems -- the family that stalls a naive entering rule.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
qEMD now solves its own transportation simplex (see the prior commit),
so include/svs/third-party/pot/ (POT/LEMON's network_simplex_simple.h,
full_bipartitegraph.h, core.h) has no remaining caller. Drops the
directory and the .pre-commit-config.yaml exclusion that kept it
byte-identical to upstream.

No code from POT or LEMON is reused anywhere in this repo after this
change, so there is nothing left to attribute beyond what the new
solver's header already cites.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Reviewer feedback: the GEM router/codebook/vector-set unit tests were
way too extensive. Consolidates duplicate setup into shared helpers,
merges per-assigner/per-thread-count and per-variant checks into single
loops/sections sharing one fixture, folds validation-failure checks
into tables, and drops hand-picked route_query cases that were strict
subsets of the brute-force reference check already in the suite.

Line counts (before -> after, clang-format applied):
- tests/svs/core/distances/gem_router.cpp:      302 -> 239
- tests/svs/quantization/gem/codebook.cpp:      563 -> 480
- tests/svs/core/data/vector_set.cpp:            261 -> 222
- tests/svs/core/data/vector_set_view.cpp:       137 -> 124
- total:                                        1263 -> 1004 (16% fewer)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ans tests

Second pass, on top of the first consolidation: one shared type/ISA matrix
for build_packed/column_max (float32/Float16/BFloat16 x NONE/AVX2/AVX512),
a compacted mmap guard-page RAII helper, one AMX forwarding+tile-kernel
TEST_CASE, one AMX-permission TEST_CASE, one kmeans_training TEST_CASE with
three sections, and a trimmed MaxSim suite. Also applies a branch-coverage
pass: drops checks that exercise no library branch beyond another check on
the keep list (AMX's float32/Float16 forwarding is one branch, tested with
one type; MaxSim's empty-document and dimension-mismatch checks are one
branch in MaxSim::compute, tested with one document type; compute_packed's
BFloat16/Float16 overloads share one dispatch call, tested with one type;
MaxSim's hand-computed cases were strict subsets of the random-oracle
sections and are dropped).

Before/after line counts:
  tests/svs/core/distances/inner_product_matrix.cpp: 1253 -> 637
  tests/svs/core/distances/maxsim.cpp:                 325 -> 251
  tests/svs/core/kmeans_training.cpp:                  215 -> 208
  total:                                              1793 -> 1096 (38.9% cut)

Still short of the requested >=50% / <=895-line target; see the PR
description / handback report for the itemized list of what would have to
drop (and its branch-coverage cost) to close the remaining ~200-line gap.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Replaces scattered hand-computed cases with one random oracle per distance
(QEmd/QChamfer/QChamferExactQuery) checked against an independent naive
reference, each covering duplicate ids and, for QChamferExactQuery, bf16
non-representable values. Consolidates the three per-distance "generic
dispatch" sections and the three "dimensions() != 1 throws" sections into
one test case each. Compacts reference_transport_cost to a standard
min-cost-flow edge-list SPFA and merges the "many equal costs" and
"uniform-weight tiny-cost" transport-solver families into one parameterized
loop, with lower trial counts (fixed seeds).

Lines: 798 -> 725 (9% cut). Short of the requested 50%/400-line target:
the transport-solver section (independent reference + the four problem
families x per-ISA cross-check, all explicitly requested to stay) is ~195
lines on its own, and the from_centroids tiling test, traits, and Vamana
extension test cases are already near-minimal. Getting under 400 would
mean dropping one of those explicitly-requested items, which this PR does
not do unilaterally.

One real coverage loss, restored: QEmd's "identical sets give 0" invariant
is not exercised by the random oracle (which never draws a == b), so it
stays as its own hand case. Two consolidations are not drop-in equivalent
in form (only in the branches they exercise): the old hand-picked worked
examples for QChamfer/QEmd are gone, replaced by randomized reference
checks, so a reader can no longer eyeball one fixed expected number.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Moves today's big-M network simplex, unchanged in arithmetic, into
TransportSimplexReference: scalar pricing only, no Avx parameter. It
becomes the AVX_AVAILABILITY::NONE path and the test oracle for the
optimized solver that follows. Its helpers live in
transport_reference_detail so the two cannot clash.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The balance residual (up to 1e-6 * total) is folded into the last
demand entry. When that entry is smaller than the residual, the fold
left it at zero or below, which the simplex cannot solve. That input
now throws, naming ua, ub, the entry and the residual. Valid inputs are
unaffected.

The copy, balance check and fold move into prepare_marginals, which the
optimized solver shares. It takes a caller tag type so each -march
translation unit instantiates its own copy rather than sharing one weak
symbol.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…simplex

RowMin<Avx>::min_in_row finds the most negative reduced cost in one
cost row, over float costs with either float or double potentials.
Generic, AVX2 and AVX512 bodies are bit-identical: two separate
subtractions (no FMA), ascending scalar lane scan, first minimum wins.
Rows are padded to 16 lanes with +inf.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Replaces the per-pivot BFS tree rebuild, which was about 80% of the old
solve, with LEMON's incremental spanning-tree update (parent, thread,
rev_thread, succ_num, last_succ), ported by way of POT's
network_simplex_simple.h. On the sub50k neighbors workload, one thread:
246 us -> 86 us per solve (POT: 103 us). The checksum is unchanged.

Structure:
- The smaller side becomes the rows. The optimal cost is invariant.
- Costs are stored as float. CostFn must return float, so this is exact.
- The start is north-west corner, rooted at source 0, with no big-M.
- Node ids are uint16_t when max(ua, ub) <= 256, else uint32_t. The
  solver picks the bucket itself.
- Pricing is float (RowMin) on float mirrors of double potentials.
- Flows and the final cost are double.
- AVX_AVAILABILITY::NONE forwards to TransportSimplexReference.
- Every helper is a member of TransportSimplex<Avx>, so no function
  body becomes a weak symbol shared across -march translation units.

Strong feasibility. LEMON's leaving rule (strict < on the i-side, <= on
the j-side) keeps every zero-flow tree arc pointing toward the root, so
its child is a source. The start meets this:
- On a value tie at (i, j), the zero arc is (i+1, j), whose child is
  the fresh source i+1.
- Every arc with a sink child carries a fresh, strictly positive amount.
- The final cell uses whichever side is still untouched. No clamp.

Pivot gate. Let n = ua+ub, P = (n-1)*max_cost (bounds |potential|),
R = recompute_period, eps = DBL_EPSILON.
- walk_tol = 2n*eps*(max_cost + P). This bounds |r_fresh - r_exact|:
  two root-path sums of at most n-1 steps each, plus the two
  subtractions that form r. Potentials walked from the root equal a
  fresh recompute bit for bit, and so does RowMin's double r, because
  the operation order is the same.
- drift_tol = 2(n-1)*R*9*eps*P + walk_tol. One pivot moves a tree arc's
  residual by at most 9*eps*P: sigma costs 5*eps*P and applying it
  4*eps*P. A node's error is the sum of residuals along its root path,
  so the depth factor is n-1, and there are two endpoints.
- A candidate enters only if r_drifted < -drift_tol, or, when
  borderline, r_walked < -walk_tol. So every entering arc is truly
  negative, and Cunningham's finiteness proof applies.
- Otherwise the solver recomputes fresh and prices once in double.
  Every stop happens on fresh potentials.

error_bound. Each term covers a stated rounding source.
- 2*walk_tol*total_mass: at the stop, r_exact >= -2*walk_tol, and
  complementary slackness turns that into a cost gap.
- n^2*eps*max_cost*total_mass: before the final sum, the flows are
  recomputed from the marginals in post-order. Each flow is a signed
  subtree sum, and subtree sizes sum to at most n^2.
- 2(n-1)*eps*max_cost*total_mass: the n-1 multiplies and adds of the
  final sum.

The balance residual is folded into the caller's last demand before
orienting, as in the reference.

Portions derive from POT 0.9.5 (MIT) and LEMON (Boost Software License
1.0); see the header notices and THIRD-PARTY-PROGRAMS.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…erate shapes

Every ISA result is compared with the NONE (reference) result within
TransportSimplex<Avx>::error_bound plus 2 float ulps for the float
return of compute_decoded. The reference's own double rounding (about
1e-15) is far below that margin. New fixed cases: 100x100, 4x256 and
256x4, 3x200, 5x300 and 300x5 (the uint32 bucket), 1x256 and 256x1,
and 100x100 with all-equal or {0,1} costs and uniform weights.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The SIMD RowMin overloads stored each block of reduced costs and scanned
it with a branchy scalar loop, about 40% of solve_impl's self cycles on
sub50k. Each block now tests a strict-less mask against the running best
and is skipped when the mask is empty. Only an improving block resolves
its minimum: the lowest column equal to the block minimum wins, and its
exact bits are taken from the lane itself (compress on AVX-512, a stack
readback on AVX2), so ties, signed zeros and NaN behave as in the scalar
primary template.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
update_potential, find_join_node, the leaving-arc search, change_flow,
update_tree_structure and recompute_potentials index std::vector members
of the workspace in their loops. A store through one vector's data may
alias another vector's pointer member, so the compiler reloaded the base
pointers each iteration. The loops now use __restrict locals. Reads,
writes and operation order are unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The dense K x K table is 1 GiB at K=16384, and qEMD's cost fill does
scattered lookups into it, which are TLB- and cache-miss bound. It now
lives in SimpleData with HugepageAllocator, the same storage as
VectorSetData and SimpleGraph. It takes 1 GiB or 2 MiB pages when
available and falls back to normal pages otherwise.

CentroidTable becomes copyable (deep copy) because SimpleData is.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
When ua > ub, TransportSimplex transposes and its cost fill walks ids_a
in the inner loop, so every lookup landed in a different 64 KiB table
row. A table built by from_centroids is exactly symmetric, since both
halves are written from one float. For such a table compute_decoded now
swaps the lookup to table(ids_b[j], ids_a[i]), which keeps the row fixed.
Results are bit-identical. Tables from the raw constructor or load keep
the old lookup.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Adds TransportSimplex<Avx>::thread_pivot_count(), a thread-local pivot
counter that exists only when SVS_ENABLE_STATS is set
(SVS_EXPERIMENTAL_ENABLE_STATS). Normal builds compile no extra code. The
counter is per ISA instantiation, so a caller must read it through the same
Avx as the dispatched solver. It lets a benchmark report pivots / (m + k - 1).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The northwest-corner start now walks columns in an order grouped by each
column's cheapest row (col_argmin, ties broken by ascending column id) instead
of column-id order, so the start tree is biased toward low-cost arcs before
any pivot runs.

Strong feasibility still holds, because this is the same NW-corner walk
(the existing construction's argument) applied to a relabelled column axis;
the walk's argument only ever reasons about *position* (i, p), never about a
column's real id j = col_order[p]:
- The forced last-row/last-col steps and the final cell are unchanged: they
  fire on p == k - 1 exactly as they fired on j == k - 1, and the final cell
  still hands the fresh side's untouched, strictly-positive value to the one
  remaining arc.
- A value tie (supply and demand exhausted together) still adds a zero arc to
  row i + 1 with a source child, because that branch never reads j except to
  pass it to add_tree_arc; which column j happens to be is irrelevant to the
  branch's own logic.
- Every other zero-flow arc claim is unaffected: the tie arc is still the only
  arc add_tree_arc is ever called with flow == 0, since every other call
  passes min(supply[i], demand[j]) once one side is still positive, or a
  fresh side's full remaining value.
- Permuting columns cannot create a zero sink-child arc: sink-child flow
  values are exactly the same set of quantities (supply/demand residuals) the
  identity-order walk would have produced, just assigned to relabelled j's;
  relabelling a column changes which array slot a value lands in, not the
  value itself or whether it is positive.
- Row identity is untouched (rows still visit 0..m-1 in order, root stays row
  0), which is what the proof actually depends on; only columns are relabelled.

Extra cost: one compare per cost cell while filling (fill_cost_matrix also
tracks col_min/col_argmin), plus an O(m + k) counting sort for col_order. On qEMD solves from a 50k-document subset (K=16384), pivots per
(m + k - 1) drop 5-9% in every set-size bucket, and 1-thread solve time
drops 2.4-4.4%. Results are unchanged. The start therefore runs
unconditionally, with no size gate.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…check

Adds a section to the random cross-check case: all-equal and {0,1} costs,
uniform and repeated ({1,2,3}) weights, 1xn, nx1 and nxn shapes up to
256x256, plus a cost pattern where row 0 is the cheapest row for every
column (every column in one group). Existing sections draw the same random
numbers as before.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
svs::kmeans::emeans_train in svs/core/kmeans_emeans.h implements Algorithm 1
of Bessa et al., arXiv:2611.09501:
- EMA per-cluster counts and sums (centroid = S / N), with a 1-based batch
  counter as in Algorithm 1.
- A cosine-annealed step from alpha0 down to 0.1 * alpha0.
- Batch size and alpha0 from the paper's Table 2 scaling laws when unset,
  including the /8 batch law above n = 2M.
- Dead-cluster revival by splitting every 5 batches at tau = 0.1.
- A final pass that sets each centroid to the mean of its last-epoch points.

It reuses centroid_assignment, generate_norms and init_centroids from
kmeans_training.h. Assignment runs in chunks of at most assignment_chunk rows,
so the score buffer is assignment_chunk x k floats rather than B x k. The
state (k * d double sums, k counts) and all scratch are allocated once.
Revival costs O(k + dead * log k) per round.

Deviations and readings where the paper is silent are commented in the code:
- the tail batch is trained;
- power-of-two rounding is done in log2 space;
- the donor pool excludes this round's dead clusters;
- delta = 2^-7;
- epochs defaults to 2, an SVS choice.

The scaling laws were fitted on DEEP (L2, 256 <= k <= 65536). Outside that
range they are applied as published.

The returned objective is the MSE of the final centroids. Tests cover:
determinism; revival of a forced-dead cluster; the scaling laws against
hand-computed values; a tail batch; the k > n error and the other parameter
checks.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
CodebookParameters gains kmeans_algorithm (KMeansAlgorithm::Lloyd by
default) and emeans_epochs, which defaults to EMeansParameters' own default.
When e-means is selected, train_centroids runs emeans_train:
- kmeans_iterations is ignored;
- minibatch_size caps the assignment chunk;
- the objective is reported as a sum, the same convention as the Lloyd path.

Default behaviour is unchanged. The faiss 256-points-per-centroid bound now
applies only to Lloyd. The coarse-cluster k-means stays Lloyd.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The back-edge prune ran DynamicPartition over the BackedgeBuffer buckets,
1000 consecutive node ids each. With an expensive distance, a few buckets
holding hub nodes with thousands of overflow edges ran alone at the end of
every batch.

The prune now flattens the buckets into one list of overflow nodes. The list
is sorted by overflow set size descending, ties by id. Threads claim entries
one at a time from an atomic cursor. The per-thread setup (scratch vectors,
build adaptor, stats collector) still runs once per thread. Generation and its
bucket locks are unchanged. Each node's prune reads only its own adjacency and
the data, so the per-node result does not depend on the schedule.

Measured on a 50k-document qEMD sub-build (R=24, L=80, alpha=1.2, 112
threads, one NUMA node): 755 s before, 290 s after (2.6x). The build is not deterministic even
before the change: two baseline builds share 99.9% of edges. Against that
noise, degree statistics and recall@10 are unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
CodebookParameters::kmeans_algorithm now defaults to KMeansAlgorithm::EMeans
(2 epochs, scaling-law batch size and step). Lloyd stays selectable. The
switch was made by the 1M MS MARCO run's criteria: the e-means objective is at
most Lloyd's at no more training time, and MRR@10 is within 0.005 of the Lloyd
codebook at every search window. The numbers are in the run's decision
record.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Two one-thread builds of data/test_dataset (L2, R 16, window 32) must give
identical adjacency lists. With more threads, back-edge lock order changes
the graph, so this is the strongest determinism the static build offers.

It also records how 5c650da (one back-edge prune per work unit, largest
first) was shown not to change the graph, beyond matching statistics:
- With one thread the generation phase has no lock-order effect, so the
  bucket schedule and the per-node schedule must agree byte for byte.
  qEMD, 1,000 docs of the 50k subset, R 24, L 80, alpha 1.2, unrouted:
  705115b against 705115b plus the 5c650da vamana_build.h, and against
  HEAD, give graph files that differ only in the 16-byte file UUID.
- L2 on data/test_dataset via utils/build_index (window 64, R 32,
  alpha 1.2, one thread): 705115b, HEAD and a second HEAD run likewise
  differ only in the UUID.
- ThreadSanitizer on a multi-threaded static build at HEAD (2,000 to
  200,000 points, degree 2 to 32, 8 to 112 threads): no reports, in the
  prune phase or elsewhere. TSan sees only races unordered by
  happens-before, and the per-vertex locks order most pairs, so this is
  supporting evidence; the one-thread identity carries the proof.
- [vamana] (49 cases) and [integration][build][vamana] pass at HEAD.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Adds enum class TransportStart { NorthWest, CheapestRow }. QEmd takes it as
an optional third constructor argument (default NorthWest) and passes it
through QEmdImpl<Avx>::compute / compute_decoded to TransportSimplex::solve.
The choice is not saved with an index: it changes how fast a solve reaches
the optimum, not the optimum.

NorthWest walks columns in id order and skips the column-min fold, so it
reproduces the solver before 8a648b2. This changes the default:
CheapestRow, hard-wired since 8a648b2, is now opt-in. Both starts share
one strong-feasibility argument, since the walk reasons only about
positions and rows keep their order.

The random cross-check, including its start-degeneracy section, runs under
both starts. On 200k qEMD solves (50k subset, K 16384), pivots per
(m + k - 1) and checksums equal 705115b's for NorthWest and 5c650da's
for CheapestRow on all four datasets. At 1 thread, NorthWest is within 1%
of 705115b, and CheapestRow is 3-5% faster than NorthWest.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant