[NoopDog]

Closes #770
Closes #711

What changed

• A sample's donors are read from its own row, by two routes.
  • IGVF's submitter sample.donors: a list cell, declared in the entity map as collected_from on the sample table.
  • AnVIL's anvil_biosample.donor_id: a list of AnVIL donor UUIDs, written as a new entity evidence field, collected_from_repository_id.
    • At reconcile, each UUID is read as the donor record it ties to through repository_ids (Record each AnVIL donor's and biosample's identifiers by namespace #669), after every file is read, so a UUID tied in another dataset still resolves.
    • It is cited by the identifier rule that recognised that donor.
  • One loop for both: reconcile_entities._read_donor_links.
• A sample may be collected from several donors (a pooled sample).
  • SampleRecord.collected_from is now {donors: [...], named_by}, replacing {donor, named_by}. The list has one donor for most samples and several for a pooled one, in identifier order.
  • Each source names a set of donors: an identifier rule names a set of one. Sources naming different sets give none, listed in collected_from_conflicts[].named.
  • A line naming a donor no rule recognises, or a UUID no donor record ties to, names none of its donors (all or none) and is counted. These go in unrecognised_links and the new unresolved_repository_links.
• Report: entities.samples.pooled counts the samples linked to more than one donor.
• Entity map loader:
  • A submitter table may read collected_from, and only that (EntityTable.kinds: a table's slots are one kind's).
  • The published sample may read collected_from_repository_id, but only beside a published donor table.
  • check-entity-map reads a sample table's donor column as donors, list cells split, as reconcile reads it.
• Schema: CollectedFrom.donors (multivalued), and the field pattern takes collected_from_repository_id. The pydantic model is regenerated.
• Small reuses: source_evidence.cell_values (a list cell's elements, else the cell) and sample_identity.in_identifier_order, renamed from _sorted.
• Docs: the claims contract's collected_from passage, an "Extended by IGVF: link each sample to its donor (the submitter's sample.donors, or AnVIL's anvil_biosample.donor_id) #770" note on ADR-0002 decision 3, CLAUDE.md and the entity map's comments.

Why

None of AnVIL_IGVF_Mouse_R1's 6,786 files reached a donor at reconcile, though AnVIL links them. Measured on the anvil15 manifests, 2026-10-28:

• Both sources agree: the submitter column and AnVIL's donor_id name the same donors on all 1,935 IGVF samples.
• 305 of those samples are pooled, 8 to 96 mice each, and every single-sample IGVF file (6,717) is on one of them. A one-donor collected_from could link only about 40 files. That's why the field became a set.
• Other datasets: AnVIL's link names the same donor as the Coriell identifier rule on every biosample whose donor resolves. Here it is a second source, not a new link.

Assumptions I made

• A pooled sample's donors are one answer. A source naming 64 mice is one source with one set. Two sources agree only if they name exactly the same set. A subset is a conflict, never a merge.
• All or none per line. A pooled sample linked to only some of its mice would understate its donors, so one unresolvable value drops the whole line, counted.
• An AnVIL donor UUID cites the identifier rule that recognised the donor record it ties to. No new code rule was added: the UUID is matched by a dictionary lookup, which involves no judgement a rule would have to declare.
• The rendered reconcile report is unchanged. It shows none of the donor and sample counts except the list no file reaches. Donor reach on a dashboard is Coverage dashboard: what meta-disco adds over what AnVIL publishes, per dataset and slot, and file-to-donor links #769's job.
• The explorer must be updated after this merges. meta-disco-browser's loader reads collected_from.donor (clevercanary/meta-disco-browser#221). Until it reads donors, a run reconciled at this commit won't load there.
• Don't import the new entity evidence into the shared data/entity_evidence until this merges. Other clones' schema would refuse the new field. I imported it into a scratch root for the measurement below.

How to verify

The issue's checklist and #711's, each with its step:

• IGVF's donor_reach shows about 6,757 files with every sample having a donor, and its 183 mouse donors are each reached.
  1. make check-entity-map. It should report 18 tables across 9 dataset(s) and no problems.
  2. make import-anvil-entities, then make reconcile.
  3. In <run>/reconciled/reconcile_report.json, read entities.donor_reach.AnVIL_IGVF_Mouse_R1. Expect {"every_sample_has_a_donor": 6757, "files_with_samples": 6757}. Main had 6,717 "one sample without donor" and 40 "several samples, some without donor".
  4. Read entities.unreached.AnVIL_IGVF_Mouse_R1. It should have no donor key (main: 183 donors).
• No other dataset's sample gains a donor that contradicts the one its identifier rule gives.
  • entities.samples.collected_from_conflicts should be [].
  • collected_from_by_source should show published_value: 5146, AnVIL's link agreeing with the rule.
  • unresolved_repository_links should list ANVIL_HPRC's 2 UUIDs that tie to no donor record anywhere, NA20300's and HG01442's. Both samples keep their donor through the Coriell rule.
• (Settle a sample's donor from AnVIL's biosample-to-donor links: IGVF samples name no donor #711) Each IGVF sample record whose AnVIL biosample names a donor carries collected_from pointing to that donor's record.
  • entities.samples.collected_from.igvf should be 1935, and pooled should be 305.
  • In reconciled/samples.ndjson.gz, IGVFSM9197PTER should have collected_from.donors == [{"id": "IGVFDO5085MJSP", "namespace": "igvf"}], named by both repository_metadata (sample.donors) and published_value (anvil_biosample.donor_id).
• (Settle a sample's donor from AnVIL's biosample-to-donor links: IGVF samples name no donor #711) The report counts the samples still without collected_from, by dataset.
  • entities.samples.no_donor lists them per dataset. Only ANVIL_T2T_CHRY's 1 remains, as on main.
• Tests: make test, plus tests/test_sample_donors.py for:
  • the pooled set
  • a conflict between sets
  • all-or-none for a line with an unrecognised donor and for one with an untied UUID
  • a UUID tied by a file read later
  • the loader's refusals
  • the check's reading of the donor column

Measured on a scratchpad copy of run 20261027_210224, reconciled at 152aa32 against a fresh entity import. The full make classify run at the PR head is due after review.

🤖 Generated with Claude Code

https://claude.ai/code/session_01VF7tbFa5ZTmYWZ6JWKiBBS
🐦 Built with cc-claude-tools

[Copilot]

🔵 Needs a closer look

The breaking collected_from.donor-to-donors schema change requires human confirmation of browser-loader compatibility and rollout sequencing, especially since browser issue #221 was inaccessible.

No open issues 🎉

What changed in this PR

This PR extends entity reconciliation to link samples to their donors through submitter metadata and AnVIL biosample records, addressing #770 and #711 and supporting pooled samples.

Changes:

• Reads donor links from IGVF’s sample.donors and AnVIL’s anvil_biosample.donor_id, resolving repository identifiers across datasets.
• Replaces collected_from.donor with collected_from.donors; requires sources to agree on the complete donor set and reports pooled samples and unresolved links.
• Updates schema, map validation, tests, and documentation. The browser loader needs the corresponding plural-field update.

File	Description
tests/​test_sgdp_entities.py	Updates SGDP assertions for donor lists and set conflicts.
tests/​test_sample_donors.py	Adds pooled-donor, resolution, conflict, and validation tests.
tests/​test_entities.py	Updates entity tests for the expanded map and output shape.
src/​meta_disco/​value_map.py	Reuses the shared cell-value helper.
src/​meta_disco/​sources/​anvil_entity_map.yaml	Maps submitter and published sample-to-donor links.
src/​meta_disco/​source_evidence.py	Adds a shared scalar/list-cell helper.
src/​meta_disco/​schema/​classification.yaml	Defines donor lists and the repository-link evidence field.
src/​meta_disco/​schema/​classification_model.py	Regenerates models for the schema changes.
src/​meta_disco/​sample_identity.py	Exposes canonical identifier ordering for reuse.
src/​meta_disco/​reconcile.py	Updates the reconciliation docstring for multiple donors.
src/​meta_disco/​reconcile_samples.py	Reuses the shared cell-value helper.
src/​meta_disco/​reconcile_entities.py	Resolves donor sets and updates conflict and reach reporting.
src/​meta_disco/​models.py	Declares the new identity-link field and its kind.
src/​meta_disco/​entity_map.py	Validates sample-link mappings and published donor requirements.
src/​meta_disco/​entity_evidence.py	Documents the expanded identity-link evidence.
src/​meta_disco/​anvil_entities.py	Checks sample donor columns and table identifier kinds.
docs/​claims-contract.md	Documents donor-set agreement semantics.
docs/​adr/​0002-derivation-graph.md	Records the pooled-sample design extension.
CLAUDE.md	Updates architecture guidance for sample-to-donor links.

_{Review effort: Balanced}