Skip to content

Map EDF digital endpoints to their physical endpoints - #1920

Open
antonsoo wants to merge 4 commits into
NeuralEnsemble:masterfrom
antonsoo:fix/edf-physical-calibration
Open

antonsoo wants to merge 4 commits into
NeuralEnsemble:masterfrom
antonsoo:fix/edf-physical-calibration

Conversation

@antonsoo

@antonsoo antonsoo commented Oct 18, 2026 •

Copy link
Copy Markdown

EDFRawIO divides the physical range by digital_max - digital_min + 1, introducing a gain and offset error. The EDF specification defines corresponding physical and digital endpoints, so the denominator needs the difference between those endpoints. For an identity range of -8092 to 8092, a stored value of -5 currently reads as approximately -5.49966 rather than -5.

Remove the extra interval. Update the existing EDFIO comparison to use PyEDFlib physical reads rather than repeating the old denominator. As requested in review, retain the recorded-data comparison and drop the synthetic TestEDFCalibration class. Read raw integers through RawIO so the existing assertion also supports the fixture's corrected nonzero physical offset.

Validation after the review change: 631 native core/IO tests pass; Black passes. Thirteen external-fixture tests are skipped because the GIN fixture downloads return HTTP403. The existing EDFIO comparison method also passes on an explicitly generated five-channel fixture. Recorded EEGMMIDB S001/R04 samples now agree exactly with independent EDF decoding across 16 raw/eager/lazy conversions; raw integers, timestamps and channel ordering stay unchanged.

The contributor-list commit is shared with #1919. The two runtime fixes are independent.

Use the difference between the digital extrema for calibration. Counting both endpoints as an additional interval introduces a systematic gain and offset error. Test real generated EDF endpoint samples and compare physical IO reads against pyedflib instead of repeating the incorrect denominator.
@zm711
zm711 requested a review from JuliaSprenger October 18, 2026 16:52
@zm711

zm711 commented Oct 18, 2026

Copy link
Copy Markdown
Contributor

@JuliaSprenger, it appears you wrote the initial implementation here and I don't know this reader at all, so could you look over this?

@h-mayorquin h-mayorquin left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The fix looks good to me. The EDF specification maps digital_min to physical_min and digital_max to physical_max. That is, the gain is the ratio between the physical range and the digital range. The + 1 was counting the digital values instead of measuring the distance between the endpoints.

I have one request, can you drop TestEDFCalibration? The change to test_compare_data already tests the fix on real data: it fails on master and passes here. TestEDFCalibration builds synthetic files with the pyedflib writer and I would like to move our EDF tests away from depending on pyedflib anyway at some point.

@h-mayorquin h-mayorquin self-assigned this Oct 28, 2026
@antonsoo

Copy link
Copy Markdown
Author

Removed TestEDFCalibration and its imports as requested. The existing test_compare_data change remains. The 631 local core/IO tests pass with the same 13 unavailable-fixture skips; the recorded EEG replay still matches independently decoded samples in all 16 raw/eager/lazy conversions, with raw samples and timing unchanged.

@antonsoo

Copy link
Copy Markdown
Author

The hosted EDF failure also exposed an old test assumption: after correcting the calibration, the fixture has a nonzero physical offset, so AnalogSignalProxy.load(magnitude_mode="raw") cannot represent it. I changed the raw assertion to read stored integers through get_analogsignal_chunk; the lazy rescaled assertion still compares physical values with PyEDFlib. A local offset-bearing fixture reproduces the same exception before this test change and passes afterward; the zero-offset control also passes. The original GIN fixture remains unavailable locally (HTTP 403), so the next hosted run will verify that exact file.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants