Skip to content

Feat/station encryption - #1128

Closed
OH2JRB wants to merge 17 commits into
DJ2LS:mainfrom
OH2JRB:feat/station-encryption
Closed

OH2JRB wants to merge 17 commits into
DJ2LS:mainfrom
OH2JRB:feat/station-encryption

Conversation

@OH2JRB

@OH2JRB OH2JRB commented Oct 23, 2026

Copy link
Copy Markdown

No description provided.

Reijo Jarnstedt and others added 17 commits October 3, 2026 20:48
Adds freedata_server/docs/ with an index and six guides: architecture
(startup, AppContext, threads, managers, config, events, scheduled jobs),
REST/WebSocket API reference, modem/audio/radio control, frame types and
byte layouts, ARQ/P2P/socket interface/NORM sessions, and the message
system database. Each guide ends with the known issues that remain.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
API
- transmit endpoints return 500 when the command fails instead of 200
- FEC transmit returns 501 (FecCommand is not functional)
- message PATCH/DELETE and broadcast PATCH return 404/409 correctly
- config is written once on POST /config; /audio_rx uses audio_rx_queue
- /radio returns 503 while no radio manager is running

Core
- keep STATION.enable_stats instead of deleting it on config validation
- only one RadioManager; set_ptt uses ctx.socket_interface_manager
- set_rf_level stubs for serial_ptt and rigdummy
- frame dispatcher is actually stopped on modem stop
- scheduler sends queued messages when the channel is free (was inverted)
- channel busy events are cleared instead of replaced; start as "free"
- get_*_session return None for unknown ids; event queues drop only the
  oldest event; ended IRS sessions are deleted

Modem/audio
- buffer overflow counter index, separate TX resampler, codec2 argtypes
- 800-sample RX audio stream chunks, non-blocking audio_rx enqueue
- consistent get_device_index_from_crc return value

ARQ/P2P
- IRS crashes on CRC failure, missing statistics, bursts after FAILED
- duplicate delivery guard, stop retries during abort
- P2P connect/heartbeat/retry exhaustion, callbacks routed per session
- socket data port no longer strips or alters binary data
- NORM repairs only for our own broadcasts; QRVCommand snr

Message DB
- message sent to the wrong destination when older messages are queued
- delete guard order, station location persistence, indexes,
  float timestamp defaults, ALTER TABLE commit, timestamp parsing

Shutdown
- AppContext.shutdown() no longer calls os._exit(); the server lifespan
  does. Scheduler and service runner threads now stop on shutdown.

On-air frame formats are unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
AppContext.shutdown() used to call os._exit(0) from every tearDownClass,
so `python -m unittest discover tests` always exited 0 and CI never
reported failures. With that removed, fix the tests it was hiding:

- test_protocols: read TESTMODE_EVENTS (modem_events is drained by the
  websocket worker), match the current frame-handler event format
- test_server: launch with sys.executable, expect current API_VERSION,
  send mycall with the config update
- test_message_protocol: use ctx_ISS, wait for the session to finish and
  assert its result
- test_norm_protocol: always stop the channel threads

Add tests/testenv.py so tests use a temporary config and message
database instead of config.ini.example and the user's app directory.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- architecture.md: AppContext.shutdown() no longer exits the process
  (os._exit moved to the server lifespan), daemon service runner with
  wake-up on shutdown, interruptible scheduler stop, channel busy
  conditions start as free
- api.md / modem-audio-radio.md: lifespan exit, stop_modem guard
- testing.md: running the suite, test isolation via tests/testenv.py,
  TESTMODE mechanics, guidelines for writing tests, known gaps

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Dependencies no code imports:
- runtime: chardet, ordered-set, websocket-client
- test extra: pick
- requirements.txt dev tools autopep8, black, isort, pycodestyle,
  replaced by ruff (what CI lints and formats with)
colorama stays: structlog's ConsoleRenderer(colors=True) needs it on
Windows.

Modules nothing imports:
- b64.py (scratch script)
- command_fec.py (non-functional; /modem/fec_transmit returns 501)
- command_qrv.py (no callers; CQFrameHandler builds QRV itself)
- norm/norm_transmission_resend.py (comment only)

Also remove an unused import in a test, apply ruff format so the CI lint
workflow passes, and update docs that referenced the removed modules.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Core / API
- REST calls are no longer dropped at startup or blocked by an error on
  one websocket; loadAllData runs once per reconnect (on /states)
- loading the config no longer POSTs it back (each POST restarted the
  modem); failed saves reload the server config instead of throwing
- server errors are shown as toasts (with the server's message) instead
  of being silently turned into undefined
- handle the server's {"freedata_server": ...} lifecycle events
- ARQ speed/SNR histogram arrays are replaced properly, so charts update
- getVersion returns modem_version; escape server text in toasts;
  merge stored local settings with defaults; valid config defaults;
  single Pinia instance; html lang follows the selected language;
  tooltip/popover delegates no longer collide

Screens / settings
- "Test Hamlib" no longer transmits a CQ; it reads the radio status
- invalid grid locators and empty number fields are not posted
- distance unit guard, unknown rig control mode, duplicate DOM ids,
  dead UI, Wavelog API key as password field

Dashboard
- validate uploaded presets; recover from a broken stored layout
- schedule RX audio on the AudioContext clock, reset backlog on start,
  stop on unmount, tolerate empty/odd frames
- remove waterfall instances and window listeners on unmount
- map, meter, and chart fixes

Chat / broadcasts
- UTF-8 safe broadcast encoding with a 390-byte limit and counter
- remove broadcast ADIF (no server endpoint exists)
- delete conversation skips queued/transmitting messages with a notice
- UTC times shown as UTC, message info, preview fallback, METAR station
  parsing and ICAO lookup, 100 KiB attachment limit, stable list keys

Verified with eslint, a production build, and a manual run against a
local server (chat send, error toasts, settings load).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
displayToast inserts HTML and the callsign can come from a remote
station's message.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Adds freedata_gui/docs/ with an index and four guides: architecture
(build, bootstrap, server discovery, WebSockets and event routing, Pinia
stores, api.js reference and a cross-check against the server API),
screens and settings, the dashboard grid (widgets, presets, waterfall,
RX audio, charts, map), and chat and broadcasts. The docs describe the
code after the GUI bug fixes and list the issues that remain open.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- GET /freedata/messages/{id} returns an object (was a JSON-encoded
  string) and 404 for unknown ids
- DELETE /freedata/broadcasts/{id} returns 404 for unknown ids/domains
- POST /freedata/broadcasts returns 409 when group chat is disabled
  instead of reporting a transmission that never happens
- broadcast priority is an int (0-7, default 1) and is included in the
  per-domain broadcast list; priority 0 no longer blocks a retransmit
- reading a broadcast domain no longer marks it read; new
  PATCH /freedata/broadcasts/{domain}/read, called by the GUI when the
  user selects a domain
- POST /config only restarts the modem when a non-GUI section changed
- POST /freedata/station/{callsign} returns 500 when the update fails
- state frames include channel_busy
- GUI: ARQ progress shows 100% after ENDED instead of NaN; composer
  sends a numeric priority; message info handles the object response

Adds tests/test_api_fixes.py and server integration tests for the new
status codes. Docs for server and GUI updated.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The lockfile was already on uuid 14 while package.json still asked for
^13, so `npm ci` refused to install. Only v4() is used, which is
unchanged in uuid 14. Also syncs the lockfile's app version (0.18.1).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Server (MODEM_VERSION, also the pip package version) and GUI
(package.json / package-lock.json) were at 0.18.2 and 0.18.1;
both now 0.19.0.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Packages nothing imports or runs:
- bootstrap-vue-next, gettext-parser, core-js
- dev: @babel/core, @babel/eslint-parser, @eslint/js
@popperjs/core stays (Bootstrap peer dependency).

Files:
- babel.config.js: pointed at an uninstalled preset; Babel never ran
- src/assets/countries-10m.json, countries-110m.json (map uses 50m)
- src/assets/logo.png (unreferenced)
- src/assets/waterfall/colormap.js (duplicate of the colormaps in
  spectrum.js), index.html, waterfall.css, make_colormap.py (upstream
  demo leftovers); LICENSE and README kept for attribution

None of these were in the built bundle. npm ci, eslint and the
production build pass. Docs updated.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Bundled builds could not start on a machine without an existing config:
the server bootstraps %APPDATA%\FreeDATA\config.ini from
config.ini.example next to the executable, but all three Nuitka
workflows bundled the template as config.ini. Bundle it as
config.ini.example.

Installer script:
- linear install section; the old label/goto flow skipped the
  uninstaller and shortcuts on upgrades and copied all files twice
- remove the previous program folder before installing (no stale files)
- migrate config.ini and freedata-messages.db from the program folder
  (where versions up to 0.18.1 kept them) to %APPDATA%\FreeDATA;
  existing data there is kept and the old file saved as a .pre-0.18.2
  backup
- per-user install (RequestExecutionLevel user) matching the
  %LOCALAPPDATA% / HKCU / desktop locations it writes to
- register in "Apps & features" with version; write Install_Dir
- uninstaller removes program files, shortcuts and registry entries,
  keeps user data, and no longer deletes the install folder recursively
- the NSIS workflow passes the version (/DVERSION) from constants.py

Tested locally: fresh-user start of the bundled server, silent install
over a simulated 0.18.1 installation (migration, stale-file removal,
shortcuts, registry) and silent uninstall.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ubuntu-latest (24.04) no longer ships python3-libhamlib2, so the
dependency step failed. Nothing imports the Hamlib Python bindings, and
the build uses setup-python's 3.13, which could not load them anyway;
build_server.yml already installs the same packages without it.

Also point AppRun at config.ini.example, the name the template is now
bundled under, and create the config directory with mkdir -p.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
P2P chat messages can now be encrypted per station with a pre-shared
key. Encryption is off by default; turning it on in the GUI shows a
legal warning (most amateur radio licences prohibit encrypted traffic)
and requires a confirmation.

Server:
- encryption.py: AES-256-GCM via the cryptography package, keys as 64
  hex chars or scrypt-stretched passphrases, fingerprints, in-memory
  StationKeyStore (keys are never written to disk or sent over the air)
- new ARQ type p2pmsg_zlib_aesgcm (21): compress, then encrypt
- messages to an encrypted station without a key get status
  key_required and don't block the queue; received messages without a
  (fitting) key are kept encrypted in pending_encrypted_message
- key_required event asks the GUI for the key; entering it requeues
  waiting messages and decrypts pending ones
- /freedata/station/{callsign}/encryption endpoints (status, on/off,
  enter/forget key)
- database schema 2: station.encryption_enabled, p2p_message.encrypted,
  pending_encrypted_message table

GUI:
- lock button in the chat header, encryption dialog with legal warning,
  key entry/generation, fingerprint and waiting-message counts
- dialog opens automatically when the server needs a key
- key-required badge and lock icons on encrypted messages

Tests and developer docs (freedata_server/docs/encryption.md) included.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@OH2JRB

OH2JRB commented Oct 23, 2026

Copy link
Copy Markdown
Author

Trying to cancel this ... went into wrong repo.

@OH2JRB OH2JRB closed this Oct 23, 2026
@OH2JRB
OH2JRB deleted the feat/station-encryption branch October 25, 2026 10:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant