[difanz]

Add V4L2 support in divinus.

The FH8626 SoC port, merged recently, uses V4L2 as its video backend, and ALSA as the audio backend. This set of patches adds support for it, and will be useful for future SoC ports that don't use a vendor SDK.

What it adds

• Video: a V4L2 capture node feeding V4L2 memory-to-memory encoders. Every encoder reads the shared capture frame. It imports the frame by DMABUF when the layouts match and copies it otherwise.
• Audio: ALSA capture, feeding the existing MP3 and G.711 encoders.
• OSD: text and image regions drawn into ARGB1555 buffers on V4L2 output (overlay) nodes. They are positioned with the compose selection, and their opacity is set through V4L2_CID_ALPHA_COMPONENT.

Impact on other platforms

None. The backend is only built with -DDIVINUS_V4L2; without it every call is a stub and platform detection is unchanged. The core changes are one case HAL_PLATFORM_V4L2 per existing switch (plat). The only other change is in MP4 muxing, which uses the capture timestamps for V4L2, as it already does for AK. Builds with the flag also need -lasound. Toolchains with kernel headers older than 4.10 lack V4L2_PIX_FMT_HEVC, so the backend defines it itself and builds against them too (e.g. the FH8852V200 4.9 toolchain).

Device discovery

Devices are matched by capability:

• Capture: the first single-planar NV12 capture node.
• Encoders: M2M nodes that take NV12 and produce the requested codec.
• Overlays: output nodes that accept ARGB555 and that no other file handle owns.

Optional environment overrides for boards that need them:

Variable	Purpose
DIVINUS_V4L2_CAPTURE	Capture node to use, e.g. /dev/video0
DIVINUS_ALSA_DEVICE	ALSA PCM device (default: default)
DIVINUS_V4L2_OSD_CARD	Only use overlay nodes whose QUERYCAP card name matches exactly. Useful when overlay nodes belong to different scaler channels.

Commits

1. media: set audioOn before starting the audio threads: a generic fix that is also part of Fix audio, RTSP, OSD and config bugs in Divinus #73. The V4L2 audio thread needs it, otherwise it exits before audioOn is set. Whichever PR merges second drops it on rebase.
2. hal: add an opt-in V4L2 video backend
3. hal: capture V4L2 backend audio through ALSA
4. hal: draw V4L2 backend regions on overlay output nodes

Testing

On FH8626V100 hardware (GC1054 sensor, 1280x720 NV12):

• RTSP: H.264 video with an IDR every 30 frames, and PCMA audio.
• /video.mp4 with MP3 audio, including after a sample-rate change through /api/audio.
• /image.jpg snapshots.
• Encoder reconfiguration through /api/mp4 while the stream is running.
• Text OSD and an uploaded PNG region on the channel 0 overlay.

[qodo-free-for-open-source-projects]

PR Summary by Qodo

Add opt-in V4L2 video, ALSA audio, and overlay support

✨ Enhancement 🐞 Bug fix 🕐 40+ Minutes

AI Description

• Add opt-in V4L2 capture, encoding, and overlays for SoCs without a vendor SDK.
• Capture ALSA audio through existing encoders and use capture timestamps for MP4.
• Route media and snapshots through the new HAL and fix an audio-thread startup race.

Diagram

graph TD
  Core["Media dispatch"] --> Backend(["V4L2 HAL"]) --> Capture["NV12 capture"] --> Encoder["M2M encoders"] --> Consumers["Stream consumers"]
  Backend --> Audio["ALSA capture"] --> Consumers
  Backend --> Overlay["OSD outputs"]

Loading

High-Level Assessment

The following are alternative approaches to this PR:

1. Use a GStreamer pipeline

• ➕ Could delegate format negotiation and buffer handling to an established framework.
• ➖ Adds a substantial embedded dependency and requires adapting existing HAL callbacks, encoder controls, and overlays.

Recommendation: Keep the direct, opt-in V4L2 HAL: it fits the existing backend model and avoids a new media framework. Verify that flagged builds link ALSA with -lasound, since the diff does not change build configuration.

Files changed (9) +1107 / -6

Enhancement (8) +1078 / -2

support.cProbe the V4L2 backend during platform detection +1/-0

Probe the V4L2 backend during platform detection

• Attempts V4L2 identification before vendor-specific probes. Without the build flag, identification is a stub that does not claim the platform.

src/hal/support.c

support.hExpose V4L2 HAL declarations to platform dispatch +1/-0

Expose V4L2 HAL declarations to platform dispatch

• Includes the new backend header so existing platform-dispatch code can call its entry points.

src/hal/support.h

types.hDefine the V4L2 platform identifier +2/-1

Define the V4L2 platform identifier

• Adds HAL_PLATFORM_V4L2 to the platform enumeration for detection and dispatch.

src/hal/types.h

v4l2_hal.cImplement opt-in V4L2 and ALSA media backend +1022/-0

Implement opt-in V4L2 and ALSA media backend

• Discovers devices by capability and manages shared NV12 capture, M2M encoding, snapshots, overlays, and ALSA audio. Uses DMABUF when layouts permit, falls back to copying, and provides disabled-build stubs.

src/hal/v4l2_hal.c

v4l2_hal.hDeclare the V4L2 HAL interface +39/-0

Declare the V4L2 HAL interface

• Defines callbacks and entry points for platform detection, audio, channels, capture, video encoding, snapshots, and regions.

src/hal/v4l2_hal.h

jpeg.cRoute JPEG snapshots through V4L2 +3/-0

Route JPEG snapshots through V4L2

• Adds V4L2 dispatch for JPEG encoder creation, destruction, and snapshot retrieval.

src/jpeg.c

region.cRender existing regions through V4L2 overlays +9/-0

Render existing regions through V4L2 overlays

• Adds V4L2 region creation and bitmap updates for text and image overlays, plus region destruction.

src/region.c

server.cUse V4L2 capture timestamps in MP4 +1/-1

Use V4L2 capture timestamps in MP4

• Passes V4L2 video-pack timestamps to the MP4 muxer, following the existing AK behavior.

src/server.c

Other (1) +29 / -4

media.cIntegrate V4L2 media lifecycle and fix audio startup +29/-4

Integrate V4L2 media lifecycle and fix audio startup

• Routes channel, encoder, pipeline, IDR, grayscale, and audio operations to the new backend. Sets audioOn before starting audio threads so an early-scheduled thread does not exit immediately.

src/media.c

[qodo-free-for-open-source-projects]

Code Review by Qodo

🐞 Bugs (7) 📘 Rule violations (0) 📎 Requirement gaps (0) 🎨 UX issues (0) 🔗 Cross-repo conflicts (0) 📜 Skill insights (0)

1. Smaller video streams have wrong dimensions 🐞 Bug ≡ Correctness

Description

v4l2_video_create() configures every encoder with the shared capture dimensions rather than the
channel's requested config.width and config.height. When the MP4 and MJPEG sizes differ, the
smaller channel encodes the larger image while MP4 configuration and other consumers still use its
requested dimensions.

Code

src/hal/v4l2_hal.c[R680-682]

+    fmt.fmt.pix.width = _v4l2_width;
+    fmt.fmt.pix.height = _v4l2_height;
+    fmt.fmt.pix.pixelformat = V4L2_PIX_FMT_NV12;

Evidence

Startup chooses the maximum of MP4 and MJPEG dimensions for capture; channel creation only warns
about a mismatch, and encoder format setup always uses capture dimensions. MP4 configuration
independently uses the requested MP4 dimensions.

src/media.c[968-974]
src/hal/v4l2_hal.c[298-305]
src/hal/v4l2_hal.c[670-691]
src/media.c[814-816]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
V4L2 channels silently encode at the shared capture size even when their configured sizes differ.
## Fix Focus Areas
- src/hal/v4l2_hal.c[298-305]
- src/hal/v4l2_hal.c[670-691]
- src/media.c[968-974]
- src/media.c[814-816]
## Recommended Fix
Scale the input separately for channels requesting other dimensions, or reject unsupported dimension combinations before enabling those channels. Ensure the dimensions supplied to MP4 configuration match the resulting encoded frames.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

2. Shared buffers can include uncaptured data 🐞 Bug ≡ Correctness

Description

v4l2_video_create() permits DMABUF import based on allocation length and stride without checking
that the capture payload size covers enc->inSize. If the allocation is padded beyond the captured
sizeimage, v4l2_encode() declares those additional bytes valid through in.bytesused and passes
them to the encoder.

Code

src/hal/v4l2_hal.c[R703-707]

+    char exported = _v4l2_capCount > 0;
+    for (unsigned int i = 0; i < _v4l2_capCount; i++)
+        if (_v4l2_capMap[i].dmabuf < 0 || _v4l2_capMap[i].length < enc->inSize)
+            exported = 0;
+    if (exported && enc->inStride == _v4l2_stride) {

Evidence

Capture sizeimage and mmap allocation length are tracked separately. Import eligibility checks the
latter, but the queued input's bytesused is always the encoder's negotiated sizeimage.

src/hal/v4l2_hal.c[384-388]
src/hal/v4l2_hal.c[689-691]
src/hal/v4l2_hal.c[702-715]
src/hal/v4l2_hal.c[911-917]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
DMABUF eligibility checks allocation capacity but not the amount of valid captured image data.
## Fix Focus Areas
- src/hal/v4l2_hal.c[384-388]
- src/hal/v4l2_hal.c[689-691]
- src/hal/v4l2_hal.c[702-715]
- src/hal/v4l2_hal.c[911-917]
## Recommended Fix
Require the capture payload to cover the encoder's queued input size before selecting DMABUF import. Otherwise use the copy path and set its queued payload consistently with the bytes initialized there.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

3. Long video frames lose NAL boundaries 🐞 Bug ≡ Correctness

Description

v4l2_deliver() stops scanning when the eight-entry NAL array fills, then extends the eighth NAL's
length to the end of the access unit. When an encoder emits more than eight NALs in one frame,
downstream streaming and recording consumers receive the remaining NALs merged into that eighth
entry rather than as separate units.

Code

src/hal/v4l2_hal.c[R858-860]

+            if (pack.naluCnt == sizeof(pack.nalu) / sizeof(*pack.nalu)) break;
+            if (pack.naluCnt)
+                pack.nalu[pack.naluCnt - 1].length = pos - pack.nalu[pack.naluCnt - 1].offset;

Evidence

The pack provides eight entries; the parser breaks before assigning the eighth entry's boundary at
the ninth start code, then makes that entry cover the full remainder. HTTP and MP4 consumers use the
recorded lengths directly.

src/hal/types.h[110-122]
src/hal/v4l2_hal.c[852-870]
src/server.c[186-205]
src/server.c[240-260]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
The parser merges all remaining NALs into its eighth entry when an access unit exceeds the fixed entry capacity.
## Fix Focus Areas
- src/hal/v4l2_hal.c[852-870]
- src/hal/types.h[110-122]
## Recommended Fix
Represent all NALs in an access unit, or explicitly reject an over-capacity unit without delivering incorrectly bounded NAL data. Keep offsets and lengths consistent for every downstream consumer.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

4. Failed audio capture blocks a retry 🐞 Bug ☼ Reliability

Description

media_audio_enable() continues into its thread setup after v4l2_audio_init() fails and sets
audioOn despite the null PCM handle. If opening or configuring the ALSA device fails, the capture
thread exits immediately and a later enable call treats audio as already enabled instead of retrying
initialization.

Code

src/media.c[558]

+        case HAL_PLATFORM_V4L2: ret = v4l2_audio_init(app_config.audio_srate); break;

Evidence

The new V4L2 initialization branch can leave _v4l2_pcm null; media setup logs its error but still
sets audioOn and starts threads. The V4L2 capture loop requires a non-null PCM handle, while the
next enable call exits early when audioOn is set.

src/hal/v4l2_hal.c[229-265]
src/media.c[547-584]
src/media.c[598-615]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
An ALSA initialization error leaves V4L2 audio marked on without a capture device.
## Fix Focus Areas
- src/media.c[550-584]
- src/media.c[598-615]
- src/hal/v4l2_hal.c[229-265]
## Recommended Fix
On HAL audio initialization failure, release resources already allocated by media_audio_enable and return without setting audioOn or starting either thread. Preserve a state from which a later enable request can retry.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

View review recommended (3)

5. Audio thread failures report success 🐞 Bug ☼ Reliability

Description

media_audio_enable() clears audioOn when either pthread_create() fails but neither updates its
return value nor unwinds the resources and any thread already started. The caller receives the
earlier HAL initialization result, normally success, while media_audio_disable() refuses cleanup
because audioOn is false.

Code

src/media.c[R631-634]

+                        &aencPid, &thread_attr, (void *(*)(void *))aenc_thread, NULL)) {
+            audioOn = 0;
         HAL_ERROR("media", "Starting the audio encoding thread failed!\n");
+        }

Evidence

Both new failure branches only clear audioOn and log, and the function returns ret. Normal
cleanup starts with an audioOn guard and otherwise joins both threads and releases the ring,
encoder, and HAL device.

src/media.c[601-640]
src/media.c[513-539]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
A failed audio thread creation leaves resources allocated and returns the earlier initialization result.
## Fix Focus Areas
- src/media.c[601-640]
- src/media.c[513-539]
## Recommended Fix
Track which threads were created. On either creation failure, stop and join only those threads, close the encoder and HAL audio device, destroy the ring, leave audioOn false, and return a failure code.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

6. Overlay update failures appear successful 🐞 Bug ☼ Reliability

Description

v4l2_region_setbitmap() discards the result of its final v4l2_osd_reclaim() call and returns
success. If polling times out or dequeuing the queued frame fails, the region remains marked queued,
yet the caller clears its update state and subsequent updates must attempt to reclaim that frame
again.

Code

src/hal/v4l2_hal.c[R576-579]

+    v4l2_osd_reclaim(osd);
+
+    pthread_mutex_unlock(&_v4l2_osdLock);
+    return EXIT_SUCCESS;

Evidence

Reclaim returns -1 without clearing queued on poll or dequeue failure, but the new bitmap setter
ignores that result and returns EXIT_SUCCESS. The region caller does not retain the update for a
retry.

src/hal/v4l2_hal.c[475-485]
src/hal/v4l2_hal.c[566-585]
src/region.c[393-399]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
The post-queue dequeue can fail without the bitmap update reporting or recovering from the failure.
## Fix Focus Areas
- src/hal/v4l2_hal.c[475-485]
- src/hal/v4l2_hal.c[549-585]
- src/region.c[393-399]
## Recommended Fix
Check the final reclaim result and maintain a usable queue state on timeout or dequeue error. Propagate failure to the region update path so the update can be retried or the overlay recreated.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

7. Encoders fail on drivers that need more buffers 🐞 Bug ☼ Reliability

Description

v4l2_map_buffers rejects a successful VIDIOC_REQBUFS allocation when req.count > wanted, even
if the returned count fits the V4L2_BUF_NUM map arrays. When DMABUF sharing is unavailable, the
copy-input path requests one OUTPUT buffer and fails encoder creation on drivers requiring two or
more; the CAPTURE request for four buffers is subject to the same check, affecting MP4, MJPEG and
JPEG.

Code

src/hal/v4l2_hal.c[R131-132]

+    if (v4l2_ioctl(fd, VIDIOC_REQBUFS, &req) || !req.count) return -1;
+    if (req.count > wanted) return -1;

Evidence

v4l2_map_buffers rejects counts above wanted at line 132, while v4l2_video_create requests one
buffer for copied input at lines 715–718 and four for CAPTURE at lines 721–723. A driver-raised
count therefore reaches the creation failure path despite the maps having V4L2_BUF_NUM entries at
lines 42–44; copied input uses only buffer 0 at lines 919–923.

src/hal/v4l2_hal.c[119-153]
src/hal/v4l2_hal.c[715-723]
src/hal/v4l2_hal.c[36-46]
src/hal/v4l2_hal.c[119-150]
src/hal/v4l2_hal.c[702-719]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
`v4l2_map_buffers` treats a driver-returned buffer count above the requested count as an error, even when it fits the map arrays. Drivers may raise requests to their minimum, causing encoder creation to fail for the one-buffer copied-input request or the four-buffer CAPTURE request.
## Fix Focus Areas
- src/hal/v4l2_hal.c[119-153]
- src/hal/v4l2_hal.c[715-723]
- src/hal/v4l2_hal.c[918-924]
## Recommended Fix
Add a `capacity` argument to `v4l2_map_buffers`, separate from the requested count. Initialize `map[0..capacity-1]`; reject `req.count` only when it exceeds capacity, then map and, where requested, queue all returned buffers. Keep the callers’ requests at one for copied input and four for CAPTURE, passing `V4L2_BUF_NUM` as capacity. The synchronous copied-input path can continue using buffer 0.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

Tip of the day

💡 Did you know, you can see how risky a PR is at a glance with blast radius labels

More tips ↗ | Customize Qodo ↗ | Qodo docs ↗

[difanz]

Addressed in the updated push:

• NAL packing: fixed. An access unit with more than 8 NALs now continues in further packs instead of merging the tail into the last entry.
• Driver-raised buffer counts: fixed. REQBUFS results up to the map size are accepted.

Not changed:

• Channel sizes: intentional. The backend has no scaler, so every encoder reads the shared capture frame. A mismatch is logged, and the default config uses equal sizes.
• DMABUF padding: the encoder's format has the same width, height and stride as the capture. Any extra rows are encoder alignment padding, which the bitstream crops.
• Audio init and thread failures: HAL_ERROR returns EXIT_FAILURE, so media_audio_enable() exits at the failing step.
• Overlay reclaim: the frame is already queued when reclaim runs, so the update is shown. Reclaim only takes the buffer back early, and the next update retries it.

Also in this push: V4L2 builds no longer use the 32-bit-offset mmap. They link no vendor libraries, and with it libasound's mmap calls failed with EINVAL on musl.