React Native Document Scanner

Android-only React Native Nitro library with an Expo scanner app in example/: a camera view with native live document guidance, a crop editor, a zoomable image view, and a processor for corner detection, perspective rendering with filters, and PDF export.

Out of scope: auto-capture, OCR, automatic corner correction and finger removal.

API

Props, callbacks and options are documented in the specs: src/DocumentScannerCamera.nitro.ts, src/DocumentCropEditor.nitro.ts, src/DocumentZoomImage.nitro.ts, src/DocumentZoomPager.nitro.ts, src/DocumentScannerProcessor.nitro.ts.

const cameraRef = useRef<CameraRef>(null)
<Camera ref={cameraRef} />
const photo = await cameraRef.current?.takePhoto()
const analysis = await DocumentScanner.analyzePhoto(photo!.originalUri, photo!.liveCandidate?.quad)
const page = await DocumentScanner.renderDocument(photo!.originalUri, quad, { paperSize: { width: 21, height: 29.7 }, dpi: 200 })
await DocumentScanner.removeCachedImages([photo!.originalUri, page.outputUri])

Camera

• Live detection runs natively; no frames reach JS, and detection never triggers capture.
• mode picks the camera's profile: 'scanner' (default) is tuned for documents, 'camera' is a general-purpose camera, and 'off' releases the camera's preview, analysis and sensors while it stays mounted; a capture in flight finishes first. guide={false} turns off guidance only.
• While guide is on, focus follows the detected page and holds a lock until the page moves nearer or farther, loses sharpness or disappears. Taps take priority for five seconds; focus does not change while a photo is being taken, so a tap then is ignored.
• In scanner mode the lens parks at a fixed distance instead of running continuous autofocus: about 30 cm at first, then wherever the last lock was. The guide finds a page at once, and a valid detected page gets a focus scan; detection runs independently of lens movement. Losing the page, a tap's five seconds or a failed scan park the lens again, so focus never wanders to the background. The shutter never focuses on its own: a photo waits for document detection to find the page and lock on it. camera mode keeps continuous autofocus between scans, as do cameras without manual focus and captures that do not wait for focus.
• The detection model loads and runs one warm-up inference while the camera opens, not on the first frame.
• With the quality gate on (default), takePhoto waits for focus up to qualityGateConfig.focusTimeout (2000 ms), in scanner mode for a confirmed page and its focus lock, then rejects with capture/focus-timeout unless requireFocus: false. It then waits for the autofocus, exposure and white-balance states CameraX requires of a zero-shutter-lag frame, and for steady framing: four consecutive analyzed frames with little movement, little smear in each exposure, and the page holding its position, scale and shape. It shoots as soon as all of that aligns, with no settling delay, and rejects after 1.5 seconds without it. CameraX picks the buffered frame. It takes one shot.
• retainCapturePreview (default off) keeps the frozen shutter frame after a successful capture, even with mode="off". Set it back to false once the review image loads; failed captures always release it. ZoomImage.onLoad(uri) reports when its decoded image is installed, and previous images remain visible while replacements load.
• captureFeedback (default on) animates each takePhoto around the guide outline, or the viewfinder area without one: a pulse and a find-the-page, focusing, lighting or hold-steady hint while it waits, a line running around while it shoots, then a flash and a haptic tick once the user can move, or a red blink on failure. captureColor sets its color (default white), and onCaptureStage reports waiting, shooting, captured and failed for custom UI.
• torch is off (default), on or auto. Auto lights it after about a second in a dark room and keeps it lit until the camera stops or the mode changes.
• Live geometry checks require valid normalized convex corners and minimum page area. Detection runs independently of focus; capture still waits for readiness.
• adaptiveDetection (default off) pauses model inference while optical flow tracks a confirmed page, including tilt. Lost tracking, a covered region, or a large shape change bring detection back.
• debugCaptures (default off) writes, per resolved photo, a folder with the full-resolution photo, the live analysis frame behind the guide at the shutter, and JSON with its corners; analyzePhoto adds the photo's own corners. DocumentScanner.pickDebugCaptureFolder() lets the user pick where, such as a folder in Downloads; the library keeps that grant across launches, getDebugCaptureFolder() reads it and clearDebugCaptureFolder() releases it. Without one, folders go to the app's external files debug-captures/.

Processing

• analyzePhoto returns zero or one quad; its score is raw model output, not a calibrated confidence.
• Live guidance uses one full-image model inference; photo analysis uses one inference on the selected region. A valid live result is displayed immediately; adaptive inference can then pause while optical flow tracks it. Framing movement gates detection; exposure-smear checks belong to capture. Analysis requests 640x480, with a fixed 256x256 model input. Saved photos are decoded at up to 2000 px. Pass the optional live quad to analyzePhoto(uri, liveQuad) to infer on its bounding region with 10% margin per side, expanded to the photo aspect ratio and shifted within the photo. Results map back to full-photo coordinates. Without a valid quad, analysis uses the whole photo. The example passes the live guide and also uses it for the initial crop preview.
• removeShadows (default true) lifts shaded paper toward the page's bright paper before the filter, up to twice its brightness. It is deterministic, with no model: the paper's brightness comes from a reduced copy with ink closed away, and one gain serves all channels, so colours keep their ratios. Without a filter, a shadow lit by differently coloured light keeps a slight tint; fingers and solid dark areas wider than a text line are lifted too. Every crop, filter or option change renders anew from the original.
• Render filters are none, enhance, grayscale and blackWhite (high-contrast grayscale, not binary). All but none also deepen faded thin ink toward the page's darkest ink and sharpen softer areas more, so glare or distance does not leave part of a page washed out. Output is JPEG by default; format: 'png' with a monochrome filter gives 4-bit grayscale, which PDF export embeds as is.
• paperSize (cm) and dpi (default 200) set the render's resolution: the page's longer side becomes the paper's longer side at that dpi, keeping the detected proportions and never enlarging the photo's pixels. Without them a render keeps the page's own size in the photo, up to 4096 px.
• thumbnailSize (16–1024 px) also writes thumbnailUri, the finished page reduced for page lists. A capture's frameUri is a stand-in from the live analysis frame at the shutter, the guide's page straightened, to show until the first render.
• jpegQuality is 0–1 for capture, rendering and PDF. Omit it to use the capture mode's quality, the renderer's 0.92, or to keep eligible JPEGs unchanged in a PDF.
• Photos and renders are written to the app cache. Copy or save anything that must outlive it.

Views

• CropEditor edits a quad over the original photo. ZoomImage shows an image (such as a rendered page) contain-fitted, with native pinch, pan and double tap between fit and 2.5x; a new uri fits again. ZoomPager swipes between such images natively (a RecyclerView snapping one page at a time); paging is blocked while the shown page is zoomed, and only the shown page and its neighbours hold a decoded image.

Errors

• Failures reject with ScannerError and a stable code, such as capture/focus-timeout or editor/filter-failed; the onError of CropEditor and ZoomImage receives the same class. The app chooses display text. Camera hints use overridable document_scanner_* Android string resources.

Development

From example/: npm run start (Metro on 8088), then npm run android (arm64) or npm run android:emulator (x86_64). Run npm run check from the root for typechecks and contract tests, and npm run nitrogen after changing a spec in src/.

Maintainer guidance: .agents/skills/document-scanner-camera/SKILL.md. Model export and validation: tools/models/README.md.

Captured-photo quality

takePhoto() returns a required defects: ImageDefect[] field. The array contains typed lowercase codes: blurred, too-dark, and overexposed. An empty array means no problems were detected, not a guarantee that the image is good. Unknown results add no issue.

Assessment runs on the saved JPEG before the capture promise resolves, off the UI thread. Blur uses the existing full-resolution central-region reblur score (threshold 0.8); a featureless region cannot be assessed. Exposure uses a sampled whole-image luminance histogram: at least 90% below level 40 flags darkness, and at least 98% at levels 253–255 flags clipping. These are initial heuristics, not calibrated document-readability guarantees. Document presence and shape remain separate photo-analysis results. Issues are advisory and never reject or delete a capture.

const photo = await cameraRef.current!.takePhoto()
if (photo.defects.length > 0) {
  // Keep review open and explain each issue.
}

Alternative corner models

The host app downloads and manages model files, then supplies a completed download to the camera:

<Camera
  cornerNetModel={{
    uri: downloadedModelUri,
    inputWidth: 320,
    inputHeight: 256,
  }}
/>

cornerNetModel.uri accepts an absolute local path or file:// URI. Dimensions are optional positive integers and describe the model's input tensor:

Supplied dimensions	Resolved input
Neither	256 × 256
inputWidth: 320 only	320 × 320
inputHeight: 192 only	192 × 192
inputWidth: 320, inputHeight: 256	320 × 256

Omit cornerNetModel to restore the bundled 256 × 256 model. Changes to the URI or dimensions load asynchronously without rebinding the camera hardware. The native resizer stretches the upright image to the resolved width and height; it does not letterbox or crop to that ratio. The resolved dimensions must match the model's existing input shape; this does not resize a fixed-shape model's tensor or change the camera's capture resolution.

The previous model remains active during loading and after failures (the bundled model is the initial fallback). Invalid dimensions, inaccessible files and incompatible models are logged natively, with no JS callbacks or user feedback. Only the latest request on a mounted view can activate a model. Equal URI/dimension values do not reload, even with a new JS object: use a new versioned URI for each download, or remount to retry a failed selection.

The selected model applies to the single shared live/photo interpreter, including subsequent DocumentScanner.analyzePhoto calls. Selection persists across camera inactivity and unmount, until another camera selects a model or the process exits. Without a mounted camera, photo analysis uses the last selected model (the bundled model initially). With multiple cameras, the latest successful selection wins. Reapply the host's saved configuration on startup.

Live and photo inference take turns on one serial model worker. An inference already running finishes before the next uses the replacement. Live tracking resets on its next frame. Model bytes are retained, so changing or deleting the source after activation cannot alter the active model. The module does not download, persist or delete model files. Validation briefly opens a candidate interpreter before activation; normal inference retains only one interpreter, which closes when its last camera/processor user releases it.

Replacement models must implement the existing DocCornerNet contract: one float32 NHWC input [1, inputHeight, inputWidth, 3] with ImageNet RGB normalization, and one float32 output [1, 9] using the same corner order, coordinate decoding and score semantics as the bundled model. Shape and dtype checks cannot establish semantic compatibility; the host must supply a compatible export.