Skip to content

[WIP][DO NOT MERGE] Soft DOP-weighted GPS + Kalman/RTS smoothing for video frame sampling#820

Draft
caglarpir wants to merge 1 commit into
mapillary:mainfrom
caglarpir:gps-frame-sampling-smoothing
Draft

[WIP][DO NOT MERGE] Soft DOP-weighted GPS + Kalman/RTS smoothing for video frame sampling#820
caglarpir wants to merge 1 commit into
mapillary:mainfrom
caglarpir:gps-frame-sampling-smoothing

Conversation

@caglarpir
Copy link
Copy Markdown
Contributor

[WIP][DO NOT MERGE] Soft DOP-weighted GPS + Kalman/RTS smoothing for video frame sampling

🚧🚧 DO NOT MERGE — INCOMPLETE WORK IN PROGRESS 🚧🚧

This pull request is intentionally incomplete and must NOT be merged.
It is opened as a draft purely to gather early, directional feedback on the approach.

Blocking gaps before this could ever be considered for merge (details in "Scope & limitations" and "Follow-ups" below):

  • ❌ The native CAMM video upload path is not handled — it currently muxes the raw, unsmoothed track.
  • ❌ Adds an undeclared numpy dependency (degrades gracefully to unsmoothed, but must be resolved first).
  • ❌ Constant-velocity model rounds sharp (~90°) corners; cold-start / loop-closure correction not included.

Please do not approve or merge this PR. Directional comments welcome.

Summary

Improves the GPS positions written to video frames sampled on the client (video_process, sample_video, and geotag-images-from-video) for cameras that carry per-sample GPS quality information (GoPro GPMF).

Instead of hard-deleting "noisy" GPS points, it:

  1. computes a per-sample horizontal uncertainty (σ) from HDOP + fix type + Doppler-speed consistency, then
  2. runs a σ-weighted constant-velocity Kalman filter + RTS smoother over the whole track,

then samples frames along the smoothed track and records a per-frame horizontal accuracy
(MAPGPSAccuracyMeters, round-tripped through EXIF GPSHPositioningError).

⚠️ Scope & limitations (important)

  • This only covers client-side frame sampling. It does NOT handle the native CAMM video upload path.
    For native video upload, the muxed CAMM GPS track is not improved by this change. In fact this PR
    replaces the GoPro extractor's hard noise filter with soft weighting, so the native CAMM path currently
    muxes the raw, unsmoothed track. Cleaning the native/CAMM path — and propagating per-point accuracy
    into CAMM telemetry — is explicit follow-up work and is intentionally out of scope here.
  • numpy dependency. The smoother uses numpy for the small (4×4) linear algebra; mapillary_tools does
    not currently depend on numpy. The smoother degrades gracefully (falls back to the unsmoothed points)
    when numpy is unavailable, but a pure-python port (or adding numpy as a dependency) is needed before this
    should merge.
  • Rounded corners. The constant-velocity model slightly rounds sharp (~90°) turns. A GPS cold-start /
    loop-closure correction and corner-aware smoothing were prototyped separately but are not included here.

What changed

File Change
gpmf/gps_weigher.py (new) Per-sample σ/weight model + weighted-median interpolation kernel
gpmf/gps_smoother.py (new) Speed-gate + σ-weighted Kalman/RTS track smoother (numpy, graceful fallback)
gpmf/gpmf_gps_filter.py Add weight_points() (keeps all points, attaches σ/weights) alongside the existing filter
geotag/video_extractors/native.py GoPro extractor attaches per-point σ/weights to VideoMetadata
sample_video.py Frame distance-sampler smooths the track (selection + per-frame positions) and writes accuracy
geotag/geotag_images_from_video.py Geotag-from-video consumer smooths the track before interpolation
exif_read.py / exif_write.py / geotag/image_extractors/exif.py Read/write GPS horizontal accuracy (GPSHPositioningErrorMAPGPSAccuracyMeters)
types.py VideoMetadata gains point_sigma_xys / point_weights

Validation

On a GoPro MAX car-mount clip (a ~700 m closed city-block loop), comparing the GPS track used for frame sampling:

track length total turning
raw / unfiltered ~1766 m ~9000° (heavy jitter)
old hard filter ~643 m ~3200° (jittery; corner-cutting; loop left open)
this PR (smoothed) ~620 m ~370° (≈ a clean rectangle's 360°)

≈ 24× less direction jitter than the raw track and ≈ 9× less than the old filter, at a plausible length.

Follow-ups (not in this PR)

  • Native CAMM video path handling (the headline gap).
  • numpy removal / pure-python smoother (or declare the dependency).
  • Cold-start / loop-closure correction and corner-aware (sharp-turn) smoothing.

…r video frame sampling

Client-side frame sampling only; does NOT handle the native CAMM video path. Adds an undeclared numpy dependency (graceful fallback to unsmoothed points). Opened as a DRAFT for directional feedback only -- incomplete, NOT for merge. See PR description for scope and limitations.
@meta-cla meta-cla Bot added the cla signed label Jun 5, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant