Skip to content

feat(fuzz): add egfx_avc420_decode oracle and target#1326

Open
Greg Lamberson (glamberson) wants to merge 1 commit into
Devolutions:masterfrom
lamco-admin:feat/egfx-avc420-decode-fuzz
Open

feat(fuzz): add egfx_avc420_decode oracle and target#1326
Greg Lamberson (glamberson) wants to merge 1 commit into
Devolutions:masterfrom
lamco-admin:feat/egfx-avc420-decode-fuzz

Conversation

@glamberson
Copy link
Copy Markdown
Contributor

This change adds an assertion-or-panic fuzz oracle for the AVC
length-prefix to Annex-B conversion that runs inside
OpenH264Decoder::decode before any OpenH264 entry point. The same
change refactors the conversion from a private method on
OpenH264Decoder into two public free functions in
ironrdp_egfx::pdu::avc. The first function, avc_to_annex_b,
returns a fresh Vec<u8> and is symmetric with the existing
annex_b_to_avc. The second function, avc_to_annex_b_into, writes
into a caller-provided buffer and preserves the per-frame buffer-reuse
optimization that OpenH264Decoder relied on.

The conversion is now available unconditionally to any consumer of
ironrdp-egfx. The previous #[cfg(feature = "openh264")] gating
went with the location, not the bytes-to-bytes logic, so lifting the
function out of decode.rs removed the gate too.

The new oracle exercises two input distributions on every fuzz call:

  • Direct path: the oracle calls avc_to_annex_b(data) on the raw fuzz
    input. This exercises the wrapper on arbitrary byte distributions,
    including inputs that do not parse as Avc420BitmapStream.
  • Decode-chain path: the oracle tries Avc420BitmapStream::decode(data);
    on success it calls avc_to_annex_b(stream.data). This exercises the
    wrapper on the realistic post-decode payload distribution.

The oracle catches panics in the wrapper, OOM allocation from
attacker-controlled NAL length encoding, and contract violations on the
produced Annex-B byte stream. The oracle does NOT catch OpenH264
internal bugs (OSS-Fuzz coverage), the YUV-to-RGBA conversion path
downstream of OpenH264 (separate workstream), or AVC444 luma plus
chroma split (sibling target).

Smoke fuzz ran 10,922,695 iterations in 31 seconds at ~352K exec/s
sustained with zero crashes. Coverage settled at 158 lines, 456
features, 69 corpus entries.

The new target auto-discovers into CI via the cargo xtask fuzz list
dynamic fan-out mechanism. The new check_egfx_avc420_decode
regression-replay test in
crates/ironrdp-testsuite-core/tests/fuzz_regression.rs runs against
the seed corpus and passes.

Refs #1316.

This change adds an assertion-or-panic fuzz oracle for the AVC
length-prefix to Annex-B conversion that runs inside
`OpenH264Decoder::decode` before any OpenH264 entry point. The same
change refactors the conversion from a private method on
`OpenH264Decoder` into two public free functions in
`ironrdp_egfx::pdu::avc`. The first function, `avc_to_annex_b`,
returns a fresh `Vec<u8>` and is symmetric with the existing
`annex_b_to_avc`. The second function, `avc_to_annex_b_into`, writes
into a caller-provided buffer and preserves the per-frame buffer-reuse
optimization that `OpenH264Decoder` relied on.

The conversion is now available unconditionally to any consumer of
`ironrdp-egfx`. The previous `#[cfg(feature = "openh264")]` gating
went with the location, not the bytes-to-bytes logic, so lifting the
function out of `decode.rs` removed the gate too.

The new oracle exercises two input distributions on every fuzz call:

- Direct path: the oracle calls `avc_to_annex_b(data)` on the raw fuzz
  input. This exercises the wrapper on arbitrary byte distributions,
  including inputs that do not parse as `Avc420BitmapStream`.
- Decode-chain path: the oracle tries `Avc420BitmapStream::decode(data)`;
  on success it calls `avc_to_annex_b(stream.data)`. This exercises the
  wrapper on the realistic post-decode payload distribution.

The oracle catches panics in the wrapper, OOM allocation from
attacker-controlled NAL length encoding, and contract violations on the
produced Annex-B byte stream. The oracle does NOT catch OpenH264
internal bugs (OSS-Fuzz coverage), the YUV-to-RGBA conversion path
downstream of OpenH264 (separate workstream), or AVC444 luma plus
chroma split (sibling target).

Smoke fuzz ran 10,922,695 iterations in 31 seconds at ~352K exec/s
sustained with zero crashes. Coverage settled at 158 lines, 456
features, 69 corpus entries.

The new target auto-discovers into CI via the `cargo xtask fuzz list`
dynamic fan-out mechanism. The new `check_egfx_avc420_decode`
regression-replay test in
`crates/ironrdp-testsuite-core/tests/fuzz_regression.rs` runs against
the seed corpus and passes.

Refs Devolutions#1316.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

1 participant