Skip to content

v5.1: Add nextField helper for nicer decoding code#144

Merged
mourner merged 5 commits into
mainfrom
read-fields-helper
May 29, 2026
Merged

v5.1: Add nextField helper for nicer decoding code#144
mourner merged 5 commits into
mainfrom
read-fields-helper

Conversation

@mourner
Copy link
Copy Markdown
Member

@mourner mourner commented May 26, 2026

Adds a new method to PbfReader, pbf.nextField(end), that advances to the next field and returns its number, which enables fast and ergonomic message reading code. Codegen in compile.js now emits nextField-based loops:

export function readTileFeature(pbf) {
    const obj = {id: 0, tags: [], type: 0, geometry: []};
    let field;
    while ((field = pbf.nextField(end))) {
        if (field === 1) obj.id = pbf.readVarint();
        else if (field === 2) pbf.readPackedVarint(obj.tags);
        else if (field === 3) obj.type = pbf.readVarint();
        else if (field === 4) pbf.readPackedVarint(obj.geometry);
    }
    return obj;
}

vs the previous v5 inline-while form, which required every caller to remember tag >>> 3, pbf.type = tag & 7 (only for messages with packable fields — easy to get wrong), and the matching pbf.skip(tag) capturing the loop-local tag.

readFields is also reimplemented in terms of nextField, but kept for backward compatibility.

Why

The post-v5 inline-while reader has a quiet footgun: hand-written readers must set pbf.type = tag & 7 to make readPackedVarint etc. correctly distinguish packed vs unpacked encodings. Forgetting it silently desyncs the cursor and surfaces as a misleading Unimplemented type: N error several frames deep. Real-world report from mapbox-gl-js's pbf v5 migration.

nextField makes the pbf.type invariant impossible to violate, hides the magic shifts (>>> 3, & 7), and gives codegen and hand-written readers a single, identical shape.

Performance

Mapbox vector-tile workload (439 tiles, 39.3 MB), Node v26, 9-run medians × 3 trials:

variant decode median
v4 readFields callback ~289 ms
v5 inline-while (pre-change) ~179 ms
nextField (this PR) ~179 ms

Within noise of v5; no regression on readFields either (V8 inlines nextField through the callback path).

@mourner mourner changed the title v5.1: Add nextField and skipField helpers v5.1: Add nextField helper for nicer decoding code May 26, 2026
stepankuzmin
stepankuzmin previously approved these changes May 28, 2026
Copy link
Copy Markdown

@stepankuzmin stepankuzmin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice!

@mourner mourner merged commit f85f990 into main May 29, 2026
6 checks passed
@mourner mourner deleted the read-fields-helper branch May 29, 2026 09:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants