Natvis: implement view() specifier, IncludeView/ExcludeView filtering, and CustomListItems loop engine by lugerard · Pull Request #1580 · microsoft/MIEngine

lugerard · 2026-06-10T13:04:56Z

This PR enables MIEngine to evaluate two natvis features:
view-based filtering (IncludeView/ExcludeView on DisplayString
and Expand elements, activated via the view() format specifier) and
<CustomListItems> loop bodies (<Loop>, <Item>, <Exec>,
<If>/<ElseIf>/<Else>, <Break>), which are the standard way to walk
pointer-based data structures such as linked lists and trees.

The two commits are bundled together because <CustomListItems> loop bodies
can contain IncludeView/ExcludeView guards on their <Item> elements,
so commit 1 is a functional prerequisite for commit 2.

Commit 1 adds view() specifier parsing and IncludeView/ExcludeView
filtering, plus fixes to EvalCondition error handling and expression
pre-processing that are prerequisites for correct evaluation on GDB/LLDB.

Commit 2 implements the <CustomListItems> loop engine, including
expression-growth prevention after <Exec> assignments and a $i
word-boundary fix.

All new code is covered by unit tests.

Add view support to the MIEngine natvis evaluator (steps 1, 2, and 3): 1. DisplayString IncludeView/ExcludeView filtering - Add IncludeView and ExcludeView properties to DisplayStringType (NatvisXsdTypes.cs) so the XML attributes are deserialized. - Filter DisplayString entries in FormatDisplayString() using the new IsIncludeViewMatch() and IsExcludeViewMatch() helpers before evaluating the Condition. 2. {expr,view(name)} inline specifier in DisplayString text - ExtractViewName() detects a view() format specifier in an inline expression block such as {this,view(RecZone)na}. - When detected, FormatValue() calls GetExpressionValue() with the view name, which re-enters FormatDisplayString() selecting only DisplayString entries whose IncludeView matches. - ExtractViewName() returns null for view() with an empty name (not a valid specifier). 3. View support on Expand elements - Add IncludeView and ExcludeView properties to ExpandedItemType, ItemType, and CustomListItemsType in NatvisXsdTypes.cs. - Thread currentView through Expand() and ExpandVisualized() so that view context propagates into recursive expand calls. - Filter Item and ExpandedItem elements by IncludeView/ExcludeView before evaluating their Condition. - Strip a view() specifier from an ExpandedItem expression before evaluating it, and pass the extracted view name into the recursive Expand() call so that IncludeView guards on the target type's Expand elements match correctly. - Add a CustomListItemsType stub case that applies IncludeView/ ExcludeView filtering; loop body execution to follow. 4. EvalCondition fallback on failure - Wrap condition evaluation in try/catch: MIException (debugger rejected the expression, e.g. too long) is caught silently and returns false; other exceptions are caught and logged at Warning before also returning false, so natvis never surfaces errors as debug-session failures. - Prerequisite: natvis files may use a lightweight condition as a platform probe (always true for valid data, but too long to expand on GDB/LLDB); without this fallback the condition would surface as an error rather than falling through to the next DisplayString. 5. Strip format specifier before name substitution in GetExpressionValue() - A specifier such as ",d" was being matched by ProcessNamesInString as a child-variable name, corrupting the expression. Strip it first, then re-attach after substitution. - Prerequisite: without this fix, an expression such as {call(),d} would have its specifier matched as a variable name, corrupting the expression before evaluation. 6. Intrinsic expansion before dll! stripping in ReplaceNamesInExpression() - Intrinsic bodies can contain dll!-qualified type casts. Moving expansion before the dll!-strip regex ensures those references are also cleaned up. - Prerequisite: without this fix, a dll!-qualified cast inside an intrinsic body would survive into the expression sent to GDB/LLDB. Note: items 4, 5, and 6 are bugfixes that are prerequisites for the view() feature to work correctly on GDB/LLDB. They are bundled in this commit because they share the same test infrastructure and were discovered during view() development. Unit tests: Add 18 tests in NatvisFormatSpecifierTest covering the three new static helpers (IsIncludeViewMatch, IsExcludeViewMatch, ExtractViewName) and the view() specifier parsing patterns used by Expand elements, including the empty-name edge case for ExtractViewName.

Add full execution support for the <CustomListItems> expand element, which was previously deserialized but produced no children. NatvisXsdTypes.cs: - Add seven new types covering the loop body: CustomListLoopType (<Loop>), CustomListLoopItemType (<Item>), CustomListBreakType (<Break>), CustomListExecType (<Exec>), CustomListIfType (<If>), CustomListElseIfType (<ElseIf>), CustomListElseType (<Else>). If/ElseIf/Else bodies and Loop bodies share the same element set, so nested loops and conditional branches are fully supported. - Extend CustomListItemsType with a Loop[] field. - Add Condition attribute to CustomListLoopType: acts as a while-guard, stopping the loop as soon as the condition evaluates to false. - Add missing [GeneratedCode], [Serializable] and [DebuggerStepThrough] attributes to the seven new types for consistency with the rest of the file. Natvis.cs: - Replace the view-filter stub with a complete execution engine: * <Variable> declarations initialise a local-variable table (name -> current expression string) via ReplaceNamesInExpression. * Optional <Size> sets the totalSize upper bound for pagination. * The loop driver evaluates an optional Loop Condition before each iteration (while-guard), then calls ExecuteCustomListBody() once per iteration. * <Break>: stops the loop when the condition holds. * <Item>: substitutes local vars, resolves field names and intrinsics, emits the child; increments the $i counter (GlobalIndex). * <Exec>: supports "varName = rhs" assignment and ++/-- shorthand (prefix and postfix) to update the local-variable table. After each update, the new expression is evaluated and normalised to a scalar literal to prevent unbounded expression growth across iterations (e.g. i++ stays "1", "2", "3" rather than accumulating nested parentheses). * <If>/<ElseIf>/<Else>: evaluates conditions in order and executes the first matching branch; all siblings are consumed in one pass so the idx cursor never re-processes them. * Pagination: fast-forwards through startIndex items, then collects up to MAX_EXPAND children; adds a [More...] node when the page is full and more items remain. * Infinite-loop guard: caps iterations at min(startIndex+51, 10000). - Add helpers (internal static for testability): ExecuteCustomListBody, SubstituteLocalVars, ApplyExecToLocalVars, FormatCustomListItemName ($i / {$i} in names with word-boundary guard to avoid corrupting tokens like $item), CustomListLoopContext (shared mutable loop state), s_execAssignment static Regex (with =(?!=) to avoid matching == operators), s_execIncrDecr static Regex for ++/-- shorthand, s_dollarI static Regex (\$i\b) for safe bare-$i replacement. NatvisFormatSpecifierTest.cs: - 25 new unit tests covering SubstituteLocalVars (empty, no-vars, single substitution, word-boundary, multi-var, parens), ApplyExecToLocalVars (assignment, unknown LHS, empty, counter increment, == not matched, prefix/postfix ++/--, unknown var with ++, whitespace tolerance), and FormatCustomListItemName (null template, {$i}, bare $i, no special tokens, local-var substitution).

Copilot

Pull request overview

This PR extends MIEngine’s Natvis evaluator to support view-based formatting/expansion (view() specifier plus IncludeView/ExcludeView filtering) and adds a CustomListItems loop execution engine to enable pointer-based collection traversal during expansion.

Changes:

Adds parsing/propagation of view(name) and applies IncludeView/ExcludeView filtering for DisplayString, Item, ExpandedItem, and CustomListItems.
Improves Natvis expression handling (specifier stripping before name substitution, intrinsic/module-prefix preprocessing ordering, and safer EvalCondition failure handling).
Implements CustomListItems loop-body execution (Loop/Item/Exec/If/ElseIf/Else/Break) with helper utilities and unit tests.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 4 comments.

File	Description
src/MIDebugEngineUnitTests/NatvisFormatSpecifierTest.cs	Adds unit tests for `view()` extraction, view matching helpers, and CustomListItems helper utilities.
src/MIDebugEngine/Natvis.Impl/NatvisXsdTypes.cs	Extends Natvis XSD-derived types with view attributes and new CustomListItems loop-body element models.
src/MIDebugEngine/Natvis.Impl/Natvis.cs	Implements view filtering, `view()` propagation through expansion/formatting, expression preprocessing fixes, and the CustomListItems loop engine.

+                        {
+                            if (string.IsNullOrEmpty(v.Name)) continue;
+                            string initVal = v.InitialValue ?? "0";
+                            // Resolve field names, template parameters and intrinsics in the initial value.
+                            localVars[v.Name] = ReplaceNamesInExpression(initVal, variable, visualizer.ScopedNames, visualizer.Intrinsics);
+                        }


+                            string normalized = GetExpressionValue(localVars[updatedVar], variable, visualizer.ScopedNames, visualizer.Intrinsics);
+                            if (!string.IsNullOrEmpty(normalized) &&
+                                !normalized.TrimStart().StartsWith("0x", StringComparison.OrdinalIgnoreCase))
+                            {
+                                localVars[updatedVar] = normalized;
+                            }


gregg-miskelly · 2026-06-21T05:16:11Z

+        }
+
+        /// <remarks/>
+        [System.Xml.Serialization.XmlElementAttribute("Item",   typeof(CustomListLoopItemType))]


CustomListLoopItemType

Can you rename this to CustomListItemType to match the main natvis.xsd

gregg-miskelly · 2026-06-21T05:16:41Z

+
+        /// <remarks/>
+        [System.Xml.Serialization.XmlElementAttribute("Item",   typeof(CustomListLoopItemType))]
+        [System.Xml.Serialization.XmlElementAttribute("Break",  typeof(CustomListBreakType))]


CustomListBreakType

Similar request for the rest. Here is how they are declared in the .xsd:

<xs:element name="Loop" minOccurs="0" maxOccurs="unbounded" type="LoopType"></xs:element> <xs:sequence minOccurs="0" maxOccurs="unbounded"> <xs:element name="If" minOccurs="1" maxOccurs="1" type="IfType"></xs:element> <xs:element name="Elseif" minOccurs="0" maxOccurs="unbounded" type="IfType"></xs:element> <xs:element name="Else" minOccurs="0" maxOccurs="1" type="ElseType"></xs:element> </xs:sequence> <xs:element name="Exec" minOccurs="0" maxOccurs="unbounded" type="ExecType"></xs:element> <xs:element name="Break" minOccurs="0" maxOccurs="unbounded" type="BreakType"></xs:element>

gregg-miskelly · 2026-06-21T05:22:03Z

+        [System.Xml.Serialization.XmlElementAttribute("Break",  typeof(CustomListBreakType))]
+        [System.Xml.Serialization.XmlElementAttribute("Exec",   typeof(CustomListExecType))]
+        [System.Xml.Serialization.XmlElementAttribute("If",     typeof(CustomListIfType))]
+        [System.Xml.Serialization.XmlElementAttribute("ElseIf", typeof(CustomListElseIfType))]


ElseIf

The casing of this is incorrect. It should be Elseif

gregg-miskelly · 2026-06-21T05:48:08Z

Can you add an end-to-end test that uses this? Example: #1559

gregg-miskelly · 2026-06-21T05:53:35Z

+                            progress = true;
+                        }
+                    }
+                }


}

else -> write a warning for unexpected element?

gregg-miskelly · 2026-06-21T05:54:16Z

+                            bool innerProgress = ExecuteCustomListBody(nestedLoop.Items, ctx, variable, visualizer, localVars, children);
+                            if (!innerProgress && !ctx.Done) break;
+                            progress = true;
+                        }


Can we share more code with the top level loop?

gregg-miskelly · 2026-06-21T06:01:06Z

+                            continue;
+                    }
+                    string execExpr = SubstituteLocalVars(exec.Value?.Trim() ?? "", localVars);
+                    string updatedVar = ApplyExecToLocalVars(execExpr, localVars);


ApplyExecToLocalVars

Not sure if you want to handle this, but the .natvis files that ship with VS will often assign to multiple variables in an <Exec> block. Example: <Exec>++idx, ++statptr</Exec>. Feels like we need to at least log a warning, but it probably isn't too hard to support.

gregg-miskelly · 2026-06-21T06:03:12Z

+                            }
+                        }
+                        catch (Exception) { /* keep the expression as-is if evaluation fails */ }
+                    }


}

Should we be logging a warning here and breaking?

gregg-miskelly

Thanks for doing this. Aside from that, this seems about right to me.

lugerard and others added 3 commits June 10, 2026 14:38

Merge branch 'main' into natvis-view-customlistitems

8337b7f

gregg-miskelly requested a review from Copilot June 21, 2026 00:49

Copilot started reviewing on behalf of gregg-miskelly June 21, 2026 00:50 View session

Copilot AI reviewed Jun 21, 2026

View reviewed changes

gregg-miskelly reviewed Jun 21, 2026

View reviewed changes

gregg-miskelly requested changes Jun 21, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Natvis: implement view() specifier, IncludeView/ExcludeView filtering, and CustomListItems loop engine#1580

Natvis: implement view() specifier, IncludeView/ExcludeView filtering, and CustomListItems loop engine#1580
lugerard wants to merge 3 commits into
microsoft:mainfrom
lugerard:natvis-view-customlistitems

lugerard commented Jun 10, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

gregg-miskelly Jun 21, 2026

Uh oh!

gregg-miskelly Jun 21, 2026 •

edited

Loading

Uh oh!

gregg-miskelly Jun 21, 2026

Uh oh!

gregg-miskelly commented Jun 21, 2026

Uh oh!

gregg-miskelly Jun 21, 2026

Uh oh!

gregg-miskelly Jun 21, 2026

Uh oh!

gregg-miskelly Jun 21, 2026 •

edited

Loading

Uh oh!

gregg-miskelly Jun 21, 2026 •

edited

Loading

Uh oh!

gregg-miskelly left a comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

lugerard commented Jun 10, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

gregg-miskelly Jun 21, 2026

Choose a reason for hiding this comment

Uh oh!

gregg-miskelly Jun 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

gregg-miskelly Jun 21, 2026

Choose a reason for hiding this comment

Uh oh!

gregg-miskelly commented Jun 21, 2026

Uh oh!

gregg-miskelly Jun 21, 2026

Choose a reason for hiding this comment

Uh oh!

gregg-miskelly Jun 21, 2026

Choose a reason for hiding this comment

Uh oh!

gregg-miskelly Jun 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

gregg-miskelly Jun 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

gregg-miskelly left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

gregg-miskelly Jun 21, 2026 •

edited

Loading

gregg-miskelly Jun 21, 2026 •

edited

Loading

gregg-miskelly Jun 21, 2026 •

edited

Loading