From 89653df8ec7af76b9aa5eeb6027a7de72f6ed766 Mon Sep 17 00:00:00 2001 From: Mitchell Hwang Date: Tue, 10 Mar 2026 18:43:22 -0400 Subject: [PATCH] Update diagnostics docs for dotnet-trace collect-linux Update 3 docs to reflect dotnet-trace collect-linux capabilities: - dotnet-trace.md: Add symbol resolution section for collect-linux - eventpipe.md: Add EventPipe (user_events) column to comparison table, document how EventPipe can emit events as user_events for unified managed + native trace collection on Linux - debug-highcpu.md: Integrate collect-linux as Linux alternative, clarify that safe-point bias and managed-only callstacks apply on all platforms Key points documented: - Native debug symbols must be on disk for symbol resolution - No environment variables or process restarts needed - EventPipe (user_events) enables unified tracing on Linux (.NET 10+) - Still framed as preview feature Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- docs/core/diagnostics/debug-highcpu.md | 2 +- docs/core/diagnostics/dotnet-trace.md | 24 ++++++++++++++++++++++++ docs/core/diagnostics/eventpipe.md | 16 +++++++++------- 3 files changed, 34 insertions(+), 8 deletions(-) diff --git a/docs/core/diagnostics/debug-highcpu.md b/docs/core/diagnostics/debug-highcpu.md index 2d42a26f3e990..504713cfe71f7 100644 --- a/docs/core/diagnostics/debug-highcpu.md +++ b/docs/core/diagnostics/debug-highcpu.md @@ -173,7 +173,7 @@ At this point, you can safely say the CPU is running higher than you expect. Ide ## Analyze High CPU with Profiler -When analyzing an app with high CPU usage, you need a diagnostics tool that can provide insights into what the code is doing. The usual choice is a profiler, and there are different profiler options to choose from. `dotnet-trace` can be used on all operating systems, however, its limitations of safe-point bias and managed-only callstacks result in more general information compared to a kernel-aware profiler like 'perf' for Linux or ETW for Windows. If your performance investigation involves only managed code, generally `dotnet-trace` will be sufficient. +When analyzing an app with high CPU usage, use a profiler to understand what the code is doing. `dotnet-trace` works on all operating systems, but safe-point bias and managed-only callstacks limit it to more general information than a kernel-aware profiler like ETW for Windows or `perf` for Linux. On Linux, [`dotnet-trace collect-linux`](dotnet-trace.md#dotnet-trace-collect-linux) eliminates these limitations by combining EventPipe with OS-level perf_events in a single unified trace. If your performance investigation involves only managed code, `dotnet-trace collect` is generally sufficient. ### [Linux](#tab/linux) diff --git a/docs/core/diagnostics/dotnet-trace.md b/docs/core/diagnostics/dotnet-trace.md index 227e1be1d89f0..28ec39e24fc98 100644 --- a/docs/core/diagnostics/dotnet-trace.md +++ b/docs/core/diagnostics/dotnet-trace.md @@ -699,6 +699,30 @@ However, when you want to gain a finer control over the lifetime of the app bein ## (Linux-only) Collect a machine-wide trace using dotnet-trace +### Get symbols for native runtime frames + +`collect-linux` captures native frames in callstacks. To resolve native method names for runtime libraries (such as `libcoreclr.so`), place the corresponding debug symbol files on disk beside the libraries. Without these symbols, native frames appear as unresolved addresses in the trace. + +Unlike [`perfcollect`](./trace-perfcollect-lttng.md), `collect-linux` doesn't require you to set environment variables like `DOTNET_PerfMapEnabled` or `DOTNET_EnableEventLog` before starting your application. `collect-linux` dynamically enables perfmap generation for JIT-compiled code when the trace begins, so you don't need to restart any .NET processes. + +To download native runtime symbols, use [dotnet-symbol](./dotnet-symbol.md): + +1. Install `dotnet-symbol`: + + ```bash + dotnet tool install -g dotnet-symbol + ``` + +1. Download the debug symbols for your runtime version. For example, if your runtime is installed at `/usr/share/dotnet/shared/Microsoft.NETCore.App/10.0.0`: + + ```bash + dotnet-symbol --symbols /usr/share/dotnet/shared/Microsoft.NETCore.App/10.0.0/lib*.so + ``` + +1. Place the downloaded `.so.dbg` files beside the runtime libraries they correspond to (for example, `libcoreclr.so.dbg` next to `libcoreclr.so`). If you run `dotnet-symbol` from the runtime directory, it places the symbols there automatically. + +After you place the symbols, `collect-linux` resolves native method names when it processes the trace. + This example captures CPU samples for all processes on the machine. Any processes running .NET 10+ will also include some additional lightweight events describing GC, JIT, and Assembly loading behavior. ```output diff --git a/docs/core/diagnostics/eventpipe.md b/docs/core/diagnostics/eventpipe.md index 8ab863fc3a0ce..467b14e6f53e8 100644 --- a/docs/core/diagnostics/eventpipe.md +++ b/docs/core/diagnostics/eventpipe.md @@ -25,18 +25,20 @@ To learn more about the NetTrace format, see the [NetTrace format documentation] EventPipe is part of the .NET runtime and is designed to work the same way across all the platforms .NET Core supports. This allows tracing tools based on EventPipe, such as `dotnet-counters`, `dotnet-gcdump`, and `dotnet-trace`, to work seamlessly across platforms. -However, because EventPipe is a runtime built-in component, its scope is limited to managed code and the runtime itself. EventPipe events include stacktraces with managed code frame information only. If you want events generated from other unmanaged user-mode libraries, CPU sampling for native code, or kernel events you should use OS-specific tracing tools such as ETW or perf_events. On Linux the [perfcollect tool](./trace-perfcollect-lttng.md) helps automate using perf_events and [LTTng](https://en.wikipedia.org/wiki/LTTng). +However, because EventPipe is a runtime built-in component, its scope is limited to managed code and the runtime itself. Without other tracing tools, EventPipe events include stacktraces with managed code frame information only. To get events from other unmanaged user-mode libraries, CPU sampling for native code, or kernel events, use OS-specific tracing tools such as ETW or perf_events. On Linux, the [perfcollect tool](./trace-perfcollect-lttng.md) helps automate using perf_events and [LTTng](https://en.wikipedia.org/wiki/LTTng). + +Starting in .NET 10, EventPipe on Linux can emit events as [user_events](https://docs.kernel.org/trace/user_events.html), enabling collection of managed events, OS/kernel events, and native callstacks in a single unified trace. This mode requires admin/root privileges and Linux kernel 6.4+. For more information, see [`dotnet-trace collect-linux`](./dotnet-trace.md#dotnet-trace-collect-linux). Another major difference between EventPipe and ETW/perf_events is admin/root privilege requirement. To trace an application using ETW or perf_events you need to be an admin/root. Using EventPipe, you can trace applications as long as the tracer (for example, `dotnet-trace`) is run as the same user as the user that launched the application. The following table is a summary of the differences between EventPipe and ETW/perf_events. -|Feature|EventPipe|ETW|perf_events| -|-------|---------|---|-----------| -|Cross-platform|Yes|No (only on Windows)|No (only on supported Linux distros)| -|Require admin/root privilege|No|Yes|Yes| -|Can get OS/kernel events|No|Yes|Yes| -|Can resolve native callstacks|No|Yes|Yes| +|Feature|EventPipe|EventPipe (user_events)|ETW|perf_events| +|-------|---------|----------------------|---|-----------| +|Cross-platform|Yes|No (Linux only)|No (only on Windows)|No (only on supported Linux distros)| +|Require admin/root privilege|No|Yes|Yes|Yes| +|Can get OS/kernel events|No|Yes|Yes|Yes| +|Can resolve native callstacks|No|Yes|Yes|Yes| ## Use EventPipe to trace your .NET application