diff --git a/docs/source/index.md b/docs/source/index.md index b65139319a7..41a9f585bc0 100644 --- a/docs/source/index.md +++ b/docs/source/index.md @@ -11,6 +11,64 @@ --- +## πŸ—ΊοΈ Find Your Path + +Not sure where to start? Use the guided pathways to navigate ExecuTorch based on your experience level, goal, and target platform. + +::::{grid} 3 +:gutter: 3 + +:::{grid-item-card} 🟒 New to ExecuTorch +:class-header: bg-success text-white +:link: pathway-beginner +:link-type: doc + +Step-by-step learning sequence from installation to your first on-device deployment. Includes concept explanations and worked examples. + ++++ +**Beginner Pathway β†’** +::: + +:::{grid-item-card} 🟑 Get Running Fast +:class-header: bg-warning text-dark +:link: pathway-quickstart +:link-type: doc + +Skip the theory β€” get a model running in 15 minutes. Includes export cheat sheets, backend selection tables, and platform quick starts. + ++++ +**Quick Start Pathway β†’** +::: + +:::{grid-item-card} πŸ”΄ Production & Advanced +:class-header: bg-danger text-white +:link: pathway-advanced +:link-type: doc + +Quantization, custom backends, C++ runtime, LLM deployment, and compiler internals for production-grade systems. + ++++ +**Advanced Pathway β†’** +::: + +:::: + +::::{grid} 1 + +:::{grid-item-card} πŸ”€ Decision Matrix β€” Route by Goal, Platform & Model +:link: user-pathways +:link-type: doc + +Not sure which pathway fits? The decision matrix routes you by experience level, target platform, model status, and developer role to the exact documentation you need. + ++++ +**View Decision Matrix β†’** +::: + +:::: + +--- + ## 🎯 Wins & Success Stories ::::{grid} 1 @@ -186,6 +244,7 @@ Profile, debug, and inspect your models with comprehensive tooling. intro-section quick-start-section +user-pathways edge-platforms-section backends-section llm/working-with-llms diff --git a/docs/source/intro-section.md b/docs/source/intro-section.md index 2f6f3c57c88..d4d7428c4fb 100644 --- a/docs/source/intro-section.md +++ b/docs/source/intro-section.md @@ -6,14 +6,39 @@ Overview, architecture, and core concepts of ExecuTorch. ExecuTorch is PyTorch's solution for training and inference on the Edge, providing portability, productivity, and performance for edge computing platforms. +--- + +## New to ExecuTorch? Start Here + +::::{grid} 2 +:gutter: 2 + +:::{grid-item-card} 🟒 Beginner Pathway +:link: pathway-beginner +:link-type: doc + +A structured, step-by-step learning sequence for engineers new to on-device deployment. Covers concepts, setup, export, and your first device deployment. +::: + +:::{grid-item-card} πŸ”€ Decision Matrix +:link: user-pathways +:link-type: doc + +Route yourself to the right documentation based on your experience level, target platform, model status, and developer role. +::: + +:::: + +--- + ## Getting Started with ExecuTorch New to ExecuTorch? Start with these foundational topics: -- **{doc}`intro-overview`** - High-level overview of ExecuTorch capabilities -- **{doc}`intro-how-it-works`** - Technical overview of the ExecuTorch workflow -- **{doc}`getting-started-architecture`** - System architecture and components -- **{doc}`concepts`** - Core concepts and terminology +- **{doc}`intro-overview`** β€” High-level overview of ExecuTorch capabilities +- **{doc}`intro-how-it-works`** β€” Technical overview of the ExecuTorch workflow +- **{doc}`getting-started-architecture`** β€” System architecture and components +- **{doc}`concepts`** β€” Core concepts and terminology ```{toctree} :hidden: diff --git a/docs/source/pathway-advanced.md b/docs/source/pathway-advanced.md new file mode 100644 index 00000000000..b1b3968844b --- /dev/null +++ b/docs/source/pathway-advanced.md @@ -0,0 +1,490 @@ +(pathway-advanced)= +# Advanced Pathway + +This pathway is for engineers building production-grade deployments, implementing custom backends, optimizing for constrained hardware, or working with large language models on edge devices. It assumes familiarity with the ExecuTorch export pipeline and at least one successful model deployment. + +--- + +## Advanced Topic Areas + +Select the area most relevant to your current work. Each section provides a curated sequence of documentation with dependencies noted. + +--- + +### Quantization and Optimization + +Quantization is the most impactful optimization available in ExecuTorch, reducing model size by 2–8Γ— and improving latency significantly on supported backends. + +::::{grid} 2 +:gutter: 2 + +:::{grid-item-card} Quantization Overview +:link: quantization +:link-type: doc + +Introduction to ExecuTorch's quantization framework, including supported schemes (INT8, INT4, FP16) and the relationship between quantization and backend selection. + +**Difficulty:** Intermediate +::: + +:::{grid-item-card} Quantization & Optimization (Advanced) +:link: quantization-optimization +:link-type: doc + +Advanced quantization techniques including mixed-precision, per-channel quantization, and calibration workflows for production models. + +**Difficulty:** Advanced +::: + +:::{grid-item-card} Model Export and Lowering +:link: using-executorch-export +:link-type: doc + +Full reference for `to_edge_transform_and_lower`, including quantization integration, dynamic shapes, and multi-backend lowering. + +**Difficulty:** Advanced +::: + +:::{grid-item-card} Backend Dialect +:link: compiler-backend-dialect +:link-type: doc + +Understanding the Backend Dialect IR and how it differs from Edge Dialect β€” essential for backend developers and advanced export customization. + +**Difficulty:** Advanced +::: + +:::: + +--- + +### Memory Planning and Runtime Optimization + +Memory planning is critical for constrained devices. ExecuTorch provides ahead-of-time memory planning to eliminate runtime allocations. + +::::{grid} 2 +:gutter: 2 + +:::{grid-item-card} Memory Planning +:link: compiler-memory-planning +:link-type: doc + +How ExecuTorch plans tensor memory at compile time, including memory hierarchy, buffer reuse strategies, and how to customize the planner. + +**Difficulty:** Advanced +::: + +:::{grid-item-card} Memory Planning Inspection +:link: memory-planning-inspection +:link-type: doc + +Tools for inspecting memory plans and diagnosing memory-related issues in exported programs. + +**Difficulty:** Advanced +::: + +:::{grid-item-card} Managing Tensor Memory in C++ +:link: extension-tensor +:link-type: doc + +The `TensorPtr` and `from_blob` APIs for zero-copy tensor management in C++ runtime integrations. + +**Difficulty:** Intermediate +::: + +:::{grid-item-card} Portable C++ Programming +:link: portable-cpp-programming +:link-type: doc + +Guidelines for writing ExecuTorch C++ code that runs on bare-metal and RTOS environments without dynamic allocation or standard library dependencies. + +**Difficulty:** Advanced +::: + +:::: + +--- + +### Custom Compiler Passes and Kernel Registration + +ExecuTorch's compiler pass interface allows you to transform the exported graph before lowering, enabling model-specific optimizations and operator fusion. + +::::{grid} 2 +:gutter: 2 + +:::{grid-item-card} Custom Compiler Passes +:link: compiler-custom-compiler-passes +:link-type: doc + +Writing and registering custom graph transformation passes that run during the export and lowering pipeline. + +**Difficulty:** Advanced +::: + +:::{grid-item-card} Kernel Registration +:link: kernel-library-custom-aten-kernel +:link-type: doc + +Registering custom ATen kernel implementations to replace or supplement the portable operator library with hardware-optimized versions. + +**Difficulty:** Advanced +::: + +:::{grid-item-card} Kernel Library Overview +:link: kernel-library-overview +:link-type: doc + +Architecture of ExecuTorch's kernel library system, including the portable library, custom kernels, and selective build. + +**Difficulty:** Advanced +::: + +:::{grid-item-card} Selective Build +:link: kernel-library-selective-build +:link-type: doc + +Reduce binary size by including only the operators required by your specific model using the selective build system. + +**Difficulty:** Advanced +::: + +:::: + +--- + +### Backend Delegate Development + +Implementing a new hardware backend for ExecuTorch requires understanding the delegate interface, partitioner API, and runtime integration. + +::::{grid} 2 +:gutter: 2 + +:::{grid-item-card} Backend Development Guide +:link: backend-development +:link-type: doc + +Complete guide to implementing a new ExecuTorch backend delegate, including the `BackendInterface`, `preprocess`, and `execute` methods. + +**Difficulty:** Advanced +::: + +:::{grid-item-card} Integrating a Backend Delegate +:link: backend-delegates-integration +:link-type: doc + +Step-by-step walkthrough of integrating an existing backend delegate into the ExecuTorch build system and runtime. + +**Difficulty:** Beginner (for integration) / Advanced (for implementation) +::: + +:::{grid-item-card} Delegate and Partitioner +:link: compiler-delegate-and-partitioner +:link-type: doc + +The `Partitioner` interface for selecting which subgraphs to delegate, including pattern matching and constraint specification. + +**Difficulty:** Advanced +::: + +:::{grid-item-card} Backend Delegate Implementation and Linking +:link: runtime-backend-delegate-implementation-and-linking +:link-type: doc + +Linking backend delegate implementations into the ExecuTorch runtime, including static and dynamic registration patterns. + +**Difficulty:** Advanced +::: + +:::{grid-item-card} Lowering a Model as a Delegate +:link: examples-end-to-end-to-lower-model-to-delegate +:link-type: doc + +End-to-end example of using `to_backend` to lower a model subgraph to a custom delegate. + +**Difficulty:** Advanced +::: + +:::{grid-item-card} Debugging Backend Delegates +:link: debug-backend-delegate +:link-type: doc + +Techniques for debugging delegate execution, including intermediate output comparison and delegate-specific logging. + +**Difficulty:** Advanced +::: + +:::: + +--- + +### C++ Runtime Integration + +For embedded, mobile native, and server deployments, the C++ runtime APIs provide full control over model loading, execution, and memory management. + +::::{grid} 2 +:gutter: 2 + +:::{grid-item-card} Module Extension (High-Level API) +:link: extension-module +:link-type: doc + +The `Module` class provides a high-level C++ API for loading and running `.pte` files with minimal boilerplate. Recommended for most C++ integrations. + +**Difficulty:** Intermediate +::: + +:::{grid-item-card} Detailed C++ Runtime APIs +:link: running-a-model-cpp-tutorial +:link-type: doc + +Low-level C++ runtime APIs for fine-grained control over memory allocation, operator dispatch, and execution planning. Required for bare-metal targets. + +**Difficulty:** Intermediate +::: + +:::{grid-item-card} Using ExecuTorch with C++ +:link: using-executorch-cpp +:link-type: doc + +CMake integration, target linking, cross-compilation setup, and C++ API reference for production deployments. + +**Difficulty:** Advanced +::: + +:::{grid-item-card} Runtime Platform Abstraction Layer +:link: runtime-platform-abstraction-layer +:link-type: doc + +The PAL interface for porting ExecuTorch to new operating systems and bare-metal environments. + +**Difficulty:** Advanced +::: + +:::: + +--- + +### Large Language Models on Edge + +Deploying LLMs to edge devices involves additional complexity around quantization, tokenization, KV-cache management, and platform-specific optimizations. + +::::{grid} 2 +:gutter: 2 + +:::{grid-item-card} LLM Overview +:link: llm/working-with-llms +:link-type: doc + +Complete overview of the ExecuTorch LLM workflow, supported models, and platform-specific deployment paths. + +**Difficulty:** Intermediate +::: + +:::{grid-item-card} Exporting LLMs +:link: llm/export-llm +:link-type: doc + +The `export_llm` module for exporting supported LLMs (Llama, Qwen, Phi, SmolLM) with quantization and optimization. + +**Difficulty:** Intermediate +::: + +:::{grid-item-card} Exporting Custom LLMs +:link: llm/export-custom-llm +:link-type: doc + +Adapting the export pipeline for custom LLM architectures beyond the officially supported models, using nanoGPT as a worked example. + +**Difficulty:** Intermediate +::: + +:::{grid-item-card} Running LLMs with C++ +:link: llm/run-with-c-plus-plus +:link-type: doc + +C++ runtime integration for LLM inference, including tokenizer setup, KV-cache configuration, and streaming output. + +**Difficulty:** Intermediate +::: + +:::{grid-item-card} Llama on Qualcomm Android +:link: llm/build-run-llama3-qualcomm-ai-engine-direct-backend +:link-type: doc + +Deploying Llama 3 3B Instruct on Android using the Qualcomm AI Engine Direct backend with hardware acceleration. + +**Difficulty:** Advanced +::: + +:::{grid-item-card} ExecuTorch on Raspberry Pi +:link: raspberry_pi_llama_tutorial +:link-type: doc + +Deploying Llama models on Raspberry Pi 4/5 edge devices using the ExecuTorch runtime. + +**Difficulty:** Intermediate +::: + +:::: + +--- + +### Developer Tools and Debugging + +ExecuTorch provides a comprehensive suite of profiling and debugging tools for diagnosing performance and correctness issues. + +::::{grid} 2 +:gutter: 2 + +:::{grid-item-card} Developer Tools Overview +:link: devtools-overview +:link-type: doc + +Overview of the ExecuTorch developer tools suite, including ETRecord, ETDump, and the Inspector API. + +**Difficulty:** Intermediate +::: + +:::{grid-item-card} Profiling a Model +:link: devtools-tutorial +:link-type: doc + +Step-by-step tutorial for profiling model execution using ETRecord and ETDump to identify performance bottlenecks. + +**Difficulty:** Intermediate +::: + +:::{grid-item-card} Profiling and Debugging +:link: using-executorch-troubleshooting +:link-type: doc + +Comprehensive debugging guide covering numerical debugging, operator-level profiling, and common failure modes. + +**Difficulty:** Advanced +::: + +:::{grid-item-card} Delegate Debugging +:link: delegate-debugging +:link-type: doc + +Techniques specific to debugging backend delegate execution, including output comparison and delegate-level tracing. + +**Difficulty:** Advanced +::: + +:::: + +--- + +### IR and Compiler Internals + +For contributors and advanced backend developers who need to understand ExecuTorch's compiler internals. + +::::{grid} 2 +:gutter: 2 + +:::{grid-item-card} Export Overview +:link: export-overview +:link-type: doc + +The complete export pipeline from `torch.export` to `.pte`, including the role of each compilation stage. + +**Difficulty:** Intermediate +::: + +:::{grid-item-card} Compiler Entry Points +:link: compiler-entry-points +:link-type: doc + +The public API surface for the ExecuTorch compiler, including `to_edge`, `to_edge_transform_and_lower`, and `to_executorch`. + +**Difficulty:** Intermediate +::: + +:::{grid-item-card} IR Specification +:link: ir-specification +:link-type: doc + +Formal specification of the ExecuTorch IR, including operator semantics, type system, and serialization format. + +**Difficulty:** Advanced +::: + +:::{grid-item-card} Compiler & IR (Advanced) +:link: compiler-ir-advanced +:link-type: doc + +Advanced IR topics including graph transformations, custom dialects, and the relationship between Export IR and Edge Dialect. + +**Difficulty:** Advanced +::: + +:::: + +--- + +## Contributing to ExecuTorch + +If you are working on ExecuTorch internals or want to contribute upstream, start with the contributor guide. + +::::{grid} 2 +:gutter: 2 + +:::{grid-item-card} New Contributor Guide +:link: new-contributor-guide +:link-type: doc + +Development environment setup, code style, testing requirements, and the pull request process for ExecuTorch contributors. + +**Difficulty:** Advanced +::: + +:::{grid-item-card} API Life Cycle and Deprecation Policy +:link: api-life-cycle +:link-type: doc + +How ExecuTorch manages API stability, deprecation timelines, and backward compatibility across releases. + +**Difficulty:** Intermediate +::: + +:::: + +--- + +## Advanced Learning Sequence + +If you prefer a structured progression rather than topic-based navigation, follow this sequence for a comprehensive advanced curriculum. + +```{list-table} +:header-rows: 1 +:widths: 10 30 60 + +* - **Order** + - **Topic** + - **Goal** +* - 1 + - {doc}`export-overview` + - Understand the full compilation pipeline +* - 2 + - {doc}`using-executorch-export` + - Master advanced export options +* - 3 + - {doc}`quantization-optimization` + - Apply production-grade quantization +* - 4 + - {doc}`compiler-memory-planning` + - Optimize memory for constrained devices +* - 5 + - {doc}`compiler-custom-compiler-passes` + - Write custom graph transformations +* - 6 + - {doc}`backend-development` + - Implement a custom backend delegate +* - 7 + - {doc}`running-a-model-cpp-tutorial` + - Master the low-level C++ runtime +* - 8 + - {doc}`devtools-tutorial` + - Profile and debug production models +``` diff --git a/docs/source/pathway-beginner.md b/docs/source/pathway-beginner.md new file mode 100644 index 00000000000..264dff4dca1 --- /dev/null +++ b/docs/source/pathway-beginner.md @@ -0,0 +1,244 @@ +(pathway-beginner)= +# Beginner Pathway + +**Welcome to ExecuTorch.** This pathway is designed for engineers who are comfortable with PyTorch but are new to on-device deployment. You will follow a structured, step-by-step sequence that builds foundational knowledge before introducing more complex topics. + +**Estimated time to complete:** 2–4 hours for the core sequence. Individual steps can be done independently. + +--- + +## What You Will Learn + +By following this pathway, you will be able to: + +1. Understand what ExecuTorch is and why it exists +2. Install ExecuTorch and set up your development environment +3. Export a PyTorch model to the `.pte` format +4. Run inference using the Python runtime +5. Deploy a model to Android or iOS +6. Know where to go next based on your use case + +--- + +## Core Learning Sequence + +Work through these steps in order. Each step builds on the previous one. + +### Step 1 β€” Understand ExecuTorch (15 min) + +Before writing any code, read the conceptual overview to understand the ExecuTorch workflow and its key benefits. + +::::{grid} 2 +:gutter: 2 + +:::{grid-item-card} Overview of ExecuTorch +:link: intro-overview +:link-type: doc + +High-level introduction to ExecuTorch's purpose, design principles, and where it fits in the PyTorch ecosystem. + +**Difficulty:** Beginner +::: + +:::{grid-item-card} How ExecuTorch Works +:link: intro-how-it-works +:link-type: doc + +A technical walkthrough of the three-stage pipeline: export, compilation, and runtime execution. + +**Difficulty:** Beginner +::: + +:::: + +--- + +### Step 2 β€” Set Up Your Environment (20 min) + +Install ExecuTorch and verify your setup before attempting to export a model. + +::::{grid} 1 + +:::{grid-item-card} Getting Started with ExecuTorch +:link: getting-started +:link-type: doc + +Install the ExecuTorch Python package, export a MobileNet V2 model using XNNPACK, and run your first inference. This is the canonical entry point for all new users. + +**Difficulty:** Beginner | **Prerequisites:** Python 3.10–3.13, PyTorch, g++7+ or clang5+ +::: + +:::: + +> **Tip:** If you encounter build errors or platform-specific issues during installation, consult the {doc}`using-executorch-faqs` page before proceeding. + +--- + +### Step 3 β€” Understand Core Concepts (20 min) + +A brief review of the key concepts and terminology used throughout ExecuTorch documentation. + +::::{grid} 1 + +:::{grid-item-card} Core Concepts and Terminology +:link: concepts +:link-type: doc + +Definitions for Export IR, Edge Dialect, delegates, partitioners, `.pte` files, and other ExecuTorch-specific terms you will encounter throughout the documentation. + +**Difficulty:** Beginner +::: + +:::: + +--- + +### Step 4 β€” Export Your First Model (30 min) + +Learn the standard export workflow using `torch.export` and `to_edge_transform_and_lower`. + +::::{grid} 2 +:gutter: 2 + +:::{grid-item-card} Model Export and Lowering +:link: using-executorch-export +:link-type: doc + +The complete guide to exporting a PyTorch model for ExecuTorch, including backend selection, quantization basics, and handling dynamic shapes. + +**Difficulty:** Intermediate | **Builds on:** Step 2 +::: + +:::{grid-item-card} Visualize Your Model +:link: visualize +:link-type: doc + +Use ModelExplorer to inspect your exported model graph and verify the export result before deployment. + +**Difficulty:** Beginner +::: + +:::: + +--- + +### Step 5 β€” Deploy to Your Target Platform (30–60 min) + +Choose the platform you are targeting and follow the appropriate guide. + +::::{grid} 3 +:gutter: 2 + +:::{grid-item-card} πŸ€– Android +:link: android-section +:link-type: doc + +Integrate ExecuTorch into an Android app using the Java/Kotlin bindings. Includes Gradle dependency setup and the `Module` API. + +**Difficulty:** Intermediate +::: + +:::{grid-item-card} 🍎 iOS +:link: ios-section +:link-type: doc + +Add ExecuTorch to an iOS or macOS project via Swift Package Manager. Covers Objective-C and Swift integration. + +**Difficulty:** Intermediate +::: + +:::{grid-item-card} πŸ’» Desktop / Python +:link: getting-started +:link-type: doc + +Run inference directly from Python using the ExecuTorch runtime bindings β€” the fastest way to validate a model before mobile deployment. + +**Difficulty:** Beginner +::: + +:::: + +--- + +### Step 6 β€” Explore a Complete Example (optional, 30 min) + +Seeing a complete end-to-end example reinforces the concepts from the previous steps. + +::::{grid} 2 +:gutter: 2 + +:::{grid-item-card} Pico2: MNIST on a Microcontroller +:link: pico2_tutorial +:link-type: doc + +A self-contained tutorial that exports an MNIST model and runs it on a Raspberry Pi Pico2. Excellent for understanding the full pipeline on constrained hardware. + +**Difficulty:** Beginner (hardware required) +::: + +:::{grid-item-card} MobileNet V2 β€” Colab Notebook +:link: https://colab.research.google.com/drive/1qpxrXC3YdJQzly3mRg-4ayYiOjC6rue3?usp=sharing +:link-type: url + +An interactive Colab notebook covering the complete export, lowering, and verification workflow for MobileNet V2. No local setup required. + +**Difficulty:** Beginner +::: + +:::: + +--- + +## Frequently Encountered Issues + +New users commonly encounter the following issues. Consult these resources before opening a support request. + +```{list-table} +:header-rows: 1 +:widths: 40 60 + +* - **Issue** + - **Resource** +* - Installation fails or package not found + - {doc}`using-executorch-faqs` β€” Installation section +* - Export fails with unsupported operator error + - {doc}`using-executorch-export` β€” Operator support section +* - Model produces incorrect output after export + - {doc}`devtools-tutorial` β€” Numerical debugging +* - Build errors on Windows + - {doc}`getting-started` β€” Windows prerequisites note +* - Backend not accelerating as expected + - {doc}`backends-overview` β€” Backend selection guide +``` + +--- + +## Where to Go Next + +Once you have completed the core sequence, choose your next direction based on your use case. + +::::{grid} 3 +:gutter: 2 + +:::{grid-item-card} Work with LLMs +:link: llm/working-with-llms +:link-type: doc + +Export and deploy Llama, Phi, Qwen, and other LLMs to mobile and edge devices. +::: + +:::{grid-item-card} Hardware Acceleration +:link: backends-overview +:link-type: doc + +Use XNNPACK, Core ML, Qualcomm, Vulkan, and other backends for hardware-accelerated inference. +::: + +:::{grid-item-card} Advanced Topics +:link: pathway-advanced +:link-type: doc + +Quantization, memory planning, custom compiler passes, and backend development. +::: + +:::: diff --git a/docs/source/pathway-quickstart.md b/docs/source/pathway-quickstart.md new file mode 100644 index 00000000000..9272a6161ec --- /dev/null +++ b/docs/source/pathway-quickstart.md @@ -0,0 +1,263 @@ +(pathway-quickstart)= +# Quick Start Pathway + +This pathway is for engineers who want to get a model running on a device as quickly as possible. It assumes you are familiar with PyTorch model development and have some prior exposure to mobile or edge deployment concepts. Steps are kept concise and link directly to the most actionable documentation. + +**Estimated time to first inference:** 15–30 minutes. + +--- + +## Choose Your Scenario + +Select the scenario that most closely matches what you are trying to accomplish right now. + +::::{grid} 2 +:gutter: 3 + +:::{grid-item-card} πŸš€ I have a PyTorch model and want to run it on device +:class-header: bg-primary text-white + +**Fastest path: Export β†’ Run** + +1. Install: `pip install executorch` +2. Export with {doc}`getting-started` (Exporting section) +3. Run with Python runtime or deploy to {doc}`android-section` / {doc}`ios-section` + +**Time:** ~15 min +::: + +:::{grid-item-card} πŸ“¦ I want to use a pre-exported model +:class-header: bg-primary text-white + +**Fastest path: Download β†’ Run** + +Pre-exported `.pte` files for Llama 3.2, MobileNet, and other models are available on [HuggingFace ExecuTorch Community](https://huggingface.co/executorch-community). + +Skip export entirely and go directly to the runtime section of {doc}`getting-started`. + +**Time:** ~10 min +::: + +:::{grid-item-card} πŸ€— I have a HuggingFace model +:class-header: bg-primary text-white + +**Fastest path: Optimum ExecuTorch** + +Use the `optimum-executorch` CLI for a one-command export of HuggingFace models. + +See {doc}`llm/export-llm-optimum` for installation and usage. + +**Time:** ~20 min +::: + +:::{grid-item-card} πŸ¦™ I want to run Llama on my phone +:class-header: bg-primary text-white + +**Fastest path: Llama on ExecuTorch** + +Follow the {doc}`llm/llama` guide for the complete Llama export and deployment workflow, including quantization and platform-specific setup. + +**Time:** ~45 min (model download included) +::: + +:::: + +--- + +## The 5-Minute Setup + +If you have not yet installed ExecuTorch, run the following in a Python 3.10–3.13 virtual environment: + +```bash +pip install executorch +``` + +Then verify the installation with a minimal export: + +```python +import torch +from executorch.exir import to_edge_transform_and_lower +from executorch.backends.xnnpack.partition.xnnpack_partitioner import XnnpackPartitioner + +# Define a simple model +class Add(torch.nn.Module): + def forward(self, x, y): + return x + y + +model = Add() +sample_inputs = (torch.ones(1), torch.ones(1)) + +et_program = to_edge_transform_and_lower( + torch.export.export(model, sample_inputs), + partitioner=[XnnpackPartitioner()] +).to_executorch() + +with open("add.pte", "wb") as f: + f.write(et_program.buffer) + +print("Export successful: add.pte created") +``` + +If this runs without error, your environment is correctly configured. + +--- + +## Quick Reference: Export Cheat Sheet + +```{list-table} +:header-rows: 1 +:widths: 30 70 + +* - **Task** + - **Code / Command** +* - Install ExecuTorch + - `pip install executorch` +* - Export with XNNPACK (mobile CPU) + - `to_edge_transform_and_lower(torch.export.export(model, inputs), partitioner=[XnnpackPartitioner()])` +* - Export with Core ML (iOS) + - Replace `XnnpackPartitioner` with `CoreMLPartitioner` β€” see {doc}`ios-coreml` +* - Export with Qualcomm (Android NPU) + - See {doc}`android-qualcomm` for QNN SDK setup and partitioner usage +* - Run from Python + - `Runtime.get().load_program("model.pte").load_method("forward").execute([input])` +* - Run from C++ + - See {doc}`extension-module` for the high-level `Module` API +* - Export an LLM + - `python -m executorch.examples.models.llama.export_llm ...` β€” see {doc}`llm/export-llm` +``` + +--- + +## Platform Quick Start Guides + +Jump directly to the platform-specific setup guide for your target. + +::::{grid} 2 +:gutter: 2 + +:::{grid-item-card} Android Quick Start +:link: android-section +:link-type: doc + +Gradle dependency, Java `Module` API, and XNNPACK / Vulkan / Qualcomm backend selection for Android. +::: + +:::{grid-item-card} iOS Quick Start +:link: ios-section +:link-type: doc + +Swift Package Manager setup, Objective-C runtime API, and Core ML / MPS / XNNPACK backend selection for iOS. +::: + +:::{grid-item-card} Desktop / Linux / macOS +:link: desktop-section +:link-type: doc + +Python runtime, C++ CMake integration, and XNNPACK / Core ML / MPS backends for desktop platforms. +::: + +:::{grid-item-card} Embedded Systems +:link: embedded-section +:link-type: doc + +Bare-metal and RTOS deployment, Arm Ethos-U, Cadence, NXP, and other embedded backends. +::: + +:::: + +--- + +## Backend Selection Guide + +Choosing the right backend has the largest impact on performance. Use this table to select the appropriate backend for your hardware. + +```{list-table} Backend Selection by Platform and Hardware +:header-rows: 1 +:widths: 20 20 20 40 + +* - **Platform** + - **Hardware Target** + - **Backend** + - **Documentation** +* - Android + - CPU (Arm/x86) + - XNNPACK + - {doc}`android-xnnpack` +* - Android + - GPU (Vulkan) + - Vulkan + - {doc}`android-vulkan` +* - Android + - Qualcomm NPU/DSP + - QNN + - {doc}`android-qualcomm` +* - Android + - MediaTek APU + - MediaTek + - {doc}`android-mediatek` +* - iOS / macOS + - Neural Engine / GPU + - Core ML + - {doc}`ios-coreml` +* - iOS / macOS + - Metal GPU + - MPS + - {doc}`ios-mps` +* - iOS / macOS + - CPU (Arm) + - XNNPACK + - {doc}`ios-xnnpack` +* - Desktop + - Intel CPU/GPU/NPU + - OpenVINO + - {doc}`desktop-openvino` +* - Desktop + - Apple Silicon + - Core ML / MPS + - {doc}`desktop-coreml` +* - Embedded + - Arm Cortex-M / Ethos-U + - Arm Ethos-U + - {doc}`embedded-arm-ethos-u` +* - Embedded + - Cadence DSP + - Cadence + - {doc}`embedded-cadence` +* - Embedded + - NXP eIQ Neutron + - NXP + - {doc}`embedded-nxp` +``` + +--- + +## Troubleshooting Quick Fixes + +```{list-table} +:header-rows: 1 +:widths: 40 60 + +* - **Symptom** + - **Quick Fix** +* - `ImportError: No module named executorch` + - Run `pip install executorch` in your active virtual environment +* - Export fails with `torch._dynamo` error + - Ensure your model is `export`-compatible; see {doc}`export-overview` +* - `.pte` file runs but produces wrong output + - Use {doc}`devtools-tutorial` to compare intermediate activations +* - Android Gradle sync fails + - Check `executorch_version` in `build.gradle.kts` matches your installed version +* - iOS build fails with missing xcframework + - Verify the Swift PM branch name matches your ExecuTorch version (format: `swiftpm-X.Y.Z`) +``` + +--- + +## Going Deeper + +Once your model is running, explore these topics to optimize performance and expand capabilities. + +- {doc}`quantization` β€” Reduce model size and improve latency with INT8/INT4 quantization +- {doc}`using-executorch-troubleshooting` β€” Profiling and debugging tools +- {doc}`using-executorch-export` β€” Advanced export options including dynamic shapes +- {doc}`pathway-advanced` β€” Full advanced user pathway for production deployments diff --git a/docs/source/quick-start-section.md b/docs/source/quick-start-section.md index b6940d2acef..625eaeb9c20 100644 --- a/docs/source/quick-start-section.md +++ b/docs/source/quick-start-section.md @@ -5,19 +5,54 @@ Get started with ExecuTorch in just a few steps. This section walks you through the essential steps to get ExecuTorch up and running, from initial setup to exporting your first model for edge deployment. +--- + +## Choose Your Starting Point + +::::{grid} 3 +:gutter: 2 + +:::{grid-item-card} 🟒 First Time Here? +:class-header: bg-success text-white +:link: pathway-beginner +:link-type: doc + +Follow the structured **Beginner Pathway** for a guided, step-by-step introduction to ExecuTorch β€” from concepts to your first deployment. +::: + +:::{grid-item-card} 🟑 Want to Move Fast? +:class-header: bg-warning text-dark +:link: pathway-quickstart +:link-type: doc + +Jump to the **Quick Start Pathway** for a 15-minute path to running a model, with export cheat sheets and backend selection tables. +::: + +:::{grid-item-card} πŸ”€ Not Sure? +:class-header: bg-info text-white +:link: user-pathways +:link-type: doc + +Use the **Decision Matrix** to find the right path based on your experience, platform, and model type. +::: + +:::: + +--- + ## What You'll Learn Follow these guides in order to get started with ExecuTorch: -- **{doc}`getting-started`** - Initial Setup: Set up your development environment and run your first ExecuTorch example. +- **{doc}`getting-started`** β€” Initial Setup: Set up your development environment and run your first ExecuTorch example. -- **{doc}`using-executorch-export`** - Exporting your model: Export for Edge deployment. +- **{doc}`using-executorch-export`** β€” Exporting your model: Export for Edge deployment. -- **{doc}`using-executorch-building-from-source`** - Building from Source: Build ExecuTorch from source for custom configurations and development. +- **{doc}`using-executorch-building-from-source`** β€” Building from Source: Build ExecuTorch from source for custom configurations and development. ## Prerequisites -- Python 3.10-3.13 +- Python 3.10–3.13 - PyTorch 2.9+ - Basic familiarity with PyTorch model development @@ -25,8 +60,9 @@ Follow these guides in order to get started with ExecuTorch: After completing the quick start, explore: -- **{doc}`edge-platforms-section`** - Deploy to specific platforms (Android, iOS, Desktop, Embedded) -- **{doc}`backends-section`** - Choose the right acceleration backend for your hardware +- **{doc}`edge-platforms-section`** β€” Deploy to specific platforms (Android, iOS, Desktop, Embedded) +- **{doc}`backends-section`** β€” Choose the right acceleration backend for your hardware +- **{doc}`pathway-advanced`** β€” Advanced topics: quantization, custom backends, LLM deployment ```{toctree} :hidden: diff --git a/docs/source/user-pathways.md b/docs/source/user-pathways.md new file mode 100644 index 00000000000..074ce4ed123 --- /dev/null +++ b/docs/source/user-pathways.md @@ -0,0 +1,207 @@ +(user-pathways)= +# Find Your Path in ExecuTorch + +**ExecuTorch** serves a wide range of users β€” from ML engineers taking their first steps in on-device inference, to embedded systems developers targeting bare-metal microcontrollers, to researchers pushing the boundaries of LLM deployment. This page helps you navigate directly to the content most relevant to your experience level, goals, and target platform. + +--- + +## Step 1: What best describes your experience? + +::::{grid} 3 +:gutter: 3 + +:::{grid-item-card} 🟒 New to ExecuTorch +:class-header: bg-success text-white +:link: pathway-beginner +:link-type: doc + +**Beginner** + +You are familiar with PyTorch but have not yet deployed a model to an edge device. You want a clear, guided path from installation to your first on-device inference. + ++++ +β†’ **Beginner Pathway** +::: + +:::{grid-item-card} 🟑 I know the basics +:class-header: bg-warning text-dark +:link: pathway-quickstart +:link-type: doc + +**Quick Start** + +You have some experience with model export or mobile ML, and want to get a model running as fast as possible without reading through every concept first. + ++++ +β†’ **Quick Start Pathway** +::: + +:::{grid-item-card} πŸ”΄ Experienced / Production +:class-header: bg-danger text-white +:link: pathway-advanced +:link-type: doc + +**Advanced** + +You are building production systems, integrating custom backends, optimizing for constrained hardware, or working with LLMs on edge devices. + ++++ +β†’ **Advanced Pathway** +::: + +:::: + +--- + +## Step 2: What is your primary goal? + +Use the decision matrix below to jump directly to the most relevant section based on your goal and target platform. + +```{list-table} ExecuTorch Decision Matrix +:header-rows: 1 +:widths: 25 20 20 20 15 + +* - **Goal** + - **Android** + - **iOS / macOS** + - **Desktop / Server** + - **Embedded / MCU** +* - Run a pre-exported model quickly + - {doc}`android-section` + - {doc}`ios-section` + - {doc}`getting-started` + - {doc}`embedded-section` +* - Export my own PyTorch model + - {doc}`using-executorch-export` + - {doc}`using-executorch-export` + - {doc}`getting-started` + - {doc}`using-executorch-export` +* - Deploy an LLM (Llama, Phi, etc.) + - {doc}`llm/llama` + - {doc}`llm/run-on-ios` + - {doc}`llm/working-with-llms` + - β€” +* - Use hardware acceleration (NPU/GPU) + - {doc}`android-qualcomm` + - {doc}`ios-coreml` + - {doc}`desktop-backends` + - {doc}`embedded-backends` +* - Integrate a custom backend delegate + - {doc}`backend-development` + - {doc}`backend-development` + - {doc}`backend-development` + - {doc}`backend-development` +* - Profile and debug my model + - {doc}`devtools-tutorial` + - {doc}`devtools-tutorial` + - {doc}`devtools-tutorial` + - {doc}`devtools-tutorial` +* - Build from source + - {doc}`using-executorch-building-from-source` + - {doc}`using-executorch-building-from-source` + - {doc}`using-executorch-building-from-source` + - {doc}`using-executorch-building-from-source` +``` + +--- + +## Step 3: What is your role? + +Different roles have different entry points into ExecuTorch. Select the one that best matches your background. + +::::{grid} 2 +:gutter: 3 + +:::{grid-item-card} πŸ€– ML Engineer +:class-header: bg-primary text-white + +You work primarily in Python, train models with PyTorch, and want to deploy them efficiently to edge devices. + +**Start here:** +- {doc}`getting-started` β€” Installation and first export +- {doc}`using-executorch-export` β€” Model export and lowering +- {doc}`backends-overview` β€” Choose the right backend +- {doc}`quantization` β€” Reduce model size and latency +::: + +:::{grid-item-card} πŸ“± Mobile Developer +:class-header: bg-primary text-white + +You build Android or iOS applications and need to integrate an on-device ML model into your app. + +**Start here:** +- {doc}`android-section` β€” Android integration guide +- {doc}`ios-section` β€” iOS integration guide +- {doc}`getting-started` β€” Export a model for your platform +- {doc}`using-executorch-android` β€” Full Android API reference +::: + +:::{grid-item-card} βš™οΈ Backend / Systems Developer +:class-header: bg-primary text-white + +You are implementing a hardware backend, writing C++ runtime integrations, or contributing to ExecuTorch internals. + +**Start here:** +- {doc}`backend-development` β€” Backend delegate development +- {doc}`backend-delegates-integration` β€” Integrating a backend +- {doc}`running-a-model-cpp-tutorial` β€” C++ runtime APIs +- {doc}`new-contributor-guide` β€” Contributing to ExecuTorch +::: + +:::{grid-item-card} πŸ”Œ Embedded Developer +:class-header: bg-primary text-white + +You target microcontrollers, DSPs, or other resource-constrained hardware where memory and compute are tightly limited. + +**Start here:** +- {doc}`embedded-section` β€” Embedded platforms overview +- {doc}`embedded-backends` β€” Available embedded backends +- {doc}`portable-cpp-programming` β€” Portable C++ for constrained devices +- {doc}`pico2_tutorial` β€” MNIST on Raspberry Pi Pico2 +::: + +:::: + +--- + +## Step 4: What is your model's status? + +The right workflow depends on whether you are starting from scratch, using a supported model, or working with a custom architecture. + +```{list-table} Model Status Routing +:header-rows: 1 +:widths: 30 70 + +* - **Model Status** + - **Recommended Path** +* - Using a supported LLM (Llama, Phi, Qwen, SmolLM) + - Use the {doc}`llm/export-llm` script for a streamlined export with quantization and optimization built in. Pre-exported models are also available on `HuggingFace ExecuTorch Community `_. +* - Using a HuggingFace model + - Use {doc}`llm/export-llm-optimum` (Optimum ExecuTorch) for broad HuggingFace model support with familiar APIs. +* - Using a custom PyTorch model + - Follow {doc}`getting-started` for the standard export flow, then consult {doc}`using-executorch-export` for advanced lowering options. +* - Model requires dynamic shapes + - See the dynamic shapes section in {doc}`using-executorch-export` and the {doc}`export-overview` for constraints. +* - Model uses unsupported operators + - Consult {doc}`kernel-library-custom-aten-kernel` to register custom kernels, or {doc}`compiler-custom-compiler-passes` for graph-level transformations. +* - Pre-exported `.pte` file available + - Skip export entirely and go directly to {doc}`getting-started` (Running on Device section) or your platform guide. +``` + +--- + +## Not sure where to start? + +If you are completely new to ExecuTorch, the recommended entry point is the **{doc}`getting-started`** guide, which walks through installation, exporting a MobileNet V2 model, and running inference in under 15 minutes. + +For a high-level conceptual overview before diving into code, read {doc}`intro-overview` and {doc}`intro-how-it-works`. + +```{toctree} +:hidden: +:maxdepth: 1 +:caption: User Pathways + +pathway-beginner +pathway-quickstart +pathway-advanced +```