From 5897e7841a67e1edede4782b3a89b3f86fee8850 Mon Sep 17 00:00:00 2001 From: Wei Zhou Date: Tue, 12 May 2026 11:30:10 +0200 Subject: [PATCH 1/2] Doc for network extension --- source/adminguide/extensions.rst | 25 +++++- .../adminguide/extensions/custom_actions.rst | 19 ++++- source/adminguide/extensions/developer.rst | 79 ++++++++++++++++--- .../adminguide/extensions/troubleshooting.rst | 28 +++++++ 4 files changed, 138 insertions(+), 13 deletions(-) diff --git a/source/adminguide/extensions.rst b/source/adminguide/extensions.rst index 2c6b7308e0..692a548528 100644 --- a/source/adminguide/extensions.rst +++ b/source/adminguide/extensions.rst @@ -17,7 +17,7 @@ Extensions ========== -Extensions are a new mechanism introduced in Apache CloudStack to allow administrators to extend the platform's functionality by integrating external systems or custom workflows. Currently, CloudStack supports a single extension type called Orchestrator. +Extensions are a new mechanism introduced in Apache CloudStack to allow administrators to extend the platform's functionality by integrating external systems or custom workflows. Currently, CloudStack supports two extension types: Orchestrator and NetworkOrchestrator. In the UI, extensions can be managed under *Extensions* menu. @@ -26,7 +26,7 @@ In the UI, extensions can be managed under *Extensions* menu. Overview ^^^^^^^^ -An extension in CloudStack is defined as an external binary (written in any programming language) that implements specific actions CloudStack can invoke. This allows operators to manage resource lifecycle operations outside CloudStack, such as provisioning VMs in third-party systems or triggering external automation pipelines. +An extension in CloudStack is defined as an external binary (written in any programming language) that implements specific actions CloudStack can invoke. This allows operators to manage resource lifecycle operations outside CloudStack, such as provisioning VMs in third-party systems, orchestrating network and VPC services on external devices, or triggering external automation pipelines. Extensions are managed through the API and UI, with support for configuration, resource mappings, and action execution. @@ -41,7 +41,7 @@ Administrators can define and manage the following components of an extension: - Configuration Details: Key-value properties used by the extension at runtime. - - Resource Mappings: Association between extensions and CloudStack resources such as clusters, etc. + - Resource Mappings: Association between extensions and CloudStack resources such as clusters and physical networks. Path and Availabilty ^^^^^^^^^^^^^^^^^^^^ @@ -80,6 +80,25 @@ An Orchestrator extension enables CloudStack to delegate VM orchestration to an |extension.png| +NetworkOrchestrator Extension +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +A NetworkOrchestrator extension enables CloudStack to delegate guest network and VPC service orchestration to an external network system. Key features include: + + - Physical Network Mapping: NetworkOrchestrator extensions are registered with a CloudStack physical network instead of a cluster. + + - Provider-based Integration: When a NetworkOrchestrator extension is registered with a physical network, CloudStack creates an external network service provider using the extension name. Network and VPC offerings can then use that provider. + + - Capability-driven Services: Supported services are declared through the extension details ``network.services`` and optional per-service capabilities in ``network.service.capabilities``. CloudStack uses these declarations when exposing supported services and validating offering capabilities. + + - Network and VPC Lifecycle: Depending on the declared services, the extension can handle operations for guest networks, VPCs, public IPs, NAT, load balancing, DHCP, DNS, userdata, network ACLs, and related restart or reapply flows. + + - Registration Details: Resource-specific details such as device endpoints, credentials, host lists, or interface mappings can be stored on the physical-network registration and updated later through the UI or the ``updateRegisteredExtension`` API. + + - Network and VPC Custom Actions: Admins can define custom actions for ``Network`` and ``Vpc`` resources when the extension advertises the ``CustomAction`` service. + + - Reference Implementation: A Linux network namespace based implementation is available in `cloudstack-extensions `_. This reference backend has been validated with KVM-based smoke tests. + CloudStack provides built-in Orchestrator Extensions for Proxmox, Hyper-V, and MaaS, which work with their respective environments out of the box. diff --git a/source/adminguide/extensions/custom_actions.rst b/source/adminguide/extensions/custom_actions.rst index c250019678..504395892b 100644 --- a/source/adminguide/extensions/custom_actions.rst +++ b/source/adminguide/extensions/custom_actions.rst @@ -17,7 +17,9 @@ Custom Actions ^^^^^^^^^^^^^^ -In addition to standard instance operations, extensions support custom actions. These can be configured via UI in the extension details view or the addCustomAction API. The extension binary or script must implement handlers for these action names and process any provided parameters. +In addition to standard lifecycle operations, extensions support custom actions. These can be configured via UI in the extension details view or the addCustomAction API. The extension binary or script must implement handlers for these action names and process any provided parameters. + +For Orchestrator extensions, custom actions typically target ``VirtualMachine`` resources. For NetworkOrchestrator extensions, custom actions can target ``Network`` and ``Vpc`` resources when the extension advertises the ``CustomAction`` network service. |add-custom-action.png| @@ -43,12 +45,27 @@ A single parameter can have the following details: - **valueoptions**: Options for the value of the parameter. This is allowed only for NUMBER and STRING type. +Supported Resource Types +~~~~~~~~~~~~~~~~~~~~~~~~ + +Custom actions can be attached to the following resource types: + + - ``VirtualMachine`` for Orchestrator extensions. + + - ``Network`` for NetworkOrchestrator extensions. + + - ``Vpc`` for NetworkOrchestrator extensions. + +For network and VPC custom actions, CloudStack dispatches the action to the external provider that serves the ``CustomAction`` service for the selected resource. + Running Custom Action ~~~~~~~~~~~~~~~~~~~~~ All enabled custom actions can then be triggered for a resource of the type the action is defined for or provided while running, using the **Run Action** view or runCustomAction API. +The same API is used for VM, network, and VPC custom actions. The available actions shown in the UI depend on the selected resource type and the extension bound to that resource. + |run-custom-action-instance.png| |run-custom-action.png| diff --git a/source/adminguide/extensions/developer.rst b/source/adminguide/extensions/developer.rst index cd9bc976b4..b9088bf84d 100644 --- a/source/adminguide/extensions/developer.rst +++ b/source/adminguide/extensions/developer.rst @@ -16,7 +16,18 @@ Writing Extensions for CloudStack ================================= -The CloudStack Extensions Framework allows developers and operators to write extensions using any programming language or script. From CloudStack’s perspective, an extension is simply an executable capable of handling specific actions and processing input payloads. CloudStack invokes the executable by passing the action name and the path to a JSON-formatted payload file as command-line arguments. The extension processes the payload, performs the required operations on an external system, and returns the result as a JSON response written to `stdout`. +The CloudStack Extensions Framework allows developers and operators to write extensions using any programming language or script. From CloudStack’s perspective, an extension is an executable capable of handling CloudStack actions and integrating with an external system. + +Extension Types +^^^^^^^^^^^^^^^ + +CloudStack currently supports two extension types: + +- ``Orchestrator`` for external instance lifecycle management. These extensions are registered with ``Cluster`` resources. + +- ``NetworkOrchestrator`` for external guest network and VPC service orchestration. These extensions are registered with ``PhysicalNetwork`` resources. + +Both types share the same extension lifecycle in the UI and API, but their invocation model and supported resource mappings differ. Create a New Extension @@ -26,11 +37,12 @@ You must first register a new extension using the API or UI: .. code-block:: bash - cloudmonkey createExtension name=myext path=myext-executable + cloudmonkey createExtension name=myext type=Orchestrator path=myext-executable Arguments: - ``name``: Unique name +- ``type``: ``Orchestrator`` or ``NetworkOrchestrator`` - ``path``: Relative path to the executable. Root path will be `/usr/share/cloudstack-management/extensions/` The path must be: @@ -43,10 +55,28 @@ If no explicit path is provided during extension creation, CloudStack will scaff CloudStack checks extension readiness periodically and shows its state in the UI/API. -Extension Structure -^^^^^^^^^^^^^^^^^^^ +For ``NetworkOrchestrator`` extensions, define supported services in the extension detail ``network.services`` and optionally declare per-service capabilities in ``network.service.capabilities``. CloudStack uses these values when exposing supported providers and validating network and VPC offerings. + +Register Extension With a Resource +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +After creating an extension, register it with the CloudStack resource it serves: + +- ``Orchestrator`` extensions are registered with ``Cluster`` resources. + +- ``NetworkOrchestrator`` extensions are registered with ``PhysicalNetwork`` resources. + +Resource-level details supplied during registration are useful for endpoints, credentials, host lists, interface mappings, or other device-specific settings. These registration details can later be changed with the ``updateRegisteredExtension`` API without removing the existing mapping. + +When a ``NetworkOrchestrator`` extension is registered with a physical network, CloudStack creates a network service provider using the extension name. Network and VPC offerings can then use that provider name. + +Invocation Model +^^^^^^^^^^^^^^^^ + +Orchestrator Invocation +~~~~~~~~~~~~~~~~~~~~~~~ -Your extension must support the following invocation structure: +Orchestrator extensions must support the following invocation structure: .. code-block:: bash @@ -64,10 +94,23 @@ Sample Invocation: /usr/share/cloudstack-management/extensions/myext/myext.py deploy /var/lib/cloudstack/management/extensions/myext/162345.json 60 +NetworkOrchestrator Invocation +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +NetworkOrchestrator extensions use service-specific command names and named CLI arguments instead of the payload-file format above. Depending on the declared services, CloudStack can invoke commands such as ``ensure-network-device``, ``implement-network``, ``shutdown-network``, ``destroy-network``, ``assign-ip``, ``add-static-nat``, ``add-port-forward``, ``apply-fw-rules``, ``apply-lb-rules``, ``apply-network-acl``, ``implement-vpc``, ``shutdown-vpc``, or ``custom-action``. + +CloudStack always passes registration and runtime state using named arguments such as: + +- ``--physical-network-extension-details`` for details stored on the physical-network registration. + +- ``--network-extension-details`` for per-network or per-VPC state maintained by the extension. + +Some commands may also receive temporary file paths for larger payloads. + Input Format (Payload) ^^^^^^^^^^^^^^^^^^^^^^ -CloudStack provides input via a JSON file, which your executable must read and parse. +For Orchestrator extensions, CloudStack provides input via a JSON file, which your executable must read and parse. Example: @@ -107,10 +150,12 @@ Example: The schema varies depending on the resource and action. Use this to perform context-specific logic. +For NetworkOrchestrator extensions, CloudStack passes command-specific CLI arguments such as network IDs, VPC IDs, VLAN tags, IP addresses, action parameters, and JSON strings in named options. The exact input depends on the command being invoked and the services declared in ``network.services``. + Output Format ^^^^^^^^^^^^^ -Your extension should write a response JSON to ``stdout``. Example: +Orchestrator extensions should write a response JSON to ``stdout``. Example: .. code-block:: json @@ -122,11 +167,20 @@ Your extension should write a response JSON to ``stdout``. Example: For custom actions, CloudStack will display the ``message`` in the UI if the output JSON includes ``"printmessage": "true"``. The ``message`` field can be a string, a JSON object or a JSON array. +For NetworkOrchestrator extensions, most commands signal success or failure by exit code. Some commands return command-specific data on ``stdout``, such as the JSON returned by ``ensure-network-device`` or the output of a ``custom-action``. Keep the output aligned with the command contract implemented by the extension. + +Declaring Network Services and Capabilities +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +``NetworkOrchestrator`` extensions should declare their supported services in the extension detail ``network.services`` as a comma-separated list. Example values include ``SourceNat``, ``StaticNat``, ``PortForwarding``, ``Firewall``, ``Lb``, ``Dhcp``, ``Dns``, ``UserData``, ``NetworkACL``, ``Gateway``, ``Vpn``, and ``CustomAction``. + +Use ``network.service.capabilities`` to provide a JSON object describing capability values for the declared services. CloudStack uses these capabilities when listing supported providers and validating network or VPC offerings. + Action Lifecycle ^^^^^^^^^^^^^^^^ 1. A CloudStack action (e.g., deploy VM) triggers a corresponding extension action. -2. CloudStack invokes the extension’s executable with appropriate parameters. +2. CloudStack invokes the extension’s executable using the protocol defined by the extension type. 3. The extension processes the input and responds within the timeout. 4. CloudStack continues action workflow based on the result. @@ -182,9 +236,12 @@ Custom Actions You can define new custom actions for users or admin-triggered workflows. - Register via UI or ``addCustomAction`` API +- Choose a resource type that matches the extension type: ``VirtualMachine`` for Orchestrator, ``Network`` or ``Vpc`` for NetworkOrchestrator - Define input parameters (name, type, required) - Implement the handler for the custom action in your executable. +For NetworkOrchestrator extensions, advertise the ``CustomAction`` service in ``network.services`` if the extension should receive custom actions for network or VPC resources. + CloudStack UI will render forms dynamically based on these definitions. Best Practices @@ -195,6 +252,8 @@ Best Practices - Avoid hard dependencies on CloudStack internals - Implement logging for troubleshooting - Use exit code and ``stdout`` for signaling success/failure +- Keep ``network.services`` and ``network.service.capabilities`` aligned with the services your NetworkOrchestrator implementation actually handles +- Use resource registration details for environment-specific settings and rotate them with ``updateRegisteredExtension`` instead of recreating mappings when possible Extension Examples ^^^^^^^^^^^^^^^^^^ @@ -231,10 +290,12 @@ Extension Examples else: print(json.dumps({"success": False, "result": {"message": "Unknown action"}})) -For a clearer understanding of how to implement an extension, developers can refer to the base shell script scaffolded by CloudStack for orchestrator-type extensions. This script is located at: +For a clearer understanding of how to implement an Orchestrator extension, developers can refer to the base shell script scaffolded by CloudStack for orchestrator-type extensions. This script is located at: /usr/share/cloudstack-common/scripts/vm/hypervisor/external/provisioner/provisioner.sh It serves as a template with minimal required action handlers, making it a useful starting point for building new extensions. +For NetworkOrchestrator extensions, refer to the Network Extension Script Protocol in the CloudStack source tree and the reference implementation in `cloudstack-extensions `_. + Additionally, CloudStack includes in-built extensions for Proxmox and Hyper-V that demonstrate how to implement extensions in different languages - Bash and Python. diff --git a/source/adminguide/extensions/troubleshooting.rst b/source/adminguide/extensions/troubleshooting.rst index 815da5d352..ada94483fd 100644 --- a/source/adminguide/extensions/troubleshooting.rst +++ b/source/adminguide/extensions/troubleshooting.rst @@ -28,6 +28,8 @@ Validate the Extension Path - Ensure files are stored at: `/usr/share/cloudstack-management/extensions/` + - For NetworkOrchestrator extensions, ensure the configured path resolves either to an executable file or to a directory that contains an executable named `.sh`. + - CloudStack runs a background task at regular intervals to verify path readiness. If the path is not ready, its state will appear as Not Ready in the UI or API responses. - Alerts are generated if the extension path is not ready. @@ -45,6 +47,28 @@ Verify Payload Handling - Improper parsing of the payload is a common cause of failure—log any parsing errors in your extension binary for debugging. + - NetworkOrchestrator extensions use command-specific CLI arguments in addition to JSON strings such as ``--physical-network-extension-details`` and ``--network-extension-details``. Verify that the script accepts the expected command names and argument format for the services it implements. + +Check Resource Registration and Provider State +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + + - Orchestrator extensions must be registered with a ``Cluster`` resource. NetworkOrchestrator extensions must be registered with a ``PhysicalNetwork`` resource. + + - If resource-specific configuration changes are needed after registration, use the UI or the ``updateRegisteredExtension`` API instead of unregistering and recreating the mapping. + + - For NetworkOrchestrator extensions, confirm that a network service provider with the same name as the extension exists on the target physical network and is in the expected state before creating network or VPC offerings. + +Verify Declared Network Services +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + + - NetworkOrchestrator extensions should declare supported services in the extension detail ``network.services``. + + - Per-service capabilities should be declared in ``network.service.capabilities``. + + - If the expected provider services do not appear while creating a network or VPC offering, verify that these details were saved correctly and that any JSON value was quoted correctly when creating or updating the extension. + + - Common declared services include ``SourceNat``, ``StaticNat``, ``PortForwarding``, ``Firewall``, ``Lb``, ``Dhcp``, ``Dns``, ``UserData``, ``NetworkACL``, and ``CustomAction``. Additional services such as ``Gateway`` or ``Vpn`` can also be declared when supported by the implementation. + Refer to Base Extension Scripts ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ @@ -52,6 +76,8 @@ Refer to Base Extension Scripts - For Orchestrator-type extensions, see: `/usr/share/cloudstack-common/scripts/vm/hypervisor/external/provisioner/provisioner.sh` + - For NetworkOrchestrator-type extensions, refer to the Network Extension Script Protocol in the CloudStack source tree and the reference implementation in `cloudstack-extensions `_. + - These scripts provide examples of how to handle standard actions like start, stop, status, etc. Check Logs for Errors @@ -67,4 +93,6 @@ Check Logs for Errors 3. Output parsing + 4. Provider and service resolution for network and VPC operations + - Any exceptions or exit code issues. From 80ba7fa5154fbe68880cefbe208060564160ac16 Mon Sep 17 00:00:00 2001 From: Wei Zhou Date: Wed, 13 May 2026 09:23:46 +0200 Subject: [PATCH 2/2] Update doc: use JSON payload --- .../adminguide/extensions/custom_actions.rst | 12 ++- source/adminguide/extensions/developer.rst | 75 +++++++++++++++++-- .../adminguide/extensions/troubleshooting.rst | 12 ++- 3 files changed, 89 insertions(+), 10 deletions(-) diff --git a/source/adminguide/extensions/custom_actions.rst b/source/adminguide/extensions/custom_actions.rst index 504395892b..6933951e36 100644 --- a/source/adminguide/extensions/custom_actions.rst +++ b/source/adminguide/extensions/custom_actions.rst @@ -58,13 +58,21 @@ Custom actions can be attached to the following resource types: For network and VPC custom actions, CloudStack dispatches the action to the external provider that serves the ``CustomAction`` service for the selected resource. +For ``NetworkOrchestrator`` extensions, the action is executed as ``custom-action`` using the standard payload-file invocation model: + +.. code-block:: bash + + /path/to/.sh custom-action + +The payload file contains top-level keys such as ``action``, ``action-params``, ``physical-network-extension-details``, and ``network-extension-details``. Unlike other network extension commands, the custom action request does not wrap its command-specific values inside a nested ``payload`` object. + Running Custom Action ~~~~~~~~~~~~~~~~~~~~~ -All enabled custom actions can then be triggered for a resource of the type the action is defined for or provided while running, using the **Run Action** view or runCustomAction API. +All enabled custom actions can then be triggered for a resource of the type the action is defined for or provided while running, using the **Run Action** view or the relevant custom action API. -The same API is used for VM, network, and VPC custom actions. The available actions shown in the UI depend on the selected resource type and the extension bound to that resource. +For network and VPC custom actions, CloudStack passes the full request in the payload file and returns the script's ``stdout`` to the caller. The available actions shown in the UI depend on the selected resource type and the extension bound to that resource. |run-custom-action-instance.png| diff --git a/source/adminguide/extensions/developer.rst b/source/adminguide/extensions/developer.rst index b9088bf84d..6a50ca6de4 100644 --- a/source/adminguide/extensions/developer.rst +++ b/source/adminguide/extensions/developer.rst @@ -97,15 +97,21 @@ Sample Invocation: NetworkOrchestrator Invocation ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -NetworkOrchestrator extensions use service-specific command names and named CLI arguments instead of the payload-file format above. Depending on the declared services, CloudStack can invoke commands such as ``ensure-network-device``, ``implement-network``, ``shutdown-network``, ``destroy-network``, ``assign-ip``, ``add-static-nat``, ``add-port-forward``, ``apply-fw-rules``, ``apply-lb-rules``, ``apply-network-acl``, ``implement-vpc``, ``shutdown-vpc``, or ``custom-action``. +NetworkOrchestrator extensions also use the payload-file invocation model: -CloudStack always passes registration and runtime state using named arguments such as: +.. code-block:: bash + + /path/to/.sh + +CloudStack resolves the executable in this order: -- ``--physical-network-extension-details`` for details stored on the physical-network registration. +- ``/.sh`` -- ``--network-extension-details`` for per-network or per-VPC state maintained by the extension. +- ```` itself, if it is an executable file -Some commands may also receive temporary file paths for larger payloads. +Depending on the declared services, CloudStack can invoke commands such as ``ensure-network-device``, ``implement-network``, ``shutdown-network``, ``destroy-network``, ``restore-network``, ``implement-vpc``, ``shutdown-vpc``, ``update-vpc-source-nat-ip``, ``assign-ip``, ``release-ip``, ``add-static-nat``, ``delete-static-nat``, ``add-port-forward``, ``delete-port-forward``, ``apply-fw-rules``, ``apply-network-acl``, ``add-dhcp-entry``, ``remove-dhcp-entry``, ``config-dhcp-subnet``, ``remove-dhcp-subnet``, ``set-dhcp-options``, ``add-dns-entry``, ``config-dns-subnet``, ``remove-dns-subnet``, ``save-vm-data``, ``save-password``, ``save-userdata``, ``save-sshkey``, ``save-hypervisor-hostname``, ``apply-lb-rules``, and ``custom-action``. + +The command name determines the operation. The payload file carries registration details, stored network or VPC state, and command-specific fields. Input Format (Payload) ^^^^^^^^^^^^^^^^^^^^^^ @@ -150,7 +156,21 @@ Example: The schema varies depending on the resource and action. Use this to perform context-specific logic. -For NetworkOrchestrator extensions, CloudStack passes command-specific CLI arguments such as network IDs, VPC IDs, VLAN tags, IP addresses, action parameters, and JSON strings in named options. The exact input depends on the command being invoked and the services declared in ``network.services``. +For NetworkOrchestrator extensions, CloudStack writes a JSON payload file and passes its path as the second argument. For standard network and VPC commands, the payload has this envelope: + +.. code-block:: json + + { + "physical-network-extension-details": {}, + "network-extension-details": {}, + "payload": {} + } + +``physical-network-extension-details`` contains the registration details stored against the physical network, enriched with values such as the physical network name. ``network-extension-details`` contains the per-network or per-VPC state saved by the extension, including the output previously returned by ``ensure-network-device``. ``payload`` contains the command-specific fields. + +Frequently used payload fields include ``network_id``, ``vpc_id``, ``vlan``, ``gateway``, ``cidr``, ``extension_ip``, ``public_ip``, ``public_cidr``, ``public_vlan``, ``public_gateway``, ``private_ip``, ``nic_uuid``, ``dns``, and ``domain``. + +For ``custom-action``, CloudStack still uses a payload file, but the command-specific values are placed at the top level instead of under ``payload``. The request includes keys such as ``action``, ``action-params``, ``physical-network-extension-details``, and ``network-extension-details``. Output Format ^^^^^^^^^^^^^ @@ -167,7 +187,7 @@ Orchestrator extensions should write a response JSON to ``stdout``. Example: For custom actions, CloudStack will display the ``message`` in the UI if the output JSON includes ``"printmessage": "true"``. The ``message`` field can be a string, a JSON object or a JSON array. -For NetworkOrchestrator extensions, most commands signal success or failure by exit code. Some commands return command-specific data on ``stdout``, such as the JSON returned by ``ensure-network-device`` or the output of a ``custom-action``. Keep the output aligned with the command contract implemented by the extension. +For NetworkOrchestrator extensions, all commands must exit with code ``0`` on success and a non-zero code on failure. ``ensure-network-device`` is special: it must write a single-line JSON object to ``stdout`` and CloudStack persists that JSON for later calls. ``custom-action`` may also return output on ``stdout``. Other commands should not rely on ``stdout`` for normal operation because CloudStack ignores it apart from debug logging. Declaring Network Services and Capabilities ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ @@ -176,6 +196,44 @@ Declaring Network Services and Capabilities Use ``network.service.capabilities`` to provide a JSON object describing capability values for the declared services. CloudStack uses these capabilities when listing supported providers and validating network or VPC offerings. +For example, ``Firewall`` capabilities can declare values such as ``SupportedProtocols``, ``SupportedEgressProtocols``, and ``SupportedTrafficDirection``; ``Lb`` can declare ``SupportedLBAlgorithms`` and ``SupportedProtocols``; and ``SourceNat`` can declare ``SupportedSourceNatTypes``. + +CloudStack Setup for NetworkOrchestrator +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +After creating a ``NetworkOrchestrator`` extension, the usual setup flow is: + +1. Deploy the executable to the path returned by ``listExtensions`` on every management server. +2. Register the extension with a ``PhysicalNetwork`` using ``registerExtension`` and pass any device-specific details required by the script. +3. Enable the generated network service provider on that physical network. +4. Create network or VPC offerings that map supported services to the extension name. +5. When registration details need to change, update them with ``updateRegisteredExtension`` instead of deleting and recreating the mapping. + +Service-to-Command Mapping +^^^^^^^^^^^^^^^^^^^^^^^^^^ + +NetworkOrchestrator scripts only need to implement the commands for the services they advertise in ``network.services``. + +- ``SourceNat`` and ``Gateway`` trigger ``assign-ip`` and ``release-ip``. +- ``StaticNat`` triggers ``add-static-nat`` and ``delete-static-nat``. +- ``PortForwarding`` triggers ``add-port-forward`` and ``delete-port-forward``. +- ``Firewall`` triggers ``apply-fw-rules``. +- ``Lb`` triggers ``apply-lb-rules``. +- ``NetworkACL`` triggers ``apply-network-acl``. +- ``Dhcp`` triggers ``add-dhcp-entry``, ``remove-dhcp-entry``, ``config-dhcp-subnet``, ``remove-dhcp-subnet``, and ``set-dhcp-options``. +- ``Dns`` triggers ``add-dns-entry``, ``config-dns-subnet``, and ``remove-dns-subnet``. +- ``UserData`` triggers ``save-vm-data``, ``save-password``, ``save-userdata``, ``save-sshkey``, and ``save-hypervisor-hostname``. +- Network lifecycle operations always include ``ensure-network-device``, ``implement-network``, ``shutdown-network``, ``destroy-network``, and ``restore-network``. +- VPC lifecycle operations include ``ensure-network-device``, ``implement-vpc``, ``shutdown-vpc``, and ``update-vpc-source-nat-ip``. +- Operator-triggered actions use ``custom-action``. + +VPC and Extension IP Notes +^^^^^^^^^^^^^^^^^^^^^^^^^^ + +For VPC-backed networks, CloudStack includes ``vpc_id`` in tier-level payloads and invokes VPC-level lifecycle commands without a ``network_id``. Use that value when the implementation needs to keep all tiers of a VPC on the same device. + +``extension_ip`` is the IP address the external device presents on the guest side. When the extension provides ``SourceNat`` or ``Gateway``, it typically matches the network gateway. When the extension only provides services such as ``Dhcp``, ``Dns``, or ``UserData``, CloudStack can allocate a dedicated guest-side IP for the device and pass it as ``extension_ip``. + Action Lifecycle ^^^^^^^^^^^^^^^^ @@ -242,6 +300,8 @@ You can define new custom actions for users or admin-triggered workflows. For NetworkOrchestrator extensions, advertise the ``CustomAction`` service in ``network.services`` if the extension should receive custom actions for network or VPC resources. +For network extension custom actions, the script should read the payload file passed on the command line and parse the top-level keys rather than looking for a nested ``payload`` object. + CloudStack UI will render forms dynamically based on these definitions. Best Practices @@ -254,6 +314,7 @@ Best Practices - Use exit code and ``stdout`` for signaling success/failure - Keep ``network.services`` and ``network.service.capabilities`` aligned with the services your NetworkOrchestrator implementation actually handles - Use resource registration details for environment-specific settings and rotate them with ``updateRegisteredExtension`` instead of recreating mappings when possible +- Keep non-``ensure-network-device`` commands quiet on ``stdout`` unless the command contract explicitly returns output, such as ``custom-action`` Extension Examples ^^^^^^^^^^^^^^^^^^ diff --git a/source/adminguide/extensions/troubleshooting.rst b/source/adminguide/extensions/troubleshooting.rst index ada94483fd..f10a35d373 100644 --- a/source/adminguide/extensions/troubleshooting.rst +++ b/source/adminguide/extensions/troubleshooting.rst @@ -47,7 +47,13 @@ Verify Payload Handling - Improper parsing of the payload is a common cause of failure—log any parsing errors in your extension binary for debugging. - - NetworkOrchestrator extensions use command-specific CLI arguments in addition to JSON strings such as ``--physical-network-extension-details`` and ``--network-extension-details``. Verify that the script accepts the expected command names and argument format for the services it implements. + - NetworkOrchestrator extensions receive `` ``. Verify that the script accepts the command name and reads the JSON payload file rather than expecting named CLI options. + + - For standard network and VPC commands, confirm that the payload file contains the expected top-level keys: ``physical-network-extension-details``, ``network-extension-details``, and ``payload``. + + - For ``custom-action``, confirm that the request uses top-level keys such as ``action`` and ``action-params`` instead of a nested ``payload`` object. + + - For ``ensure-network-device``, confirm that the script prints a single-line JSON object to ``stdout``. For other commands, unexpected ``stdout`` output is usually a sign that the script is not following the current command contract. Check Resource Registration and Provider State ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ @@ -58,6 +64,8 @@ Check Resource Registration and Provider State - For NetworkOrchestrator extensions, confirm that a network service provider with the same name as the extension exists on the target physical network and is in the expected state before creating network or VPC offerings. + - If the provider exists but operations still fail, verify that the provider was enabled after registration and that the intended offering maps each supported service to the extension name. + Verify Declared Network Services ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ @@ -95,4 +103,6 @@ Check Logs for Errors 4. Provider and service resolution for network and VPC operations + 5. Failures while decoding nested payload values such as firewall rules, ACL rules, restore data, or VM metadata blobs + - Any exceptions or exit code issues.