Skip to content

feat: Add dynamic workflow support#127

Draft
tadhgdowdall wants to merge 12 commits into
harmont-dev:mainfrom
tadhgdowdall:feat/dynamic-workflow-support
Draft

feat: Add dynamic workflow support#127
tadhgdowdall wants to merge 12 commits into
harmont-dev:mainfrom
tadhgdowdall:feat/dynamic-workflow-support

Conversation

@tadhgdowdall

Copy link
Copy Markdown
Contributor

Summary

This PR is linked to #47

Introduces deferred target evaluation for Python and TypeScript workflows. Dynamic targets are represented as placeholders during pipeline rendering and expanded into concrete steps when local execution reaches them.

What’s Done

  • Added static and dynamic evaluation variants to pipeline IR.

  • Added Python @hm.target(dynamic=True) support.

  • Added TypeScript target(..., { dynamic: true }) support.

  • Added runtime target evaluation for both DSL engines.

  • Added hm.env() and env() helpers for runtime conditionals.

  • Added local execution of dynamically generated steps.

  • Supports:

    • Single generated steps
    • Linear command chains
    • Groups with multiple branches
    • Explicit group continuation
    • Snapshot continuation from one terminal step
    • Fresh-image continuation for multiple terminal steps
  • Preserved caching, environment, image, and runner metadata.

  • Added IR, SDK, engine, and executor tests.

Example

@hm.target(dynamic=True)
def choose_build() -> hm.Step:
    if hm.env("BUILD_TYPE") == "release":
        return hm.sh("npm run build:release")

    return hm.sh("npm run build:debug")
const chooseBuild = target(
  "choose-build",
  () =>
    env("BUILD_TYPE") === "release"
      ? sh("npm run build:release")
      : sh("npm run build:debug"),
  { dynamic: true },
);

Remaining Work

  • Execute independent branches inside dynamic groups concurrently.
  • Add committed CLI and Docker end-to-end tests.
  • Improve build event and step-count reporting for generated steps.
  • Define and implement cloud-side dynamic evaluation.
  • Document dynamic targets and group continuation semantics.
  • Confirm the proposed API and multiple-terminal behavior with maintainers.

Current Limitations

  • Dynamic execution currently works only with the local backend.
  • Steps inside dynamic groups execute sequentially.
  • Initial build step counts include the placeholder rather than the eventual generated steps.
  • A dynamic group with multiple terminal leaves cannot provide one snapshot for a subsequent step. The subsequent step therefore starts from the pipeline’s default image unless the group defines an explicit merged continuation.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants