Adds retry support to the Amazon.Lambda.DurableExecution

[GarrettBeatty]

Stacked PRs:

• ->Adds retry support to the Amazon.Lambda.DurableExecution #2363
• Add durable execution Step + Wait end-to-end #2360

---

#2216

What

Adds retry support to the Amazon.Lambda.DurableExecution SDK on top of the foundation in #2360. After this PR a step that throws can be retried with configurable backoff and jitter; durable executions resume after the retry timer elapses without billing Lambda compute during the wait.

Public API introduced:

Type	Purpose
IRetryStrategy	Decides whether a failed step should retry, with what delay.
RetryDecision	Output of IRetryStrategy.ShouldRetry — ShouldRetry flag plus Delay.
RetryStrategy	Static factory: Default, Transient, None, Exponential(...), FromDelegate(...).
JitterStrategy	None / Half / Full for exponential backoff.
StepSemantics	AtLeastOncePerRetry (default) / AtMostOncePerRetry.
StepConfig.RetryStrategy, StepConfig.Semantics	Per-step retry configuration.

Why

Real workflows fail. A step that calls a flaky downstream service or hits a transient throttle needs to retry without restarting the whole workflow. Durable execution makes service-mediated retries possible: the SDK checkpoints a RETRY operation with a NextAttemptDelaySeconds, suspends the Lambda, and the service re-invokes us when the timer fires. The user's compute isn't billed during the wait.

AtMostOncePerRetry semantics handle non-idempotent steps (e.g. charging a card): a START checkpoint is durably persisted before user code runs, so a Lambda crash mid-execution can be detected on replay and routed through the retry strategy rather than re-executing.

How

Retry control flow. When a step throws, StepOperation.HandleStepFailureAsync consults the configured IRetryStrategy.ShouldRetry(ex, attemptNumber). If the decision says retry, the SDK enqueues a RETRY checkpoint carrying NextAttemptDelaySeconds, then suspends via TerminationManager.SuspendAndAwait so RunAsync returns Pending to the service. On the next invocation, StepOperation.ReplayAsync sees Status == PENDING and either re-suspends (timer not yet elapsed) or re-executes (timer fired) with the carried-forward attempt counter.

At-most-once semantics. For non-idempotent steps, Semantics = AtMostOncePerRetry writes a START checkpoint and blocks until the batcher flushes it before user code runs. If Lambda crashes between user code and the SUCCEED flush, replay sees STARTED with no terminal record and routes through HandleStepFailureAsync as a failed attempt instead of re-executing — the side effect runs at most once per attempt.

Retry strategy contract. IRetryStrategy.ShouldRetry(Exception, int attemptNumber) returns a RetryDecision. ExponentialRetryStrategy supports configurable max attempts, initial/max delay, backoff rate, jitter (None/Half/Full), and exception filtering by type or message regex. Built-in factories: RetryStrategy.Default (6 attempts, 5s/60s, 2× backoff, full jitter), Transient (3 attempts, 1s/5s, half jitter), None. RetryStrategy.FromDelegate(...) for arbitrary policies.

Key files:

• Config/IRetryStrategy.cs — strategy interface + RetryDecision value type
• Config/RetryStrategy.cs — built-in strategies, ExponentialRetryStrategy, JitterStrategy, StepSemantics, DelegateRetryStrategy
• Config/StepConfig.cs — adds RetryStrategy and Semantics properties
• Internal/StepOperation.cs — adds PENDING (retry timer) and STARTED (AtMostOnce crash recovery) replay arms; HandleStepFailureAsync decision tree
• Internal/TerminationManager.cs — adds RetryScheduled reason

Testing

21 new unit tests in Amazon.Lambda.DurableExecution.Tests (130 total, up from 109 in #2360):

• RetryStrategyTests (14 tests) — exponential backoff math, jitter strategies, max-attempt exhaustion, exception-type and message-pattern filtering, delegate strategies
• DurableContextTests retry block (6 tests) — FailsWithRetryStrategy_CheckpointsRetryAndSuspends, FailsNoRetryStrategy_CheckpointsFail, RetryExhausted_CheckpointsFail, PendingWithFutureTimestamp_Suspends, PendingWithPastTimestamp_ReExecutes, AtMostOnce_FlushesStartBeforeExecution, AtMostOnce_StartedReplay_TriggersRetryHandler

Integration tests (Amazon.Lambda.DurableExecution.IntegrationTests) — RetrySucceeds and RetryExhausts end-to-end against the real durable-execution service.

Out of scope (follow-up PRs)

• MapAsync / ParallelAsync / RunInChildContextAsync / WaitForConditionAsync
• CallbackAsync, InvokeAsync
• DefaultJsonCheckpointSerializer
• DurableLogger replay-suppression (currently NullLogger)
• Annotations source-generator integration / [DurableExecution] attribute
• DurableTestRunner / Amazon.Lambda.DurableExecution.Testing package
• dotnet new lambda.DurableFunction blueprint

[Copilot]

Pull request overview

Builds on PR #2360 to add retry support to the Amazon.Lambda.DurableExecution SDK. Failed steps can now be retried with configurable backoff and jitter via service-mediated retries (the SDK checkpoints a RETRY operation and suspends the Lambda so the user is not billed during backoff). Adds at-most-once semantics for non-idempotent steps via a synchronously-flushed START checkpoint that allows crash detection on replay.

Changes:

• New public retry API: IRetryStrategy, RetryDecision, RetryStrategy factories (Default/Transient/None/Exponential/FromDelegate), JitterStrategy, StepSemantics, and StepConfig.RetryStrategy/StepConfig.Semantics.
• StepOperation adds PENDING (retry-timer) and STARTED (AtMostOnce crash-recovery) replay arms, a HandleStepFailureAsync decision tree, and START-checkpoint emission (sync for AtMostOnce, fire-and-forget for AtLeastOnce).
• 21 new unit tests plus integration-test updates asserting StepStarted events and richer history logging.

Reviewed changes

Copilot reviewed 15 out of 15 changed files in this pull request and generated no comments.

Show a summary per file

File	Description
Config/IRetryStrategy.cs	New strategy interface + RetryDecision struct
Config/RetryStrategy.cs	ExponentialRetryStrategy, DelegateRetryStrategy, JitterStrategy, StepSemantics, factories
Config/StepConfig.cs	Adds RetryStrategy and Semantics properties
Internal/StepOperation.cs	PENDING/STARTED replay arms, retry decision tree, START-checkpoint emission
Internal/TerminationManager.cs	Adds RetryScheduled termination reason
Internal/CheckpointBatcher.cs	Doc-only update describing fire-and-forget semantics
Tests/RetryStrategyTests.cs	14 unit tests for exponential math/jitter/filters/delegate
Tests/DurableContextTests.cs	6 retry/AtMostOnce/Pending replay tests
Tests/DurableFunctionTests.cs	Updated to assert START + SUCCEED + WAIT-START flat sequence
IntegrationTests/*.cs	Add StepStarted-event assertions; richer history dump in DurableFunctionDeployment

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[GarrettBeatty]

now that we are emitting START steps (which are needed for retries) we are asserting them in the IT tests

[GarrettBeatty]

in previous PR only SUCCEEDED or FAILED mattered. But now for replays, we need to keep track of how many times the function was executed, which is done via the number of STARTED steps.