Add cross-SDK RPC E2E coverage#1424
Open
stephentoub wants to merge 5 commits into
Open
Conversation
Add non-Canvas RPC E2E coverage across C#, Node, Python, Go, and Rust. The tests exercise server and session RPC surfaces with assertions for stable return shapes, state transitions, events, no-op behavior, and capability-gated error paths. Java is intentionally excluded. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
![github-code-quality[bot]](https://avatars.githubusercontent.com/u/9919?s=60&v=4){kind=small}
| ) | ||
|
|
||
|
|
||
| async def _wait_for_command_in_pending_items(session, command: str) -> QueuePendingItems: |
| finally: | ||
| try: | ||
| await client.stop() | ||
| except ExceptionGroup: |
| finally: | ||
| try: | ||
| await client.stop() | ||
| except ExceptionGroup: |
This comment has been minimized.
This comment has been minimized.
Comment on lines
+164
to
+169
| catch (Exception ex) | ||
| { | ||
| var message = ex.ToString(); | ||
| AssertNotUnhandledMethod(message); | ||
| AssertSamplingError(message); | ||
| } |
Comment on lines
+79
to
+83
| catch (Exception ex) | ||
| { | ||
| Assert.DoesNotContain($"Unhandled method {method}", ex.ToString(), StringComparison.OrdinalIgnoreCase); | ||
| return null; | ||
| } |
Comment on lines
+92
to
+95
| catch (Exception ex) | ||
| { | ||
| Assert.DoesNotContain($"Unhandled method {method}", ex.ToString(), StringComparison.OrdinalIgnoreCase); | ||
| } |
Comment on lines
+83
to
+86
| catch (Exception ex) | ||
| { | ||
| readError = ex; | ||
| } |
Contributor
There was a problem hiding this comment.
Pull request overview
This PR expands end-to-end JSON-RPC coverage across the SDKs (C#, Node/TS, Python, Go, Rust) by adding/porting non-Canvas RPC E2E tests that validate typed request/response shapes, stable semantics (no-ops/idempotency), and capability-gated error behavior.
Changes:
- Added new cross-SDK E2E tests for additional RPC groups (event log, queue, remote, schedule, workspace checkpoints, compaction handoff summary, commands, MCP apps/skills, and expanded tasks/UI handlers).
- Updated existing E2E harness/helpers (Node wait helper; Python harness directory prep) to support more reliable multi-RPC tests.
- Extended existing RPC E2E suites (tasks/handlers, permissions, server-scoped RPC, MCP/skills) with additional typed coverage paths.
Show a summary per file
| File | Description |
|---|---|
| rust/tests/e2e/rpc_workspace_checkpoints.rs | New Rust E2E coverage for workspace checkpoints/diff/large paste. |
| rust/tests/e2e/rpc_tasks_and_handlers.rs | Expands Rust tasks + UI pending-handler RPC coverage. |
| rust/tests/e2e/rpc_schedule.rs | New Rust E2E coverage for session.schedule RPC. |
| rust/tests/e2e/rpc_remote.rs | New Rust E2E coverage for session.remote RPC and steerable persistence. |
| rust/tests/e2e/rpc_queue.rs | New Rust E2E coverage for session.queue RPC. |
| rust/tests/e2e/rpc_mcp_and_skills.rs | Expands Rust MCP apps + skills RPC coverage. |
| rust/tests/e2e/rpc_event_log.rs | New Rust E2E coverage for session.eventLog RPC. |
| rust/tests/e2e/compaction.rs | Adds Rust E2E coverage for handoff summary + compaction cancel/abort. |
| rust/tests/e2e/commands.rs | Adds Rust E2E coverage for commands list/invoke/execute/queue response. |
| rust/tests/e2e.rs | Registers newly added Rust E2E modules. |
| python/e2e/testharness/context.py | Ensures test harness base directories exist before cleanup. |
| python/e2e/test_rpc_workspace_checkpoints_e2e.py | New Python E2E coverage for workspace checkpoints/diff/large paste. |
| python/e2e/test_rpc_tasks_and_handlers_e2e.py | Expands Python tasks + UI pending handler + elicitation/auto-mode coverage. |
| python/e2e/test_rpc_server_e2e.py | Expands Python server-scoped RPC coverage (sessions, secrets, sessionFs, etc.). |
| python/e2e/test_rpc_schedule_e2e.py | New Python E2E coverage for schedule RPC. |
| python/e2e/test_rpc_remote_e2e.py | New Python E2E coverage for remote RPC + persisted steerable. |
| python/e2e/test_rpc_queue_e2e.py | New Python E2E coverage for queue pending items + mutations. |
| python/e2e/test_rpc_mcp_and_skills_e2e.py | Expands Python MCP/skills and MCP apps host-context/capability tests. |
| python/e2e/test_rpc_event_log_e2e.py | New Python E2E coverage for event log read/tail/interest/long-poll. |
| python/e2e/test_rpc_commands_e2e.py | New Python E2E coverage for commands list/invoke/execute/queued response. |
| nodejs/test/e2e/rpc_workspace_checkpoints.e2e.test.ts | New Node E2E coverage for workspace checkpoints/diff/large paste. |
| nodejs/test/e2e/rpc_tasks_and_handlers.e2e.test.ts | Expands Node tasks + UI pending handler coverage + direct auto mode switch. |
| nodejs/test/e2e/rpc_server.e2e.test.ts | Expands Node server-scoped RPC coverage and adds polling helper usage. |
| nodejs/test/e2e/rpc_schedule.e2e.test.ts | New Node E2E coverage for schedule RPC. |
| nodejs/test/e2e/rpc_remote.e2e.test.ts | New Node E2E coverage for remote RPC + persisted steerable. |
| nodejs/test/e2e/rpc_queue.e2e.test.ts | New Node E2E coverage for queue pending items + mutations. |
| nodejs/test/e2e/rpc_mcp_and_skills.e2e.test.ts | Expands Node MCP/skills and MCP apps host-context/capability tests. |
| nodejs/test/e2e/rpc_event_log.e2e.test.ts | New Node E2E coverage for event log read/tail/interest/long-poll. |
| nodejs/test/e2e/permissions.e2e.test.ts | Adds Node E2E coverage for permission paths/state/locations/folder trust. |
| nodejs/test/e2e/harness/sdkTestHelper.ts | Adds reusable waitForCondition helper for polling-based assertions. |
| nodejs/test/e2e/compaction.e2e.test.ts | Adds Node E2E coverage for handoff summary and compaction cancel/abort. |
| nodejs/test/e2e/commands.e2e.test.ts | Adds Node E2E coverage for commands list/invoke/execute/enqueue/respond. |
| go/internal/e2e/rpc_workspace_checkpoints_e2e_test.go | New Go E2E coverage for workspace checkpoints/diff/large paste. |
| go/internal/e2e/rpc_schedule_e2e_test.go | New Go E2E coverage for schedule RPC. |
| go/internal/e2e/rpc_remote_e2e_test.go | New Go E2E coverage for remote RPC + persisted steerable. |
| go/internal/e2e/rpc_queue_e2e_test.go | New Go E2E coverage for queue pending items + mutations. |
| go/internal/e2e/rpc_event_log_e2e_test.go | New Go E2E coverage for event log read/tail/interest/long-poll. |
| go/internal/e2e/rpc_coverage_helpers_test.go | Adds shared Go E2E helpers for pointers, paths, and polling. |
| go/internal/e2e/permissions_e2e_test.go | Expands Go permission RPC E2E coverage (paths/state/locations/folder trust). |
| go/internal/e2e/compaction_e2e_test.go | Expands Go compaction E2E coverage (handoff summary, cancel/abort). |
| go/internal/e2e/commands_and_elicitation_e2e_test.go | Expands Go commands E2E coverage (list/invoke/execute/enqueue/respond). |
| dotnet/test/E2E/RpcWorkspaceCheckpointsE2ETests.cs | New .NET E2E coverage for workspace checkpoints/diff/large paste. |
| dotnet/test/E2E/RpcTasksAndHandlersE2ETests.cs | Expands .NET tasks + UI pending handlers + elicitation/auto-mode coverage. |
| dotnet/test/E2E/RpcScheduleE2ETests.cs | New .NET E2E coverage for schedule RPC. |
| dotnet/test/E2E/RpcRemoteE2ETests.cs | New .NET E2E coverage for remote RPC + persisted steerable. |
| dotnet/test/E2E/RpcQueueE2ETests.cs | New .NET E2E coverage for queue pending items + mutations. |
| dotnet/test/E2E/RpcMcpAndSkillsE2ETests.cs | Expands .NET MCP/skills and MCP apps host-context/capability tests. |
| dotnet/test/E2E/RpcEventLogE2ETests.cs | New .NET E2E coverage for event log read/tail/interest/long-poll. |
| dotnet/test/E2E/PermissionE2ETests.cs | Expands .NET permission RPC E2E coverage (paths/state/locations/folder trust). |
| dotnet/test/E2E/CompactionE2ETests.cs | Expands .NET compaction E2E coverage (handoff summary, cancel/abort). |
| dotnet/test/E2E/CommandsE2ETests.cs | Expands .NET commands E2E coverage (list/invoke/execute/enqueue/respond). |
Copilot's findings
- Files reviewed: 61/61 changed files
- Comments generated: 4
Comment on lines
+6
to
+10
| async fn should_return_empty_handoff_summary_for_fresh_session() { | ||
| with_e2e_context( | ||
| "compaction", | ||
| "should_not_emit_compaction_events_when_infinite_sessions_disabled", | ||
| |ctx| { |
Comment on lines
+36
to
+40
| async fn should_report_noop_when_cancelling_compaction_without_inflight_work() { | ||
| with_e2e_context( | ||
| "compaction", | ||
| "should_not_emit_compaction_events_when_infinite_sessions_disabled", | ||
| |ctx| { |
Comment on lines
+73
to
+77
| async fn should_summarize_for_handoff_after_non_ephemeral_log_event() { | ||
| with_e2e_context( | ||
| "compaction", | ||
| "should_not_emit_compaction_events_when_infinite_sessions_disabled", | ||
| |ctx| { |
Comment on lines
+99
to
+113
| let pending = session | ||
| .rpc() | ||
| .queue() | ||
| .pending_items() | ||
| .await | ||
| .expect("pending queued command"); | ||
| assert!(pending.items.is_empty()); | ||
| assert!(pending.steering_messages.is_empty()); | ||
| let removed = session | ||
| .rpc() | ||
| .queue() | ||
| .remove_most_recent() | ||
| .await | ||
| .expect("remove after command event"); | ||
| assert!(!removed.removed); |
Apply Ruff formatting and import ordering to the new Python RPC E2E tests so the Python SDK checks pass. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This comment has been minimized.
This comment has been minimized.
Relax model switch assertions for runtime differences, remove unused Go E2E helper, align Rust fork prompt with the replay snapshot, and normalize Windows permission paths. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
| Assert.Equal(branch, contextChanged.Data.Branch); | ||
| Assert.Equal("github/copilot-sdk-e2e", contextChanged.Data.Repository); | ||
| Assert.Equal("github.com", contextChanged.Data.RepositoryHost); | ||
| Assert.Equal("github", contextChanged.Data.HostType?.Value); |
This comment has been minimized.
This comment has been minimized.
Use SendAndWaitAsync for the compaction setup turn so the test waits through the SDK send path instead of racing metadata polling and assistant event delivery. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Contributor
Cross-SDK Consistency Review ✅This PR maintains strong cross-SDK consistency. All five SDK implementations (C#/.NET, Go, Node.js/TypeScript, Python, Rust) receive equivalent E2E test coverage for the same RPC categories:
Java is explicitly noted as out of scope in the PR description, which is a conscious and documented decision rather than an oversight. No cross-SDK consistency issues found.
|
Give the Go SDK race test run enough time for the expanded E2E suite on slower Windows runners. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
The SDKs expose a large public RPC surface, but E2E coverage had gaps across several server and session RPC groups. This PR adds meaningful non-Canvas RPC E2E coverage across the supported SDKs so regressions in typed RPC behavior are caught consistently.
What changed
Validation
cargo +nightly-2026-04-14 fmt --check,cargo check --all-features --tests, and targeted E2E filters passed.git diff --checkpassed.Notes
Some remote and MCP app paths are capability-gated in the local E2E environment, so those tests assert stable implemented errors or no-op result shapes rather than pretending unavailable infrastructure exists.