Syscall rewriter improvements#812
Merged
Merged
Conversation
wdcui
force-pushed
the
wdcui/pr1-trampoline-format
branch
from
75ea7ec to
0bcb6ff
Compare
wdcui
force-pushed
the
wdcui/pr1-trampoline-format
branch
from
a897889 to
fd4fcb7
Compare
Member
Author
|
This PR is ready for review. Ignore the rtld changes since it will be removed after PR #810 is merged. |
jaybosamiya-ms
added
the
expmt:shadow-kiln
wdcui
force-pushed
the
wdcui/pr1-trampoline-format
branch
2 times, most recently
from
d083228 to
097c515
Compare
Member
Author
Contributor
|
Similar to how Linux handles syscall restart, we could update pt_regs->ip directly to where we would like to re-execute from. For hook syscall and its previous instructions, there is no post_jmp instruction; we need to update rewriter as well. |
Contributor
|
I don't see why we need the red zone. |
Member
Author
Good point. It's removed now. |
Member
Author
ctx.rip = ctx.rcx - 6 since we don't need to rerun the lea of rcx (and lea rsp is removed now). |
wdcui
changed the title
…ress The syscall rewriter now emits a 12-byte preamble before each trampoline stub: LEA RSP,[RSP-0x80] to reserve the SysV 128-byte red zone, and LEA R11,[RIP+disp32] to load the call-site restart address into R11 for future SA_RESTART support. Platform callbacks are renamed to syscall_callback_redzone to reflect the new calling convention. The callback recovers the architectural RSP with LEA R11,[RSP+128] and saves the restart address to TLS (saved_restart_addr) before clobbering R11. The rewriter also emits a size=0 header sentinel for binaries with no syscall instructions, allowing the loader to distinguish 'checked, no syscalls' from 'never processed.' The is_already_hooked check treats size=0 as already-hooked. The rtld_audit library is updated to handle both the new calling convention (red zone + R11) and size=0 binaries (via mincore probe before reading trampoline memory).
The pre-built binary had old-format trampolines that jumped to syscall_callback_redzone without reserving the red zone, causing an access violation (0xc0000005) on Windows.
…oline anchor Per CvvT's review: the R11 restart-address load and the SysV red zone preamble exist to support each other (the Windows callback's R11 shuttle was the only thing that wrote to the guest stack pre-stack-switch, and the red zone reservation existed to protect against that shuttle). Replace the scheme so every stub satisfies: pt_regs.rcx - 6 == address of JMP [syscall_entry_slot] RCX uniformly points at the in-trampoline `post_jmp`, which is now a 5-byte `JMP rel32 -> guest_text` tail in every stub. The callback's `jmp rcx` return path lands at post_jmp and falls through to guest text. The SA_RESTART handler can rewind ctx.rip with a single subtract, with no per-variant layout knowledge. Drops, both platforms: - saved_restart_addr TLS slot - Windows TEB-slot shuttle (mov gs:[0x28], r11; etc.) - lea r11, [rsp+128] architectural-RSP recovery - LEA RSP, [RSP-0x80] red zone preamble - LEA R11, [RIP+disp32] restart-address load The post_jmp tail adds 5 bytes per stub. Net: -7 bytes per stub vs. the prior approach, with the SA_RESTART invariant baked in for free. Snapshot regenerated to reflect the new 18-byte-per-stub layout. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
wdcui
force-pushed
the
wdcui/pr1-trampoline-format
branch
from
77b4540 to
9faf56b
Compare
|
🤖 SemverChecks 🤖 No breaking API changes detected Note: this does not mean API is unchanged, or even that there are no breaking changes; simply, none of the detections triggered. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR changes how the return address is saved in rcx when syscalls are patched so that it can used to calculate the restart address. It also adds an empty trampoline to ELF files that don't have syscalls so the runtime knows it's already patched.