Calling Conventions and Stack Frames: System V, Windows x64, and AArch64
A disassembly shows which registers a function reads. Only the ABI says what those registers mean, and the ABI is chosen by the object header, so this lesson reads the header first and then checks every register against the rule that put it there.
140 minutesBuildPython, C, x86-64 and AArch64 assemblyMembers
Learning Objectives
- Pick the calling convention from the object header: ELF x86-64 is the System V psABI, COFF AMD64 is Microsoft x64, ELF AArch64 is AAPCS64, Mach-O arm64 is Apple's variant.
- Decode the prologue of one function byte by byte for each ABI and place every saved register, the return address, and the frame pointer at its offset from the entry stack pointer.
- Infer the parameter registers of a function from the registers it reads before it writes them, and the stack argument slots from the loads above the entry stack pointer.
- Tell the independent register sequences of System V from the positional slots of Microsoft x64, and both from the NGRN and NSRN counters of AAPCS64.
- Check every inference against the source signature, the rule in the specification, and the compiler's own unwind data in
.pdata.xdataand.eh_frame. - Compare how Ghidra, IDA, Binary Ninja, radare2, rizin, and angr name and recover the same facts, and record where their output disagrees with the source.