Disassembly and Control Flow: From Bytes to a Graph You Can Check
A disassembly listing is a claim about which bytes are instructions, and this lesson builds the decoder and graph that check the claim.
150 minutesBuildPython, C, x86-64 assemblyMembers
Learning Objectives
- Decode the x86-64 instruction subset in the ledger fixtures from the rules in Intel SDM Volume 2, Chapter 2.
- Prove the decoder against
llvm-objdump -don every instruction of every fixture build. - Run a linear sweep and a recursive descent over the same bytes and explain where each one is wrong.
- Cut instructions into basic blocks and build a control-flow graph with typed edges.
- Compute immediate dominators with the iterative algorithm of Cooper, Harvey, and Kennedy, and find loops from back edges.
- Recover the targets of an indirect jump from the relocation entries that point at its table.