Windows Binary Exploitation
October 29, 2025
Yesterday's meeting covered the basics of Binary Exploitation of the Windows operating system, and highlighted three key concepts central to understanding the operating system's binary execution and its weaknesses. Members were then given a set of challenges to practice these techniques on.
Memory Architecture
The first topic of interest is the Memory Architecture and Anatomy of Windows. Like in all operating systems, Windows' memory structure is linear and contains a kernel, a heap, and a stack. Windows' memory architecture also contains a section for Dynamic Link Libraries (DLLs), Process Metadata (PEB), and Thread Metadata (TEB). The architecture executes commands using registers, small data holders which are used for data manipulation and can be directly modified by binaries.
Buffer Overflows
With this base knowledge in mind, buffer overflows are unsafe implementations of memory management which may allow users to interact with Windows' memory in unintended ways, namely while accepting user input. If a program attempts to write unrestricted user input to a memory buffer (a section of memory which has been allocated) which is too small, the user's input will overwrite the data stored beyond the buffer, which may damage the program or cause unintended consequences, and may be used to read hidden data or execute arbitrary code.
Mitigations
Many Windows programs are compiled with mitigations which attempt to prevent damage from binary attacks. Binaries compiled using ASLR (Address Space Layout Randomization) will randomize the location of the program instruction data before execution so as to prevent attacks which rely on raw instruction addresses. Binaries compiled with DEP (Data Execution Prevention) will mark certain areas in memory as non-executable, causing the process to panic if any code is executed in that section or the stack. Many binaries are compiled with a Stack Canary, which is a checksum stored on the stack which aims to prevent buffer overflows. And there are many other mitigations that Windows OS implements, such as CFG, CET, SEHOP, ACG, and CIG.