O'Reilly logo

The Shellcoder's Handbook: Discovering and Exploiting Security Holes, Second Edition by Gerardo Richarte, Felix FX Lindner, John Heasman, Chris Anley

Stay ahead with the world's most comprehensive technology and business learning platform.

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, tutorials, and more.

Start Free Trial

No credit card required

Chapter 7. Windows Shellcode

One author's girlfriend continually reminds him that "writing shellcode is the easy part." And, in fact, it usually is—but like anything on Windows, it can also be an insanely frustrating part. Let's review shellcode for a bit, and then delve into the oddities that make Windows shellcode so entertaining. Along the way, we'll discuss the differences between AT&T and Intel syntax, how the various bugs in the Win32 system will affect you, and the direction of advanced Windows shellcode research.

Syntax and Filters

First, few Windows shellcodes are small enough to work without an encoder/decoder. In any case, if you are writing many exploits, you may want to involve a standardized encoder/decoder API to avoid constantly tweaking your shellcodes. Immunity CANVAS uses an "additive" encoder/decoder. That is, it treats the shellcode as a list of unsigned longs, and for each unsigned long in the list, it adds a number X to it in order to create another unsigned long that has no bad characters in it. To find X, it randomly chooses numbers until one works. This sort of random structure works very well; however, other people are just as happy with XOR or any other character- or word-based operation.

It's important to remember that a decoder is just a function y=f(x) that expands x into a different character space. If x can only contain lowercase alphabetic characters, then f(x) could be a function that transforms lowercase characters into arbitrary binary characters ...

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, interactive tutorials, and more.

Start Free Trial

No credit card required