12.9. Using Function Pointers

Problem

By knowing which functions are called—either directly or indirectly—a programmer can understand the operation of a compiled program without resorting to runtime analysis.

Solution

The address of a function will always be visible in memory before it is called; however, by storing an obfuscated version of the function pointer, disassemblers and cross-reference analysis tools will fail to recognize the stored pointer as a code address. Note that this technique will not work with function pointers that require relocation, such as the addresses of functions in shared libraries.

Discussion

Function pointers can be handled like other variables. Because they are essentially compile-time constants, it is best to use a technique that obfuscates them at compile time. It is important that the functions created using the SET_FN_PTR macro presented below be inlined by the compiler so that they do not appear in the resulting executable’s symbol table; otherwise, they will be obvious tip-offs to a cracker that something is not as it should be.

#define SET_FN_PTR(func, num)                  \
        static inline void *get_##func(void) { \
          int  i, j = num / 4;                 \
          long ptr = (long)func + num;         \
          for (i = 0;  i < 2;  i++) ptr -= j;  \
          return (void *)(ptr - (j * 2));      \
        }
#define GET_FN_PTR(func) get_##func(  )

With the SET_FN_PTR macro, the pointer to a function is returned by a routine that stores the function pointer modified by a programmer-supplied value. The GET_FN_PTR macro calls this routine, ...

Get Secure Programming Cookbook for C and C++ now with the O’Reilly learning platform.

O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.