You are previewing Debugging Applications for Microsoft® .NET and Microsoft Windows®, Second Edition.
O'Reilly logo
Debugging Applications for Microsoft® .NET and Microsoft Windows®, Second Edition

Book Description

Learn application debugging techniques for Microsoft® .NET and Microsoft Windows® from a debugging expert. This updated programming guide includes new scenarios, bug-killing tools, tips, and source code and covers topics on Microsoft Visual Studio® .NET.

Table of Contents

  1. Debugging Applications for Microsoft® .NET and Microsoft Windows®
  2. Dedication
  3. A Note Regarding Supplemental Files
  4. Acknowledgments
  5. Introduction
    1. Who Should Read This Book?
    2. How to Read This Book and What's New in the Second Edition
    3. System Requirements
    4. What Comes with This Book's Sample Files?
      1. READ THIS! Windows 98/Me and ANSI Code
      2. READ THIS! The DBGHELP.DLL Symbol Engine
    5. Feedback
    6. Microsoft Press Support Information
  6. I. The Gestalt of Debugging
    1. 1. Bugs: Where They Come From and How You Solve Them
      1. Bugs and Debugging
        1. What Are Bugs?
          1. Inconsistent User Interfaces
          2. Unmet Expectations
          3. Poor Performance
          4. Crashes or Data Corruption
        2. Process Bugs and Solutions
          1. Short or Impossible Deadlines
          2. The "Code First, Think Later" Approach
          3. Misunderstood Requirements
          4. Engineer Ignorance or Improper Training
          5. Lack of Commitment to Quality
        3. Planning for Debugging
      2. Prerequisites to Debugging
        1. The Skill Set
          1. Know Your Project
          2. Know Your Language
          3. Know Your Technology/Tools
          4. Know Your Operating System/Environment
          5. Know Your CPU
        2. Learning the Skill Set
      3. The Debugging Process
        1. Step 1: Duplicate the Bug
        2. Step 2: Describe the Bug
        3. Step 3: Always Assume That the Bug Is Yours
        4. Step 4: Divide and Conquer
        5. Step 5: Think Creatively
        6. Step 6: Leverage Tools
        7. Step 7: Start Heavy Debugging
        8. Step 8: Verify That the Bug Is Fixed
        9. Step 9: Learn and Share
        10. Final Debugging Process Secret
      4. Summary
    2. 2. Getting Started Debugging
      1. Track Changes Until You Throw Away the Project
        1. Version Control Systems
          1. The Importance of Including Unit Tests
          2. Controlling Changes
          3. The Importance of Labeling
        2. Bug Tracking Systems
        3. Choosing the Right Systems for You
      2. Schedule Time for Building Debugging Systems
        1. Build All Builds with Debugging Symbols
        2. For Managed Code, Treat Warnings as Errors
        3. For Native Code, Treat Warnings as Errors—Mostly
        4. For Native Code, Know Where Your DLLs Load
        5. What About Managed Modules and Base Addresses?
        6. Design a Lightweight Diagnostic System for Release Builds
      3. Frequent Builds and Smoke Tests Are Mandatory
        1. Frequent Builds
        2. Smoke Tests
      4. Build the Installation Program Immediately
      5. QA Must Test with Debug Builds
      6. Install the Operating System Symbols and Set Up a Symbol Store
        1. Your Source and Symbol Servers
      7. Summary
    3. 3. Debugging During Coding
      1. Assert, Assert, Assert, and Assert
        1. How and What to Assert
          1. How to Assert
          2. What to Assert
        2. Assertions in .NET Windows Forms or Console Applications
          1. Controlling the TraceListener Object with Configuration Files
        3. Assertions in ASP.NET Applications and XML Web Services
        4. Assertions in Native C++ Applications
          1. The VERIFY Macro
        5. Different Types of Visual C++ Assertions
          1. assert, _ASSERT, and _ASSERTE
          2. ASSERT_KINDOF and ASSERT_VALID
        6. SUPERASSERT
          1. Using SUPERASSERT
          2. A Word About Ignoring Assertions
          3. SUPERASSERT Implementation Highlights
      2. Trace, Trace, Trace, and Trace
        1. Tracing in Windows Forms and Console .NET Applications
        2. Tracing in ASP.NET Applications and XML Web Services
        3. Tracing in Native C++ Applications
      3. Comment, Comment, Comment, and Comment
      4. Trust Yourself, but Verify (Unit Testing)
      5. Summary
  7. II. Power Debugging
    1. 4. Operating System Debugging Support and How Win32 Debuggers Work
      1. Types of Windows Debuggers
        1. User-Mode Debuggers
        2. Kernel-Mode Debuggers
          1. Kernel Debugger (KD)
          2. WinDBG
          3. SoftICE
      2. Windows Operating System Support for Debuggees
        1. Just-In-Time (JIT) Debugging
          1. Quick Break Keys with the UserDebuggerHotKey
          2. The Debugger Value
          3. Choosing the Debugger at Crash Time
        2. Automatically Starting in a Debugger (Image File Execution Options)
      3. MinDBG: A Simple Win32 Debugger
      4. WDBG: A Real Debugger
        1. Reading and Writing Memory
        2. Breakpoints and Single Stepping
        3. Symbol Tables, Symbol Engines, and Stack Walking
          1. The Different Symbol Formats
          2. Accessing Symbol Information
          3. Walking the Stack
        4. Step Into, Step Over, and Step Out
      5. So You Want to Write Your Own Debugger
      6. What's Next for WDBG?
      7. Summary
    2. 5. Advanced Debugger Usage with Visual Studio .NET
      1. Advanced Breakpoints and How to Use Them
        1. Breakpoint Tips
        2. Quickly Breaking on Any Function
        3. Location Breakpoint Modifiers
          1. Hit Counts
          2. Conditional Expressions
        4. Multiple Breakpoints on a Single Line
      2. The Watch Window
        1. Calling Methods in the Watch Window
        2. The Set Next Statement Command
      3. Summary
    3. 6. Advanced .NET Debugging with Visual Studio .NET
      1. Advanced Breakpoints for .NET Programs
        1. Conditional Expressions
      2. The Watch Window
        1. Expanding Your Own Types Automatically
      3. Tips and Tricks
        1. DebuggerStepThroughAttribute and DebuggerHiddenAttribute
        2. Mixed Mode Debugging
        3. Remote Debugging
      4. ILDASM and Microsoft Intermediate Language
        1. Getting Started with ILDASM
        2. CLR Basics
        3. MSIL, Locals, and Parameter
        4. The Important Instructions
        5. Other Reverse Engineering Tools
      5. Summary
    4. 7. Advanced Native Code Techniques with Visual Studio .NET
      1. Advanced Breakpoints for Native Applications
        1. Advanced Breakpoint Syntax
        2. Breakpoints on System or Exported Functions
        3. Conditional Expressions
        4. Data Breakpoints
          1. Better Data Breakpoints
      2. The Watch Window
        1. Formatting Data and Expression Evaluation
        2. Timing Code in the Watch Window
        3. The Undocumented Pseudoregisters
        4. Expanding Your Own Types Automatically
          1. Adding Your Own HRESULT Values
          2. Adding Super Customized Display to the Watch Window
      3. Remote Debugging
      4. Tips and Tricks
        1. Debugging Injected Code
        2. The Memory Window and Auto Memory Evaluation
        3. Exception Monitoring
        4. More Symbol Handling Tips
        5. Detaching from Windows 2000 Processes
        6. Handling Dump Files
      5. x86 Assembly Language
        1. The Basics of the CPU
          1. Registers
          2. Instruction Format and Memory Addressing
        2. A Word About the Visual C++ .NET Inline Assembler
        3. Instructions You Need to Know
          1. Stack Manipulation
          2. Very Common Simple Instructions
        4. Common Sequence: Function Entry and Exit
          1. Pointer Manipulation
        5. Calling Procedures and Returning
        6. Calling Conventions
          1. Calling Conventions Example
        7. Variable Access: Global Variables, Parameters, and Local Variables
        8. More Instructions You Need to Know
          1. Data Manipulation
          2. Comparing and Testing
          3. Jump and Branch Instructions
          4. Looping
        9. String Manipulation
        10. Common Assembly-Language Constructs
          1. FS Register Access
        11. Structure and Class References
        12. A Complete Example
        13. The Disassembly Window
          1. Navigating
          2. Viewing Parameters on the Stack
          3. The Set Next Statement Command
        14. Walking the Stack Manually
        15. Tips and Tricks
          1. Endians
          2. Garbage Code
          3. Registers and the Watch Window
          4. Learn from ASM Files
      6. Summary
    5. 8. Advanced Native Code Techniques with WinDBG
      1. Before You Begin
      2. The Basics
      3. Debugging Situations
        1. Getting Help
        2. Ensuring Correct Symbols Are Loaded
        3. Processes and Threads
          1. Debugging Child Processes
          2. Viewing Processes and Threads in the Command Window
          3. Creating Processes from the Command Window
          4. Attaching to and Detaching from Processes
          5. Noninvasive Attaching
      4. General Debugging with the Command Window
        1. Looking at and Evaluating Variables
        2. Executing, Stepping, and Tracing
          1. Trace and Watch Data
        3. Breakpoints
          1. General Breakpoints
          2. Memory Access Breakpoints
        4. Exceptions and Events
        5. Controlling WinDBG
      5. The Magical Extensions
        1. Loading and Controlling Extensions
        2. The Important Extension Commands
          1. Other Interesting Extension Commands
      6. Dealing with Dump Files
        1. Creating Dump Files
        2. Opening Dump Files
        3. Debugging the Dump
      7. Son of Strike (SOS)
        1. Using SOS
      8. Summary
  8. III. Power Tools and Techniques for .NET
    1. 9. Extending the Visual Studio .NET IDE
      1. Extending with Macros
        1. Macro Parameters
        2. Problems with Projects
        3. Code Elements
      2. CommenTater: The Cure for the Common Potato?
      3. Introduction to Add-Ins
        1. Fixing the Add-In Wizard–Generated Code
        2. Handling Toolbar Button Issues
        3. Creating Tool Windows
        4. Creating Options Property Pages with Managed Code
      4. The SuperSaver Add-In
      5. The SettingsMaster Add-In
        1. SettingsMaster Implementation Highlights
        2. SettingsMaster Future Enhancements
      6. Summary
    2. 10. Managed Exception Monitoring
      1. Introduction to the Profiling API
        1. Getting Your Profiler Started
      2. ProfilerLib
      3. ExceptionMon
        1. In-Process Debugging and ExceptionMon
      4. Exception Usage in .NET
      5. Summary
    3. 11. Flow Tracing
      1. Hooking Functions in the Profiling API
        1. Requesting Enter and Leave Notifications
        2. Implementing the Hook Functions
        3. Inlining
        4. The Function ID Mapper
      2. Using FlowTrace
      3. FlowTrace Implementation Highlights
      4. What's Next for FlowTrace
      5. Summary
  9. IV. Power Tools and Techniques for Native Code
    1. 12. Finding Source and Line Information with Just a Crash Address
      1. Creating and Reading a MAP File
        1. MAP File Contents
        2. Finding the Source File, Function Name, and Line Number
        3. PDB2MAP—Map Files After the Fact
      2. Using CrashFinder
        1. Implementation Highlights
      3. What's Next for CrashFinder?
      4. Summary
    2. 13. Crash Handlers
      1. Structured Exception Handling vs. C++ Exception Handling
        1. Structured Exception Handling
        2. C++ Exception Handling
        3. Avoid Using C++ Exception Handling
          1. Absolutely, Positively, NEVER EVER Use catch ( … )
          2. Don't Use _set_se_translator
      2. The SetUnhandledExceptionFilter API Function
      3. Using the CrashHandler API
      4. Translating EXCEPTION_POINTERS Structures
      5. Minidumps
        1. The MiniDumpWriteDump API
        2. Pacifying MiniDumpWriteDump
      6. Summary
    3. 14. Debugging Windows Services and DLLs That Load into Services
      1. Service Basics
        1. The API Dance
        2. The Security Dance
      2. Debugging Services
        1. Debugging the Core Code
          1. COM+ Services
          2. ISAPI Filters and Extensions
          3. Exchange Server
        2. Debugging the Basic Service
          1. Turn On Allow Service To Interact With Desktop
          2. Set Your Service Identity
          3. Attach to Your Service
          4. Debug Your Startup Code
      3. Summary
    4. 15. Multithreaded Deadlocks
      1. Multithreading Tips and Tricks
        1. Don't Do It
        2. Don't Overdo It
        3. Multithread Only Small, Discrete Pieces
        4. Synchronize at the Lowest Level
        5. Spin Your Critical Sections
        6. Don't Use CreateThread/ExitThread
        7. The Default Memory Manager Might Kill You
        8. Get the Dump in the Field
        9. Review the Code—And Review the Code Again
        10. Test on Multiprocessor Machines
      2. DeadlockDetection Requirements
      3. High-Level Design Issues with DeadlockDetection
      4. Using DeadlockDetection
      5. Implementing DeadlockDetection
        1. Hooking Imported Functions
        2. Implementation Highlights
      6. What's Next for DeadlockDetection?
      7. Summary
    5. 16. Automated Testing
      1. The Bane of Unit Testing: User Interfaces
      2. Tester Requirements
      3. Using Tester
        1. Tester Scripts
        2. Recording Scripts
      4. Implementing Tester
        1. The TESTER.DLL Notification and Playback Implementation
        2. Implementing TESTREC.EXE
          1. Processing Keystrokes
          2. Processing Mouse Input
      5. What's Next for Tester?
      6. Summary
    6. 17. The Debug C Run-Time Library and Memory Management
      1. Features of the Debug C Run-Time Library
      2. Using the Debug C Run-Time Library
        1. The Bug in the DCRT
        2. Useful DCRT Functions
      3. Choosing the Right C Run-Time Library for Your Application
      4. Using MemDumperValidator
        1. Using MemDumperValidator with C++
        2. Using MemDumperValidator with C
        3. Deep Validations
      5. Implementing MemDumperValidator
        1. Initialization and Termination in C++
        2. Where Have All the Leak Reports Gone?
      6. Using MemStress
        1. Interesting Stress Problems
      7. Operating System Heaps
      8. Tricks for Hunting Down Memory Problems
        1. Finding Uninitialized Memory Writes
        2. Finding Memory Overruns
          1. Application Compatibility Toolkit
      9. Killer Switches
        1. The Run-Time Check Switches
          1. Controlling Run-Time Check Output
        2. The Buffer Security Check Switch
      10. Summary
    7. 18. FastTrace: A High-Performance Tracing Tool for Server Applications
      1. The Fundamental Problem and Its Solution
      2. Using FastTrace
        1. Merging Trace Logs
      3. FastTrace Implementation
      4. Summary
    8. 19. Smoothing the Working Set
      1. Working Set Tuning
      2. Using SWS
        1. Setting Up the SWS Compiles
        2. Running Applications Using SWS
        3. Producing and Using the Order File
      3. Implementing SWS
        1. The_penter Function
        2. The .SWS File Format and Symbol Enumeration
        3. The Run Time and Tuning
      4. What's Next for SWS?
      5. Summary
  10. V. Appendixes
    1. A. Reading Dr. Watson Logs
      1. Dr. Watson Logs
    2. B. Resources for .NET and Windows Developers
      1. Books
        1. Software Development
        2. Debugging and Testing
        3. .NET Technologies
        4. C/C++ Language
        5. Windows and Windows Technologies
        6. The Intel CPU and Hardware
      2. Software Tools
      3. Web Sites
  11. Index
  12. Copyright