You are previewing Subclassing and Hooking with Visual Basic.
O'Reilly logo
Subclassing and Hooking with Visual Basic

Book Description

Subclassing & Hooking with Visual Basic offersdevelopers a unique way to customize Windows behavior. Windows is amessage-based system. Every action you request creates one or moremessages to carry out the action. These messages are passed betweenobjects and carry with them information that gives the recipientmore detail on how to interpret and act upon the message. WithSubclassing and the Windows hooking mechanism ("hooks"), you canmanipulate, modify, or even discard messages bound for otherobjects within the operating system, in the process changing theway the system behaves. What kinds of results can you achieve usingthe power of subclassing and hooking? Here are just a few of thepossibilities:

  • Determine when a window is being activated ordeactivated and respond to this change.

  • Display descriptions of menu items as the mousemoves across them.

  • Disallow a user to move or resize a window.

  • Determine where the mouse cursor is and respondaccordingly.

  • Determine when the display resolution has beenchanged.

  • Monitor the system for a low system resourcecondition.

  • Modify or disallow keystrokes sent to a window or acontrol.

  • Create an automated testing application.

  • Determine when an application is idle.

  • Along with this power comes responsibility; Windows is veryunforgiving if subclassing and hooking are used incorrectly.Subclassing & Hooking with Visual Basic demonstrates thevarious techniques for intercepting messages bound for one or morewindows or controls: the intercepted message can be left in itsoriginal state or modified; afterwards, the message can be sent toits original destination or discarded. For both VB 6 and VB.NETdevelopers, Subclassing & Hooking with Visual Basicopens up a wealth of possibilities that ordinarily would becompletely unavailable, or at least not easy to implement.

    Table of Contents

    1. Subclassing and Hooking with Visual Basic
      1. Preface
        1. Who This Book Is For
        2. How This Book Is Structured
        3. Obtaining the Sample Code
        4. Conventions Used in This Book
        5. How to Contact Us
        6. Acknowledgments
      2. I. Introducing Subclassing and Hooking
        1. 1. Introduction
          1. Subclassing
          2. The Window Hooking Mechanism
          3. Tools to Aid Us in Our Efforts
            1. Spy++
              1. Using Spy++ to examine a VB application
            2. NuMega SmartCheck
            3. Dbgwproc.dll
            4. Microsoft System Information
          4. A Word of Warning
        2. 2. Windows System-Specific Information
          1. Inside a Windows Application
            1. Window Relationships
            2. What Are All These Handles For?
              1. hWnd
              2. hInstance
              3. hHook
            3. Processes
            4. Threads
            5. The Internals of a Window
              1. The window class
              2. The window procedure
              3. Creating a simple window application with Visual C++
          2. Inside the Windows Messaging System
            1. The Raw Input Thread
            2. The System Message Queue
            3. Thread Message Queues
            4. Message Loops
            5. Messages
            6. The Different Types of Messages
          3. Performance Considerations
        3. 3. The Basics of Subclassing and Hooks
          1. What Is Subclassing?
            1. How Does Subclassing Work?
            2. The Types of Subclassing
              1. Instance subclassing
              2. Global subclassing
              3. Superclassing
            3. Why Do We Use Subclassing?
          2. What Are Hooks?
            1. How Do Hooks Work?
            2. Hook Scope
              1. Thread-specific
              2. System-wide
            3. Why Do We Use Hooks?
            4. Deciding Between Hooking or Subclassing
      3. II. Subclassing and Superclassing
        1. 4. Subclassing
          1. The AddressOf Operator
            1. Using AddressOf
            2. Callbacks and Enumeration Functions
            3. Callbacks and Asynchronous Processing
            4. AddressOf and Subclassing
          2. Some Subclassing Tips
          3. Instance Subclassing: An Example
            1. The Class Module
            2. The BAS Module
            3. The Form Module
            4. Doing Something Interesting
              1. Overriding window resizing
              2. Overriding a window’s minimize behavior
          4. Global Subclassing
            1. Changes to the Class
            2. Changes to the BAS Module
            3. Changes to the Form
            4. How It All Works
            5. Behind the Scenes with Spy++
          5. Summary of Key Points in Subclassing
        2. 5. Subclassing the Windows Common Dialog Boxes
          1. How Common Dialog BoxSubclassing Works
            1. Using the COMDLG32.DLL
            2. The Resource File
            3. Creating a Dialog Resource File UsingVB and Visual C++
            4. Manipulating Control Placement on the Dialog
          2. Using the Resource
            1. Embedding the Resource in the EXE
            2. Using a Resource in a DLL
              1. Creating a Visual C++ resource DLL
              2. Creating a VB resource DLL
          3. Creating the Subclassing Application
            1. The Code Behind the frmMain.frm Module
            2. The Code Behind the Module1.bas Module
              1. The dialog creation structure
              2. The code for the Module1.bas module
              3. The dialog hook procedure
              4. Dialog initialization
              5. Processing dialog box notification messages
              6. Processing messages from the controls on the child dialog box
              7. Processing the dialog shutdown messages
              8. Default message processing
          4. Subclassing Common Dialog Boxes Other Than Open and Save As
            1. The Dialog Template Resource
              1. Creating a dialog resource file
              2. The Color common dialog
              3. The Font common dialog
              4. The Print common dialog
              5. The Page Setup common dialog
              6. The Print Property Sheet common dialog
              7. The Find common dialog
              8. The Replace common dialog
            2. Receiving Notification and Control Messagesfrom the Dialog
          5. Problems Subclassing the Find and Replace Common Dialogs
        3. 6. ActiveX Controls and Subclassing
          1. Subclassing a Third-PartyActiveX Control
          2. Subclassing an ActiveX ControlCreated in VB
            1. Creating the Control
            2. Creating the Project to Subclass the Control
          3. Subclassing a UserControl from Within a VB-Created ActiveX Control
            1. Creating the Control
              1. The UserControl module
              2. The Module1.bas module
            2. Hosting the Control
          4. Creating an ActiveX Control That Subclasses Other Windows
            1. Creating the First Version of the Control
            2. Creating the Second Version of the Control
            3. Creating the Third Version of the Control
        4. 7. Superclassing
          1. What Is Superclassing?
            1. Similarities and Differences Between Subclassing and Superclassing
            2. When to Use Superclassing
          2. How the Superclassing Example Works
            1. The Code
              1. The module-level declarations
              2. The Sub Main procedure
              3. Message loop operation
              4. Class creation
              5. Window creation
              6. The superclass procedures
          3. Peering into the Superclassing Application with Spy++
        5. 8. Debugging Techniques for Subclassing
          1. Where to Start
            1. Subclassing Checklist
            2. Stepping Through the Application
            3. Log Files
          2. Microsoft Tools
            1. Spy++
            2. DBGWPROC.DLL
          3. NuMega’s SmartCheck
      4. III. Hooking
        1. 9. WH_CALLWNDPROC
          1. How WH_CALLWNDPROC Works
            1. Filter Function Definition
            2. Location of This Hook in the System
          2. A Thread-Specific Example
          3. Caveats
        2. 10. WH_CALLWNDPROCRET
          1. How WH_CALLWNDPROCRET Works
            1. Filter Function Definition
            2. Location of This Hook in the System
          2. A Thread-Specific Example
          3. Caveats
        3. 11. WH_GETMESSAGE
          1. How WH_GETMESSAGE Works
            1. Filter Function Definition
            2. Location of This Hook in the System
          2. A Thread-Specific Example
            1. Other Uses of This Hook
          3. Caveats
        4. 12. WH_KEYBOARD and WH_KEYBOARD_LL
          1. How Keyboard Messaging Operates
          2. Description of Hooks
            1. Filter Function Definitions
            2. Explanation of Hook Codes
            3. Location of This Hook in the System
          3. A Thread-Specific Example
          4. A System-Wide Example
          5. A Low-Level Hook Example
          6. Caveats
        5. 13. WH_MOUSE and WH_MOUSE_LL
          1. How Mouse Messaging Operates
            1. The Mouse Capture
            2. Mouse Click Messages
          2. Description of Hooks
            1. Filter Function Definitions
            2. Explanation of Hook Codes
          3. Location of This Hook in the System
          4. A Single-Thread Example
          5. A System-Wide Hook Example
            1. Low-Level Mouse Hook Example
          6. Caveats
        6. 14. WH_FOREGROUNDIDLE
          1. Description
          2. Location of This Hook in the System
          3. Background Processing Example
          4. Caveats
        7. 15. WH_MSGFILTER
          1. Description
            1. The Modal Loop
            2. Filter Function Definition
            3. Explanation of Hook Codes
            4. Location of This Hook in the System
          2. A Thread-Specific Example
          3. Caveats
        8. 16. WH_SYSMSGFILTER
          1. Description
            1. Filter Function Definition
            2. Explanation of Hook Codes
            3. Location of This Hook in the System
          2. A System-Wide Example
          3. Caveats
        9. 17. WH_SHELL
          1. Description
            1. Filter Function Definition
            2. Explanation of Hook Codes
          2. Location of This Hook in the System
          3. A Thread-Specific Example
            1. Using the Application
            2. The Code
          4. Caveats
        10. 18. WH_CBT
          1. Description
            1. Filter Function Definition
            2. Explanation of Hook Codes
            3. Comparing WH_SHELL to WH_CBT
          2. A Thread-Specific Example
            1. Using the WH_JOURNALPLAYBACK Hookwith a CBT
          3. Caveats
        11. 19. WH_JOURNALRECORD
          1. Description
            1. Stopping This Hook
            2. Explanation of Hook Codes
            3. Location of This Hook in the System
          2. The Macro Recorder Example
          3. Caveats
        12. 20. WH_JOURNALPLAYBACK
          1. Description
            1. Explanation of Hook Codes
            2. Location of This Hook in the System
          2. The Macro Recorder/Playback Example
          3. Caveats
        13. 21. WH_DEBUG
          1. Description
            1. Filter Function Definition
            2. Explanation of Hook Codes
          2. A Thread-Specific Example
          3. Some WH_DEBUG Scenarios
            1. Problem: Misbehaving System-Wide Hook
            2. Problem: Determining if Hooks Are Interfering with Each Other
            3. Problem: Performance Degradation
          4. Caveats
      5. IV. The .NET Platform
        1. 22. Subclassing .NET WinForms
          1. What Is a WinForm?
            1. Using a WinForm
          2. Instance Subclassing
            1. Technique #1: Overriding the WinProc Method
              1. The application
              2. The .VB file
            2. Technique #2: Binding to a NativeWindow Class Instance
              1. The application
              2. The .VB file
            3. Technique #3: The IMessageFilter Interface
              1. The application
              2. The .VB file
          3. Subclassing a Control
            1. The application
              1. The .VB file
          4. Superclassing
            1. The application
              1. The .VB file
          5. Superclassing a Control
            1. The application
              1. The .VB file
          6. Intercepting Keystrokes in a WinForm
            1. The application
              1. The .VB file
          7. Behind the Scenes with Spy++
        2. 23. Implementing Hooks in VB.NET
          1. Hooks and VB.NET
          2. Delegates
            1. A Simple Example Using Delegates
          3. Implementing the WH_KEYBOARD Hook
            1. The Application
            2. The .VB File
          4. Implementing the WH_MOUSE Hook
            1. The Application
            2. The .VB File
      6. V. Appendixes
        1. A. Messages
          1. Window Messages
            1. Public Const WM_NULL = &H0
            2. Public Const WM_CREATE = &H1
            3. Public Const WM_DESTROY = &H2
            4. Public Const WM_MOVE = &H3
            5. Public Const WM_SIZE = &H5
            6. Public Const WM_ACTIVATE = &H6
            7. Public Const WM_SETFOCUS = &H7
            8. Public Const WM_KILLFOCUS = &H8
            9. Public Const WM_SETTEXT = &HC
            10. Public Const WM_PAINT = &HF
            11. Public Const WM_QUIT = &H12
            12. Public Const WM_ERASEBKGND = &H14
            13. Public Const WM_SHOWWINDOW = &H18
            14. Public Const WM_SETCURSOR = &H20
            15. Public Const WM_QUEUESYNC = &H23
            16. Public Const WM_GETMINMAXINFO = &H24
            17. Public Const WM_WINDOWPOSCHANGING = &H46
            18. Public Const WM_WINDOWPOSCHANGED = &H47
            19. Public Const WM_CANCELJOURNAL = &H4B
            20. Public Const WM_NCCREATE = &H81
            21. Public Const WM_NCDESTROY = &H82
            22. Public Const WM_NCCALCSIZE = &H83
            23. Public Const WM_NCHITTEST = &H84
            24. Public Const WM_NCPAINT = &H85
            25. Public Const WM_NCACTIVATE = &H86
            26. Public Const WM_GETDLGCODE = &H87
            27. Public Const WM_NCMOUSEMOVE = &HA0
            28. Public Const WM_NCLBUTTONDOWN = &HA1
            29. Public Const WM_NCLBUTTONUP = &HA2
            30. Public Const WM_NCLBUTTONDBLCLK = &HA3
            31. Public Const WM_NCRBUTTONDOWN = &HA4
            32. Public Const WM_NCRBUTTONUP = &HA5
            33. Public Const WM_NCRBUTTONDBLCLK = &HA6
            34. Public Const WM_NCMBUTTONDOWN = &HA7
            35. Public Const WM_NCMBUTTONUP = &HA8
            36. Public Const WM_NCMBUTTONDBLCLK = &HA9
            37. Public Const WM_NCXBUTTONDOWN = &HAB
            38. Public Const WM_NCXBUTTONUP = &HAC
            39. Public Const WM_NCXBUTTONDBLCLK = &HAD
            40. Public Const WM_KEYDOWN = &H100
            41. Public Const WM_KEYUP = &H101
            42. Public Const WM_CHAR = &H102
            43. Public Const WM_SYSKEYDOWN = &H104
            44. Public Const WM_SYSKEYUP = &H105
            45. Public Const WM_SYSCHAR = &H106
            46. Public Const WM_INITDIALOG = &H110
            47. Public Const WM_COMMAND = &H111
            48. Public Const WM_SYSCOMMAND = &H112
            49. Public Const WM_ENTERIDLE = &H121
            50. Public Const WM_CTLCOLOREDIT = &H133
            51. Public Const WM_CTLCOLORLISTBOX = &H134
            52. Public Const WM_CTLCOLORBTN = &H135
            53. Public Const WM_CTLCOLORDLG = &H136
            54. Public Const WM_CTLCOLORSCROLLBAR = &H137
            55. Public Const WM_CTLCOLORSTATIC = &H138
            56. Public Const WM_MOUSEMOVE = &H200
            57. Public Const WM_LBUTTONDOWN = &H201
            58. Public Const WM_LBUTTONUP = &H202
            59. Public Const WM_LBUTTONDBLCLK = &H203
            60. Public Const WM_RBUTTONDOWN = &H204
            61. Public Const WM_RBUTTONUP = &H205
            62. Public Const WM_RBUTTONDBLCLK = &H206
            63. Public Const WM_MBUTTONDOWN = &H207
            64. Public Const WM_MBUTTONUP = &H208
            65. Public Const WM_MBUTTONDBLCLK = &H209
            66. Public Const WM_XBUTTONDOWN = &H20B
            67. Public Const WM_XBUTTONUP = &H20C
            68. Public Const WM_XBUTTONDBLCLK = &H20D
            69. Public Const WM_PARENTNOTIFY = &H210
            70. Public Const WM_ENTERMENULOOP = &H211
            71. Public Const WM_EXITMENULOOP = &H212
            72. Public Const WM_SIZING = &H214
            73. Public Const WM_CAPTURECHANGED = &H215
            74. Public Const WM_MOVING = &H216
            75. Public Const WM_ENTERSIZEMOVE = &H231
            76. Public Const WM_EXITSIZEMOVE = &H232
            77. Public Const WM_HOTKEY = &H312
          2. Button Control-Specific Messages
            1. Public Const BM_GETCHECK = &HF0
            2. Public Const BM_SETCHECK = &HF1
            3. Public Const BM_CLICK = &HF5
          3. Combo Box-Specific Messages
            1. Public Const CB_ADDSTRING = &H143
            2. Public Const CB_DELETESTRING = &H144
            3. Public Const CB_GETCOUNT = &H146
            4. Public Const CB_GETCURSEL = &H147
          4. Edit Control-Specific Messages
            1. Public Const EM_GETLINE = &HC4
            2. Public Const EM_CANUNDO = &HC6
            3. Public Const EM_UNDO = &HC7
          5. Listbox-Specific Messages
            1. Public Const LB_ADDSTRING = &H180
            2. Public Const LB_DELETESTRING = &H182
            3. Public Const LB_SETSEL = &H185
            4. Public Const LB_GETSEL = &H187
          6. Scroll Bar-Specific Messages
            1. Public Const SBM_SETPOS = &HE0
            2. Public Const SBM_GETPOS = &HE1
            3. Public Const SBM_SETRANGE = &HE2
            4. Public Const SBM_GETRANGE = &HE3
          7. Messages Specific to the CommonDialog Boxes
            1. Public Const CDM_GETFILEPATH = (CDM_FIRST + &H1)
            2. Public Const CDM_GETFOLDERPATH = (CDM_FIRST + &H2)
            3. Public Const CDM_GETSPEC = (CDM_FIRST + &H0)
            4. Public Const CDM_HIDECONTROL = (CDM_FIRST + &H5)
            5. Public Const CDM_SETCONTROLTEXT = (CDM_FIRST + &H4)
            6. Public Const CDM_SETDEFEXT = (CDM_FIRST + &H6)
          8. Messages Specific to the Font Common Dialog Box
            1. Public Const WM_CHOOSEFONT_GETLOGFONT = _USER + 1)
            2. Public Const WM_CHOOSEFONT_SETLOGFONT = (WM_USER + 101)
            3. Public Const WM_CHOOSEFONT_SETFLAGS = (WM_USER + 102)
          9. Messages Specific to the Page Setup Common Dialog Box
            1. Public Const WM_PSD_PAGESETUPDLG = (WM_USER)
            2. Public Const WM_PSD_FULLPAGERECT = (WM_USER+1)
            3. Public Const WM_PSD_MINMARGINRECT = (WM_USER+2)
            4. Public Const WM_PSD_MARGINRECT = (WM_USER+3)
            5. Public Const WM_PSD_ENVSTAMPRECT = (WM_USER+5)
            6. Public Const WM_PSD_FULLPAGERECT = (WM_USER+6)
        2. B. API Functions
          1. AttachThreadInput
          2. BroadcastSystemMessage
          3. CallNextHookEx
          4. CallWindowProc
          5. ChooseColor
          6. ChooseFont
          7. CopyMemory
          8. CreateWindowEx
          9. DefWindowProc
          10. EnumChildWindows
          11. FindText
          12. FindWindow
          13. FindWindowEx
          14. GetClassInfoEx
          15. GetDeskTopWindow
          16. GetDlgItem
          17. GetMessage
          18. GetParent
          19. GetSaveFileName
          20. GetWindow
          21. GetWindowLong
          22. GetWindowRect
          23. OutputDebugStringA
          24. PageSetupDlg
          25. PeekMessage
          26. PostMessage
          27. PostQuitMessage
          28. PostThreadMessage
          29. PrintDlg
          30. PrintDlgEx
          31. RegisterClassEx
          32. RegisterWindowMessage
          33. ReleaseCapture
          34. ReplaceText
          35. SendDlgItemMessage
          36. SendMessage
          37. SendMessageCallback
          38. SendMessageTimeout
          39. SendNotifyMessage
          40. SetCapture
          41. SetClassLongPtr
          42. SetWindowLongPtr
          43. SetWindowPos
          44. SetWindowsHookEx
          45. ShowWindow
          46. UnhookWindowsHookEx
          47. UnregisterClass
        3. C. Structures and Constants
          1. CBT_CREATEWND
            1. CBT_CREATEWND
          2. CBTACTIVATESTRUCT
            1. CBTACTIVATESTRUCT
          3. CHOOSECOLOR
            1. CHOOSECOLOR
          4. CHOOSEFONT
            1. CHOOSEFONT
          5. CWPRETSTRUCT
            1. CWPRETSTRUCT
          6. CWPSTRUCT
            1. CWPSTRUCT
          7. DEBUGHOOKINFO
            1. DEBUGHOOKINFO
          8. EVENTMSG
            1. EVENTMSG
          9. FINDREPLACE
            1. FINDREPLACE
          10. KBDLLHOOKSTRUCT
            1. KBDLLHOOKSTRUCT
          11. MOUSEHOOKSTRUCT
            1. MOUSEHOOKSTRUCT
          12. MOUSEHOOKSTRUCTEX
            1. MOUSEHOOKSTRUCTEX
          13. MSG
            1. MSG
          14. MSLLHOOKSTRUCT
            1. MSLLHOOKSTRUCT
          15. NMHDR
            1. NMHDR
          16. OPENFILENAME
            1. OPENFILENAME
          17. PAGESETUPDLG
            1. PAGESETUPDLG
          18. POINTAPI
            1. POINTAPI
          19. PRINTDLG
            1. PRINTDLG
          20. PRINTDLGEX
            1. PRINTDLGEX
          21. RECT
            1. RECT
          22. WNDCLASSEX
            1. WNDCLASSEX
          23. Message (.NET Structure)
            1. Message (.NET Structure)
          24. Constants
            1. BroadcastSystemMessage Function Masks
            2. Common Dialog Box Error Messages
            3. Common Dialog Box Notification Messages
            4. Class Styles
            5. Edit Control Notification Codes
            6. Extended Window Styles
            7. GetClassLongPtr Function Constants
            8. GetWindow API Function Constants
            9. GetWindowLongPtr Function Constants
            10. Hook Codes
              1. General
              2. Computer-based training (CBT) hook codes
              3. WH_MSGFILTER hook codes
              4. WH_SHELL hook codes
            11. HSHELL_ACCESSIBILITYSTATE Constants
            12. Key State Masks for Mouse Messages
            13. Low-Level Hook Constants
            14. Mouse Wheel Constant
            15. Open and Save As Common Dialog BoxNotification Codes
            16. PeekMessage Function Options
            17. PostMessage and SendMessage Function Constants
            18. Predefined Control ID Values for the Open andSave As Common Dialog Boxes
            19. SendMessageTimeout API Function Constants
            20. SetWindowLongPtr/GetWindowLongPtr Function Constants
            21. SetWindowPos Function Constants
            22. ShowWindow Function Constants
            23. User Button Notification Codes
            24. Virtual Key Codes
            25. Window Hook Constants
            26. Window Styles
            27. WM_KEYUP, WM_DOWN, andWM_CHAR lParam HIWORD Flags
            28. WM_NCHITTEST Codes
            29. WH_SHELL Codes for the HSHELL_APPCOMMAND Hook Code
            30. WM_SIZE Message wParam Values
            31. WM_SYSCOMMAND Message wParam Constants
            32. XButton Constants
      7. Index
      8. Colophon