WO2002103493A2 - Hook and subclass manager - Google Patents

Abstract

A data processing system comprising a processor, a memory, an application, an application stack, and a hook manager (42) for managing hooking and unhooking of functions called by the application (41). The hook manager maintains a handler chain (45) of handler functions and adjusts the application stack using a dynamically constructed data structure during processing of the handler functions for the application. The present solution is distinguishable over conventional techniques in that it only one stack is manipulated, namely the application stack, hence reducing processing overhead.

Description

TITLE

Hook and Subclass Manager

CLAIM OF PRIORITY/CROSS REFERENCE OF RELATED APPLICATION(S)

This international application claims the benefit of priority of United States Non- Provisional Application Number 09/994,132, filed November 26, 2001, which claims the benefit of United States Provisional Application Number 60/273,386, filed March 5, 2001. The entire disclosures contained in the above-mentioned applications are incorporated by reference as if set forth at length herein.

STATEMENT REGARDING FEDERALLY SPONSORED RESEARCH OR DEVELOPMENT

Not applicable.

REFERENCE OF AN APPENDIX The accompanying Appendices, which are part of the present disclosure, are incorporated herein by reference in its entirety. The Appendices include information pertaining to embodiments of the present solution as described more completely below.

BACKGROUND

1. Field of the Invention

The present invention relates generally to data processing and more particularly, to an apparatus, method and article of manufacture for managing the hooking and unhooking of arbitrary functions and for managing the subclassing of window procedures.

2. Description of Related Art

A. Hooking

Several techniques exist for hooking an Application Programming Interface (API) function, including but not limited to import table patching, simple code overwriting and extended code overwriting. The widely used import table patching technique provides for API hooking by patching a Microsoft® Windows® application's import table (set of API functions that an application calls). Unfortunately, dynamically linked APIs do not access the import table and therefore are not processed by this technique

Simple code overwriting, on the other hand, provides for the processing of dynamically linked APIs by directly manipulating an API's binary code in memory. Specifically, the first couple of bytes of the API's code, generally, five bytes of the API code, is replaced with a JMP instruction, which then transfers control to a callback function. However, even this technique has its shortcomings. The biggest disadvantage is that the original API cannot be called in the callback function as long as the API is hooked because the JMP instruction would transfer control to the callback function resulting in an endless loop.

Extended code overwriting overcomes certain limitations of simple code overwriting by copying the first couple of bytes of the API's code to another location. Thus, whenever the original API is called, the instructions are accessed from the other location. However, even this method has its disadvantages - shared APIs can only be hooked system wide and crashes may be unavoidable based upon how the API's code is structured. B. Subclassing

According to Kyle Marsh's article entitled "Safe Subclassing in Win32," Microsoft Developer Network Technology Group, January 25, 1994, "subclassing is a technique that allows an application to intercept messages destined for another window. An application can augment, monitor, or modify the default behavior of a window by intercepting messages meant for another window. Subclassing is an effective way to change or extend the behavior of a window without redeveloping the window. Subclassing the default control window classes (button controls, edit controls, list controls, combo box controls, static controls, and scroll bar controls) is a convenient way to obtain the functionality of the control and to modify its behavior." For further details, the reader is urged to review the article, which is incorporated by reference in its entirety herein and is further set forth as Appendix A.

Petter Hesselberg's article entitled 'The User Interface Programming" by, Windows Developer, which is incorporated by reference in its entirety herein and is further set forth as Appendix B, describes "wdjSUB" a general subclassing library available for download at "http://www.wd- mag.com/articles/2000/0003/0003f/0003f. htm?topic=articles", that solves several subclassing issues. For example, "to allow separate parts of an application's code to subclass the same window and yet unhook in any order, subclassings must, unfortunately but necessarily, be aware of one another. To this end, wdjSUB, maintains a linked list of subclassing descriptors for each subclassed window:

typedef struct SUBCLASSING { WNDPROC wndProc; WNDPROC wndProcSaved; void *pData; struct SUBCLASSING *pNext; } SUBCLASSING;

Subclassing is now a matter of attaching a new SUBCLASSING node to the head of the list and replacing the window function. Unhooking is a matter of removing a node from the list; restoration of the original window function takes place only when the head of the list is removed."

The above approach however, is vulnerable to harmful effects of "foreign" subclassings. SUMMARY

The present solution addresses the aforementioned problems of the prior art.

In accordance with one aspect of the present invention, there is generally provided an apparatus, method and article of manufacture embodying a Hook Manager for managing the hooking and unhooking of arbitrary functions. The Hook Manager of the present invention is distinguishable over conventional techniques in that it manipulates only one stack, namely the application stack, hence reducing processing overhead.

In accordance with a second aspect of the present invention, there is generally provided an apparatus, method and article of manufacture embodying a Subclass Manager for managing the subclassing of window procedures. In cooperation with the Hook Manager described herein and in order to avoid code destruction resulting from an application's own subclassing, Hook Manager is injected into the application's address space as part of the subclassing code. Hook Manager then installs hooks on the API functions GetWindowLong and SetWindowLong and operates in accordance with the principles described herein. The above-mentioned aspect(s) and other aspects, features and advantages of the present invention will become better understood with regard to the following description, appended claims, and accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWING(S)

Referring briefly to the drawings, embodiments of the present invention will be described with reference to the accompanying drawings in which:

Figure 1 is a block diagram of one embodiment of a data processing system constructed in accordance with the teachings herein.

Figure 2 is a block diagram of an alternative view of one embodiment of the present solution constructed in accordance with the teachings herein.

Figures 3 A and B are block diagrams of further alternative views of embodiments of the present solution constructed in accordance with the teachings herein.

Figure 4 is a block diagram of an additional alternative view of one embodiment of the present solution constructed in accordance with the teachings herein. Figures 5 A and 5B are block diagrams of further alternative views of embodiments of the present solution constructed in accordance with the teachings herein.

DETAILED DESCRIPTION

Referring more specifically to the drawings, for illustrative purposes the present invention is embodied in the system configuration, method of operation and product or computer-readable medium, such as floppy disks, conventional hard disks, CD-ROMS, Rash ROMS, nonvolatile ROM, RAM and any other equivalent computer memory device, generally shown in Figures 1 - 5. It will be appreciated that the system, method of operation and product may vary as to the details of its configuration and operation without departing from the basic concepts disclosed herein.

FIG. 1 is a block diagram of an exemplary data processing system that is suitable for practicing the present invention in accordance with the teachings presented herein. The data processing system of FIG. 1 comprises a central processing unit, a memory, a bus and an input/output (I/O) subsystem. The aforementioned components of the data processing system may be embodied within a personal computer, a portable computer, a workstation, a minicomputer or a supercomputer. As such, the details of the physical embodiment of the data processing system , such as the structure of the bus or the number of CPUs that are coupled to the bus, is not crucial to the operation of the present invention, and is not described in further detail below.

Hook Manager

In accordance with the present invention, the data processing system further includes a Hook Manager component (hereinafter, "Hook Manager") for managing the hooking and unhooking of arbitrary functions. In one embodiment of the present invention, Hook Manager is implemented as a dynamic link library of functions and related data structures. Specific functions, for example, include "Handler", "SetHook", "ReleaseHook" and "AppStackCall." However, before turning to the details of the aforementioned functions, first the data structures used herein will be described.

In managing the hooking/unhooking process, Hook Manager uses several key data structures. These data structures are described below with reference to FIG. 2. FUNCTION_CONTROL_BLOCK 34

During program execution, an application may hook a Callee function several times. Thus, as a result of multiple calls by an application to the SetHook and ReleaseHook functions, a Handler Chain is constructed for the particular Callee function. In one embodiment of the present invention, the Handler Chain is implemented as a linked list data structure. In constructing the Handler Chain, a FUNCτiON_CONTROL_BLOCK 34 data structure is created for each hook and added as an element of the linked list data structure. The FUNCTION_CONTROL_BLOCK 34 data structure may be defined, for example, as follows:

struct FUNCTION_CONTROL_BLOCK { HOOKID HookTD; // id of current hook VOID ""Handler; // references the Handler function VOID *Callee; // references the Callee FUNCTION_CONTROL_BLOCK *Next; // NULL or the next // element in the linked list } As indicated above, each FUNCTION_CONTROL_BLOCK 34 data structure includes several fields: a HookJD field, a Handler field, a Callee field and a Next field. The HookJD field uniquely identifies the current hook. The Handler field references a Handler function. The UserData field contains user-defined data. The Callee field references the Callee function and the Next field references NULL or the next FUNCΗON_CONTROL_BLOCK data structure in the list.

HANDLER CHAIN - FUNCTION_CONTROL_BLOCK_CONTAINER 33

In one embodiment of the present invention, the above-mentioned Handler Chain is further implemented as a FUNCTION_CONTROL_BLOCK CONTAINER 33 data structure, wherein the FUNCTION_CONTROL_BLOCK CONTAINER 33 data structure consists of a linked list of one or more FUNCTION_CONTROL_BLOCK 34 data structures .

As shown in the embodiment depicted in FIG. 3, Hook Manager creates and maintains a separate Handler Chain for each type of Callee function it monitors. For example, if the TextOut API function is to be hooked and monitored by Hook Manager, Hook Manager will maintain a separate Handler Chain for the TextOut function. Those skilled in the art will readily appreciate that it is equally feasible to implement a single Handler Chain irregardless of the type of function to be hooked. For purposes of clarity and simplicity herein, Hook Manager will only maintain a separate Handler Chain per type of hooked function.

HOOK_CONTROL_BLOCK_CONTAINER 31

During implementation of the certain features of the present invention, a HOOK_CONTROL_BLOCK_CONTAINER 31 data structure is constructed like that shown in Figure 3. The HOOK_CONTROL_BLOCK_CONTAINER 31 data structure is a binary search tree container comprising dynamically allocated nodes, each node defined, for example, as follows:

Struct HOOK_CONTROL_BLOCK_NODE { KEY VALUE Key Value HOOK_CONTROL_BLOCK HookControlBlock } As illustrated above, the HOOK_CONTROL_BLOCK_NODE 32 data structure has two fields, a KeyValue field and a HookControlBlock field. The KeyValue field references the Callee. The HookControlBlock field stores select information in a HOOK_CONTROL_BLOCK data structure further defined, for example, as follows:

struct HOOK_CONTROL_BLOCK { VOID *Handler; // references Handler function VOID *InstructSet; // references Callee replaced instruction's FUNCTION_CONTROL_BLOCK *fcb; // references the first // FUNCTION_CONTROL_BLOCK // data structure in the FUNCΗON_CONTROL_BLOCK_CONTAINER HOOK_CONTROL_BLOCK *Left; // NULL or the left node in the tree HOOK_CONTROL_BLOCK * Right // NULL or the right node in the tree

As illustrated above, each HOOK_CONTROL_BLOCK data structure has a number of different fields: a Handler field, an InstructSet field, an fcb field, a Left field and a Right field. The Handler field references the next Handler function in the Handler Chain or zero if the current Handler function is the first Handler in the Chain. The InstructSet field references the set of instructions which were removed from the Callee and replaced with an unconditional branch. The fcb field references the first structure in the FUNCTION_CONTROL_BLOCK CONTAINER. The Left and Right fields references either NULL of the left most or right most node in the tree.

Therefore, in creating a Handler Chain for a given Callee, Hook Manager constructs a FUNCTION_CONTROL_BLOCK CONTAINER data structure and constructs a HOOK_CONTROL_BLOCK data structure to reference the FUNCnON_CONTROL_BLOCK CONTAINER data structure. Hence, Hook Manager performs initial processing prior to passing control down the Handler Chain to the Callee.

CALL_CONTROL_BLOCK

The CALL_CONTROL_BLOCK data structure facilitates the transfer of control in a Handler Chain. The CALL_CONTROL_BLOCK data structure may be defined, for example, as follows:

Struct CALL_CONTROL_BLOCK { CONTEXT Context; // references state information HOOKTD HookID; // id of current hook }

As shown, the CALL_CONTROL_B OCK data structure includes two fields: a Context field and a HookJD field. The HookJD field uniquely identifies the current hook. The Context field stores state information/call context information in a CONTEXT data structure further defined, for example, as follows:

Struct CONTEXT { VOID *Registers; // contents of applicable registers VOID *Stack; // contents of application stack }

wherein the Registers field references the contents of applicable registers and the Stack field references the contents of the application stack..

Hook Manager maintains call context control and reenterability using the CALL_CONTROL_BLOCK data structure. The CALL_CONTROL_BLOCK data structure stores state information, that is the contents of the registers and the contents of the stack. Hook Manager constructs the CALL_CONTROL_BLOCK data structure before control is transferred to a Callee Handler function or a Callee.

In constructing the CALL_CONTROL_BLOCK data structure for a given Callee Handler function, Hook Manager first gets the address of the Callee Handler function from the FUNCTION_CONTROL_BLOCK 32 data structure. Thereafter, a pointer to the newly populated CALL_CONTROL_BLOCK data structure is stored in the application's stack as the last parameter. Next, the Callee Handler function is invoked with the additional CALL_CONTROL_BLOCK parameter. Finally, the Callee Handler function calls the Callee and execution continues accordingly.

Hook Manager manages the hooking/unhooking process using several functions/APIs, including: the SetHook, ReleaseHook and AppStackCall functions. Pre-defined Handler functions provide desired functionality, instrumentation or extensions to an application. A description of each one of the foregoing functions follows.

SetHook Hook Manager invokes the SetHook function to install a Handler function at the beginning of the Handler Chain. The SetHook function has the following exemplary syntax:

HOOKID SetHook ( VOID *Callee, VOID *Handler, DWORD *UserData )

As indicated above, the SetHook function header has three parameters, namely, Callee, Handler and UserData. The Callee parameter references a Callee function. The Handler parameter references an applicable Handler function. The UserData parameter references user-defined data.

The SetHook function returns an identification value, HOOKID, which uniquely identifies the current hook for subsequent processing.

ReleaseHook

Hook Manager invokes the ReleaseHook function to remove a Handler function from a Handler Chain. The ReleaseHook function has the following exemplary syntax:

VOID ReleaseHook ( HOOKID hid ); As indicated above, the ReleaseHook function header has one parameter, hid, of type HOOKID, which identifies the hook Handler to be removed.

AppStackCall

A Handler function invokes the AppStackCall function to call the next Handler function in a Handler Chain. Accordingly, the AppStackCall function facilitates the transfer of control within a Handler Chain.

The AppStackCall function allows a user-defined Handler function to be called within a user-defined Handler function without requiring state information to be copied from one application/process stack to another application process stack. When AppStackCall is called, Hook Manager determines the next Handler function in the Chain and calls that function. The Callee and Handler functions are not called directly from the Handler function. One of the reasons is that the same Handler function can be utilized for different Callees. The universal AppStackCall function is used for making calls to Callee and Handler functions from Handler functions.

The AppStackCall function has the following exemplary syntax: VOJD AppStackCall ( CALL_CONTROL_BLOCK *ccb )

As indicated above, the AppStackCall function header has one parameter, ccb, of type CALL_CONTROL_BLOCK (see above). The ccb parameter receives and passes the current call context, that is state information about the application stack and applicable registers as well as the current hook's identification.

Handler

During interception of a Callee, Hook Manager invokes a user-defined Handler function instead of the Callee. An exemplary prototype for the Handler function follows:

VOID Handler ( CALL_CONTROL_BLOCK *ccb )

As indicated above, the Handler function header has one parameter, ccb, of type CALL_CONTROL_BLOCK. The ccb parameter receives and passes the current call context, that is state information about the application stack and applicable registers as well as the current hook's identification. A Hand'er function has a universal prototype thereby enabling association with Callees' having different prototypes and calling conventions. For example, within its function body, a Handler function having both C and PASCAL calling conventions, can access the parameters that were pushed onto the application stack by the Caller with the help of standard macros, such as va_list, va_arg, va_end, and va_start. This implementation enables the original Callee function to be called directly from within the body of a Handler function without averse effects.

A more detailed view of exemplary code for a Handler function follows:

VOJD Handler (CALL_CONTROL_BLOCK *ccb ) { // pre-processing code AppStackCall ( ccb ); // call next Handler function in Handler Chain Callee ( Callee's parameters ); // call original function // post-processing code }

The remaining figures illustrate the process flow of an exemplary embodiment of the present invention. As an overview, suppose a Caller will call a Callee that is to be hooked/handled by Hook Manager. First, the Hook Manager DLL is injected in an application's address space. Second, interception code is applied dynamically at runtime. Third, the Caller calls the Callee. Fourth, the Callee is intercepted and control is transferred to Hook Manager. Fifth, Hook Manager determines the appropriate Handler function and transfers control to it. Sixth, if the Handler function is part of a Handler Chain, control passes down the Chain. Seventh, when the last Handler function has completed its task, control passes to the Callee.

The flow of control within a Handler Chain proceeds first from a Caller to a Callee and then back again to the Caller.

FIG. 3 A is a block diagram illustrating the downward flow of control in a Handler Chain from a Caller A to a Callee B.

At 30, the Caller A calls the Callee B.

At 31, Callee B is intercepted and control is transferred to Hook Manager. Based upon the type of Callee function being invoked, Hook Manager determines the appropriate Handler function (see below) and transfers control to the first Handler function in the Handler Chain.

At 32, the first Handler function executes its pre-processing code.

At 33, the first Handler function calls or invokes the AppStackCall function. The AppStackCall function transfers control to Hook Manager. Hook Manager calls or invokes the Callee B. The Callee B transfers control to Hook Manager via its CALL instruction to Hook Manager. Hook Manager determines the next Handler function in the Chain and transfers control to this Handler function. The current Handler function executes its pre-processing code and upon completion calls or invokes the AppStackCall function.

At 34, the prior step repeats until control reaches the last Handler function in the Handler Chain. Next, the last Handler in the Handler Chain executes its pre- processing code and then calls the AppStackCall function. Next, the AppStackCall function transfers control to Hook Manager. Hook Manager determines that there are no more Handler functions in the Handler Chain. Next, Hook Manager executes the stored Callee B's replaced-instructions. Next, Hook Manager transfers control to the Callee B. At 35, the Callee B executes its remaining statements/code and returns to its reference point - the calling statement in Caller A - as follows:

Figure 3B is a block diagram illustrating the upward flow of control in a Handler Chain from the Callee B to the Caller A.

At 36, after the Callee B completes its task, Callee B returns to its reference point, that is, Callee B returns to the same statement where it was called. In this case, Callee B's reference point is the APIStackCall function call statement of the last Handler function in the Handler Chain.

At 37, the last Handler function executes its post-processing code/statements thereby completing its task and returns to its reference point - the APIStackCall function call statement of the next to the last Handler function in the Handler Chain.

At 38, the previous step repeats for the remaining Handler functions up the Handler Chain until control returns to the first Handler in the Handler Chain. Next, the first Handler function executes its post-processing code/statements thereby completing its task and returns to its reference point - that is, the statement in Hook Manager that contains the Handler function call. At 39, Hook Manager transfers control to the Caller A. Caller A then continues executing its remaining statements.

FIG. 4 describes in detail steps 22-28. Within a patched Callee 41 is an unconditional branch/jump/call instruction to Hook Manager 42. When the Callee 41 executes the unconditional branch/jump/call instruction, the Callee's 41 return address is pushed onto the application stack and control is transferred to Hook Manager 42. Next, Hook Manager 42 pops the return address from the application stack and, based upon the return address, determines the applicable HOOK_CONTROL_BLOCK 43 data structure of the HOOK_CONTROL_BLOCK_CONTAINER data structure. The HOOK_CONTROL_BLOCK 43 data structure references a FUNCTION_CONTROL_BLOCK 44 data structure, that references a Handler 45 function. Next, Hook Manager 42 creates a CALL_CONTROL_BLOCK data structure and thereafter transfers control to the Handler 45 function. The Handler 45 function executes its preprocessing code and calls the AppStackCall function in order to transfer control to the next Handler function in the Handler Chain. When the end of the Handler Chain is reached, control is transferred to Hook Manager 42 and Hook Manager 42 transfers control to the Callee 41. The Hook Manager component ensures that a Callee and its associated Callee Handler function have the proper call context. Suppose that a Caller A calls a Callee B having two parameters. Figure 5 illustrates the application stack allocation (a) before, (b) during, and (c) after the procedure call.

At 50, the Caller A apportions a procedure frame or activation record on the application stack for the Callee B by pushing onto the stack the Callee B's saved argument registers corresponding to at least the two parameters set forth in the Callee B's function header, return address, saved saved registers (if any) and memory-resident local variables, local arrays or structures that do not fit in registers (if any).

At 52, when execution reaches the Callee B, control is transferred to Hook Manager component by way of the replacement instructions inserted in the Callee B.

At 54, Hook Manager populates a CALL_CONTROL_BLOCK data structure by storing in its data fields, the HookJD of the current hook and the current state information comprising the current contents of the applicable registers and the current contents of the application stack. At 55, Hook Manager restores the old values of the registers that were saved

At 56, Hook Manager "creates" a new return address and pushes this return address onto the application stack.

At 58, Hook Manager adjusts the application stack by pushing the CALL_CONTROL_BLOCK data structure to the application stack.

At 60, Hook Manager calls the corresponding Handler function (Callee Handler B) having the same parameters as Callee B and passes as an additional parameter the CALL_CONTROL_BLOCK data structure.

At 62, the Callee Handler B function apportions a procedure frame or activation record on the application stack to be used by it by pushing to the application stack saved parameter registers, return address, saved saved registers (if any) and memory-resident local variables, local arrays or structures that do not fit in registers (if any). In addition, the address of the pointer to the CALL_CONTROL_BLOCK data structure is pushed onto the application stack.

At 64, the Callee Handler B function performs its pre-processing tasks. At 66, the Callee Handler B function invokes or calls the AppStackCall function.

At 68, control is transferred to Hook Manager.

At 70, Hook Manager stores the current state information comprising the current contents of the registers and stack information in the CONTEXT field of the CALL_CONTROL_BLOCK data structure.

At 72, Hook Manager restores the call context by restoring the old values of the registers that were saved.

At 74, Hook Manager creates a new return address and pushes this return address onto the application stack.

At 76, Control is transferred to the Callee B.

At 78, the Callee B performs its tasks and returns to its origin point or point of reference.

At 80, Hook Manager restores the call context by restoring the old values of the registers that were saved. At 82, Control is transferred to the Callee Handler B function.

At 84, Callee Handler B function performs its post-processing tasks.

At 86, Control is transferred to Hook Manager.

At 88, Hook Manager restores the call context for the Caller B by restoring the old values of the registers that were saved.

At 90, Hook Manager adjusts the application stack by popping the CALL_CONTROL_BLOCK data structure.

At 92, Control is transferred to Caller A.

Subclass Manager

The Subclass Manager component of Hook Manager facilitates the organization of independent subclassing of windows by, at the very least, providing for the subclassing of arbitrary windows of arbitrary processes, multiple subclassing of a window thus creating a chain of subclass window procedures for said window and random installation and release of subclass window procedures from the subclass chain in any arbitrary order.

In order to avoid code destruction resulting from an application's own subclassing, Hook Manager is injected into the application's address space as part of the subclassing code. Hook Manager installs hooks on the API functions GetWindowLong and SetWindowLong.

Thus, whenever the application attempts to subclass a window or remove subclassing from a window, the application will call the SetWindowLong and SetWindowLong API functions which. Hook Manager thereafter intercepts and processes in accordance with the principles described herein. The code in Hook Manager works in such a way that the application will not know that its window has been subclassed by a foreign component.

In managing the subclassing process, Subclass Manager uses the following data structures.

With each call to the SetSubClass function, a SUBCLASS_CONTROL_BLOCK structure is created in a SUBCLASS CONTROL BLOCK CONTAINER. In one exemplary embodiment of the present solution, the SUBCLASS_CONTROL_BLOCK_CONTAINER data structure is a binary search tree container comprising dynamically allocated nodes, each node defined, for example, as follows:

Struct SUBCLASS_CONTROL_BLOCK_NODE { KEYVALUE KeyValue SUBCLASS_CONTROL_BLOCK SubclassControlBlock }

As illustrated above, the SUBCLASS_CONTROL_BLOCK_NODE data structure has two fields, a KeyValue field and a SubclassControlBlock field. The KeyValue field references hWnd , which is the window handle of the window to be subclassed. The SubclassControlBlock field stores select information pertaining to the subclass window in a SUBCLASS_CONTROL_BLOCK data structure.

Subclass Manager manages the subclassing process using several functions/APIs, including: "SetSubclass" and "ReleaseSubclass." A description of each one of the foregoing functions follows.

SetSubclass

Subclass Manager invokes the SetHook function to install a subclass procedure at the beginning of a Subclass chain. The SetHook function has the following exemplary syntax:

HSUBCL SetSubclass((HWND hWnd, WNDPROC WndProc, DWORD UserData);

As shown above, the SetSubclass function header passes three parameters, namely, hWnd of type HWND, WndProc of type WNDPROC and UserData of type DWORD. The hWnd parameter references the handle of the window to be subclassed. The WndProc parameter references an applicable subclassing procedure. The UserData parameter references user-defined data.

The SetSubclass function returns an identification value, HSUBCL, which uniquely identifies the current subclassing for subsequent processing. ReleaseSubclass

Subclass Manager invokes the ReleaseSubclass function to remove a subclass procedure from a subclass chain. The ReleaseSubclass function has the following exemplary syntax:

VOID ReleaseSubclass (HSUBCL hSubcl);

As indicated above, the ReleaseSubclass function header passes one parameter, ^' hSubcl of type HSUBCL. The hSubcl parameter identifies the subclass procedure to be removed from the subclass chain.

During execution, a call is made to the SetWindowLong API having the following syntax:

FARPROC lpfnOldWndProc = (FARPROC)SetWindowLong(hWnd, GWL_WNDPROC, (DWORD) SubclassFunc)

SetWindowLong replaces the window procedure for the window specified by hWnd with the new window procedure SubclassFunc. The old window procedure is saved within lpfnOldWndProc , which is returned by SetWindowLong.

Next, Hook Manager intercepts SetWindowLong.

Next, the following exemplary preprocessing code executes:

void Preprocessing (HWND hWnd, int nlndex,) { if ( hWnd && nlndex == GWL_WNDPROC ) { SCB *pSCB = Find(hWnd); SetWindowLongW(hWnd,GWL_WNDPROC,(long) pSCB - >OriginalWndProc); } }

Next, the SetWindowLong API with its original parameters is called. This function returns proper value lpfnOldWndProc. Next, the following postprocessing code executes:

void Postprocessing (HWND hWnd) { SetWindowLongW (hW, GWL_WNDPROC, (long)SMWndProc) }

Exemplary execution code statements follow:

void Processor (HM_CALLID CalllD)

{ va_list ap; va_start(ap,CallJD); HWND h Wnd = va_arg(ap,HWND) ; int nlndex = va_arg(ap,int); Preprocessing (HWND hWnd, int nlndex,) HM_AppStackCall (CalllD); Postprocessing (HWND hWnd,) }

Conclusion Having now described a preferred embodiment of the invention, it should be apparent to those skilled in the art that the foregoing is illustrative only and not limiting, having been presented by way of example only. All the features disclosed in this specification (including any accompanying claims, abstract, and drawings) may be replaced by alternative features serving the same purpose, and equivalents or similar purpose, unless expressly stated otherwise. Therefore, numerous other embodiments of the modifications thereof are contemplated as falling within the scope of the present invention as defined by the appended claims and equivalents thereto.

Moreover, the techniques may be implemented in hardware or software, or a combination of the two. Preferably, the techniques are implemented in computer programs executing on programmable computers that each include a processor, a storage medium readable by the processor (including volatile and non-volatile memory and/or storage elements), at least one input device and one or more output devices. Program code is applied to data entered using the input device to perform the functions described and to generate output information. The output information is applied to one or more output devices.

Each program is preferably implemented in a high level procedural or object oriented programming language to communicate with a computer system, however, the programs can be implemented in assembly or machine language, if desired. In any case, the language may be a compiled or interpreted language.

Each such computer program is preferably stored on a storage medium or device (e.g., CD-ROM, hard disk or magnetic diskette) that is readable by a general or special purpose programmable computer for configuring and operating the computer when the storage medium or device is read by the computer to perform the procedures described in this document. The system may also be considered to be implemented as a computer-readable storage medium, configured with a computer program, where the storage medium so configured causes a computer to operate in a specific and predefined manner.

handler functions;

adding a first handler function for said application to said handler chain using said hook manager;

subsequently adding a next handler function for said application program to said handler chain using said hook manager;

unhooking said first handler function for said application from said hook chain using said hook manager; and

subsequently unhooking said next handler function for said application from said handler chain using said hook manager; wherein said hook manager adjusts said application stack using a dynamically constructed data structure during processing of said handler functions for said application thereby eliminating the need to create a second stack to store call context state information.

6. A machine-readable medium as in claim 5 wherein said machine is a computer.

APPENDIX A

Safe Subclassing in Win32

Kyle Marsh

Microsoft Developer Network Technology Group

Created: January 25, 1994

This article describes subclassing in the Win32® environment, how it is done, and the rules that should be followed to make subclassing safe. Both instance and global subclassing are covered. Superclassing is described as an alternative to global subclassing.

Subclassing has not changed dramatically from Winlδ to Win32. However, there are several new subclassing rules that an application must follow in Win32. The most important (and most visible) of these is that an application cannot subclass a window or class that belongs to another process. This rule cannot be broken, but there are some workarounds that applications can use.

A Note About This Article

I originally wrote this article for Winl6. The article was modified for inclusion in the Microsoft® Win32® Software Development Kit (SDK) for Windows NT™, Programmer's API Reference, Vol. 1, Chapters 4.1.2 to 4.1.4 (see Product Documentation, SDKs, in the MSDN Library). If you read my original article or the Win32 SDK documentation, you will find the information in this article very familiar. Unfortunately, the Information in the Win32 SDK is not completely accurate for Win32. For example, the SDK contains the following statement:

An application can subclass any window in the system; however, when subclassing a window it does not own, the application must ensure the subclass procedure does not destroy the original behavior of the window.

This statement is correct for Win 16. In Win32, however, you cannot subclass a window owned by another process, so the statement is incorrect. I made this and other corrections to my original Winl6 article. The article you are now reading is the result. Please use this article instead of the subclassing chapters in the Win32 SDK.

Subclassing Defined

Subclassing is a technique that allows an application to intercept messages destined for another window. An application can augment, monitor, or modify the default behavior of a window by intercepting messages meant for another window. Subclassing is an effective way to change or extend the behavior of a window without redeveloping the window. Subclassing the default control window classes (button controls, edit controls, list controls, combo box controls, static controls, and scroll bar controls) is a convenient way to obtain the functionality of the control and to modify its behavior. For example, if a multiline edit control is included in a dialog box and the user presses the ENTER key, the dialog box closes. By subclassing the edit control, an application can have the edit control insert a carriage return and line feed into the text without exiting the dialog box. An edit control does not have to be developed specifically for the needs of the application.

The Basics

The first step in creating a window is registering a window class by filling a WNDCLASS structure and calling RegisterClass. One element of the WNDCLASS structure is the address of the window procedure for this window class. When a window is created, the 32-bit versions of the Microsoft Windows™ operating system take the address of the window procedure in the WNDCLASS structure and copy it to the new window's information structure. When a message is sent to the window, Windows calls the window procedure through the address in the window's information structure. To subclass a window, you substitute a new window procedure that receives all the messages meant for the original window by substituting the window procedure address with the new window procedure address.

When an application subclasses a window, it can take three actions with the message: (1) pass the message to the original window procedure; (2) modify the message and pass it to the original window procedure; (3) not pass the message. The application subclassing a window can decide when to react to the messages it receives The application can process the message before, after, or both before and after passing the message to the original window procedure

Types of Subclassing

The two types of subclassing are instance subclassing and global subclassing.

Instance subclassing is subclassing an individual window's information structure With instance subclassing, only the messages of a particular window instance are sent to the new window procedure

Global subclassing is replacing the address of the window procedure in the WNDCLASS structure of a window class. All subsequent windows created with this class have the substituted window procedure's address. Global subclassing affects only windows created after the subclass has occurred. At the time of the subclass, if any windows of the window class that is being globally subclassed exist, the existing windows are not affected by the global subclass. If the application needs to affect the behavior of the existing windows, the application must subclass each existing instance of the window class

Win32 Subclassing Rules

Two subclassing rules apply to instance and global subclassing in Win32.

Subclassing is allowed only within a process. An application cannot subclass a window or class that belongs to another process.

The reason for this rule is simple: Win32 processes have separate address spaces A window procedure has an address in a particular process. In a different process, that address does not contain the same window procedure As a result, substituting an address from one process with an address from another process does not provide the desired result, so the 32-bιt versions of Windows do not allow this substitution (that is, subclassing from a different process) to take place. The SetWindowLong and SetClassLong functions prevent this type of subclassing. You cannot subclass a window or class that is in another process End of story

However, there are ways you can add subclassing functionality to every process. Once you get a function inside the address space of a process, you can subclass anything in that process. There are a few ways to do this. The easiest (and most brutal) approach is to add a dynamic-link library (DLL) name to the following key in the registry-

HKEY_LOCA _MACHINE\So£tware\Mι.crosoft:\W.ndows NT\ CurrentVersιon\Wιndow3\APPINIT_Dt,LS

This key causes Windows to add your DLL to every process in the system. Your DLL would need some way to wake up after every event that the DLL would want to subclass after A WH_CBT hook usually does the trick. The DLL can watch for the HCBT_CREATEWND event, then subclass the desired windows. The CTL3D sample application uses the WH_CBT hook to do its subclassing, although it does not contain the registry entry that makes subclassing a part of every process Applications that want CTL3D can link it into their process

Another way to add your subclassing code to every process is to use a systemwide hook. When a systemwide hook is called from the context of another process, the system loads the DLL containing the hook into the process. The CTL3D code will work for a systemwide WH_CBT hook in the same way that it works for the current thread's local WH_CBT hook.

The third way to add subclassing code into another process is much more complicated- It involves using the OpenProcess, WriteProcessMemory, and CreateRemoteThread functions to inject code into the other process I don't recommend this method and won't go into any details on how to do it For developers who insist on using this method, Jeffrey Richter tells me that he is planning to describe the technique in one of his upcoming Win32 Q&A columns in the Microsoft Systems Journal

Today, many Windows 3 1 applications subclass other processes to augment the processes and to add some cool features As Windows moves toward an object-oriented system, object linking and embedding (OLE) provides a better way to achieve this capability Subclassing other processes may get more difficult in future versions of Windows, while exploiting OLE will probably get easier I recommend that, if possible, you move your applications toward OLE and away from subclassing other processes The subclassing process may not use the original window procedure address directly.

In Winlβ, an application could use the window procedure address returned from SetWindowLong or SetClassLong to call the procedure directly. After all, the return value is simply a pointer to a function, so why not just call it? In Win32, this is a definitive no-no. The value returned from SetWindowLong and GetClassLong may not be a pointer to the previous window procedure at all. Win32 may return a pointer to a data structure that it can use to call the actual window procedure. This occurs in Windows NT™ when an application subclasses a Unicode™ window with a non-Unicode window procedure, or a non-Unicode window with a Unicode window procedure. In this case, the operating system must perform a translation between Unicode and ANSI for the messages the window receives. If an application uses the pointer to this structure to directly call the window procedure, the application will immediately generate an exception. The only way to use the window procedure address returned from SetWindowLong or SetClassLong is as a parameter to CallWindowProc.

Instance Subclassing

The SetWindowLong function is used to subclass an instance of a window. The application must have the address of the subclass function. The subclass function is the function that receives the messages from Windows and passes the messages to the original window procedure. The subclass function must be exported in the application's or the DLL's module definition file.

The application subclassing the window calls SetWindowLong with the handle to the window the application wants to subclass, the GWL_WNDPROC option (defined in WINDOWS. H), and the address of the new subclass function. SetWindowLong returns a DWORD, which is the address of the original window procedure for the window. The application must save this address to pass the intercepted messages to the original window procedure and to remove the subclass from the window. The application passes the messages to the original window procedure by . calling CallWindowProc with the address of the original window procedure and the hWnd, Message, wParam, and IParam parameters used in Windows messaging. Usually, the application simply passes the arguments it receives from Windows to CallWindowProc.

The application also needs the original window procedure address for removing the subclass from the window. The application removes the subclass from the window by calling SetWindowLong again. The application passes the address of the original window procedure with the GWL_WNDPROC option and the handle to the window being subclassed.

The following code subclasses and removes a subclass to an edit control:

LONG FAR PASCAL, SubclassFunc (HWND hWnd.WORD Message, WORD wParam,

LONG IParam) FARPROC lpfnOldWndProc; HWND hEditWnd;

//

// Create an edit control and subclass it.

// The details of this particular edit control are not important.

// hEditWnd = CreateWindow( "EDIT", "EDIT Test",

WS_CHILD I WS_VISIBLE | WS_BORDER ,

0, 0, 50, 50, hWndMain,

NULL, hlnst,

NULL) ; //

// Now subclass the window that was just created. // lpfnOldWndProc = (FARPROC) SetwindowLongthEditWnd,

GWL_WNDPROC. (DWORD) SubclassFunc) .-

//

// Remove the subclass for the edit control .

//

SetWindowLong lhEdi cWnd, GWL_WNDPROC , (DWORD) lpf π ldtvridPr ) ;

//

// Here is a sample subclass function.

//

LONG FAR PASCAL SubclassFunc ( HWND hWnd,

WORD Message,

WORD wParam, LONG IParam) (

//

// When the focus is in an edit control inside a dialog box, the

// default ENTER key action will not occur

// i f ( Message == WM_GETDLGCODE ) return DLGC_WANTALLKEYS , return CallWindowProc ( .ptrOldWndFroc, hWnd, Message , wParam, IParam) , )

Potential pitfalls

Instance subclassing is generally safe, but observing the following rules ensures safety.

When subclassing a window, you must know what is responsible for the window's behavior. For example, Windows is responsible for all the controls it supplies, and applications are responsible for all the windows they define. Subclassing can be done on any window in the same process; however, when an application subclasses a window for which it is not responsible, the application must ensure that the subclass function does not break the original behavior of the window. Because the application does not control the window, it should not rely on any information about the window that the responsible component might change in the future. A subclass function should not use the extra window bytes or the class bytes for the window unless it knows exactly what they mean and how the original window procedure uses them. Even if the application knows everything about the extra window bytes or the class bytes, it should not use them unless the application is responsible for the window. If an application uses the extra window bytes of a window for which another component is responsible, and the other component decides to update the window and change some aspect of the extra bytes, the subclass procedure is likely to fail. For this reason, Microsoft suggests that you not subclass the control classes. Windows is responsible for the controls it supplies, and aspects of the controls might change from one version of Windows to the next. If your application must subclass a control supplied by Windows, it may need to be updated when a new version of Windows is released.

Because instance subclassing occurs after a window is created, the application subclassing the window cannot add any extra bytes to the window. Applications that subclass windows should store any data needed for an instance of the subclassed window in the window's property list.

The SetProp function attaches properties to a window. The application calls SetProp with the handle to a window, a string identifying the property, and a handle to the data. The handle to the data is usually obtained with a call to either LocalAlloc or GlobalAlloc. When the application uses the data in a window's property list, the application calls the GetProp function with the handle to the window and the string that identifies the property. GetProp returns the handle to the data that was set with SetProp. When the application is finished with the data or when the window is to be destroyed, the application must remove the property from the property list by calling the RemoveProp function with the handle to the window and the string identifying the property. RemoveProp the handle to the data, which the application then uses in a call to either LocalFree or GlobalFree. For information on SetProp, GetProp, and RemoveProp, see the Win32 SDK for Windows NT, Programmer's API Reference, Vols. 3 and 4 (in Product Documentation, SDKs, in the MSDN Library).

When an application subclasses a subclassed window, the subclasses must be removed in reverse of the order in which they were performed.

Global Subclassing

Global subclassing is similar to instance subclassing. The application calls SetClassLong to globally subclass a window class. As it does with instance subclassing, the application needs the address of the subclass function, and the subclass function must be exported in the application's or the DLL's module definition file.

To globally subclass a window class, the application must have a handle to a window of that class. To get a handle to a window in the desired class, most applications create a window of the class to be globally subclassed. When the application removes the subclass, it needs a handle to a window of the type the application wants to subclass, so creating and keeping a window for this purpose is the best technique. If the application creates a window of the type it wants to subclass, the window is usually hidden. After obtaining a handle to a window of the correct type, the application calls SetClassLong with the window handle, the GCL_WNDPROC option (defined in WINDOWS. H), and the address of the new subclass function. SetClassLong returns a DWORD, which is the address of the origina' window procedure for the class The original window procedure address is used in global subclassing in the same way it is used m instance subclassing . Messages are passed to the original window procedure in the same way as in instance subclassing, by calling CallWindowProc. The application removes the subclass from the window class by calling SetClassLong again. The application passes the address of the original window procedure with the GCL_WNDPROC option and a handle to the window of the class being subclassed. An application that globally subclasses a control class must remove the subclass when the application finishes.

The following code globally subclasses and removes a subclass to an edit control :

LONG FAR PASCAL SubclassFunc (HWND hWnd.WORD Message. WORD wParam,

LONG IParam) , FARPROC lpfnOldClassWndProc, HWND hEdi Wnd,

//

// Create an edit control and subclass it

// Notice that the edit control is not visible

// Other detail3 of this particular edit control are not important.

// hEditWnd » CreateWindowCEDIT", "EDIT Test",

WS_CHILD,

0, 0, 50, 50, hwndMain.

NULL, hlnst,

NULL) ; lpfnOldClassWndProc =

(FARPROC) SetClasβLongthEdltWnd, GCL_WNDPROC, (DWORD) SubclassFunc) ;

I I To remove the subclass

//

SetClassLong(hEditWnd, GWL_WNDPROC, (DWORD) lp.nOldClassWndProc) ;

DestroyWιndow(hEdιtWnd) ;

Potential pitfalls

Global subclassing has the same limitations as instance subclassing. The application should not attempt to use the extra bytes for either the class or the window instance unless it knows exactly how the original window procedure using them. If data must be associated with a window, the window's properties list should be used in the same way as in instance subclassing.

In Win32, global subclassing does not affect the classes of any other processes or the windows created from these classes. This is a significant change from the Winlδ environment. Windows keeps window class information separately for each Win32 process on the system. For a detailed look at how Windows does this, see the "Window Classes in Win32" technical article in the MSDN Library. Now that global subclassing does not affect other processes, it becomes a usable technique for developers. In Win 16, global subclassing was discouraged because it affected every window of the subclassed class— not only for the application that performed the subclass but for the entire system. This was usually not what the application wanted to do, so applications had to use less convenient and less powerful ways to change the behavior of windows created from the system classes. It is far easier to use global subclassing in Win32.

Superclassing

Subclassing a window class causes messages meant for the window procedure to be sent to the subclass function. The subclass function then passes the message to the original window procedure. Superclassing (also known as class cloning) creates a new window class. The new window class uses the window procedure from an existing class to give the new class the functionality of the existing class. The superclass is based on some other window class, known as the base class. Frequently the base class is a Windows-supplied control class, but it can be any window class.

Note Do not superclass the scroll bar control class because Windows uses the class name to produce the correct behavior for scroll bars.

The superclass has its own window procedure, the superclass procedure, which can take the same actions a subclass procedure can. The superclass procedure can take three actions with the message. ( 1) pass the message directly to the oπginal window procedure; (2) modify the message before passing it to the original window procedure? (3) not pass the message. The superclass can react to the message before, after, or both before and after passing the message to the oπginal window procedure.

Unlike a subclass procedure, a superclass procedure receives create (WM_NCCREATE, WM_CREATE, and so αη_\ messages from Windows. The superclass procedure can process these messages, but it must also pass these messages to the original base-class window procedure so that the base-class window procedure can initialize.

The application calls GetClassInfo to base a superclass on a base class. GetClassIπfo fills a WNDCLASS structure with the values from the base class's WNDCLASS structure. The application that is superclassing the base class then sets the hlnstance field of the WNDCLASS structure to the instance handle of the application. The application must also set the WNDCLASS structure's IpszClassName field to the name it wants to give this superclass. If the base class has a menu, the application superclassing the base class must supply a new menu that has the same menu IDs as the base class's menu. If the superclass intends to process the WM_COMMAND message and not pass the message to the base class's window procedure, the menu does not have to have corresponding IDs. GetClassInfo does not return the IpszMenuName, IpszClassName, or hlnstance field of the WNDCLASS structure.

The last field that must be set in the superclass's WNDCLASS structure is the IpfnWndProc field. GetClassInfo fills this field with the original class window procedure. The application must save this address so that it can pass messages to the original window procedure with a call to CallWindowProc. The application must put the address of its subclass function into the WNDCLASS structure. This address is not a procedure-instance address because RegisterClass gets the procedure-instance address. The application can modify any other fields in the WNDCLASS structure to suit the application's needs.

The application can add to both the extra class bytes and the extra window bytes because it is registering a new class. The application must follow two rules when doing this: (1) the original extra bytes for both the class and the window must not be touched by the superclass for the same reasons that an instance subclass or a global subclass should not touch these extra bytes; (2) if the application adds extra bytes to either the class or the window instance for the application's own use, it must always reference these extra bytes relative to the number of extra bytes used by the original base class. Because the number of bytes used^' by the base class may be different from one version of the base class to the next, the starting offset for the superclass's own extra bytes is also different from one version of the base class to the next.

After the WNDCLASS structure is filled, the application calls RegisterClass to register the new window class. Windows of this class can now be created and used.

Applications often used superclassing in Winl6 because global subclassing was discouraged. Now that global subclassing is no longer discouraged in Win32, superclassing has lost some of its appeal. You may still find it useful to create a superclass if your application wants to change the behavior for only a subset of the windows (instead of all windows) created from a system class, which is the effect of global subclassing.

Summing Up

Subclassing is a powerful technique that has not changed significantly in Win32. The only major change is that you can no longer subclass a window or class that belongs to another process. Although there are workarounds for this restriction, I recommend that you move your application to OLE rather than relying on subclassing if you need this capability. APPENDIX B

'••*»'

CMP

United Business Media

HOME I CONTACT US | SUBSCRIBE |

ADVERTISξ I CDROM | SOURCE CODE | GO TO.

Windows Developer's Journal is now Windows Developer Magazine. Click for more information.

User Interface Programming

⁽• Search wd-mag.com C Search WWW Petter Hesselberg

Google 1 Search J

Window subclassing is probably the most versatile tool in

Home the user-interface programmer's toolkit. It gives you total

Current Issue freedom to do wonderful things or, as the case may be, to Articles Archive Source Code shoot yourself in the foot. This month's column describes Author Guidelines wdjSUB, a general subclassing library that improves your Links chances of avoiding self-mutilation.

Editor's Comer ^•Careers Subclassing CD-ROM Development Tools

Window subclassing is subversion, pure and simple. When you subclass a window, you replace its window function with your own, and the original gets to see messages only if your function decides to pass them on. Usually you want to be good about this. If you interfere any more than necessary, the whole exercise is pointless. If you greedily keep all messages to yourself, you would have been better off developing a new window class from scratch.

Window subclassing comes in several variants:

Instance subclassing: This technique (also known as "local subclassing") dynamically replaces the window function of an existing window instance. You directly alter the window procedure by calling SetWindowLong() to alter the GWL_WNDPROC window data. This affects only a single window. Other windows are unaltered, even if they are of the same class.

Global subclassing: This means changing the window function that resides in the window class definition. You use SetClassLong() (with the GCL_WNDPROC parameter) to store your own window function, and any new window of that class will be assigned your window function instead of the one the window class was originally created with. Windows that already exist before you call SetClassLongQ are unaffected by this technique. Class cloning: Also known as superclassing, this is just another way to borrow functionality from an existing window class. You use GetClasslnfo() and RegisterClass () to register a new window class that has the same characteristics as an existing window class, but with a new window function.

This article covers the first and most dynamic of these techniques, instance subclassing. The most obvious reason for instance subclassing is to modify the behavior of standard windows such as edit controls. Less obvious, perhaps, is subclassing to listen in on the message traffic: a ToolTip control, for example, will subclass its parent if you set the TTF_SUBCLASS flag. This lets the ToolTip control act on mouse messages sent to the parent, saving said parent from having to send TTM_RELAYEVENT messages to the ToolTip control. The ToolTip control thus becomes a more self-contained widget.

The archetypal subclassing example subclasses a specific edit control in a dialog box to filter out illegal characters. Before the advent of the ES_NUMBER style, this was a popular technique for creating numeric input fields.

Basic Instance Subclassing

To subclass a window, you first need a place to store the address of the original window function: static WNDPROC orgWndProc;

Next, you need a window function to replace the original; call it myWndProc(). The main difference between myWndProc() and other window functions is that myWndProc invokes orgWndProc instead of DefWindowProc() to handle messages it doesn't want to handle itself:

CallWindowProc) orgWndProc, hwnd, msg, wParam, IParam ) ;

It's imperative that you invoke the original window function through CallWindowProc(). Never call orgWndProc directly; it may not even be a function address! If you subclass a Unicode window with a non-Unicode window function under Windows NT, the system must translate text-related message parameters, and the alleged window function is actually a handle to some black-box data structure cooked up by SetWindowLong().

You use SetWindowLong() to both retrieve the original window procedure and store your new window procedure. For example, if you were creating a dialog box that subclassed one of its edit controls, you might execute this statement when handling WMJNITDIALOG: orgWndProc = (WNDPROC) SetWindowLong!

GetDlglteml hwnd, ID_OF_EDIT_CONTROL_TO_BE_SUBCLASSED ) , GWL_WNDPR0C, (LONG) myWndProc ) ;

Finally, the original window function must be restored before the window dies. The typical myWndProc() does this in response to either WM_DESTROY or WM_NCDESTROY:

SetWindowLong! hwnd, GWL_WNDPROC, (LONG) orgWndProc ) ;

Subclassing Issues

The biggest problem with the previous code fragments is the static variable orgWndProc that holds the original window function. This works for subclassing a single window or for subclassing multiple windows that share the same window function at subclassing time. It does not work in general because of the difficulty involved in stuffing multiple function pointers into a single variable. Using multiple variables wouldn't help either, as you'd have no way of telling them apart from the context of myWndProc

The first and foremost requirement for a general subclassing library, then, is a way of associating saved window functions with HWNDs. Here are some other desirable properties:

• It should be possible to apply multiple subclassings to a window, i.e., to create a subclassing chain.

• It should be possible to subclass and unhook in any order.

• It is often convenient to associate arbitrary subclassing- specific data with a window; a general solution should offer this feature. Consider a toolbar that subclasses its parent to listen for W _SIZE messages. From the context of the subclassing window function, which receives the parent HWND, the toolbar's HWND is not immediately available without such a facility.

• The application programming interface (API) should be easy to use and protect against programming mistakes. Runtime robustness is also desirable, of course.

• It may be desirable to apply one subclassing multiple times to the same window. As an example, multiple toolbars might wish to subclass their common parent with the same subclassing to listen for WM_SIZE messages. wdjSUB satisfies all these requirements except the last. It can be done, but the resulting design is not what you'd call elegant. A better solution permitting repeated subclassing uses a single subclassing to listen for WM_SIZE and maintains its own list of interested windows (which could include status bars, as well as toolbars). Associating Data with Windows

Data can be associated with windows in two basic ways: you can associate the data with the window itself (using SetWindowLong() or window properties), or you can maintain a static list of associations (array, linked list, binary tree, hash table, Post-it notes, or engraved stone tablets).

Every window has a spare 32-bit word that you can access by passing GWL_USERDATA to SetWindowLong(), so that might appear the ideal place to store your custom data. If, however, if you don't have the source code to the window you're subclassing, you have no guarantee that the window doesn't use GWL JSERDATA for its own purposes.

You could try to work around this by having your subclass procedure dynamically restore the old value of GWL_USERDATA before passing any message on to the previous window procedure, putting your own data back in when that procedure returns. Allocating the following structure would give you room to save everything you need: struct ORIGINAL {

' LONG lOriginalUserData; WNDPROC fnOriginalWndProc; }; ^'

GWLJJSERDATA would then hold a pointer to this structure. This approach has a serious problem, though — it doesn't stand up to recursion. Consider WM_PAINT: before passing WM_PAINT to the next window function in the subclassing chain, you restore the old GWLJJSERDATA to, say, zero. Before CallWindowProc () returns, someone somewhere will call BeginPaint(), which may send a WM JΞRASE message to the window. The WM JΞRASE message arrives; you attempt to retrieve the original pointer from GWLJJSERDATA and get... zero. Oops.

Window Properties

Storing the original window function as a window property works better, the main difference being that you no longer need to restore any original values. The only requisite is that each subclassing use a unique property name. With subclassing chains, this suggests an image of saved window functions sticking out all over the window like pins from a pincushion; I hereby dub this the "porcupine technique."

Conventional wisdom has it that GetProp() is less efficient than GetWindowLong(), so I did some informal testing to see just how big a problem this might be. In my benchmarking, I found that GetWindowLongQ was faster than GetProp() by an order of magnitude when the property name was a string. When the property name was an atom, however, the tables were turned — GetProp() was now slightly faster than GetWindowLong().

Don't read too much into this benchmarking. It's only valid for a Windows NT 4.0 window with a single property. It does indicate, though, that the use of window properties shouldn't cause excessive performance problems.

Practicing Safe Unhooking

Unhooking is just a matter of reverse subclassing, with one important caveat: if there's another subclassing on top of myWndProc(), restoring the window function to orgWndProc unhooks not only myWndProc(), but one or more other subclassings as well. While the wrongness of this will vary, it certainly isn't right — a well-behaved subclassing should unhook only if it finds itself at the top of the food chain. The following condition must hold: myWndProc == GetWindowLong ( hwnd, GWL_WNDPROC ) ,-

Both the GWLJJSERDATA technique and the porcupine technique share the limitation that each subclassing lives in ignorance of all others, so unhooking must proceed in reverse order of subclassing if either technique is to practice safe unhooking. This does not meet any reasonable definition of "unhooking at will."

To allow separate parts of your code to subclass the same window and yet unhook in any order, subclassings must, unfortunately but necessarily, be aware of one another. To this end, wdjSUB maintains a linked list of subclassing descriptors for each subclassed window: typedef struct SUBCLASSING {

WNDPROC wndProc ;

WNDPROC wndProcSaved; void *pData;

Struct SUBCLASSING *pNext; } SUBCLASSING;

I'll get back to the details of this structure shortly. At any rate, subclassing is now a matter of attaching a new SUBCLASSING node to the head of the list and replacing the window function. Unhooking is a matter of removing a node from the list; restoration of the original window function takes place only when the head of the list is removed. The only thing I have to associate with the window is the head of the linked SUBCLASSING list.

This approach allows subclassing and unhooking with gay abandon, provided you stick to using the wdjSUB library. You're still vulnerable to misbehaved "foreign" subclassings, though. When you're ready to unsubclass a window, you should check first to see that the current window procedure is equal to your window procedure. If it isn't, the implication is that someone else subclassed the window after you did. Hence, it's quite possible that other subclasser thought your subclass procedure was the original window procedure and will, unbeknownst to you, later restore your subclass procedure when it unhooks. There's really no way you can protect yourself against all the possible ways someone else's subclass code could mess yours up, but you can be as defensive as possible. wdjSUB does the best it can on the assumption that most subclassing schemes unhook on either WM JDESTROY or WMJMCDESTROY. wdjSUB provides automatic unhooking on both of these messages. If unhooking succeeds on WM J0ESTROY, this opens the door for any subclassing below that wants to unhook on that message. If someone else "owns" the window at WMJ ESTROY time, then wdjSUB waits to try again at WMJMCDESTROY time. If you just go ahead and restore the previous window procedure at WMJDESTROY time, then the code that subclassed the window after you will never get called again — it's out of the loop. It's likely that that code intended to perform some processing, such as freeing up memory or other resources, when it was unhooked. Taking someone else's window subclass function out of the loop could result in resource leaks or worse problems.

If that someone else still hasn't unhooked at WMJMCDESTROY time, then there's little wdjSUB can do, since WMJMCDESTROY should be the last message that Windows sends before destroying the underlying window object.

Static Association

As mentioned previously, all that must be associated with a window is the head of its SUBCLASSING list. You may have noticed that this invalidates the killer criterion that shot down the GWLJJSERDATA technique. I'm not going to resurrect it, though — it's too risky for a general library. The remaining choices are to attach the list head to the window using a single window property or to maintain the mapping in a static table.

There is something very appealing and elegant about attaching the head of the SUBCLASSING chain to the window itself. There are drawbacks, though: window properties are exposed to the rest of the world, and it is within the realm of possibility that some Bad Person will steal your property. In other words, encapsulation is imperfect. Another drawback is that a poorly designed "foreign" subclasser could cause a window property leak (and other leaks). Suppose someone subclassed the window before you did and then decided to unhook long before receiving WM J3ESTROY. If they rudely just restore the previous window procedure instead of noticing that someone else has subclassed the window, then your subclass procedure will be out of the loop and will never get called again. In this case, that means you won't properly free up the window property you allocated.

You can avoid these problems by maintaining a static table of HWND/SUBCLASSING associations. This improves encapsulation and ensures that you can clean up after yourself. The data structure of choice here is a hash table. (Post-it notes have well-documented performance problems.)

What about multithreading? If you maintain a single global table, you must provide thread-safe access to the table itself. This is not a terribly good idea, because wdjCallOldProc() has an inner-loopy nature and thread synchronization is expensive. A better way to handle the problem is to maintain the table on a per-thread basis. Windows are thread-bound anyhow — you can subclass a window from any thread, but the subclassing function will execute in the context of the thread that created the window.

If creation and subclassing of a given window is the responsibility of a single thread, there is never any contention for the SUBCLASSING list, yet other threads are still free to send or post messages to the window. This, then, is a rule of wdjSUB programming, and to ensure compliance, both subclassing and unhooking are protected by an assertion: assert) GetCurrentThreadId( ) ==

GetWindowThreadProcessId! hwnd, 0 ) ) ;

The major disadvantage of the static table approach is more code and increased complexity. To keep the code size down, I decided to use a window property to store the SUBCLASSING list. This implementation detail does not affect the wdjSUB API.

The wdjSUB API

The interface to wdjSUB is defined by wdjsub.h (Listing 1 ). Some internal declarations are in internal. h (Listing 2). to make them available for debugging and testing, and the implementation is in wdjsub.c (Listing 3).

The wndProc member of the SUBCLASSING structure serves to identify the subclassing. Although any unique value would serve as an identifier, using the address of the subclassing's window function makes it easy for wdjUnhook() to check whether unhooking is safe.

In my original example, subclassing with wdjSUB would look like this: wd] Subclass! myWndProc , GetDlgItem( hwnd, ID_OF_ΞDIT_C0NTROL_TO_BE_SUBCLASSED ) , 0 ) ;

This function returns TRUE on success, FALSE on failure. It fails if the window has already been subclassed using myWndProc(), if it fails to add a global atom, or if it can't allocate enough memory for a SUBCLASSING structure.

The SubclassWindow() macro is defined in windowsx.h.

The application data parameter was set to zero in this example, so calling wdjGetData( myWndProc, hwnd ); from myWndProc() would yield zero. The accompanying test program (described later) demonstrates real use of this facility.

Explicit unhooking is generally unnecessary. wdjSUB unhooks automatically on window destruction, provided wdjCallOldProcO is called for either WM J3ESTROY or WMJMCDESTROY (preferably both): wdjCallOldProc ( myWndProc, hwnd, msg, wParam, IParam );

This function should be called only from myWndProc().

The allocPropertyNameO and releasePropertyName() utility functions in wdjsub.c create and free the atom that names the property that holds the head of the window's SUBCLASSING chain. Remember that atoms are reference-counted; multiple calls to GlobalAddAtom() require an equal number of calls to GlobalDeleteAtom() before the atom actually dies.

The find() utility function searches the SUBCLASSING list for a given node, while getHead() (a macro in internal.h) and setHead() handle all GetProp(), SetProp() and RemovePropO calls.

The most important functions — wdjSubclass(), wdjUnhook()_> and wdjCallOldProcO — are not as complex as they may appear; most of the code is devoted to tracing statements helpful during debugging or testing.

The Sample Program wdjSUB comes with a sample program that lets you see what's happening under the hood (Figure 1 ). You'll find the test program in subtest.c (Listing 4) and the dialog script in subtest.rc (Listing 5).

The test program lets you apply five different subclassings to a simple edit window. You can subclass and unhook these in any order, although unhooking will sometimes fail if the "Only unhook if expected wnd proc found" check box is checked. If it is not checked, subclassings may be lost instead.

Two of the subclassings (tracer() and upper()) are "foreign," while the remaining three (digits(), hex(), and beep()) use wdjSUB. As long as you stick to the last three, you can do anything you like in any order. When you involve the first two, restrictions apply. If a foreign subclassing blocks a chain of wdj subclassings, for example, the head of the wdj chain is stuck for the duration of the blockage, but the others can safely be unhooked.

What happens if a foreign subclassing appears in between wdj subclassings? Try it and see; that's what the test program is for.

The test program is in the privileged position of having perfect knowledge of all subclassing techniques employed. This lets updateStack() display the complete subclassing chain in the stack window.

The message trace window is used by the tracer() subclassing, and the diagnostics window provides a running commentary from wdjSUB and the test program.

The three wdj subclassings all demonstrate the association of subclassing-specific data with a window. Two of them store a string pointer, while the third stores a function pointer.

Finally, the "Destroy" button destroys the guinea pig; this lets you trace through automatic unhooking. Note that tracer() and upper() unhook on different messages!

All in all, this program should give you a feel for how subclassing chains work — and how they sometimes fail to work.

Conclusion wdjSUB is far from being the final word about subclassing. I've sidestepped some issues and failed to consider some implementation alternatives (dynamically generated code stubs spring to mind). Still, wdjSUB is robust and simple, and covers (according to a complicated formula I just derived) 98 percent of your daily subclassing needs. If your needs fall into the other 2 percent, or if you have subclassing insights to offer, please drop me a line.

Petter Hesselberg is a Specialist Partner with Accenture's Oslo office. He's been programming Windows for the past thirteen years and is the author of Programming Industrial Strength Windows. He can be reached at petter.hesselberg@accenture.com. Get Source Code

Copyright © 2002 Windows Developer Magazine, Pπvacv Policy

Comments/webmaster jlucca@cmp com SDMG Websites

C/C++ Users Journal, Dr Dobb's Journal, MSDN Magazine, Sys Admin

SD Expo. SD Magazine Unixreview, Windows Developer Magazine

Claims

CLAIMSWhat is claimed is:

1. In a data processing system comprising a processor, a memory, an application and an application stack, a method comprising the steps of:

providing a hook manager for managing hooking and unhooking of functions called by said application, said hook manager maintaining a handler chain of handler functions;

adding a first handler function for said application to said handler chain using said hook manager;

subsequently adding a next handler function for said application program to said handler chain using said hook manager;

unhooking said first handler function for said application from said hook chain using said hook manager; and

subsequently unhooking said next handler function for said application from said handler chain using said hook manager; wherein said hook manager adjusts said application stack using a dynamically constructed data structure during processing of said handler functions for said application thereby eliminating the need to create a second stack to store call context state information.

2. A method as in claim 1 wherein said data processing system is a computer.

3. A data processing system comprising:

an application stack;

a processor associated with said application stack, said processor configured to:

generate a hook manager for managing hooking and unhooking of functions called by said application, said hook manager maintaining a handler chain of handler functions;

add a first handler function for said application to said handler chain using said hook manager;

subsequently add a next handler function for said application program to said handler chain using said hook manager;

unhook said first handler function for said application from said hook chain using said hook manager; and

subsequently unhook said next handler function for said application from said handler chain using said hook manager; wherein said hook manager adjusts said application stack using a dynamically constructed data structure during processing of said handler functions for said application thereby eliminating the need to create a second stack to store call context state information.

4. A data processing system as in claim 3 wherein said data processing system is a computer.

5. A machine-readable medium having machine-executable instructions for performing a machine-implemented method, said machine comprising a processor, a memory, an application and an application stack, said method comprising the steps of:

providing a hook manager for managing hooking and unhooking of functions called by said application, said hook manager maintaining a handler chain of