JP2003515219A - Method and system for inhibiting application program interface - Google Patents

Abstract

(57) [Summary] A method of performing a secure function operates an operating system platform having an operating system including a process space including a user application running in a kernel space (114) and a process space (112). Computer system (100). The system call is blocked (190) and checked for validity (196) by comparing the blocked system call generation address with the range of process effective addresses associated with the process (194). A notification is provided regarding the validity of the blocked system call (166). If found to be invalid (180), the blocked system call can be terminated.

Description

Detailed Description of the Invention

[0001]

【Technical field】

The present invention relates generally to methods for detecting and preventing unauthorized or illegal access attempts within computer systems. In particular, the present invention relates to methods for detecting and preventing attempts to exploit vulnerabilities associated with buffer overflows in computer systems.

[0002]

[Background technology]

This application is an Israeli patent application number filed on November 14, 1999 It relates to "methods and systems for blocking application program interfaces".

Today's computers are designed with the requirements of human-readable high-level languages. The most essential techniques for constructing computer programs derived from high-level languages are procedures and functions.

Procedures and functions are computer programs. Procedure calls and function calls are abstract concepts that change the flow of execution of the calling program. In contrast to conventional "jump" and "goto" instructions that change the flow of execution, procedures and functions execute code they own and then return control to the instruction immediately after the call. It is used in a storage device called a stack to execute procedure calls and function calls in the manner described.

A stack is a continuous storage block that stores data. Its size is dynamically adjusted at runtime by operating system routines. The data is the central representation used in assembler language instructions such as “push” and “pop”.
It is inserted into the stack or deleted from the stack by the arithmetic processing unit (CPU).

The stack is made up of logical stack frames and procedure active records that are inserted into the stack when a function is called and removed from the stack when the function returns control to the calling program. . The stack frame itself consists of parameters, local variables,
Holds a pointer to the previous stack frame, the return address of the calling computer program. The return address is the instruction pointer of the calling program when the function is called.

Buffer overflow triggering and buffer flow attacks are known to those skilled in the art. The buffer flow attack takes advantage of the flaw in checking the size of the inputs stored in the buffer array. An array is a storage device that is predetermined and arranged in a computer system. By deliberately writing data past the end of the allocated array, an attacker may arbitrarily alter the data stored adjacent to the array. The most shared data structure that is adversely affected in this way is the stack. Therefore, this type of attack is also known as stack destruction.

A common method of exploiting buffer flow is to attack the buffers allocated on the stack. Such attacks attempt to achieve two interdependent goals. The first purpose is to insert the attack code in the form of executable binary code into the attacked device. Another purpose is to change the return address that points to the attacker-provided code that currently exists in the stack memory. The code provided by such an attacker may be used to gain enhanced privileges on the computer system.

A program attacked using this technique is a utility or daemon with a constantly high privilege that operates under the user ID that is the basis for executing essential services. The effect of a successful bufferflow attack is to give the attacker unauthorized root privileges. Obtaining root privileges on a computer system is allowing unauthorized users access to the resources under their control.

Since the maximum length of the overflowing data string is only the current depth of the stack, the inserted attack code will be insufficient in terms of code length. Writing data outside the stack limits will result in an abnormal condition that prevents the execution of attack code. Therefore, a buffer overflow attacker would have to force the writing of short code and use high-level system and library calls. Such a call will later be used to obtain an illegally enhanced privilege to access a resource under privilege.

Several methods of attempting to overcome the buffer overflow vulnerability are known to those skilled in the art. One Way to Write Past Stacked Segment Arrays
It is to design a compiler designed to prohibit computer programs from being compromised. Another method is to detect programs that are vulnerable to buffer overflows offline and be vigilant to users who may compromise system privileges.

Another known method is to use a repair program. Repair programs repair and recover those weak programs that can be used to exploit the weakness of buffer overflow. The above is not a method or apparatus for buffer overflow prevention through the controlled execution of system or other calls in a computer system.

[0013]

DISCLOSURE OF THE INVENTION

Thus, there is a compelling need to detect and prevent unauthorized or illegal access attempts in computer systems. In particular, it is a method of detecting and preventing attempts to exploit buffer overflow, which is a weakness of computer systems, by validating system and other calls made within the computer system.

Therefore, when a high-level system call, a library call, an application program interface call, or the like is illegally made, a method for preventing the induced buffer overflow attack by preventing the execution of such a call is provided. It is an object of the invention to provide.

Therefore, when a high-level system call, a library call, an application program interface call, or the like originates from an illegal area in the computer system, a buffer overflow is induced by preventing the execution of such call. It is another object of the invention to provide a method of preventing attacks.

Thus, when a call to a high level system call, library call, application program interface call occurs from outside a user process associated with the called system and other calls, execution of such call. It is a further object of the invention to provide a method of preventing induced buffer overflow attacks by preventing

Therefore, according to a preferred embodiment of the present invention, a method of performing a safety function in a computer system running an operating system platform,
The operating system including kernel space and process space;
The process space containing a user application running in a process space, the user application running to block a system call, the address at which the blocked system call is generated, and the blocked Inspecting the effectiveness of the inhibited system call by comparing the process with a range of process effective addresses associated with the process where a system call occurs and provides notification of the effectiveness of the inhibited system call. And terminating the inhibited system call.

Therefore, according to a preferred embodiment of the present invention, a method of performing a safety function in a computer system running an operating system platform,
The operating system including kernel space and process space;
The process space containing a user application running in a process space, the user application running to block a system call, the address at which the blocked system call is generated, and the blocked When a system call occurs and the application program interface obstruction module is inserted into the created process, the process is responded to, the process effective address table is updated, or the process effective address table is updated. Providing the method comprising the step of checking the validity of the interrupted system call by comparing a range of process effective addresses associated with the process responding or responding.

According to another preferred embodiment of the present invention, a method of performing a safety function in a computer system running an operating system platform, the operating system including kernel space and process space, and operating in the process space. The process space containing the user application
Provides notification of the user application that operates to inhibit a library call, the address where the inhibited library call is occurring, and the effectiveness of the inhibited library call when the inhibited library call occurs Or checking the effectiveness of the inhibited library call by comparing the range of process effective addresses associated with the process that terminates the inhibited library call. Is provided.

According to another preferred embodiment of the present invention, a method of performing a safe function in a computer system running an operating system platform, the operating system including kernel space and process space, and operating in the process space. The process space containing the user application
Connects the user application that operates to block the library call, the address where the blocked library call is generated, and the library routine associated with the dynamic link library that is generated by the blocked library call Respond to a system call that loads the dynamic link library and unloads the dynamic link library that updates the process effective address table. A range of process effective addresses associated with the process. Providing the method comprising the step of testing the effectiveness of the inhibited library call by comparing

According to another preferred embodiment of the present invention, a method of performing a safety function in a computer system running an operating system platform, the operating system including a kernel space and a process space, and operating in the process space. The process space containing the user application
The user application operating on system and function calls, the blocked system or function call, the method comprising the step of receiving a caller routine return address from the process store, the caller address routine and process enable The calling routine by comparing with an address table, providing notification as to the validity of the calling routine return address, or performing a user predetermined action associated with the validity of the calling routine address. It provides to determine if the address is valid. Further, there is a similar method that comprises the step of determining the calling routine call address by determining an address preceding the calling routine address.

According to another preferred embodiment of the present invention, a method of performing a safety function of a computer system operating an operating system platform, the operating system including a kernel space and a process space, and operating in the process space. Configuring a step of receiving a calling routine return address from the process space containing a user application, the user application operating on a system and function call, the blocked system or function call, and the process storage device. Comparing the caller address routine with the associated process stack address area, providing notification as to the validity of the caller routine return address, or the caller routine. It is provided that the user associated with the validity of the chin address performs a predetermined action to determine if the calling routine address is valid. Further, a similar method of receiving a calling routine return address from the process memory comprises determining the calling routine call address by determining an address preceding the calling routine address.

[0023]

BEST MODE FOR CARRYING OUT THE INVENTION

The present invention overcomes the deficiencies of the prior art by providing a new method.

The new method is an attempt to utilize the weak point of buffer overflow.
It detects if it is happening by enabling the use of the system or other calls within the computerized system.

Now refer to FIG. This figure is a schematic representation of a system environment in which a safety function execution system is operating, generally referred to as system 100, and provides for preferred embodiments of the present invention.

This invention was filed on November 14, 1999 in Israel Patent Application No. XXX
It relates to XXX “Methods and Systems for Inhibiting Application Program Interfaces”.

As further detailed in Israel Patent Application No. XXXXXXXXX, the system 100 of FIG. 1 comprises the following four components of a safety function execution system:

A) The security function execution system server 116 is an active component. The safety function execution server 116 is a central component in operating the safety function execution system 100. The safety function execution system 116 loads and controls the system call blocking component 124 and also loads and controls the API blocking modules 134, 140, 146, and responds to various system and library calls to the user. Operates as an interface. The security function execution server 116 is loaded into the user space storage device 112 of the computer system. The security function execution server 116 has an Israel patent application number XXX.
It incorporates the API inhibition control server operation detailed in XXX.

B) API inhibition modules 134, 140, 146, etc. are active components. The operation of the API inhibition modules 134, 140, 146, etc.
This is detailed in Israeli Patent Application No. XXXXXXXXX. The API inhibition modules 134, 140, 146, etc. are composed of a dispatch routine, a depatch routine, a hook and patch routine, a pre-entry routine, and a post-entry routine. The operation of the routine is also detailed in Israel Patent Application No. XXXXXXXXX.

C) The system call blocking component 124 is an active component. The operation of the system call block component 124 is detailed in Israel Patent Application No. XXXXXXXXX.

D) API routines 132, 138, 144, etc. are passive components. The API routines 132, 138, 144, etc. are potential objects for the safety function execution system 100 to operate. API routine 132,1
38, 144, etc. are loaded into the process address space stores 118, 120, 122 etc. in the user store 112 of the system 100.

System 1 previously detailed in Israel Patent Application No. XXXXXX
00 serves as a model of the safety function execution system in which the present invention is operating. It will be appreciated by those skilled in the art that the present invention operates under a variety of similar systems and that the system shown here is an example to further demonstrate the workings of the invention.
Will be understood.

Please refer to FIG. This figure provides a high level flow chart of the operation of the security function execution server 116. The security function execution server 116 was previously detailed in Israel Patent Application No. XXXXXXXXX. The operation of the security function execution server 116 will be summarized and described herein.

Safety function execution (abbreviated from this point in the text of this document as SFE)
The server 116 initializes the application in step 150. Therefore, the SFE server 116 is configured by the host operating system (step 15
2. Always monitor the system calls made by the various applications running in 2) and, as detailed in FIG. 4, said system calls (step 154).
The execution of step 152 begins by responding appropriately to. SF
The EServer also keeps track of library calls made by various applications running on the host operating system (step 156). The SFE server is properly responding to the library call as detailed in FIG.

Reference is now made to FIG. This figure provides a high level flow chart of the start operation of the SFE server 116 referenced as step 150 in FIG. The start operation of the SFE server 116 was previously performed in Israel Patent Application No. XX
Detailed in XXXXXX.

First, the SFE server 116 uses the system call blocking component 124.
Is loaded into kernel space storage 114 (step 184). After establishing contact with the system call block component 124, the SFE server 116 queries the list of active processes 118, 120, 122, etc. (step 186) for the system call block component 124. When using the list of active processes 118, 120, 122, etc., the SFE server 11
6 creates a list of valid address ranges for each active process 118, 120, 122, etc. This structure will be referred to in this regard as the process effective address range in the body of this document.

The process effective address range list holds an address range in a place where the processes 118, 120, 122, etc. are loaded. The process effective address range also includes various dynamic link libraries (DLL) 130, 136,
Hold all address ranges in places where 142 etc. are loaded. A dynamic link library is a set of callable subroutines linked as a binary image that can be dynamically loaded by an application that utilizes callable subroutines.

Finally, the SFE server is responsible for all active processes 118, 120, 122, etc. (as detailed in Israeli patent application number XXXXXXXXX).
For step 19), the API inhibition modules 134, 140, 146, etc. are inserted.

The SFE server operation for monitoring the system call in step 152 of FIG.
This is detailed in Israeli Patent Application No. XXXXXXXXX. Let's move on to Fig. 4. 4, which is a high level flow chart of a response, such as an SFE server, to an interrupted system call, referenced as step 154 of FIG. The SFE server 116 determines in step 160 whether the detected system call is an illegal call or a legal call. The SFE server determines whether the system call is valid by comparing the system call generation address with a range of process valid addresses associated with the process generated by the system call. If an illegal call is detected,
SFE server 116 terminates the illegal function (step 164). Alternatively, the SFE server 116 may prompt the user () for the illegal call (step 166).
Notify the system administrator). Alternatively, the SFE server 116 performs another or other set of user predetermined actions. If the detected system call is legal (step 160), the SFE server 11
6 examines the system call to determine if it is the type of process creation (step 162). If a process creation type system call is determined, the SFE server 116 inserts the API blocking module 134 into the newly created process address space 118 (step 168) and adds the process address list to the process effective address. Update the process effective address range list (step 170) by adding to the range. If the determination at step 162 is negative, the SFE server optionally performs a user predetermined or commanded action over any other. if,
When a process termination type system call is determined, the SFE server 11
6 updates the process effective address range list (step 171) by deleting the process effective address from the process effective address range list.

The operation of the SFE server monitoring the library calls in step 156 of FIG. 2 is detailed in US patent application number XXXXXXXXX. Now, let's move to FIG. FIG. 5 is a high level flow chart of the response of the SFE server or the like to the blocked library call referenced as step 158 in FIG. The SFE server 116 determines in step 172 whether the detected library call is an illegal call. By comparing the library call generation address with the process effective address range associated with the process generated by the library call, it is determined whether the library call is valid or not.
The E server judges. If an illegal call is detected, SFE server 1
16 optionally terminates the illegal library function (step 180). Alternatively, the SFE server 116 may be a user (typically a system administrator) (step 1
82). Alternatively, SFE server 116 performs a user predetermined or ordered action (step 182) more than any other.

If the detected library call is legal (step 172),
The SFE server 116 determines whether the library call is a DLL 130 load (step 174) type. If the determination at step 174 is affirmative, then the library calls (APIs) 132 present in the loaded DLL 130 are connected and patched. Applying such a connection and splicing is an Israeli patent application More on. After connecting and splicing, the API blocking module 134 already loaded in the associated process 118 will immediately act to block the call made to the library call 132. Library call is DLL load S
If the FE server type is assumed, the process valid address range list (step 178) is updated by adding the DLL address range to the process valid address range list. If the determination at step 172 is negative, the blocked library call is DL from the process effective address range list.
DLL unloaded by deleting L address range (step 1
76) Judge whether the type. When the library call is determined to be the type of DLL unload SFE server, the process valid address range list (step 178) is updated.

Now refer to FIG. FIG. 6 is a high level flow chart of the operation of the call address verification routine module. The call address verification routine module operates in conjunction with the API inhibit module pre-entry routine as described in more detail in Israeli Patent Application No. XXXXXXX.

The pre-entry routine operates when the API 132 of FIG. 1 is blocked. When operating under the SFE system 100 pre-entry routine, the calling address verification routine module executes a set of instructions designed to verify the API function 132 of the calling address (calling routine) of FIG. There is. Calling routines also include calling application program interfaces, calling system calls, calling library calls, and so on.

The calling address confirmation routine module starts its operation by reading the calling routine return address from the procedure activation record (stack frame) on the user stack segment (step 191). The stack frame is a dynamic area of the process stack segment used as a control area for function calls. The process stack segment is
A dynamic storage area belonging to a process. In step 192, the calling routine calling address is called with the aid of data in the process effective address range list which checks whether the calling routine calling address is within the limits of the valid address range (step 194). The routine call address is calculated (step 192). In step 196, it is determined whether the calling address is valid or invalid. The calling routine call address matches the effective address range limit to make a calculation as to whether the calling routine call address is within the effective address range limit.
If the calling routine call address is within the valid address range, then the calling routine call address is valid. Next, the calling address confirmation routine module by the pre-entry routine or the like uses the SFE server 11 for the test result (step 198 and step 200).
Notify 6th grade.

In this embodiment of the invention described, any fraudulent or illegal system or library calls originating from storage areas such as active process address space storage 118, 120, 122 of FIG. 1 are detected, It will be appreciated by those skilled in the art that their implementation is optionally prevented by the SFE system 100.

Now refer to FIG. FIG. 7 is a high level flow chart of a call address verification routine associated with another embodiment of the present invention. In the preferred embodiment, the call address verification routine module begins its operation by reading the caller return address from the procedure activation record (stack frame) on the process stack segment (step 202). It will be appreciated that reading the calling routine return address will be significantly faster and more accurate. In step 204,
The calling routine call address is calculated and checked with the help of system level structures to determine if the call address is within the address limits of the process stack segment (step 206). Such a determination is accomplished by comparing the calling routine call address with the address limit of the process stack segment. Next, the calling address confirmation routine module by the pre-entry routine or the like checks the inspection result (step 210
And the step 212) is notified to the SFE server 116 or the like.

It will be appreciated by those skilled in the art that, in further inventive embodiments, any illegal or illegal system and library calls originating from the process stack segment structure will be detected and optionally prevented by the SFE system 100. Will

In addition, the advantages will be readily apparent to one of ordinary skill in the art. Therefore, the invention in its broader concept is not limited to the specific details, exemplary methods, systems and examples shown or described. The invention is not particularly limited to what has been shown and described above. The scope of the general concept of the invention as applied and the claims are shown in the "Claims" section.

The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate preferred embodiments of the invention and together with the description, serve to explain the essence of the invention.

[Brief description of drawings]

FIG. 1 is a block diagram of a safety function execution system environment, generally referred to as system 100.

[Fig. 2]   FIG. 6 is a high level flow diagram of the operation of the security function execution server 116.

FIG. 3 is a high level flow diagram of the operation of the secure function execution server initialization module referenced in FIG.

FIG. 4 is a high level flow diagram of a safety function performing server etc. response to the blocked system call referenced in FIG.

5 is a high level flow diagram of the operation of the library call response module such as the secure function execution server referenced in FIG.

FIG. 6 is a high level flow diagram of the operation of the Call Address Valid Routine module.

FIG. 7 is a high level flow diagram of a call address enable routine module associated with another embodiment of the present invention.

─────────────────────────────────────────────────── ─── Continued front page    (81) Designated countries EP (AT, BE, CH, CY, DE, DK, ES, FI, FR, GB, GR, IE, I T, LU, MC, NL, PT, SE, TR), AU, C A, JP (72) Inventor Rahman, Ofya             California, United States             94040, Los Altos, Ben Loe             1570 (72) Inventor Holwitz, Oded             Hazelya 46325, Haga, Israel             Na Street 43 F term (reference) 5B042 GA23 GA33 JJ29 JJ43 JJ46                       JJ47 KK01 LA20                 5B098 AA03 GA02 GA04 JJ03 JJ08

Claims (21)

[Claims] 1. A computer system operating an operating system platform, comprising: an operating system including a kernel space and a process space; a process space including a user application operating in the process space; and a system call. Comparing the user application acting to inhibit, a method of performing a safety function, the inhibited system call generation address and a range of process effective addresses associated with the process generated by the inhibited system call. The method thereby comprising the step of testing the effectiveness of the inhibited system call. 2. The method of claim 1, further comprising the step of providing a notification regarding the validity of the blocked system call. 3. The method of claim 1, further comprising terminating the blocked system call. 4. The method of claim 2, further comprising terminating the blocked system call. 5. The method according to claim 1, further comprising the step of responding to a process generation for inserting an application program interface module into the generated process and a process generation for updating a process effective address table. the method of. 6. The method of claim 1, comprising the step of responding to process termination updating the process effective address table. 7. A computer system running an operating system platform, comprising: an operating system including a kernel space and a process space; a process space including a user application operating in the process space; and a library call. Comparing the user application operating to inhibit, a method of performing a safety function, the inhibited library call generation address and a range of process effective addresses associated with the process generated by the inhibited library call. Said method comprising the step of examining the effectiveness of said inhibited library call. 8. The method of claim 7, further comprising the step of providing a notification regarding the effectiveness of the inhibited library call. 9. The method of claim 7, further comprising terminating the inhibited library call. 10. The method of claim 8, further comprising terminating the inhibited library call. 11. A dynamic link library for updating a process effective address table is unloaded in response to a system call for loading the dynamic link library which connects and splices library routines associated with the dynamic link library. The method of claim 7, comprising the step of responding to a system call. 12. In a computer system running an operating system platform, the operating system including a kernel space and a process space, the process space including a user application running in the process space, a system and a function. The user application acting on the call, the system or function call being disturbed,
A method of performing a safety function, comprising the steps of receiving a calling routine return address from the process memory and determining whether the calling routine address is valid by comparing the calling address routine with a process effective address table. The above method. 13. The method of claim 12, further comprising the step of providing a notification regarding the validity of the calling routine return address. 14. The method of claim 1, further comprising the step of performing a user predetermined action associated with the validity of the calling routine address.
2. The method described in 2. 15. Receiving a calling routine return address from the process memory further comprising determining the calling routine calling address by determining an address preceding the calling routine address. 13. The method of claim 12, comprising: 16. In a computer system running an operating system platform, the operating system including a kernel space and a process space, the process space including a user application running in the process space, a system and a function. The user application acting on the call, the system or function call being disturbed,
A method of performing a safety function, receiving a calling routine return address from the process memory and determining whether the calling routine address is valid by comparing the calling address routine with an associated process stack address area. The method comprising the: 17. The method of claim 16, further comprising the step of providing a notification regarding the validity of the calling routine return address. 18. The method of claim 1, further comprising the step of performing a user predetermined action associated with the validity of the calling routine address.
6. The method according to 6. 19. Receiving a calling routine return address from the process memory further comprising determining the calling routine call address by determining an address preceding the calling routine address. 17. The method of claim 16, comprising: 20. A method of performing a safety function substantially as described above. 21. A method of performing a safety function as described in the drawings.