GB2572637A

GB2572637A - Detecting shared library hijacking

Info

Publication number: GB2572637A
Application number: GB1805756.2A
Authority: GB
Inventors: Suominen Mikko
Original assignee: F Secure Oyj
Current assignee: WithSecure Oyj
Priority date: 2018-04-06
Filing date: 2018-04-06
Publication date: 2019-10-09
Anticipated expiration: 2038-04-06
Also published as: GB2572637B; GB201805756D0

Abstract

A computer-implemented method of detecting shared library hijacking in which an executable application is caused to load a malicious shared library file instead of a trusted shared library file. An executable file installed in a location on a file system of a computer system is identified and a location profile for the file determined. A location profile may include digital signatures of the executable file, version information, a full path for the executed file or information about other content in the folder. A location profile for the executable file is obtained from a location profile store, the location profile being associated with a non-malicious installation of the executable file on the file system. The location profile determined for the identified executable file is compared with the location profile obtained from the location profile store to determine whether the executable file was installed maliciously and thereby detect shared library hijacking. Indications of a malicious location profile may be based on its digital signature and version information or location of executable file not being on a whitelist of acceptable locations or content that is normally stored alongside the executable file being missing.

Description

DETECTING SHARED LIBRARY HIJACKING

Technical Field

The present invention relates to the detection of malware on a computer system and, in particular, the detection of malware which hijacks loading of a shared library on a computer system.

Background

Operating systems typically provide functionality which allows computer code and other data to be stored in a shared library which can be used by multiple computer programs. This encourages re-use of computer code and more efficient memory usage. For example, computer code associated with the graphical user interface of the operating system is typically provided in one or more shared library files so that each computer program running on the operating system has a similar look and “feel”.

Pre-compiled computer code from a shared library may be incorporated into a program when the program is being compiled (so-called “static linking”). Alternatively, the library code may be accessed only when the program is executed (so-called “dynamic linking” or dynamic loading). In the latter case, the program may locate the shared library file based on a hierarchy (or “search path”) of possible locations within the file system of the operating system. For example, the program may initially search for a shared library in the directory from which the program is loaded. If the shared library cannot be found, the program may then search a “system directory” that is used by the operating system to store shared library files, e.g. on a Windows (TM) operating system (OS), a directory such as “C:\Windows\System”.

An example of a type of shared library file used by Windows™ is a DLL (Dynamic-Link Library) file, whilst for Linux-based OSs, shared library files are referred to as “shared object libraries” and carry the extension “.so”.

Malware may modify or replace the computer code of a DLL file (or other type of shared library file) with malicious code (to generate a “malicious” DLL) which may be loaded and executed by a computer program which is dynamically linked to the DLL file. Such an attack is commonly referred to as “DLL hijacking”, although this type of attack is not specific to Windows™. This attack method is used especially by Advanced Persistent Threats to execute their own code in a clandestine way.

DLL hijacking typically relies on a malicious DLL file being located at a specific location in the file system to ensure that it is loaded in preference to another (genuine) DLL with the same name. For example, an application may, at a certain point in its execution, require code contained in a DLL file called “shared_library.dll” to be loaded. The application queries the file system in order to determine whether a file with this name can be found in one of the locations specified in the application (or system) search path. In normal use, the file may be found in “C:\Windows\System”, for example. However, an attacker may place a malicious version of “shared_library.dll” in a location which is higher in the search hierarchy, so that the malicious version of the DLL file is found in preference to the genuine DLL file. The application then loads the malicious DLL to its own process instead of the DLL it should have loaded as part of its normal operation. The process will then automatically execute whatever code the malicious DLL contains. In addition, the attacker may store the clean / trusted application in a non-standard location such that when the DLL is loaded the search follows a different path from the standard path. As long as the malicious DLL is stored somewhere along this path, ahead of the genuine DLL, the malicious DLL will be loaded.

To guard against these types of attack, DLL files may be digitally signed so that computer programs which are dynamically linked to the DLL are able to verify the identity of the DLL file. However, many computer programs do not make use of this form of security or may contain unpatched DLL hijacking vulnerabilities, such as vulnerabilities caused by insufficient checks in the code that loads DLLs. Skilled attackers can make use of these programs for DLL hijacking. For example, an attacker may be able to determine that a well-known, digitally signed, reputable clean application has poorly implemented DLL loading logic. The attacker can make use of such a clean but inadequate application to launch the hijacking attacks outlined in the preceding paragraph.

By selecting a commonly trusted, digitally signed executable file as the “host” for loading their malicious DLL, attackers may easily bypass most anti-virus products as these products will not monitor anything the known clean file does. The trust of known software vendors is required for both performance and false alarm avoidance and is therefore integral to most anti-virus software. As such, even though an executable file may be installed for the sole purpose of running code from a malicious DLL, existing anti-virus products are, in general, unable to detect this threat. There is therefore a need for improved detection of malware which operates by DLL hijacking.

Summary of the Invention

According to a first aspect of the present invention there is provided a computerimplemented method of detecting shared library hijacking in which an executable application is caused to load a malicious shared library file instead of a trusted shared library file. An executable file installed in a location on a file system of a computer system is identified and a location profile for the file determined. A location profile for the executable file is obtained from a location profile store, the location profile being associated with a non-malicious installation of the executable file on the file system. The location profile determined for the identified executable file is compared with the location profile obtained from the location profile store to determine whether the executable file was installed maliciously and thereby detect shared library hijacking.

The executable file may be digitally signed and/or provided by a trusted software vendor. In other words, the executable file itself may not be malicious, but rather it is the installation of that file that has a malicious purpose as it will cause the application to call a malicious shared library file.

The method may comprise, in response to detecting that the executable file was installed maliciously, scanning the file system for a malicious shared library file. Scanning the file system for the malicious shared library file may make use of the location profile determined for the identified executable file and/or the identity of the executable file.

Each location profile may comprise a file system path for the executable file and/or data associated with one or more other objects installed in the same location as the executable file. The data may identify each of the objects.

The location profile obtained from the location profile store may comprise one or more rules for determining, based on the location profile of the identified executable file, whether the executable file was installed maliciously.

The method may comprise detecting an attempt to load a shared library, the attempt being triggered by execution of the executable file. Execution of the executable file may be terminated or prevented in response to determining that the executable file was installed maliciously.

The method may comprise determining that the executable file is installed in a location for which the computer device allows files to be written without requiring administrator rights and using this as an indication of a malicious installation.

The location profile obtained from the location profile store may be derived, at least in part, from an installation of the executable file in a virtualised environment and/or, at least in part, from data collected by an Endpoint Protection Platform.

According to a second aspect of the present invention there is provided a method of identifying a malicious shared library file. The method comprises detecting shared library hijacking using the method described above and further comprises identifying a malicious shared library file based on the location of the identified executable file.

The identified executable file may be located in a directory of the file system. Identifying the malicious shared library file may comprise identifying a shared library file in that directory or in a sub-directory of that directory. The method may further comprise comparing the identified shared library file with one or more versions of the shared library file which are known to be clean or which are located in locations of the file system which are known to be trusted.

According to a third aspect of the present invention, there is provided a computer program product. The computer program product comprises a computer storage medium having computer code stored thereon which, when executed on a computer system, causes the system to perform the method described above.

According to a fourth aspect of the present invention, there is provided a computer system for shared library hijacking in which an executable application is caused to load a malicious shared library file instead of a trusted shared library file. The system comprises a memory storing a plurality of location profiles, each location profile being associated with a non-malicious installation of a respective executable file on a file system. The system also comprises a processor configured to: identify an executable file installed in a location on a file system and determine a location profile for the file; obtain, from the memory, a location profile for the executable file; and compare the location profile determined for the identified executable file with the location profile obtained from the location profile store to determine whether the executable file was installed maliciously and thereby detect shared library hijacking.

The computer system may be a client computer on which the executable file is installed or a remote server configured to communicate with a client computer on which the executable file is installed.

Embodiments of the invention may reduce the risk of successful attacks on computers and computer systems resulting in improved and better performing computers and computer systems. For example, computers may be less prone to crashing, may operate more quickly, and may be more likely to operate in a correct fashion.

Brief Description of the Drawings

Figure 1 is a flowchart of a malware detection method according to an embodiment of the invention;

Figure 2 is a schematic diagram of a computer system according to an embodiment of the invention; and

Figure 3 is a schematic diagram of a computer system according to an embodiment of the invention.

Detailed Description

As has been described above, an attacker may be able to successfully hack, or otherwise hijack or damage, a computer system by installing a malicious DLL at a location that is loaded, by a clean / trusted application, ahead of a genuine DLL at another location. A particularly advantageous attack causes the application to be installed at a non-standard location as this gives the attacker more scope to store the malicious DLL at a location that will be accessed ahead of a location where the genuine DLL is stored.

In order to provide improved protection against such an attack, an approach is proposed which identifies whether an executable file is present in an “unusual” or suspicious location in the file system of a computer device. The executable file may be provided by a trusted software vendor and identified based on, for example, its digital signature and version information. In one example, the location of the executable file (e.g. its path within the file system) is compared with a “whitelist” of acceptable locations for that executable file. If the location of the executable file is not found on the whitelist then this can be taken as an indication that DLL hijacking malware may be present on the computer device. A location may also be characterised as being suspicious based on other content stored in (or absent from) the same location as the executable file, e.g. if files which are normally installed alongside the executable file are missing.

Operating systems typically provide one or more “standard” locations into which applications and application data may be installed. For example, most Windows applications are installed in:

• “C:\Program Files”; or • “C:\Program Files (x86)”.

On Linux systems, typical locations for executable files include:

• “/bin”;

• “/user/bin”; or • “/usr/share”.

As administrator rights are typically required to install files in these standard locations, an attacker may be forced to install a “host” application for DLL hijacking elsewhere. Therefore, if an application is not installed in one of these locations then this may be an indication that it was installed by an attacker or malware, rather than by a legitimate user or process. .

An application typically includes at least one executable file and a number of other objects which may be required for execution of the application, such as data archive files or configuration files. In particular, many applications may have a relatively large number of objects which are stored in the same folder as the executable file or in a number of sub-folders. The number of these objects, their names and how they are arranged within the file system typically remains relatively static over time and is normally the same when the application is installed.

When an application is installed maliciously for the purpose of DLL hijacking, the objects associated with the application may be different. For example, the number of files accompanying the executable file may be radically reduced for a malicious installation for reasons of simplicity and compactness, e.g. to reduce the time required to install the application.

Several methods can be used to determine which locations should be deemed nonsuspicious for a particular executable file, i.e. which locations should be considered usual or “safe” locations for the executable file. For example, monitoring software may be used to extract relevant location information during the execution of the executable file in normal use (i.e. at runtime). Alternatively (or in addition) scanning software may be used to perform a “static” analysis of the location of executable files, i.e. without requiring the files to be executed. The information can be collected in a virtualized environment where clean software is installed automatically and/or manually. The information may also be collected by analysing the applications which have been or are being installed on users’ devices (“endpoints”), e.g. information from an Endpoint Protection Platform (EPP) or Endpoint Detection and Response (EDR)-type solution running on real endpoints.

The collected information may include:

• partial or full digital signature details of the executable file, e.g. data to identify the software author;

• version information for the executable file, e.g. information stored in the executable file’s resources, to allow identification of the application to which the executable file belongs, such as the ‘Product name’, ‘Product version’, ‘Filename’ and Original filename’ fields;

• a full (or partial/relative) path for the executed file; and/or • information about other content in the folder, such as the number of files, number of folders, and/or the names of those files and folders.

The collected information may be stored in a local or a remote database as a “location profile” for the executable file.

Figure 1 is a flowchart of a method for detecting a potential attack by DLL hijacking. An executable file installed in a location on a file system of a computer device is identified (S101). A location profiles for the executable file is then accessed (S102). As discussed above, the location profile comprises data associated with one or more nonmalicious installations of the executable file in a respective location on the file system. Data associated with the location of the executable file is then compared with the location profile to determine whether the executable file was installed maliciously (S103).

In one implementation of the method, when a new process is launched by loading an executable file, signature and version information associated with the process is used to retrieve one or more location profiles which are associated with a non-suspicious location for the executable file. The path of the new process is then compared to the paths which have been seen for previous processes. The content of the folder where the execution happens may also be compared with content which was found for previous executions. If the location and contents of the folder differ significantly from previously seen data for that software, the new execution is deemed or suspected to be because it is used for DLL hijacking. When finding relevant previous executions’ data, data from any number of different endpoints or virtualized runs can be used, not just executions that have happened on the same endpoint as the new execution for which matches are being searched.

Evaluating whether locations and their contents are similar between two executions of an executable file can be done with manually defined rules (e.g. using a decision tree), machine learning or any other suitable decision-making mechanism.

As an example of a simple rule-based approach, the following rules may cover many of the most common cases:

• previously seen execution locations are all somewhere under ‘Program Files’;

• previously seen total file and folder count in the execution folder is over 10;

• new execution location includes the folder ‘appdata’ in its path; and/or • total file and folder count in the new execution location is under 5.

Decision-making does not necessarily have to include both a location and the content at the location; in some cases only one of these can be enough.

When DLL hijacking is detected or suspected, monitoring software can terminate the offending process (and also its parent if it is not a critical system process) and notify the user.

Following detection of DLL hijacking, a malicious DLL (which forms part of the attack) may be identified in a number of ways. For example, once it has been determined that an executable file has been installed in a “non-standard” directory or folder, a malicious DLL can be identified by comparing, directly or indirectly (e.g. using a comparison of hashes), the DLLs in the same directory (or in a sub-directory of the same directory) with versions of the DLLs that are known to be clean. Clean version of the DLLs may be obtained by searching locations of the file system which are known to be clean, e.g. searching for DLLs with the same name in one or more system directories. Clean versions of the DLLs, or information identifying them, such as a file hash, may also be retrieved from a database, such as a remote database. If the malicious DLL differs from all known clean versions of the DLL, then it may be concluded that the DLL is being used for hijacking.

In another implementation of the method, execution of the executable file is not required. For example, the method may be performed while carrying out a scan of the file system.

Figure 2 shows a schematic view of a client computer 200 according to an embodiment of the invention. The client computer 200 comprises a memory 201 storing a file system 202, a location profile store 203 and a processor 204. The location profile store 203 stores a plurality of location profiles, each location profile being associated with a non-malicious installation of a respective executable file on a file system 202 of the client computer 200.

Figure 3 shows an embodiment of a computer system in which the location profiles are stored in a location profile store 305 of a remote server 304 and the malware detection performed by a processor 305 of the remote server 304. In this case, the client computer 300 comprises a memory 301 storing a file system 302 on which the executable file is installed and a processor 303. The client computer 300 communicates with the server 304. The processor 303 extracts data associated with the location of the executable file and data associated with the identity of the executable file and sends these data to the server 304. The server 304 may respond to the client computer 300 with an indication that malware may be present on the client computer and/or with instructions to remove the executable file.

Although the invention has been described in terms of preferred embodiments as set forth above, it should be understood that these embodiments are illustrative only and that the claims are not limited to those embodiments. Those skilled in the art will be able to make modifications and alternatives in view of the disclosure which are contemplated as falling within the scope of the appended claims. Each feature disclosed or illustrated in the present specification may be incorporated in the invention, whether alone or in any appropriate combination with any other feature disclosed or illustrated herein.

Claims

CLAIMS:

1. A computer-implemented method of detecting shared library hijacking in which an executable application is caused to load a malicious shared library file instead of a trusted shared library file, the method comprising:

identifying an executable file installed in a location on a file system of a computer system and determining a location profile for the file;

obtaining from a location profile store a location profile for the executable file, the location profile being associated with a non-malicious installation of the executable file on the file system; and comparing the location profile determined for the identified executable file with the location profile obtained from the location profile store to determine whether the executable file was installed maliciously and thereby detect shared library hijacking.

2. A method according to any one of the preceding claims, wherein the executable file is digitally signed.

3. A method according to claim 1 or 2, wherein the executable file is provided by a trusted software vendor.

4. A method according to any one of the preceding claims, wherein each location profile comprises a file system path for the executable file.

5. A method according to any one of the preceding claims, wherein the location profile comprises data associated with one or more other objects installed in the same location as the executable file.

6. A method according to claim 5, wherein the data associated with the one or more other objects identifies each of the objects.

7. A method according to any one of the preceding claims, wherein the location profile obtained from the location profile store comprises one or more rules for determining, based on the location profile of the identified executable file, whether the executable file was installed maliciously.

8. A method according to any one of the preceding claims, wherein identification of the executable file is triggered by the file being executed.

9. A method according to any one of the preceding claims and comprising detecting an attempt to load a shared library, the attempt being triggered by execution of the executable file.

10. A method according to any one of the preceding claims and comprising terminating or preventing execution of the executable file in response to determining that the executable file was installed maliciously.

11. A method according to any one of preceding claims and comprising determining that the executable file is installed in a location for which the computer device allows files to be written without requiring administrator rights.

12. A method according to any one of the preceding claims, wherein the location profile obtained from the location profile store is derived, at least in part, from an installation of the executable file in a virtualised environment.

13. A method according to any one of the preceding claims, wherein the location profile obtained from the location profile store is derived, at least in part, from data collected by an Endpoint Protection Platform.

14. A method of identifying a malicious shared library file, the method comprising detecting shared library hijacking using the method of any one of the preceding claims and further comprising identifying a malicious shared library file based on the location profile of the identified executable file.

15. A method according to claim 14, wherein the location profile for the executable file identifies a directory in which the executable file is located and wherein identifying the malicious shared library file comprises identifying a shared library file in that directory or in a sub-directory of that directory.

16. A method according to claim 15 and comprising comparing the identified shared library file with one or more versions of the shared library file which are known to be clean or which are in locations of the file system which are trusted.

17. A computer program product comprising a computer storage medium having computer code stored thereon which, when executed on a computer system, causes the system to perform the method of any one of the preceding claims.

18. A computer system for shared library hijacking in which an executable application is caused to load a malicious shared library file instead of a trusted shared library file, the system comprising:

a memory storing a plurality of location profiles, each location profile being associated with a non-malicious installation of a respective executable file on a file system; and a processor configured to:

identify an executable file installed in a location on a file system and determine a location profile for the file;

obtain, from the memory, a location profile for the executable file; and compare the location profile determined for the identified executable file with the location profile obtained from the location profile store to determine whether the executable file was installed maliciously and thereby detect shared library hijacking.

19. A computer system according to claim 18, wherein the computer system is a client computer on which the executable file is installed.

20. A computer system according to claim 18, wherein the computer system is a remote server configured to communicate with a client computer on which the executable file is installed.