RU2821054C1 - Method and system for forming clusters of nodes in computer network - Google Patents

Abstract

FIELD: electrical communication equipment.

SUBSTANCE: invention relates to communication networks. Method of forming clusters of nodes in a computer network includes steps of collecting data on network interactions of all nodes of the computer network for a selected period of time, wherein said data contain at least local and remote IP addresses and ports of nodes, determining a list of network connections of each node of a computer network, determining, based on the data, the number of network connections of each node to services by constructing an adjacency matrix of a bipartite graph, generating a graph Laplacian based on the data, and determining eigen values and eigen vectors of each node, performing clustering of computer network based on values of eigen vectors, wherein the clustering process continues until the stop criterion, a graph representing clusters of computer network nodes is formed, network interaction policies are applied in accordance with the obtained clusters of computer network nodes.

EFFECT: increased network security due to accuracy of network nodes clustering.

4 cl, 4 dwg

Description

TECHNICAL FIELD

[0001] The claimed technical solution generally relates to the field of communication networks, and in particular to a method and system for clustering a communication network based on network interaction data.

BACKGROUND OF THE ART

[0002] Currently, the world is paying much attention to the field of network security. Solutions aimed at ensuring network security make it possible to protect both data, including critical data, transmitted within the network, as well as to ensure protection from a variety of threats and the performance of the network as a whole, which, accordingly, is an integral part of the performance of any modern enterprise or organization.

[0003] One of the important tools in managing network interactions in enterprises and organizations is the establishment of network security policies. Network interaction policies allow you to detect both anomalies in the behavior of network devices and ensure the isolation of certain groups from other network participants, for example, to prevent leakage of critical data. As a rule, to formulate network interaction policies, it is necessary to form clusters within the network, i.e. divide all network devices into specific groups, which is a complex and non-trivial task. The accuracy and quality of such partitioning will directly affect the security of the network.

[0004] Thus, in the prior art, approaches based on clustering the graph of connections between network hosts and the intensity of interactions between network hosts are used to form clusters in a network.

[0005] The disadvantages of this approach are the low accuracy and efficiency of network clustering due to the impossibility of determining clusters in a network when the number of clusters in this network is unknown in advance. In addition, due to the construction of clusters only based on the interaction of network hosts, the formed groups may include nodes that do not actually belong to the specified group, which accordingly leads to a decrease in network security.

[0006] The prior art also knows the solution disclosed in US patent No. US 7945668 B1 (NARUS INC), publ. 05/17/2011. This solution, in particular, discloses a method for profiling network activity based on network clustering. Thus, to profile network activity, an association matrix and a reduced matrix are formed from the collected data on network activity, on the basis of which intermediate clusters are generated, which are subsequently combined into joint clusters in accordance with the degree of similarity of objects. Network operations are performed on the specified clusters.

[0007] The disadvantage of this solution is the low accuracy of the clustering process itself due to the splitting of intermediate clusters based on the connectivity metric and the further merging of these clusters into a joint one based on the threshold value, which, accordingly, can lead to an incorrect final cluster, and, as a result, fail to provide required network security.

[0008] The general disadvantages of existing solutions is the lack of an effective method for clustering network nodes, providing an accurate process for dividing nodes into groups due to additional data about the services with which the hosts interact and the intensity of interaction between hosts and services, which, accordingly, allows for more efficient formation of network policies security, detect anomalies in network nodes and increases the security of the network and data. In addition, this kind of solution should provide the ability to determine in advance an unknown number of clusters in the network.

DISCLOSURE OF INVENTION

[0009] The claimed technical solution proposes a new approach to the formation of clusters of nodes in a computer network. This solution uses data on the interaction of services and hosts to form clusters of nodes, which accordingly increases the accuracy of the formation of such clusters and provides the ability to apply network operations to these clusters.

[0010] Thus, the technical problem of ensuring the accuracy of clustering of network nodes is solved.

[0011] The technical result achieved by solving this problem is to increase network security due to the accuracy of clustering of network nodes.

[0012] The specified technical result is achieved by implementing a method for forming clusters of nodes in a computer network, performed by at least one computing device, and containing the steps of:

a) collect data on network interactions of all nodes of a computer network for a selected period of time, and the specified data contains at least: local and remote IP addresses and ports of nodes;

b) determine, based on the data obtained at stage a), a list of network connections for each node of the computer network;

c) determine, based on the data obtained in step b), the number of network connections of each node to services by constructing an adjacency matrix of a bipartite graph;

d) forming a Laplacian of the graph based on the data obtained in step c) and determining the eigenvalues and eigenvectors of each node;

e) perform clustering of the computer network based on the values of the eigenvectors obtained in step e), and the clustering process continues until the stopping criterion;

f) form a graph representing clusters of computer network nodes;

g) apply network interaction policies in accordance with the resulting clusters of computer network nodes.

[0013] In one particular embodiment, the services are at least one of an application, a database, or a software tool running on a node.

[0014] In another particular embodiment, the network communication policies are at least one of the following:

a) limiting network interactions between clusters of nodes;

b) limiting network interactions between certain clusters of nodes;

c) ensuring network interaction of network nodes within the cluster;

d) limiting anomalous network interactions of certain network nodes.

[0015] In addition, the claimed technical results are achieved through a system for forming clusters of nodes, containing:

at least one processor;

at least one memory coupled to the processor that contains computer-readable instructions that, when executed by at least one processor, implement a method for forming clusters of nodes in a computer network.

BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Features and advantages of the present invention will become apparent from the following detailed description of the invention and the accompanying drawings.

[0017] FIG. 1 illustrates a block diagram of the claimed method.

[0018] FIG. Figure 2 illustrates the intensity of interaction between hosts and services on a computer network.

[0019] FIG. 3 illustrates the result of network clustering.

[0020] FIG. 4 illustrates an example of a general view of a computing system that ensures the implementation of the claimed solution.

IMPLEMENTATION OF THE INVENTION

[0021] The claimed technical solution offers a new approach that improves the accuracy of clustering of network nodes, which, as a result, provides more flexible and efficient management of network security policies, and also increases the security of the network as a whole. In addition, the specified technical solution provides the ability to identify anomalous nodes based on the formed clusters.

[0022] The technical solution can be implemented in the form of a distributed computer system.

[0023] In this solution, a system means a computer system, a computer (computer), CNC (computer numerical control), PLC (programmable logic controller), computerized control systems and any other devices capable of performing a given, well-defined sequence of computing operations (actions, instructions).

[0024] By command processing device we mean an electronic unit or an integrated circuit (microprocessor) that executes machine instructions (programs).

[0025] A command processing device reads and executes machine instructions (programs) from one or more storage devices, such as devices such as random access memory (RAM) and/or read only memory (ROM). ROM can be, but is not limited to, hard drives (HDD), flash memory, solid-state drives (SSD), optical storage media (CD, DVD, BD, MD, etc.), etc.

[0026] Program - a sequence of instructions intended for execution by a computer control device or command processing device.

[0027] The term "instructions" as used in this application may refer generally to software instructions or software commands that are written in a given programming language to perform a specific function, such as, for example, retrieving hardware solution artifacts, generating digital standard for a software and hardware solution, generation of test results for a software and hardware solution, data analysis, etc. Instructions can be implemented in a variety of ways, including, for example, object-oriented methods. For example, instructions can be implemented using the programming language C++, Java, Python, various libraries (for example, MFC; Microsoft Foundation Classes), etc. Instructions that carry out the processes described in this solution can be transmitted over both wired and wireless data channels, such as Wi-Fi, Bluetooth, USB, WLAN, LAN, etc.

[0028] In FIG. 1 shows a block diagram of a method 100 for forming clusters of nodes in a computer network, which is described step by step in more detail below. Said method 100 involves performing steps to process various digital data. Processing is typically performed by a system, such as system 400, which may be, for example, a server, a computer, a mobile device, a computing device, etc. The elements of the system 400 are disclosed in more detail in FIG. 4

[0029] At step 110, data is collected about network interactions of all nodes of the computer network for a selected period of time, and the specified data contains at least: local and remote IP addresses and ports of the nodes.

[0030] At this step 110, data about the network activity of each network node is collected. A network node can be a computing device connected to an enterprise computer network. A computing device can be, for example, a server, a personal computer, a tablet, a peripheral device connected to a network, etc. A computer network can be a local area network of an enterprise, a corporate network, etc. An enterprise can be understood as any organization, company, firm, etc., with a structural communication network. Thus, the network activity of nodes can be obtained using network traffic analyzers. Network traffic analyzers are software and hardware tools for intercepting and analyzing network network traffic. In addition, network equipment, for example, network routers, can be used as an agent collecting network activity in an enterprise. Also, other means of obtaining network activity can be used, for example, the Netftow protocol from Cisco, Hadoop, etc. Collected network activity data is stored on a system, such as system 400, in the form of logs, such as Hadoop cluster parquet files. Thus, in one embodiment, network activity data is stored in an unstructured form and retrieved using regular expressions. This activity can be collected at certain periods of time, for example once an hour, and contain information about network communication flows, including the following fields: local IP address of the network device, local port, remote IP address, remote port, connection status, date and connection time, protocol type, host name, etc. In this solution, a host should be understood as any device that provides client-server format services in server mode over any interfaces and is uniquely defined on these interfaces. So a host can be a server, a computer connected to a communication network, etc.

[0031] At step 120, a list of network connections for each node on the computer network is determined, based on the data obtained at step 110. Based on data on network interactions containing at least local and remote IP addresses and ports of network hosts, records are filtered out in which the local IP address and remote IP address are equal to 127.0.0.1 or 0.0.0.0, which means that the host interacts exclusively within its domain. The specified filtering is necessary, in frequency, to exclude data about network interactions of internal host processes, i.e. data not related to the clustering process. The filtered data set is divided into two fields containing data about interactions of each host and data about interactions with services. In this solution, services should be understood as any software and/or application running on the host that was accessed. So, a service can be a database, a collaboration application, a video conferencing application, etc.

[0032] At step 130, the number of network connections of each node to services is determined, based on the data obtained at step 120, by constructing a bipartite graph adjacency matrix.

[0033] In FIG. 2 shows an example of a bipartite graph. The specified graph displays all interactions of hosts (circles) with services (squares) on a computer network. Because a graph is a data structure that represents a network with connections inside, then to ensure that all connections between network nodes can be determined, a graph is constructed at step 130. The specified graph can be generated in the security management system and displayed through input/output interfaces. To form the specified graph characterizing the intensity of interaction of nodes in the network, at the specified stage 130, from two fields of data on network interactions obtained at stage 120, a rectangular matrix is formed in which all occurring values of services are in the rows, and all occurring values are in the columns hosts. Based on the specified matrix, an adjacency matrix of a bipartite graph is formed. It is square, consisting of four blocks: two square blocks along the main diagonal and two rectangular blocks - above the main diagonal, below the diagonal. The size of the matrix is equal to the number of hosts and services. The matrix cells contain the number of network connections from the host to the service for the period under consideration. In the specified adjacency matrix, row and column indices represent nodes, and entries represent the absence or presence of an edge between nodes. The weights on the edges can represent the intensity of interactions between the hosts they connect.

[0034] The specified matrix is stored in the memory of the system 200. Based on the specified matrix, a bipartite graph is generated. Thus, at the specified stage 130, the number of network interactions of each host with services is formed, i.e. provides the ability to determine the total number of hosts and the total number of services, as well as the intensity of their interactions.

[0035] Next, based on the received data, the network clustering process is performed. In one particular embodiment, clustering of network nodes is performed using a spectral clustering algorithm. The spectral clustering algorithm is disclosed in more detail in steps 140-160.

[0036] The spectral clustering algorithm provides the ability to identify clusters in computer networks based on the interactions that occur in those networks.

[0037] At step 140, a Laplacian of the graph is generated based on the data obtained at step 130, and the eigenvalues and eigenvectors of each node are determined.

[0038] It is important to understand the concept of eigenvalues and eigenvectors of matrices. For a square matrix A, if there is a nonzero vector x and some scalar λ such that Ax = λx, then x is called an eigenvector of matrix A with a corresponding eigenvalue λ. Matrix A is essentially a function that maps vectors into new vectors. Most vectors will change in a completely random way when A is applied to them, but eigenvectors only change in magnitude. The eigenvector is scaled according to the eigenvalue λ. Eigenvectors help describe the dynamics of systems represented by matrices, namely the dynamics of a communication network. The specified eigenvectors are necessary to implement the spectral clustering algorithm.

[0039] Thus, at said step 140, the eigenvalues and eigenvectors of the matrix are found. The specified eigenvectors reflect data about the node's membership in the cluster. Since the matrix is symmetric about the main diagonal and consists of real numbers, all eigenvalues are non-negative real numbers. And eigenvectors consist of real numbers. Next, we consider the eigenvalues in ascending order and their corresponding eigenvectors. In this case, the first eigenvalue of the matrix is always equal to 0 (by construction of the matrix, the degree of each vertex is equal to the sum of the row and column of the adjacency matrix of the graph). In general, the number of zero eigenvalues is equal to the number of connected components of the graph. Thus, zero eigenvalues represent connected components, so eigenvalues near zero tell us that there is a separation of two components. So, for example, having four eigenvalues before the break would mean that there are four clusters. The vectors associated with the first three positive eigenvalues should tell us which three cuts need to be made in the graph to assign each node to one of the four approximate components.

[0040] Thus, determining the eigenvalues makes it possible to determine the number of clusters in the network. Therefore, at this step 140, data on the number of clusters in the network is obtained.

[0041] At step 150, the computer network is clustered based on the eigenvector values obtained at step 140, with the clustering process continuing until the stopping criterion.

[0042] To directly determine the number of clusters in the network and the belonging of each node to a specific cluster, eigenvectors are considered in a cycle, starting from the second. Look at the signs of the numbers included in the eigenvector. If it is greater than zero, then the corresponding vertex of the graph belongs to one cluster; if it is less than zero, to another. If the value is close to zero, then such vertices belong to either the first or second cluster or are separated into a separate cluster (poorly distinguishable vertices). Consideration of each subsequent eigenvector will result in the division of the already obtained clusters into smaller parts. An empirically selected stopping criterion allows us to consider eigenvectors corresponding to eigenvalues less than 1. Typically these are 4-10 eigenvectors. Further, in the spectrum of the matrix there are several tens or hundreds of eigenvalues equal to 1. If we consider the partitions using the corresponding eigenvectors, then the resulting partitions do not affect the partitioning of hosts into clusters, but make increasingly detailed partitions of services. Thanks to this feature, it is possible to accurately divide nodes into network clusters. At the same time, it is worth noting that the time and required expended computing resources are also reduced, because In prior art algorithms, all vectors are considered, which leads to an inaccurate division of nodes into clusters.

[0043] At step 160, a graph is formed that represents clusters of computer network nodes;

[0044] Based on the data obtained at step 150, a graph is generated that represents clusters of nodes. Thus, the specified process can be performed in the system 400. The result of the graph formation can be presented to the network administrator, for example, in a graphical interface.

[0045] So, FIG. 3 reveals the result of clustering of enterprise network nodes, obtained during the creation of the specified technical solution. Services are marked with squares, and hosts are marked with circles. This column reflects the actual structure of the enterprise. To form the graph, a simulation of the physical interaction of the graph vertices was carried out (vertices belonging to one cluster are attracted to a certain distance, vertices belonging to different clusters repel, attractive forces act along the edges like stretched elastic bands).

[0046] At step 170, network communication policies are applied in accordance with the resulting clusters of computer network nodes.

[0047] At step 170, network security policies are applied to the clustered enterprise network in accordance with established requirements.

[0048] Thus, network communication policies are at least one of the following: limiting network communications between clusters of nodes; limiting network interactions between certain clusters of nodes; ensuring network interaction of network nodes within the cluster; limiting anomalous network interactions of certain network nodes.

[0049] Since an enterprise network may have a large number of hosts and services, individually applying security policies to each node is an impossible task. The specified method 100, thanks to the implementation of the clustering algorithm, provides the ability to effectively manage network security, and due to the accuracy of the algorithm achieved in the specified solution, the security of the network as a whole is also increased, because The network security policy is installed on the relevant cluster.

[0050] Thus, continuing with step 170, the network clustering data obtained at step 160 can be compared with cluster data contained in an accounting system, such as an enterprise security system. Thus, the specified security system may contain, for example, in a database, information about the membership of nodes in a specific cluster. As mentioned above, a cluster can be a group of computer network nodes, for example, a department, department, etc. Based on the results of comparing the actual and specified structure, anomalies are detected when the actual and accounting clusters coincide, say, by 80-90 percent, and 10-20 percent of hosts fall into other clusters (actually constitute anomalies). Such cases characterize atypical behavior of a node on the network, for example, accessing databases that are not used in its cluster, which, accordingly, may indicate anomalous behavior. Security measures can then be applied to identified anomalous nodes, for example, automatic temporary blocking, restriction of network interactions, etc.

[0051] Additionally, in yet another particular embodiment, based on the results of identified network clusters (step 160), the security system may automatically enforce network policies, such as allowing network interactions within clusters and restricting network interactions between clusters. It will be apparent to one skilled in the art that the limitations may depend on the sensitivity of the data exchanged by each cluster. For example, a cluster belonging to a department that has access to bank secrets may be limited to data exchange only within the enterprise network. In another example, the public relations department might have a policy that allows Internet use but prohibits downloading files.

[0052] Thus, the above materials of the claimed technical solution disclose an effective and accurate method for clustering an enterprise communications network to establish network security policies, which improves network security due to the ability to establish group policies and identify anomalies based on accurate network clustering. In addition, as follows from the above application materials, the specified method of clustering a communication network may be part of a method for limiting network interactions in an enterprise, i.e. protecting the communication network by establishing network security policies and/or identifying anomalous nodes in the specified communication network.

[0053] In FIG. 4 shows an example of a general view of a computer system 400, which provides the implementation of the claimed method or is part of a computer system, for example, a server, a personal computer, or part of a computing cluster that processes the necessary data to implement the claimed technical solution.

[0054] In general, system 400 includes components such as: one or more processors 401, at least one memory 402, data storage 403, input/output interfaces 404, I/O 405, networking 406, which are connected via a universal bus.

[0055] The processor 401 performs the basic computational operations necessary to process data when executing the method 100. The processor 401 executes the necessary computer-readable instructions contained in the main memory 402.

[0056] Memory 402 is typically in the form of RAM and contains the necessary software logic to provide the required functionality.

[0057] The data storage medium 403 can be in the form of HDD, SSD drives, raid array, flash memory, optical storage devices (CD, DVD, MD, Blue-Ray disks), etc. The means 403 allow for long-term storage of various types of information, for example, data on network activity of nodes, etc.

[0058] To organize the operation of system components 400 and organize the operation of external connected devices, various types of I/O interfaces 404 are used. The choice of appropriate interfaces depends on the specific design of the computing device, which can be, but is not limited to: PCI, AGP, PS/2, IrDa, Fire Wire, LPT, COM, SATA, IDE, Lightning, USB (2.0, 3.0, 3.1, micro, mini, type C), TRS/Audio jack (2.5, 3.5, 6.35), HDMI, DVI, VGA, Display Port, RJ45, RS232, etc.

[0059] The choice of interfaces 404 depends on the specific implementation of the system 400, which can be implemented on a wide class of devices, for example, a personal computer, mainframe, laptop, server cluster, thin client, smartphone, server, etc.

[0060] The following can be used as I/O data 405: keyboard, joystick, display (touch display), monitor, touch display, touchpad, mouse, light pen, stylus, touchpad, trackball, speakers, microphone, augmented reality, optical sensors, tablet, light indicators, projector, camera, biometric identification tools (retina scanner, fingerprint scanner, voice recognition module), etc.

[0061] Network communication means 406 are selected from devices that provide network reception and transmission of data, for example, an Ethernet card, WLAN/Wi-Fi module, Bluetooth module, BLE module, NFC module, IrDa, RFID module, GSM modem, etc. . Using tools 405, the organization of data exchange is ensured between, for example, a system 400, represented in the form of a server, and a user’s computing device, on which the received data can be displayed (clustering of network nodes) via a wired or wireless data transmission channel, for example, WAN, PAN, LAN (LAN), Intranet, Internet, WLAN, WMAN or GSM.

[0062] The specific selection of elements of the device 400 to implement various software and hardware architectural solutions may vary while maintaining the required functionality provided.

[0063] The submitted application materials disclose preferred examples of implementation of a technical solution and should not be interpreted as limiting other, particular examples of its implementation that do not go beyond the scope of the requested legal protection, which are obvious to specialists in the relevant field of technology. Thus, the scope of the present technical solution is limited only by the scope of the attached formula.

Claims (19)

1. A method for forming clusters of nodes in a computer network, performed by at least one computing device and containing the steps of: a) collect data on network interactions of all nodes of a computer network for a selected period of time, and the specified data contains at least: local and remote IP addresses and ports of nodes; b) determine, based on the data obtained in step a), a list of network connections for each node of the computer network; c) filter network connections, during which they exclude connections that characterize the interaction of a node within its domain; d) determine, based on the data obtained in step c), the number of network connections of each node to services by constructing an adjacency matrix of a bipartite graph; e) form, based on the data obtained at stage d), a bipartite graph characterizing the total number of nodes, the total number of services and the intensity of their interactions; f) forming a Laplacian of the graph based on the data obtained in step e) and determining the eigenvalues and eigenvectors of each node; g) perform clustering of the computer network based on the eigenvector values obtained in step f), and the clustering process continues until the stopping criterion; h) form a graph representing clusters of computer network nodes; i) apply network interaction policies in accordance with the resulting clusters of computer network nodes. 2. The method according to claim 1, characterized in that the services represent at least one of: an application, a database, a software tool running on the node. 3. The method according to claim 1, characterized in that the network interaction policies are at least one of the following: a) limiting network interactions between clusters of nodes; b) limiting network interactions between certain clusters of nodes; c) ensuring network interaction of network nodes within the cluster; d) limiting anomalous network interactions of certain network nodes. 4. A system for forming clusters of nodes in a computer network, containing: at least one processor; at least one memory coupled to the processor, which contains machine-readable instructions that, when executed by at least one processor, enable the method of any one of claims to be carried out. 1-3.