CN110149234B - Graph data compression method, device, server and storage medium - Google Patents
Graph data compression method, device, server and storage medium Download PDFInfo
- Publication number
- CN110149234B CN110149234B CN201910446525.0A CN201910446525A CN110149234B CN 110149234 B CN110149234 B CN 110149234B CN 201910446525 A CN201910446525 A CN 201910446525A CN 110149234 B CN110149234 B CN 110149234B
- Authority
- CN
- China
- Prior art keywords
- node
- network graph
- distance
- graph
- nodes
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/12—Discovery or management of network topologies
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention provides a method and a device for compressing image data, a server and a storage medium, and belongs to the technical field of internet. The method comprises the following steps: determining the importance of each first node in a first network graph to be compressed; selecting a plurality of second nodes from the first network graph according to the importance of each first node; forming a second network graph from the plurality of second nodes, and forming at least one node pair from two different second nodes in the plurality of second nodes; determining a first distance of each node pair in the first network graph; for each node pair, determining a second distance of the node pair in the second network graph, and adding distance information in the second network graph according to the first distance of the node pair, the second distance of the node pair and the graph parameter. Due to the fact that the graph parameters are set, the number of nodes reserved in the second network graph can be limited through the graph parameters, so that excessive nodes and edges cannot be increased, graph data are effectively compressed, and the obtained second network graph is accurate.
Description
Technical Field
The present invention relates to the field of internet technologies, and in particular, to a method and an apparatus for compressing image data, a server, and a storage medium.
Background
With the rapid development of internet technology, a great deal of data is accumulated in various applications. The graph is used as an effective data structure for describing big data, and is widely applied to the fields of internet analysis, social network analysis, recommendation network analysis and the like. In the structure of graph data, data and the relationship between data are generally stored using the form of a network graph. The network graph comprises a plurality of nodes, the nodes are used for storing data, and edges among the nodes are used for representing relations among the data. Because the network map includes large-scale data. Therefore, in the process of analyzing, processing and applying large-scale graph data, a large number of nodes and edges included in the network graph need to be compressed.
In order to solve the above problems, a compression idea is currently adopted to reserve some important nodes and some edges in the network graph to ensure connectivity of the network graph. For example, one graph compression algorithm is to determine important nodes in the network graph, construct a minimum spanning tree with the important nodes, and use the minimum spanning tree as a compressed network graph.
The method has the problems that the distance information between the nodes in the original graph is lost by only constructing the minimum spanning tree, so that the obtained second network graph is inaccurate.
Disclosure of Invention
The embodiment of the invention provides a graph data compression method, a graph data compression device, a server and a storage medium, which are used for solving the problem that the distance information between nodes in an original graph is lost and the obtained second network graph is inaccurate because the minimum spanning tree is constructed only by important nodes in the current graph compression algorithm. The technical scheme is as follows:
in one aspect, a method for compressing graph data is provided, where the method includes:
determining the importance of each first node in a first network graph to be compressed;
selecting a plurality of second nodes from the first network graph according to the importance of each first node;
forming a second network graph by the plurality of second nodes, and forming a node pair by two different second nodes in the plurality of second nodes to obtain at least one node pair;
determining a first distance of each node pair in the first network graph;
for each node pair, determining a second distance of the node pair in the second network graph, adding distance information of the node pair in the first network graph in the second network graph according to the first distance of the node pair in the first network graph, the second distance of the node pair in the second network graph and a graph parameter, wherein the size of the graph parameter is inversely related to the number of nodes reserved in the second network graph.
In one possible implementation, the adding, in the second network graph, distance information of the node pair in the first network graph according to a first distance of the node pair in the first network graph, a second distance of the node pair in the second network graph, and a graph parameter includes:
determining a product of a first distance of the node pair in the first network graph and the graph parameter to obtain a third distance;
when the second distance of the node pair in the second network graph is larger than the third distance, adding nodes and edges included in a shortest path between the node pair in the first network graph into the second network graph, wherein the shortest path is a path corresponding to the first distance.
In another possible implementation, the determining a first distance of each node pair in the first network graph includes:
when the first network graph is a non-weight graph, determining the shortest distance of the node pair in the first network graph, and taking the shortest distance as the first distance of the node pair in the first network graph;
when the first network graph is a weight graph, determining the weighted sum of each path of the node pair in the first network graph, and taking the smallest weighted sum of the weighted sums of each path as the first distance of the node pair in the first network graph.
In another possible implementation, the determining a second distance of the node pair in the second network graph includes:
determining a fourth distance of a shortest path between the pair of nodes when a path between the pair of nodes exists in the second network graph;
taking a fourth distance of the shortest path as a second distance of the node pair in the second network graph.
In another possible implementation manner, before determining a product of the first distance of the node pair in the first network graph and the graph parameter to obtain a third distance, the method further includes:
when a path between a node pair exists in the second network graph, performing the step of determining a product of a first distance of the node pair in the first network graph and the graph parameter to obtain a third distance;
when no path between pairs of nodes exists in the second network graph, performing the step of adding nodes and edges in the shortest path between pairs of nodes to the second network graph.
In another possible implementation manner, the determining the importance of each first node in the first network graph to be compressed includes:
for each first node, determining the centrality of the first node in the first network graph, and taking the centrality of the first node as the importance of the first node.
In another possible implementation manner, the determining the centrality of each first node in the first network graph includes:
determining a total number of nodes included in the first network graph and a degree of each first node in the first network graph;
for each first node, determining the centrality of the first node according to the degree of the first node and the total number, or determining the centrality of the first node according to the degree of the first node, the degree of a neighbor node of the first node and the total number.
In another possible implementation manner, before adding, in the second network graph, distance information of the node pair in the first network graph according to the first distance of the node pair in the first network graph, the second distance of the node pair in the second network graph, and the graph parameter, the method further includes:
determining an execution order of each node pair according to a first distance of each node pair in the first network graph;
according to the execution sequence, the step of adding distance information of the node pairs in the first network graph in the second network graph according to the first distance of the node pairs in the first network graph, the second distance of the node pairs in the second network graph and the graph parameters is executed.
In another aspect, an apparatus for compressing image data is provided, the apparatus comprising:
a first determination module configured to determine an importance of each first node in a first network graph to be compressed;
a selection module configured to select a plurality of second nodes from the first network graph according to the importance of each first node;
a combining module configured to combine the plurality of second nodes into a second network graph and to combine two different second nodes of the plurality of second nodes into a node pair, resulting in at least one node pair;
a second determination module configured to determine a first distance of each node pair in the first network graph;
a third determination module configured to determine, for each node pair, a second distance of the node pair in the second network graph, add distance information of the node pair in the first network graph in the second network graph according to the first distance of the node pair in the first network graph, the second distance of the node pair in the second network graph, and a graph parameter, a size of the graph parameter being inversely related to a number of nodes retained in the second network graph.
In a possible implementation, the third determining module is further configured to determine a product of the first distance of the node pair in the first network graph and the graph parameter, resulting in a third distance;
when the second distance of the node pair in the second network graph is larger than the third distance, adding nodes and edges included in a shortest path between the node pair in the first network graph into the second network graph, wherein the shortest path is a path corresponding to the first distance.
In another possible implementation manner, the second determining module is further configured to determine, when the first network graph is a non-weight graph, a shortest distance of the node pair in the first network graph, and use the shortest distance as the first distance of the node pair in the first network graph; when the first network graph is a weight graph, determining the weighted sum of each path of the node pair in the first network graph, and taking the smallest weighted sum of the weighted sums of each path as the first distance of the node pair in the first network graph.
In another possible implementation, the third determining module is further configured to determine a fourth distance of the shortest path between the node pairs when a path between the node pairs exists in the second network graph; taking a fourth distance of the shortest path as a second distance of the node pair in the second network graph.
In another possible implementation, the third determining module is further configured to determine, when a path between a node pair exists in the second network graph, a product of a first distance of the node pair in the first network graph and the graph parameter to obtain a third distance; adding nodes and edges in a shortest path between pairs of nodes to the second network graph when no path between pairs of nodes exists in the second network graph.
In another possible implementation manner, the first determining module is further configured to determine, for each first node, a centrality of the first node in the first network graph, and use the centrality of the first node as an importance of the first node.
In another possible implementation, the first determining module is further configured to determine a total number of nodes included in the first network graph and a degree of each first node in the first network graph; for each first node, determining the centrality of the first node according to the degree of the first node and the total number, or determining the centrality of the first node according to the degree of the first node, the degree of a neighbor node of the first node and the total number.
In another possible implementation manner, the apparatus further includes:
a fourth determination module configured to determine an order of execution for each node pair based on the first distance of each node pair in the first network graph;
the third determination module is configured to add distance information of the node pairs in the first network graph in the second network graph according to the execution order, the first distance of the node pairs in the first network graph, the second distance of the node pairs in the second network graph, and a graph parameter.
In another aspect, a server is provided, which includes a processor and a memory, where at least one instruction, at least one program, a set of codes, or a set of instructions is stored in the memory, and the instruction, the program, the set of codes, or the set of instructions is loaded and executed by the processor to implement the operations performed in the graph data compression method in the embodiment of the present invention.
In another aspect, a computer-readable storage medium is provided, in which at least one instruction, at least one program, a set of codes, or a set of instructions is stored, which is loaded and executed by a processor to implement the operations as performed in the data compression method in the embodiment of the present invention.
The technical scheme provided by the embodiment of the invention has the following beneficial effects:
in the embodiment of the invention, the importance of each first node in the first network graph to be compressed is determined; selecting a plurality of second nodes from the first network graph according to the importance of each first node; forming a second network graph by the plurality of second nodes, and forming a node pair by two different second nodes in the plurality of second nodes to obtain at least one node pair; determining a first distance of each node pair in the first network graph; and for each node pair, determining a second distance of the node pair in the second network graph, and adding distance information of the node pair in the first network graph in the second network graph according to the first distance of the node pair in the first network graph, the second distance of the node pair in the second network graph and the graph parameters, thereby obtaining the compressed second network graph. Adding distance information between node pairs between second nodes through the first distance and the second distance in combination with the graph parameters, and keeping the distance information in the first network graph as much as possible; and because the graph parameters are set, the number of nodes reserved in the second network graph can be limited through the graph parameters, so that excessive nodes and edges are not increased, graph data are effectively compressed, and the obtained second network graph is more accurate.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings needed to be used in the description of the embodiments will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.
FIG. 1 is a system architecture diagram illustrating data compression provided by an embodiment of the present invention;
FIG. 2 is a flow chart of a method of graph data compression provided in accordance with an exemplary embodiment;
FIG. 3 is a flow chart of another method of graph data compression provided in accordance with an exemplary embodiment;
FIG. 4 is a schematic diagram of a first network diagram provided in accordance with an example embodiment;
FIG. 5 is a flow chart of another method of graph data compression provided in accordance with an exemplary embodiment;
FIG. 6 is a schematic diagram of a first network diagram and a second network diagram provided in accordance with an example embodiment;
FIG. 7 is a comparison diagram of graph data compression provided in accordance with an exemplary embodiment;
FIG. 8 is a block diagram of a graph data compression apparatus provided in accordance with an exemplary embodiment;
fig. 9 is a schematic diagram of a server according to an exemplary embodiment.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, embodiments of the present invention will be described in detail with reference to the accompanying drawings.
Reference will now be made in detail to the exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, like numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the exemplary embodiments below are not intended to represent all implementations consistent with the present disclosure. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present disclosure, as detailed in the appended claims.
The embodiment of the invention provides a method for compressing graph data, wherein the graph data refers to data stored in a graph data structure. In the structure of graph data, a network graph is generally used to store data and a relationship between data. The network graph comprises a plurality of nodes, each node is used for storing data, and edges among the nodes are used for identifying the relation among the data. Compression of graph data can be divided into lossy compression and lossless compression. For convenience of description, the network map to be compressed is referred to as a first network map, and the network map obtained after compression is referred to as a second network map. Lossless compression refers to the way the first network graph itself is compressed. Lossy compression refers to a compression scheme that preserves some of the nodes and edges in the first network graph. The graph data compression provided by the embodiment of the invention is lossy compression.
Fig. 1 is a system architecture diagram for graph data compression according to an embodiment of the present invention, where the system architecture diagram includes an important node extraction module and a graph generation module. The output end of the important node extraction module is connected with the input end of the generation module. The important node extraction module is used for acquiring the first network graph to be compressed, determining the importance of each first node in the first network graph, selecting a plurality of second nodes from the first network graph according to the importance of each first node, and outputting the plurality of second nodes to the graph generation module. And the graph generating module is used for receiving the plurality of second nodes, forming the plurality of second nodes into a second network graph, and adding the distance information of the node pair in the first network graph in the second network graph by using a greedy algorithm K-spanner calculation.
Wherein the distance information may include an edge on a shortest path of the pair of nodes; a third node on the shortest path of the node pair and an edge between the third node may also be included.
And when the distance information only comprises the edge on the shortest path of the node pair, generating a graph module, which is used for adding the edge on the shortest path between second nodes in the second network graph so as to update the second network graph and obtain a final compressed second network graph.
When the distance information further includes a third node on the shortest path of the node pair and an edge between the third nodes, the graph generation module is further configured to calculate the third node on the shortest path, the edge between the third nodes, and the edge between the second node and the third node using the K-spanner, so that the determined third node, the edge between the third nodes, and the edge between the second node and the third node are added to the second network graph, thereby updating the second network graph, and obtaining the finally compressed second network graph.
The graph data compression method provided by the embodiment of the invention can be applied to the fields of data storage, internet analysis, social network analysis and recommended network analysis. When the graph data compression method is applied to the field of data storage, the first network graph can be compressed to obtain a second network graph when the first network graph is stored, and the second network graph is stored, so that the storage space is saved.
When the graph data compression method is applied to the field of application internet analysis, application internet data are stored in the first network graph. When the first network diagram is subjected to the application internet analysis, the first network diagram is compressed to obtain a second network diagram, and the second network diagram is subjected to the application internet analysis, so that the calculation amount is saved, and the analysis efficiency is improved.
When the graph data compression method is applied to the field of social network analysis, social network data are stored in the first network graph. When the social network analysis is performed on the first network graph, the first network graph is compressed to obtain a second network graph, and the social network analysis is performed on the second network graph, so that the calculation amount is saved, and the analysis efficiency is improved.
When the graph data compression method is applied to the field of recommended network analysis, reference data are stored in the first network graph. When the recommendation network analysis is performed on the first network graph, the first network graph is compressed to obtain a second network graph, network data to be recommended are obtained according to the data in the second network graph, and the network recommendation to be recommended is displayed, so that the calculation amount is saved, and the recommendation efficiency is improved.
It should be noted that the graph data compression method can be applied to a terminal and can also be applied to a server; in the embodiment of the present invention, the data compression method is applied to a server as an example. When the graph data compression method is applied to a terminal, a server can be directly modified into the terminal, and the processing logics of the server are the same, which is not described again in the embodiment of the invention.
Fig. 2 is a flowchart of a graph data compression method according to an exemplary embodiment. As shown in fig. 2, the present invention provides a method for compressing graph data, which can be applied in a server. Comprises the following steps.
In step 201, the importance of each first node in the first network graph to be compressed is determined.
At step 204, a first distance of each node pair in the first network graph is determined.
In the embodiment of the invention, the importance of each first node in the first network graph to be compressed is determined; selecting a plurality of second nodes from the first network graph according to the importance of each first node; forming a second network graph by the plurality of second nodes, and forming a node pair by two different second nodes in the plurality of second nodes to obtain at least one node pair; determining a first distance of each node pair in the first network graph; for each node pair, determining a second distance of the node pair in a second network graph, and adding distance information between the node pairs between the second nodes according to a first distance of the node pair in the first network graph, a second distance of the node pair in the second network graph and graph parameters, so that the distance information in the first network graph is kept as much as possible; and because the graph parameters are set, the number of nodes reserved in the second network graph can be limited through the graph parameters, so that excessive nodes and edges are not increased, graph data are effectively compressed, and the obtained second network graph is more accurate.
Fig. 3 is a flow chart of another graph data compression method provided in accordance with an exemplary embodiment, as shown in fig. 3, the method including the steps of:
in step 301, the server determines the importance of each first node in the first network graph to be compressed.
The first network graph to be compressed may be a network graph having a plurality of nodes and a plurality of edges, and reflects a connection relationship between the nodes. The server may determine the importance of each first node in the first network graph by different importance indicators, for example, the importance of a node may be characterized by the centrality of the node.
When the importance of the node is characterized by using the centrality of the node, for each first node in the first network graph, the server determines the centrality of the first node in the first network graph, and takes the centrality of the first node as the importance of the first node.
The centrality of the first node may be at least one of a degree centrality of the first node, a near centrality of the first node, an intermediary centrality of the first node, a feature value centrality of the first node, or a markov centrality of the first node, etc.
In the embodiments of the present invention, the above centrality is taken as an example of centrality. In this step, the server may determine the centrality of the first node according to the degree of the first node itself, that is, the following first implementation manner; the server may also determine the centrality of the first node according to the degree of the first node itself and in combination with the degrees of the neighboring nodes of the first node, that is, the following second implementation manner.
For the first implementation, the step of the server determining the centrality of each first node in the first network graph may be:
the server determines the total number of nodes included in the first network graph and the degree of the first node in the first network graph, and determines the centrality of the first node, namely the degree centrality, according to the degree of the first node and the total number of the nodes for each first node.
The degree of the first node may be a number of nodes directly connected to the first node in the first network graph. Accordingly, for each first node, the step of the server determining the degree of the first node may be: the server determines a number of nodes directly connected to the first node in the first network graph, and takes the number as the degree of the first node.
The centrality of the first node is negatively related to the first quantity and positively related to the degree of the first node. The server may determine the centrality of the first node by any algorithm that negatively correlates with the first number and positively correlates with the degree of the first node. For example, the server determines the centrality of the first node based on the degree of the first node and the first number of nodes by the following formula one:
the formula I is as follows:
wherein v represents the identity of the first node, c (v) represents the centrality of the first node, deg (v) represents the degree of the first node, and N represents the first number of nodes in the first network graph.
For example, referring to fig. 4, fig. 4 is a schematic diagram of a first network diagram provided in accordance with an example embodiment. The figure has 10 nodes, which are nodes A-J. The server determines the centrality of each first node as c (a) ═ c (b) ═ c (c) ((d) ((f) ((5/9)), c (e) ((6/9), c (g) ((4/9), and c (h) ((i) ((c) ((j) ((1/9), according to the above formula one.
For the second implementation, the step of the server determining the centrality of each first node in the first network graph may be:
the server determining a total number of nodes included in the first network graph and a degree of each node in the first network graph; for each first node, determining the centrality of the first node according to the degree of the first node, the degrees of the neighbor nodes of the first node and the total number.
The step of the server determining the centrality of the first node according to the degree of the first node, the degrees of the neighbor nodes of the first node and the total number may be: the server determines the product of the degree of the first node and the selection parameter to obtain a first numerical value; counting the number of nodes with the intermediate degree of the neighbor nodes being larger than the first numerical value; and determining the centrality of the first node according to the number and the total number of the nodes. For convenience of description, the total number is referred to as a first number, and the number of nodes is referred to as a second number.
Wherein the centrality of the first node is negatively related to the first number and positively related to the second number. The server may determine the centrality of the first node by any algorithm that negatively correlates with the first number and positively correlates with the second number. For example, the server determines the centrality of the first node based on the degrees of the first node, the degrees of the first node's neighbor nodes, and the total number by the following formula two:
the formula II is as follows:
wherein v represents the identity of the first node, u represents the identity of the neighbor nodes of the first node, c (v) represents the centrality of the first node, n (v) represents the set of neighbor nodes of the first node, u ∈ n (v) represents the neighbor nodes belong to the set of nodes n (v), deg (u) represents the degree of the node u, β is a selection parameter, and β is a positive number greater than zero.
In the embodiment of the invention, when the degree centrality is used for selecting the important node, a selection parameter beta can be added, and for any first node, the centrality of the first node can be determined through the relation between the degrees of the neighbor nodes of the first node and the product of the degrees of the first node and the beta, so that the selected important node is prevented from being too concentrated.
For example, with continued reference to fig. 4, when β is 0.8, the server determines, by the above formula two, that c (g) ═ 3/9, c (e) ═ 1/9, c (a) ═ c (b) ═ c (c) (d) ((f) ═ c (h) ═ c (i) ((j)) 0.
It should be noted that, when the above-mentioned centrality is near centrality, the server may determine the centrality of each first node in the following two ways. For the first implementation, the step of the server determining the centrality of each first node in the first network graph may be:
for each first node, the server determines the shortest distance between the first node and each of the other nodes; determining the sum of the shortest distances between the first node and each other node to obtain a second numerical value; the centrality of the first node is determined from the second value.
Wherein the centrality of the first node is inversely related to the second value. The server may determine the centrality of the first node by any algorithm that is inversely related to the second value; for example, the server determines the centrality of the first node through formula three;
the formula III is as follows:
wherein V represents the identifier of the first node, w represents the identifiers of the other nodes, c (V) represents the centrality of the first node, d (w, V) represents the shortest distance between the first node and the other nodes, and V represents the set of other nodes.
For the second implementation, the step of the server determining the centrality of each first node in the first network graph may be:
for each first node, the server determines a shortest distance between the first node and each of the other nodes and a total number of nodes in the first network graph; determining the sum of the shortest distances between the first node and each other node to obtain a second numerical value; and determining the centrality of the first node according to the second value and the total number.
Wherein the centrality of the first node is negatively correlated with the second value and positively correlated with the total number. The server may determine the centrality of the first node by any algorithm that is negatively correlated with the second value, positively correlated with the total number; for example, the server determines the centrality of the first node by equation four.
The formula four is as follows:
wherein V represents the identity of the first node, w represents the identities of the other nodes, c (V) represents the centrality of the first node, d (w, V) represents the shortest distance between the first node and the other nodes, V represents the set of other nodes, and N represents the total number.
It should be further noted that, when the centrality is an intermediary centrality, the step of the server determining the centrality of each first node in the first network graph may be:
for each first node, the server determines at least one node pair of nodes consisting of nodes other than the first node. For each node pair, the server determines a third number of paths between the node pair and a fourth number of paths for the node pair through the first node. And determining the centrality of the first node according to the third number and the fourth number of each node pair. The path between the pair of nodes may be the shortest path between the pair of nodes.
Wherein the centrality of the first node is negatively correlated with the third quantity and positively correlated with the fourth quantity. The server may determine the centrality of the first node by any algorithm that is negatively correlated with the third quantity and positively correlated with the fourth quantity; for example, the server determines the centrality of the first node by equation five.
The formula five is as follows:
wherein V represents the identifier of the first node, s and t represent the unique identifiers of other nodes, c (V) represents the centrality of the first node, V represents the set of all nodes in the first network graph, σ (s, t) represents the third number of shortest paths from the node s to the node t, and σ (s, t | V) represents the fourth number of shortest paths from the node s to the node t passing through the first node.
It should be noted that, when the centrality of the first node includes a plurality of centralities, which may be a degree centrality of the first node, a proximity centrality of the first node, an intermediary centrality of the first node, a feature value centrality of the first node, or a markov centrality of the first node, the server may perform a weighted summation of the plurality of centralities to obtain the centrality of the first node.
In step 302, the server selects a plurality of second nodes from the first network graph according to the importance of each first node.
The server may select a target number of second nodes with the highest importance according to the importance of each first node, to obtain a plurality of second nodes.
When the importance is represented by the centrality of the nodes, the server may select a target number of nodes with higher centrality as the second nodes, that is, the important nodes, according to the centrality of each first node. The target number may be a positive integer greater than 1, such as 3, 10, 25, and 40, etc. The target number can be set and changed according to the requirement; for example, the target number may be a fixed number, or may be set according to the total number of nodes in the first network map.
It should be noted that the server may also select, according to the importance of each first node, a plurality of second nodes whose importance exceeds a preset threshold from the first network graph, or the server may also sort each first node according to the importance of each first node and select the target proportion second nodes.
For example, when c (a), (b), (c), (d), (f), (5/9), (e), (6/9), (g), (4/9), (h), (c), (i), (j), (1/9). With a target number of 2, the server selects node E and optionally one of A, B, C, D and F.
For example, when c (g) ═ 3/9, c (e) ═ 1/9, c (a) ═ c (b) ═ c (c) ((d) ((f) ((h) ((i)) ((j) () 0. With a target number of 2, the server selects nodes E and G.
The server makes up a plurality of second nodes into a second network graph, that is, the plurality of second nodes can be regarded as initial nodes of the second network graph, and the second network graph is a graph obtained after the first network graph is compressed, but the subsequent steps can also update the topological structure in the second network graph, that is, the current second network graph is not a graph obtained by final compression.
After the server selects the plurality of second nodes, any two different second nodes in the plurality of second nodes can form a node pair; it is also possible to group each second node with other nodes into node pairs and then keep only one pair for the same node pair. When the number of the second nodes is two, one node pair is obtained, and when the number of the second nodes is more than two, a plurality of node pairs are obtained, for example, three second nodes may form three node pairs, and four second nodes may form six node pairs.
At step 304, the server determines a first distance for each node pair in the first network graph.
For each node pair, the server may determine a first distance between each node pair according to the type of the first network graph, determine a shortest distance of the node pair in the first network graph when the first network graph is a non-weight graph, and take the shortest distance as the first distance of the node pair in the first network graph; when the first network graph is a weight graph, the server determines the weighted sum of each path of the node pair in the first network graph, the node pair has a plurality of paths in the first network graph, each path comprises at least one edge with weight, and the first distance of the node pair in the first network graph is the smallest weighted sum of the weight sum of each path.
It should be noted that the path between the node pairs may include not only the edge but also the node on the path.
The server may sort each node pair from small to large according to a first distance of each node pair in the first network graph, and obtain an execution order of each node pair.
The server may assign a unique identifier, such as an ID (Identification), a Name, or a Number, to each node pair in sequence from small to large according to a first distance of each node pair in the first network diagram, where the ID may be represented as "0, 1, 2, 3" or "i, ii, iii," and the like, and the server represents an execution order of each node pair according to an order of the unique identifiers.
After determining the execution order for each node pair, the server may perform step 306 according to the execution order.
In step 306, the server determines, for each node pair, a second distance of the node pair in the second network graph, and adds distance information of the node pair in the first network graph to the second network graph according to the first distance of the node pair in the first network graph, the second distance of the node pair in the second network graph, and the graph parameter.
This step can be realized by the following steps (1) to (2).
(1) The server determines a second distance of the node pair in the second network graph, the second distance being the distance of the shortest path of the node pair in the second network graph.
Correspondingly, before determining the second distance, the server needs to determine whether a path between the node pair exists in the second network, when the path between the node pair exists in the second network graph, a fourth distance of the shortest path between the node pair is determined, and the fourth distance of the shortest path is used as the second distance of the node pair in the second network graph; when the path between the node pair does not exist in the second network graph, a second distance of the node pair in the second network graph is set to infinity.
(2) After determining the second distance of the node pair in the second network graph, the server adds the distance information of the node pair in the first network graph in the second network graph according to the first distance of the node pair in the first network graph, the second distance of the node pair in the second network graph and the graph parameter.
When a path between a node pair exists in the second network graph, the server determines a product of a first distance of the node pair in the first network graph and the graph parameter to obtain a third distance. When the second distance of the node pair in the second network graph is greater than the third distance, adding nodes and edges included in a shortest path between the node pair in the first network graph into the second network graph, wherein the shortest path is a path corresponding to the first distance; when the second distance of the node pair in the second network graph is not greater than the third distance, the distance information of the node pair in the first network graph is not added in the second network graph, and the original topological relation of the second network graph is kept unchanged.
When a path between a node pair does not exist in the second network graph, at which point the second distance of the node pair in the second network graph is infinite, the nodes and edges in the shortest path between the node pair are added to the second network graph.
The size of the map parameter may be determined according to actual conditions, and the map parameter may be represented by K, where K is any positive number not less than 1. The size of the graph parameter is inversely related to the number of nodes reserved in the second network graph, i.e. the larger the value of the graph parameter, the smaller the number of nodes reserved by the corresponding additional nodes other than the second node; as the value of the graph parameter is smaller, the number of the corresponding additional nodes other than the second node is reserved more. Through the adjustment of the K value, the retention degree of additional nodes between the second nodes can be customized, and all shortest path nodes do not need to be retained, so that the customizability is strong, the flexibility is high, the application range is wider, and the compression result is controllable.
It should be further noted that, the server repeatedly executes the above step (1) and step (2) from the node pair with the smallest first distance, updates the topological relation between the node pair in the second network graph after adding the distance information of the node pair in the first network graph in the second network graph, and acquires the updated second distance when the next node acquires the second distance in the second network graph until all the node pairs are executed. For example, referring to fig. 5, fig. 5 is a flow chart of another graph data compression method provided according to an example embodiment.
Fig. 6 is a schematic diagram of a first network diagram and a second network diagram provided in accordance with an example embodiment. In the first network graph, A, B, C, D four significant nodes are identified, resulting in six node pairs of (A, B), (A, C), (A, D), (B, C), (B, D), and (C, D). The connections between the four points in the first network diagram are shown in the left diagram of fig. 6, and the first network diagram is a weight diagram, and the values on each side in the left diagram are the weight of the side. According to the content of step 304, the sum of the weights of the edges at the shortest distance of each node pair is used as the first distance between each node pair. And according to the distances between the node pairs, sequencing the six node pairs from small to large: { (A, C): 3, (B, D): 3, (a, B): 4, (C, D): 4, (a, D): 5, (B, C): and 5, setting the graph parameter to be K-2, starting from the node pair (A, C) with the shortest distance, namely the first distance, the shortest distance, wherein the node pair (A, C) is not connected in the second network graph, so that the first distance between the node pair (A, C) which is larger than K times is larger than the first distance between the node pair (A, C), and the edge between the node A and the node C is added in the second network graph. For the second node pair (B, D), since (B, D) is not connected in the second network graph, the first distance between the node pair (B, D) that is greater than K times is increased in the second network graph by the edge between node B and node D. For the third node pair (a, B), (a, B) is not connected in the second network graph, and thus is greater than K times the first distance between the node pair (a, B), increasing the edge between node a and node B in the second network graph. For the fourth node pair (C, D), (C, D) is connected in the second network graph, the distance in the second network graph is 3+4+3 ═ 10, (C, D) the distance in the first network graph is 4, 10>2 × 4, i.e. (C, D) the distance in the second network graph is greater than K times the distance in the first network graph, the edges between node C and node D are added in the second network graph. For the fifth node pair (a, D), the distance of (a, D) in the second network graph is 3+4 — 7, and the distance of (a, D) in the first network graph is 5, 7<2 × 5, so no edge in the first network graph is added between node a and node D. For the sixth node pair (B, C), the distance of (B, C) in the second network graph is 3+4 — 7, and the distance of (B, C) in the first network graph is 5, 7<2 × 5, so no edge in the first network graph is added between node B and node C. And after the six node pairs are compressed, obtaining a second network graph as shown in the right graph in fig. 6.
In the embodiment of the invention, the importance of each first node in the first network graph to be compressed is determined; selecting a plurality of second nodes from the first network graph according to the importance of each first node; forming a second network graph by the plurality of second nodes, and forming a node pair by two different second nodes in the plurality of second nodes to obtain at least one node pair; determining a first distance of each node pair in the first network graph; for each node pair, determining a second distance of the node pair in a second network graph, and adding distance information between the node pairs between the second nodes according to a first distance of the node pair in the first network graph, a second distance of the node pair in the second network graph and graph parameters, so that the distance information in the first network graph is kept as much as possible; and because the graph parameters are set, the number of nodes reserved in the second network graph can be limited through the graph parameters, so that excessive nodes and edges are not increased, graph data are effectively compressed, and the obtained second network graph is more accurate.
FIG. 7 is a comparison diagram of graph data compression provided in accordance with an exemplary embodiment. Referring to fig. 7, three different results of a second network graph obtained by compressing the same first network graph through three different algorithms are exemplarily listed, where the left graph is the second network graph obtained by compressing the first network graph through the keepal algorithm, the middle graph is the second network graph obtained by compressing the first network graph through the KeepOne algorithm, and the right graph is the second network graph obtained by compressing the first network graph through the graph data compression method provided by the present invention. As can be seen from fig. 7, compared to keepal and KeepOne, the method proposed by the present invention is a trade-off between keepal and KeepOne, i.e. distance information between important nodes is preserved, and excessive redundant edges and nodes are not added.
FIG. 8 is a block diagram of a graph data compression apparatus provided in accordance with an exemplary embodiment. The apparatus is used for executing the steps executed by the graph data compression method, and referring to fig. 8, the apparatus comprises: a first determination module 801, a selection module 802, a combination module 803, a second determination module 804, and a third determination module 805.
A first determining module 801 configured to determine the importance of each first node in the first network graph to be compressed;
a selection module 802 configured to select a plurality of second nodes from the first network graph according to the importance of each first node;
a combining module 803 configured to combine the plurality of second nodes into a second network graph and to combine two different second nodes of the plurality of second nodes into a node pair, resulting in at least one node pair;
a second determining module 804 configured to determine a first distance of each node pair in the first network graph;
a third determining module 805 configured to determine, for each node pair, a second distance of the node pair in the second network graph, add distance information of the node pair in the first network graph in the second network graph according to the first distance of the node pair in the first network graph, the second distance of the node pair in the second network graph, and a graph parameter, a size of the graph parameter being inversely related to a number of nodes reserved in the second network graph.
In one possible implementation, the third determining module 805 is further configured to determine a product of the first distance of the node pair in the first network graph and the graph parameter, resulting in a third distance;
and when the second distance of the node pair in the second network graph is greater than the third distance, adding nodes and edges included in the shortest path between the node pair in the first network graph into the second network graph, wherein the shortest path is a path corresponding to the first distance.
In another possible implementation, the second determining module 804 is further configured to determine, when the first network graph is a non-weight graph, a shortest distance of the node pair in the first network graph, and use the shortest distance as the first distance of the node pair in the first network graph; when the first network graph is the weight graph, determining the weighted sum of each path of the node pair in the first network graph, and taking the smallest weighted sum of the weighted sums of each path as the first distance of the node pair in the first network graph.
In another possible implementation, the third determining module 805 is further configured to determine, when a path between a pair of nodes exists in the second network graph, a fourth distance of the shortest path between the pair of nodes; the fourth distance of the shortest path is taken as the second distance of the node pair in the second network graph.
In another possible implementation, the third determining module 805 is further configured to determine, when a path between a node pair exists in the second network graph, a product of the first distance of the node pair in the first network graph and the graph parameter to obtain a third distance; when no path between node pairs exists in the second network graph, nodes and edges in the shortest path between node pairs are added to the second network graph.
In another possible implementation manner, the first determining module 801 is further configured to determine, for each first node, a centrality of the first node in the first network graph, and use the centrality of the first node as the importance of the first node.
In another possible implementation, the first determining module 801 is further configured to determine a total number of nodes included in the first network graph and a degree of each first node in the first network graph; for each first node, determining the centrality of the first node according to the degree of the first node and the total number, or determining the centrality of the first node according to the degree of the first node, the degree of a neighbor node of the first node and the total number.
In another possible implementation manner, the apparatus further includes:
a fourth determination module configured to determine an execution order of each node pair according to the first distance of each node pair in the first network graph;
a third determination module 805 configured to add distance information of node pairs in the first network graph in the second network graph according to the execution order, the first distance of the node pairs in the first network graph, the second distance of the node pairs in the second network graph, and the graph parameters.
In the embodiment of the invention, the importance of each first node in the first network graph to be compressed is determined; selecting a plurality of second nodes from the first network graph according to the importance of each first node; forming a second network graph by the plurality of second nodes, and forming a node pair by two different second nodes in the plurality of second nodes to obtain at least one node pair; determining a first distance of each node pair in the first network graph; and for each node pair, determining a second distance of the node pair in the second network graph, and adding distance information of the node pair in the first network graph in the second network graph according to the first distance of the node pair in the first network graph, the second distance of the node pair in the second network graph and the graph parameters, thereby obtaining the compressed second network graph. Adding distance information between node pairs between second nodes through the first distance and the second distance in combination with the graph parameters, and keeping the distance information in the first network graph as much as possible; and because the graph parameters are set, the number of nodes reserved in the second network graph can be limited through the graph parameters, so that excessive nodes and edges are not increased, graph data are effectively compressed, and the obtained second network graph is more accurate.
It should be noted that: in the above embodiment, when the graph data compression apparatus runs an application program, only the division of the functional modules is described as an example, and in practical applications, the function distribution may be completed by different functional modules according to needs, that is, the internal structure of the apparatus is divided into different functional modules to complete all or part of the functions described above. In addition, the graph data compression apparatus provided in the above embodiment and the graph data compression method embodiment belong to the same concept, and specific implementation processes thereof are detailed in the method embodiment and are not described herein again.
Fig. 9 is a schematic structural diagram of a server according to an embodiment of the present invention, where the server 900 may generate a relatively large difference due to different configurations or performances, and may include one or more processors (CPUs) 901 and one or more memories 902, where the memory 902 stores at least one instruction, and the at least one instruction is loaded and executed by the processors 901 to implement the methods provided by the foregoing method embodiments. Of course, the server may also have components such as a wired or wireless network interface, a keyboard, and an input/output interface, so as to perform input/output, and the server may also include other components for implementing the functions of the device, which are not described herein again.
The embodiment of the present invention also provides a computer-readable storage medium, which is applied to a server, and the computer-readable storage medium stores at least one instruction, at least one program, a set of codes, or a set of instructions, where the instruction, the program, the set of codes, or the set of instructions is loaded and executed by a processor to implement the operations performed by the server in the graph data compression method according to the foregoing embodiment.
It will be understood by those skilled in the art that all or part of the steps for implementing the above embodiments may be implemented by hardware, or may be implemented by a program instructing relevant hardware, where the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a magnetic disk or an optical disk, etc.
The above description is only for the purpose of illustrating the preferred embodiments of the present invention and is not to be construed as limiting the invention, and any modifications, equivalents, improvements and the like that fall within the spirit and principle of the present invention are intended to be included therein.
Claims (10)
1. A method for compressing graph data, the method comprising:
determining the importance of each first node in a first network graph to be compressed;
selecting a plurality of second nodes from the first network graph according to the importance of each first node;
forming a second network graph by the plurality of second nodes, and forming a node pair by two different second nodes in the plurality of second nodes to obtain at least one node pair;
determining a first distance of each node pair in the first network graph;
for each node pair, determining a second distance of the node pair in the second network graph;
determining a product of a first distance of the node pair in the first network graph and a graph parameter to obtain a third distance;
when the second distance of the node pair in the second network graph is larger than the third distance, adding nodes and edges included in a shortest path between the node pair in the first network graph into the second network graph, wherein the shortest path is a path corresponding to the first distance.
2. The method of claim 1, wherein determining a first distance for each node pair in the first network graph comprises:
when the first network graph is a non-weight graph, determining the shortest distance of the node pair in the first network graph, and taking the shortest distance as the first distance of the node pair in the first network graph;
when the first network graph is a weight graph, determining the weighted sum of each path of the node pair in the first network graph, and taking the smallest weighted sum of the weighted sums of each path as the first distance of the node pair in the first network graph.
3. The method of claim 1, wherein determining the second distance of the node pair in the second network graph comprises:
determining a fourth distance of a shortest path between the pair of nodes when a path between the pair of nodes exists in the second network graph;
taking a fourth distance of the shortest path as a second distance of the node pair in the second network graph.
4. The method of claim 1, wherein before determining a product of a first distance of the node pair in the first network graph and the graph parameter, resulting in a third distance, the method further comprises:
when a path between a node pair exists in the second network graph, performing the step of determining a product of a first distance of the node pair in the first network graph and a graph parameter to obtain a third distance;
when no path between node pairs exists in the second network graph, performing the step of adding nodes and edges included in the shortest path between the node pairs in the first network graph to the second network graph.
5. The method of claim 1, wherein determining the importance of each first node in the first network graph to be compressed comprises:
for each first node, determining the centrality of the first node in the first network graph, and taking the centrality of the first node as the importance of the first node.
6. The method of claim 5, wherein determining the centrality of each first node in the first network graph comprises:
determining a total number of nodes included in the first network graph and a degree of each first node in the first network graph;
for each first node, determining the centrality of the first node according to the degree of the first node and the total number, or determining the centrality of the first node according to the degree of the first node, the degree of a neighbor node of the first node and the total number.
7. The method of claim 1, further comprising:
determining an execution order of each node pair according to a first distance of each node pair in the first network graph;
according to the execution sequence, executing the step of determining the product of the first distance of the node pair in the first network graph and the graph parameter to obtain a third distance; a step of adding, when a second distance of the node pair in the second network graph is greater than the third distance, a node and an edge included in a shortest path between the node pair in the first network graph to the second network graph.
8. An apparatus for compressing image data, the apparatus comprising:
a first determination module configured to determine an importance of each first node in a first network graph to be compressed;
a selection module configured to select a plurality of second nodes from the first network graph according to the importance of each first node;
a combining module configured to combine the plurality of second nodes into a second network graph and to combine two different second nodes of the plurality of second nodes into a node pair, resulting in at least one node pair;
a second determination module configured to determine a first distance of each node pair in the first network graph;
a third determination module configured to determine, for each node pair, a second distance of the node pair in the second network graph; determining a product of a first distance of the node pair in the first network graph and a graph parameter to obtain a third distance; when the second distance of the node pair in the second network graph is larger than the third distance, adding nodes and edges included in a shortest path between the node pair in the first network graph into the second network graph, wherein the shortest path is a path corresponding to the first distance.
9. A server, comprising a processor and a memory, wherein the memory has stored therein at least one instruction, at least one program, set of codes, or set of instructions, which is loaded and executed by the processor to implement the operations performed in the graph data compression method according to any one of claims 1 to 7.
10. A computer-readable storage medium for storing at least one program for executing the graph data compression method according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910446525.0A CN110149234B (en) | 2019-05-27 | 2019-05-27 | Graph data compression method, device, server and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910446525.0A CN110149234B (en) | 2019-05-27 | 2019-05-27 | Graph data compression method, device, server and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110149234A CN110149234A (en) | 2019-08-20 |
| CN110149234B true CN110149234B (en) | 2021-10-08 |
Family
ID=67592000
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910446525.0A Active CN110149234B (en) | 2019-05-27 | 2019-05-27 | Graph data compression method, device, server and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110149234B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111046002B (en) * | 2019-12-06 | 2022-08-02 | 浪潮(北京)电子信息产业有限公司 | Data compression method, decompression method, system and related device of graph data |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105530011A (en) * | 2014-09-30 | 2016-04-27 | 华东师范大学 | Graph data compression method and query method based on triangular statistics |
| CN106301868A (en) * | 2015-06-12 | 2017-01-04 | 华为技术有限公司 | The method and apparatus determining the importance of network node |
| CN107564075A (en) * | 2017-09-29 | 2018-01-09 | 桂林电子科技大学 | A kind of dense graph adjacency list compression method |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2016194224A1 (en) * | 2015-06-05 | 2016-12-08 | 株式会社日立製作所 | Information processing apparatus and graph data writing device |
-
2019
- 2019-05-27 CN CN201910446525.0A patent/CN110149234B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105530011A (en) * | 2014-09-30 | 2016-04-27 | 华东师范大学 | Graph data compression method and query method based on triangular statistics |
| CN106301868A (en) * | 2015-06-12 | 2017-01-04 | 华为技术有限公司 | The method and apparatus determining the importance of network node |
| CN107564075A (en) * | 2017-09-29 | 2018-01-09 | 桂林电子科技大学 | A kind of dense graph adjacency list compression method |
Non-Patent Citations (1)
| Title |
|---|
| 图数据表示与压缩技术综述;张宇等;《软件学报》;20140915;第1937-1952页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110149234A (en) | 2019-08-20 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11349824B2 (en) | Block sequencing method and system based on tree-graph structure, and data processing terminal | |
| CN110719106B (en) | Social network graph compression method and system based on node classification and sorting | |
| CN112667860A (en) | Sub-graph matching method, device, equipment and storage medium | |
| US11074274B2 (en) | Large scale social graph segmentation | |
| CN112100450A (en) | Graph calculation data segmentation method, terminal device and storage medium | |
| CN110097581B (en) | Method for constructing K-D tree based on point cloud registration ICP algorithm | |
| CN110149234B (en) | Graph data compression method, device, server and storage medium | |
| CN111159577A (en) | Community division method and device, storage medium and electronic device | |
| CN108614932B (en) | Edge graph-based linear flow overlapping community discovery method, system and storage medium | |
| CN113111351B (en) | Test method, test device and computer readable storage medium | |
| CN111539517A (en) | Graph convolution neural network generation method based on graph structure matrix characteristic vector | |
| CN116151384B (en) | Quantum circuit processing method and device and electronic equipment | |
| CN109245948B (en) | Security-aware virtual network mapping method and device | |
| CN109710633B (en) | Method and device for determining man-in-middle information and intelligent terminal | |
| JP2018156458A (en) | Creation device, creation method, and creation program | |
| CN108683599B (en) | Preprocessing-based method and system for determining maximum flow of flow network | |
| CN110224847B (en) | Social network-based community division method and device, storage medium and equipment | |
| CN109993338B (en) | Link prediction method and device | |
| CN112784962B (en) | Super-network training method and device, electronic equipment and storage medium | |
| CN110555158A (en) | mutually exclusive data processing method and system, and computer readable storage medium | |
| CN113704309A (en) | Graph data processing method and device, computer equipment and storage medium | |
| CN115361293B (en) | Network topology graph segmentation method and device, terminal equipment and storage medium | |
| Sowjanya et al. | A cluster feature-based incremental clustering approach to mixed data | |
| CN112751690B (en) | Network representation learning method and device, electronic equipment and readable storage medium | |
| CN113064720B (en) | Object allocation method, device, server and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |