CN115422245A - Reordering method for graph data locality mining - Google Patents

Abstract

The invention relates to a reordering method for graph data locality mining, which comprises the following steps: step S1, converting an original image data set into a corresponding tree structure; s2, performing hierarchical traversal on the nodes of the first m layers of the tree structure based on a deep branch reordering algorithm to obtain a hierarchical traversal result; wherein m is a designated number of layers, and m is a positive integer less than the total number of layers of the tree structure; s3, performing depth-first traversal on the rest nodes of the tree structure except the nodes of the previous m layers based on a depth branch reordering algorithm to obtain a depth-first traversal result; and S4, storing the hierarchy traversal result and the depth-first traversal result into the same result set array to obtain a reordering result of the original image data set. By means of the technical scheme, the method and the device for processing the graph application program can effectively reduce the computation overhead of the running time and the algorithm running of the graph application program.

Description

Reordering method for graph data locality mining

Technical Field

The invention relates to the technical field of image processing, in particular to a reordering method for graph data locality mining.

Background

The efficiency improvement of the graph computing system is limited by the problems of unstructured and irregular large-scale graph data, strong randomness and poor locality of memory access and the like, and the graph computing system has huge performance gap due to input dependency caused by the multi-type input graph data. And, current distributed graph computing systems increase locality by studying data organization formats, reordering graph data, or partitioning graphs.

Therefore, a reordering method for graph data locality mining is urgently needed so as to effectively reduce the running time of graph application program and the calculation overhead of algorithm running.

Disclosure of Invention

Technical problem to be solved

In view of the above-mentioned shortcomings and drawbacks of the prior art, the present invention provides a reordering method for graph data locality mining, which can effectively reduce the computation overhead of graph application running time and algorithm running.

(II) technical scheme

In order to achieve the purpose, the invention adopts the main technical scheme that:

in a first aspect, an embodiment of the present invention provides a reordering method for graph data locality mining, including: step S1, converting an original image data set into a corresponding tree structure; s2, performing hierarchical traversal on the front m layers of nodes of the tree structure based on a deep branch reordering algorithm to obtain a hierarchical traversal result; wherein m is a designated number of layers, and m is a positive integer less than the total number of layers of the tree structure; s3, performing depth-first traversal on the remaining nodes of the tree structure except the nodes of the previous m layers based on a depth branch reordering algorithm to obtain a depth-first traversal result; and S4, storing the hierarchical traversal result and the depth-first traversal result into the same result set array to obtain a reordering result of the original image data set.

Optionally, step S2 includes the steps of: step S21, determining whether the current first accessed node is accessed; the current first visited node is one of the nodes in the previous m layers; step S22, if the current first accessed node is determined not to be accessed, storing the current first accessed node in a pre-established access record array by using a breadth first search method, and accessing the current first accessed node; wherein, the access record array is used for recording the accessed nodes; step S23, if it is determined that the current first visited node is visited, the current first visited node is updated to a node next to the current first visited node, and the process returns to step S21.

Optionally, after accessing the current first visited node, the step S22 further includes the following steps: step S221, determining whether the specified layer number m is reached; step S222, if the specified layer number m is not reached, updating the current first accessed node to be the next node of the current first accessed node, and returning to the step S21; in step S223, when the number of layers reaches the designated number m, step S3 is executed.

Optionally, step S3 includes the steps of: step S31, accessing the current second accessed node; the current second accessed node is one of the rest nodes; step S32, determining whether the current second accessed node has an adjacent point; the adjacent point is a child node adjacent to the current second accessed node; step S33, if the second accessed node is determined to have the adjacent point, updating the second accessed node according to the adjacent point; in step S34, if it is determined that the current second visited node does not have the adjacent point, the current second visited node is updated to be the next node of the current second visited node.

Optionally, the number of the adjacent points is multiple; updating the current second visited node according to the point of adjacency, comprising the steps of: step S331, determining whether the current adjacent point is accessed; step S332, if the current adjacent point is determined to be accessed, the current second accessed node is updated to be the next adjacent point, and the step S32 is returned; in step S333, if it is determined that the current adjacency point has not been accessed, the current second accessed node is updated to the current adjacency point, and the process returns to step S31.

Optionally, after step S34, the reordering method further comprises: step S35, determining whether the updated current second accessed node is the last node; in step S36, if it is determined that the updated current second visited node is the last node, the depth-first traversal is ended.

Optionally, the reordering method further comprises: step S37, if the updated current second accessed node is not the last node, determining whether the updated current second accessed node is accessed; step S38, if the updated current second accessed node is determined to be accessed, the updated current second accessed node is updated to be the next node of the updated current second accessed node, and the step S35 is returned; in step S39, if it is determined that the updated current second visited node has not been visited, the process returns to step S31.

Optionally, the start node set of the depth-first traversal in step S3 is a child node set of the mth layer node.

In a second aspect, an embodiment of the present application provides a storage medium, where a computer program is stored, and when the computer program is executed by a processor, the computer program performs the method of the first aspect or any optional implementation manner of the first aspect.

In a third aspect, an embodiment of the present application provides an electronic device, including: a processor, a memory and a bus, the memory storing processor-executable machine-readable instructions, the processor and the memory communicating via the bus when the electronic device is running, the machine-readable instructions, when executed by the processor, performing the method of the first aspect or any of the optional implementations of the first aspect.

(III) advantageous effects

The invention has the beneficial effects that:

according to the reordering method for graph data locality mining, due to the fact that the DBR algorithm combines the advantages of hierarchical community mining and deep community mining, the running time of a graph application program and the calculation cost of algorithm running can be effectively reduced, and therefore retrieval efficiency can be improved.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings that are required to be used in the embodiments of the present application will be briefly described below, it should be understood that the following drawings only illustrate some embodiments of the present application and therefore should not be considered as limiting the scope, and that those skilled in the art can also obtain other related drawings based on the drawings without inventive efforts.

FIG. 1 is a flow chart illustrating a reordering method for graph data locality mining according to an embodiment of the present disclosure;

FIG. 2 is a flow chart illustrating a hierarchical traversal section of a reordering method for graph data locality mining according to an embodiment of the present application;

FIG. 3 is a flow chart illustrating a depth-first traversal section of a reordering method for graph data locality mining according to an embodiment of the present application;

FIG. 4a is a schematic diagram of a tree structure provided by an embodiment of the present application;

FIG. 4b is a diagram illustrating a tree structure sorted according to a layer of child nodes according to an embodiment of the present application;

fig. 4c is a schematic diagram illustrating a reordering result after being ordered according to a layer of child nodes according to an embodiment of the present application;

FIG. 5a is a schematic diagram illustrating a tree structure sorted according to two layers of child nodes according to an embodiment of the present application;

fig. 5b is a schematic diagram illustrating a reordering result after being ordered according to two layers of child nodes according to an embodiment of the present application;

FIG. 6a is a schematic diagram of another tree structure provided by an embodiment of the present application;

fig. 6b is a schematic diagram illustrating another tree structure sorted according to two layers of child nodes according to the embodiment of the present application;

fig. 6c is a schematic diagram illustrating a reordering result after being ordered according to two layers of child nodes according to an embodiment of the present application.

Detailed Description

For the purpose of better explaining the present invention and to facilitate understanding, the present invention will be described in detail by way of specific embodiments with reference to the accompanying drawings.

At present, complex graph reordering techniques can effectively reduce the runtime of graph applications, but the reordering step also incurs a corresponding computational overhead.

Based on this, the embodiment of the present application provides a reordering method for graph data locality mining, because a Depth-Branch-Reorder (DBR) algorithm combines advantages of hierarchical community mining (or hierarchical traversal) and deep community mining (or Depth-first traversal), starting from a source node, proceeding downward according to a hierarchical community mining manner, traversing a child node of a specified level m according to a Depth-first manner when a specified number of layers m is reached, and traversing other branches after the traversal of a Branch of a child node according to the Depth-first manner is completed until the whole graph is completely traversed, thereby effectively reducing the running time of a graph application program and the computational overhead of algorithm operation.

In order to better understand the above technical solution, exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. While exemplary embodiments of the invention are shown in the drawings, it should be understood that the invention may be embodied in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art.

Referring to fig. 1, fig. 1 is a flowchart illustrating a reordering method for graph data locality mining according to an embodiment of the present disclosure. As shown in fig. 1, the reordering method may be performed by an image reordering device, and the specific device of the image reordering device may be set according to actual requirements, which is not limited to the embodiments of the present application. For example, the image reordering device may be a computer, a server, or the like. Specifically, the reordering method comprises:

step S110, converting the original image data set into a corresponding tree structure;

step S120, performing hierarchical traversal on the first m layers of nodes of the tree structure based on a deep branch reordering algorithm to obtain a hierarchical traversal result; wherein m is a designated number of layers, and m is a positive integer less than the total number of layers of the tree structure;

step S130, depth-first traversal is carried out on the rest nodes of the tree structure except the nodes of the previous m layers based on a depth branch reordering algorithm, and a depth-first traversal result is obtained;

step S140, storing the hierarchical traversal result and the depth-first traversal result in the same result set array to obtain a reordering result of the original image data set.

Therefore, the DBR algorithm combines the advantages of hierarchical community mining and deep community mining, the running time of the graph application program and the calculation cost of algorithm running can be effectively reduced, and therefore the retrieval efficiency can be improved.

To facilitate understanding of the specific process of fig. 1, the following description is made with reference to fig. 2 and 3.

Referring to fig. 2, fig. 2 is a flowchart illustrating a hierarchical traversal section in a reordering method for graph data locality mining according to an embodiment of the present application. Specifically, as shown in fig. 2, the related method of the hierarchical traversal part includes:

step S211, the original image data set is converted into its corresponding tree structure.

Specifically, in the case where the original image dataset contains image compression formats from an initial vertex to an end vertex (e.g., (1, 2), (1, 3), and (1, 4), etc.), the original image dataset may be converted into a tree structure of a specified form (specifically, parent node: set of leaf nodes, e.g., (1.

Step S212, respectively creating an access record array, a to-be-accessed node queue and a result set array. Wherein, the access record array is used for recording the accessed nodes; the to-be-accessed node queue is used for recording nodes needing to be accessed; the result set array is used for storing the hierarchical traversal result and the depth-first traversal result.

Specifically, an access record array DBR _ visited, a to-be-accessed node queue DBR _ queue, and a result set array DBR _ Graph may be created separately.

It should be noted here that the access record array DBR _ visited, the to-be-accessed node queue DBR _ queue, and the result set array DBR _ Graph can be used not only in the hierarchical traversal part, but also in the depth-first traversal part.

In step S213, the designated number m of layers input by the user is acquired.

It should be understood that the specific value of the designated number m of layers may be set according to actual requirements, and the embodiment of the present application is not limited thereto.

For example, m may be 1 or 2.

Step S214, the current first visited node is taken out from the queue of nodes to be visited. Wherein the current first visited node may be one of the first m-level nodes of the tree structure

Specifically, the current first visited node Vi may be taken out from the queue of nodes to be visited, and may be initialized as the source node.

In step S215, it is determined whether the current first visited node has been visited.

If it is determined that the current first visited node is not visited, execute step S216; if it is determined that the first visited node is currently visited, step S218 is performed.

In particular, whether the current first visited node has been visited may be determined by querying the access record array. The subsequent related steps are similar and are not described one by one.

Step S216, the current first accessed node is stored in the access record array created in advance by using the breadth first search method, the current first accessed node is accessed, and the message accessed by the current first accessed node is updated to the access record array.

In step S217, it is determined whether the specified number m of layers is reached.

If it is determined that the specified number m of layers is reached, executing step S219; if the number of layers m is not reached, step S218 is executed.

Step S218, the current first visited node is updated to be the next node of the current first visited node, and the process returns to step S215.

Step S219, the hierarchical community mining is finished.

That is, the tree structure may be initialized, hierarchical local community mining may be started from a source node of the tree structure, traversal may be performed while storing accessed node data into the result set array, and after all child node sets (e.g., leaf node sets) are accessed except for the source node, the number of traversed layers may be added by 1, and after the number of traversed layers reaches a specified number m, deep community traversal may be performed on the remaining nodes.

Referring to fig. 3, fig. 3 is a flowchart illustrating a depth-first traversal section in a reordering method for graph data locality mining according to an embodiment of the present application. Specifically, as shown in fig. 3, the related method of the depth-first traversal part includes:

step S311, accessing the current second visited node, and identifying the current second visited node by using the target identifier. The current second visited node is one of the remaining nodes, and the current second visited node may be taken out from the queue of the node to be visited, or may be an updated node, or the like; the target identifier is used to indicate that the current second visited node is a node mined by the deep community, and a specific identifier of the target identifier may be set according to an actual requirement, which is not limited in this embodiment of the application.

In step S312, it is determined whether an adjacent point exists in the current second visited node. And the adjacent point is a child node adjacent to the current second accessed node.

If it is determined that the current second visited node has an adjacent point, step S313 is executed; if it is determined that the current second visited node does not have the adjacency point, step S316 is performed.

In step S313, it is determined whether the current adjacency point has been accessed.

If it is determined that the current adjacency point is not accessed, go to step S314; if it is determined that the current neighbor point has been accessed, step S315 is executed.

Step S314, update the current second visited node to the current neighbor point, and return to step S311.

Step S315, update the current second visited node to the next neighbor point, and return to step S312.

Step S316, the current second visited node is updated to be the next node of the current second visited node.

In step S317, it is determined whether the updated current second visited node is the last node.

If it is determined that the updated current second visited node is not the last node, performing step S318; if it is determined that the current second visited node is the last node, step S319 is executed.

Step S318, determine whether the updated current second visited node has been visited.

If it is determined that the current second visited node has been visited, then step S315 is executed; if it is determined that the current second visited node has not been visited, the step S311 is returned to.

And step S319, ending.

That is, during the deep community traversal, the data for the accessed nodes is also stored into the result set array until all nodes are accessed, and the algorithm ends.

It should be noted here that the start node set of the depth community traversal is a leaf node set at the end of the last traversal when the hierarchical community traversal is ended, that is, the start node set of the depth-first traversal is a child node set of the mth layer node.

It should be noted here that, because the traversal data in the embodiment of the present application can be cached (for example, the traversal result is cached through the result set array; for example, the accessed node is recorded through accessing the record array, etc.), a large number of repeated traversal processes can be avoided, and the traversal time can be greatly reduced by performing the sequence of hierarchical community mining and deep community mining first.

In order to facilitate understanding of the embodiments of the present application, the following description will be given by way of specific examples.

Specifically, please refer to fig. 4a to 4c, which illustrate an algorithmic process of DBR reordering of one layer of sub-nodes. As shown in fig. 4a, there are 16 nodes in the graph, and the graph is sorted by one level of child nodes and converted into a tree structure, and the result is shown in fig. 4 b.

And traversing from the node 1, if the depth-first traversal is performed from the source node one-level child node, if all five layers of data exist, the result is shown in fig. 4 c. The node 1 traverses one level downwards according to a level community mining mode, and accesses the nodes 5, 8 and 12. Then, traversing down in a depth-first traversal from the child node of node 1, node 5 accesses nodes 2, 11, and 14, node 8 accesses nodes 3, 7, and 15, and node 12 accesses nodes 4, 9, 13, 10, 16, and 6, respectively, traversing through the entire graph in sequence.

In addition, please refer to fig. 5a, which is a tree structure for ordering the transformation according to two levels of child nodes. And traversing from node 1, dividing according to two layers, traversing the first two layers according to the hierarchical communities, and accessing the nodes 5, 8 and 12 from node 1 by the traversing sequence. Then, accessing the nodes 2, 14, 3, and 4, after two layers are finished, performing a single-branch deep traversal downwards according to the child node in the second layer, first accessing the child node 11 of the node 2, then sequentially traversing downwards other branches according to the node sequence in the second layer, and sequentially accessing the nodes 7, 15, 9, 13, 10, 16, and 6 until the entire graph is traversed, with the result shown in fig. 5 b.

In addition, please refer to fig. 6a to 6c, which illustrate DBR sorting algorithm process diagrams of two layers of child nodes of a 64-node graph community. And traversing from the node 1, and dividing according to two layers. The first two layers traverse according to the hierarchical communities, the traversal sequence accesses the nodes 2,3 and 4 and the nodes 5, 6, 7, 8 and 9 from the node 1, after the two layers are finished, the nodes are traversed downwards according to the depth of the sub-nodes of the second layer but branches, the sub-node of the node 5 is accessed firstly, and then other branches are traversed downwards once according to the node sequence of the second layer, as shown in fig. 6b, until the whole graph is traversed, and the result is shown in fig. 6 c.

It should be understood that the above data protection method is only exemplary, and those skilled in the art can make various modifications according to the above method, and the solution after the modification also belongs to the protection scope of the present application.

The present application provides a storage medium having stored thereon a computer program which, when executed by a processor, performs the method of an embodiment.

The present application also provides an electronic device comprising a processor, a memory, and a computer program stored on the memory, the processor executing the computer program to perform the method of the embodiments.

As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.

The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions.

It should be noted that in the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. The word "comprising" does not exclude the presence of elements or steps not listed in a claim. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The invention can be implemented by means of hardware comprising several distinct elements, and by means of a suitably programmed computer. In the claims enumerating several means, several of these means may be embodied by one and the same item of hardware. The use of the terms first, second, third, etc. are used for convenience only and do not denote any order. These words are to be understood as part of the name of the component.

Furthermore, it should be noted that in the description of the present specification, the description of the term "one embodiment", "some embodiments", "examples", "specific examples" or "some examples", etc., means that a specific feature, structure, material or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the terms used above are not necessarily intended to refer to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples. Moreover, various embodiments or examples and features of various embodiments or examples described in this specification can be combined and combined by one skilled in the art without being mutually inconsistent.

While preferred embodiments of the present invention have been described, additional variations and modifications in those embodiments may occur to those skilled in the art once they learn of the basic inventive concepts. Therefore, the claims should be construed to include preferred embodiments and all changes and modifications that fall within the scope of the invention.

It will be apparent to those skilled in the art that various modifications and variations can be made in the present invention without departing from the spirit or scope of the invention. Thus, if such modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention should also include such modifications and variations.

Claims (8)

1. A reordering method for graph data locality mining, comprising:

step S1, converting an original image data set into a corresponding tree structure;

s2, performing hierarchical traversal on the nodes of the first m layers of the tree structure based on a deep branch reordering algorithm to obtain a hierarchical traversal result; wherein m is a designated number of layers, and m is a positive integer less than the total number of layers of the tree structure;

s3, performing depth-first traversal on the rest nodes of the tree structure except the nodes of the previous m layers based on the depth branch reordering algorithm to obtain a depth-first traversal result;

and S4, storing the hierarchical traversal result and the depth-first traversal result into the same result set array to obtain a reordering result of the original image data set.

2. A reordering method according to claim 1 wherein said step S2 comprises the steps of:

step S21, determining whether the current first accessed node is accessed; wherein the current first visited node is one of the previous m-layer nodes;

step S22, if the current first accessed node is determined not to be accessed, storing the current first accessed node in a pre-established access record array by using a breadth first search method, and accessing the current first accessed node; wherein, the access record array is used for recording the nodes which have been accessed;

step S23, if it is determined that the current first visited node is visited, updating the current first visited node to be a node next to the current first visited node, and returning to the step S21.

3. A reordering method according to claim 2 wherein, after accessing said current first accessed node, said step S22 further comprises the steps of:

step S221, determining whether a specified number m of layers is reached;

step S222, if the designated number m of layers is not reached, updating the current first visited node to a node next to the current first visited node, and returning to the step S21;

and step S223, if the designated number m of layers is reached, executing step S3.

4. A reordering method according to claim 1 wherein said step S3 comprises the steps of:

step S31, accessing the current second accessed node; wherein the current second visited node is one of the remaining nodes;

step S32, determining whether the current second accessed node has an adjacent point; the adjacent point is a child node adjacent to the current second visited node;

step S33, if the adjacent point of the current second accessed node is determined to exist, updating the current second accessed node according to the adjacent point;

step S34, if it is determined that the adjacent point does not exist in the current second visited node, updating the current second visited node to be a node next to the current second visited node.

5. The reordering method according to claim 4 wherein said abutment points are plural in number; said updating said current second visited node in accordance with said adjacency point, comprising the steps of:

step S331, determining whether the current adjacent point is accessed;

step S332, if it is determined that the current adjacency point is accessed, updating the current second accessed node to be the next adjacency point, and returning to step S32;

in step S333, if it is determined that the current adjacent point has not been accessed, the current second accessed node is updated to the current adjacent point, and the process returns to step S31.

6. The reordering method of claim 4, wherein after step S34, the reordering method further comprises:

step S35, determining whether the updated current second accessed node is the last node;

step S36, if it is determined that the updated current second visited node is the last node, ending the depth-first traversal.

7. The reordering method of claim 6, wherein said reordering method further comprises:

step S37, if it is determined that the updated current second visited node is not the last node, determining whether the updated current second visited node has been visited;

step S38, if the updated current second accessed node is determined to be accessed, updating the updated current second accessed node to be the next node of the updated current second accessed node, and returning to the step S35;

and step S39, if it is determined that the updated current second visited node is not visited, returning to step S31.

8. A reordering method according to claim 1 wherein said set of starting nodes traversed depth-first in step S3 is a set of sub-nodes of said mth layer node.

Images