WO2023273171A1 - 图像处理方法、装置、设备和存储介质 - Google Patents
图像处理方法、装置、设备和存储介质 Download PDFInfo
- Publication number
- WO2023273171A1 WO2023273171A1 PCT/CN2021/136446 CN2021136446W WO2023273171A1 WO 2023273171 A1 WO2023273171 A1 WO 2023273171A1 CN 2021136446 W CN2021136446 W CN 2021136446W WO 2023273171 A1 WO2023273171 A1 WO 2023273171A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- connected graph
- graph
- edge
- reconstructed
- image
- Prior art date
Links
- 238000003672 processing method Methods 0.000 title claims abstract description 16
- 238000000034 method Methods 0.000 claims abstract description 35
- 238000012545 processing Methods 0.000 claims abstract description 17
- 238000004590 computer program Methods 0.000 claims description 18
- 238000012360 testing method Methods 0.000 claims description 18
- 238000013527 convolutional neural network Methods 0.000 claims description 16
- 238000003062 neural network model Methods 0.000 description 8
- 239000013598 vector Substances 0.000 description 7
- 238000002372 labelling Methods 0.000 description 6
- 230000008569 process Effects 0.000 description 5
- 230000011218 segmentation Effects 0.000 description 5
- 238000004364 calculation method Methods 0.000 description 4
- 238000004422 calculation algorithm Methods 0.000 description 3
- 238000010586 diagram Methods 0.000 description 3
- 238000012549 training Methods 0.000 description 3
- 102100033620 Calponin-1 Human genes 0.000 description 2
- 102100033591 Calponin-2 Human genes 0.000 description 2
- 102100033592 Calponin-3 Human genes 0.000 description 2
- 101000945318 Homo sapiens Calponin-1 Proteins 0.000 description 2
- 101000945403 Homo sapiens Calponin-2 Proteins 0.000 description 2
- 101000945410 Homo sapiens Calponin-3 Proteins 0.000 description 2
- 238000013473 artificial intelligence Methods 0.000 description 2
- 238000013135 deep learning Methods 0.000 description 2
- 230000004927 fusion Effects 0.000 description 2
- 239000011159 matrix material Substances 0.000 description 2
- 235000005338 Allium tuberosum Nutrition 0.000 description 1
- 244000003377 Allium tuberosum Species 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000003058 natural language processing Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000005192 partition Methods 0.000 description 1
- 238000000638 solvent extraction Methods 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 238000000547 structure data Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/23—Clustering techniques
- G06F18/232—Non-hierarchical techniques
- G06F18/2323—Non-hierarchical techniques based on graph theory, e.g. minimum spanning trees [MST] or graph cuts
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
Definitions
- the present application relates to the technical field of computer processing, in particular to an image processing method, device, computer equipment and storage medium.
- each image represented by each node of the connected graph included in the node connection graph is regarded as the same type of image.
- An image processing method comprising:
- the reconstructed connected graph of the first connected graph is obtained; the reconstructed connected graph is obtained according to the reconstruction of the outer graph of the first connected graph;
- the outer graph includes a graph other than the target connection edge in the first connected graph;
- a second connected graph of the image set is determined according to the first connected graph and the reconstructed connected graph.
- the determining the second connected graph of the image set according to the first connected graph and the reconstructed connected graph includes:
- the connectivity of the reconstructed connected graph is disconnected, and each sub-connected graph contained in the reconstructed connected graph is dense in structure, then the sub-connected graphs that have not been reconstructed in each sub-connected graph and /or reconstruct the sub-connected graph as the second connected graph of the image set; the reconstructed sub-connected graph does not contain connection edges belonging to wrong connection edges or the number of nodes in the reconstructed connected graph is lower than the node number threshold .
- the method also includes:
- the sub-connected graph is reconstructed.
- the determining the second connected graph of the image set according to the first connected graph and the reconstructed connected graph includes:
- the connectivity of the reconstructed connected graph is disconnected, and the reconstructed connected graph contains at least one sub-connected graph with a sparse structure, and the target connected edge is the last connected edge predicted sequentially, then the The first connected graph is used as the second connected graph of the image set.
- the determining the second connected graph of the image set according to the first connected graph and the reconstructed connected graph includes:
- the reconstructed connected graph is used as the second connected graph of the image set.
- predicting whether each connected edge belongs to a wrongly connected edge according to the order of betweenness centrality measures of each connected edge included in the first connected graph from large to small includes:
- connection edges determining the connection edges to be predicted among the connection edges according to the order
- the feature information includes the connection relationship between nodes in the node set corresponding to the connection edge to be predicted, and the label value of each node in the node set;
- the node set includes The two target nodes connected by the connection edge to be predicted, the respective first-order neighbor nodes and second-order neighbor nodes of the two target nodes; for any two neighbor nodes in the node set, the neighbor node with the smaller label value
- the sum of the shortest paths to each target node is less than the sum of the shortest paths from the neighbor nodes with a larger label value to each target node; or, for any two neighbor nodes in the node set, the neighbor with a smaller label value
- the shortest path from neighbor nodes with smaller label values to each target node is the shortest The shortest path is smaller than the shortest path in the
- the acquiring the first connected graph obtained by clustering the image set includes:
- the method also includes:
- An image processing device comprising:
- a connected graph acquisition module configured to obtain the first connected graph obtained by clustering the image set
- connection edge prediction module used to predict whether each connection edge is a wrong connection edge according to the order of betweenness centrality measures of each connection edge included in the first connected graph from large to small;
- the reconstructed connected graph acquisition module is used to obtain the reconstructed connected graph of the first connected graph if the target connected edge belonging to the wrong connected graph is predicted; the reconstructed connected graph is based on the first connected graph Obtained by reconstructing the outer edge graph; the outer edge graph includes graphs other than the target connection edge in the first connected graph;
- a connected graph determining module configured to determine a second connected graph of the image set according to the first connected graph and the reconstructed connected graph.
- a computer device includes a memory and a processor, the memory stores a computer program, and the processor implements the above method when executing the computer program.
- a computer-readable storage medium on which a computer program is stored, and the above-mentioned method is realized when the computer program is executed by a processor.
- the above image processing method, device, computer equipment, and storage medium obtain the first connected graph obtained by clustering the image set; according to the betweenness centrality measure of each connecting edge included in the first connected graph Sequence, predicting whether each connected edge belongs to an incorrectly connected edge; if the target connected edge belonging to an incorrectly connected edge is predicted, then obtain the reconstructed connected graph of the first connected graph; the reconstructed connected graph is based on the Obtained by reconstructing the edge outer graph of the first connected graph; the edge outer graph includes graphs other than the target connection edge in the first connected graph; according to the first connected graph and the reconstructed connected graph, A second connected graph of the set of images is determined.
- the prediction order of each connecting edge is determined according to the size of the betweenness centrality measure of each connecting edge of the first connected graph. If the betweenness centrality measure of the connecting edge is larger, it indicates that the connecting edge may connect different clusters. Therefore, the connection edge with a large betweenness centrality measure is preferentially predicted to save computing resources and reduce time costs; if the connection edge is predicted to be a wrong connection edge, it means that the connection edge should not exist and the Connecting edges may connect different clusters, and then according to the graph reconstruction of the first connected graph except for the wrong connected edges, the reconstructed connected graph is obtained, and the second connected graph of the image set is determined according to the first connected graph and the reconstructed connected graph , the image represented by each node in the second connected graph can be subsequently regarded as the same type of image, and an image clustering result with high accuracy can be obtained.
- Fig. 1 is a schematic flow chart of an image processing method in an embodiment
- Fig. 2 is a schematic flow chart of an image processing method in an embodiment
- Fig. 3 is a schematic flow chart of an image processing method in an embodiment
- Fig. 4 is a structural block diagram of an image processing device in an embodiment
- Figure 5 is an internal block diagram of a computer device in one embodiment.
- Deep learning has made many breakthroughs in the field of artificial intelligence, such as computer vision, natural language processing and many other artificial intelligence directions.
- Various neural network models obtained through deep learning are used to complete tasks such as classification and recognition.
- a supervised training process is required, that is, the parameters of the neural network model are continuously adjusted using labeled sample images to achieve the required performance.
- One of the ways of intelligent sample image labeling may be: clustering the sample images to separate the sample images belonging to different categories, where the categories are also called clusters. Based on this, by marking the sample images in the same cluster with the same virtual label, training data samples can be provided for the neural network model learning task.
- Graph clustering method is a clustering algorithm based on graph partition theory. Compared with other clustering methods, graph clustering algorithm has obvious advantages: this type of method is more adaptable to data distribution and can identify clusters of arbitrary shapes. kind.
- the core of the image clustering algorithm based on graph partitioning is to regard the image clustering problem as a graph segmentation problem.
- the first part is to construct the node connection graph of the sample image (the node connection graph can be obtained through Extract the feature vector of the sample image to construct); the second part is to determine the optimal segmentation target and solve the problem of node connection graph division.
- the relationship between nodes can be evaluated through prior knowledge, and the accuracy of the generated adjacency matrix is related to the value of the hyperparameter.
- the collected sample images are often non-ideal datasets with unbalanced samples. If the hyperparameters set are unreasonable, it is likely that some connection edges in the node connection graph will be permanently deleted or there will be many wrong connection edges, which will affect the image clustering performance.
- this application inputs the unlabeled sample image sets into different CNNs to obtain different neighbors of the same node Structural representation, local topological fusion is performed on this basis, and the sample image is described more accurately from multiple different angles, making it more discriminative and feature expression ability stronger.
- CNN Convolution Neural Network
- the computer device obtains the first connected graph obtained by clustering the image set (that is, S201 in FIG. 2 ), it can be realized through the following steps: input the image set into different convolutional neural networks; The first similar atlas corresponding to each image in the image set determined by each convolutional neural network; according to the first similar atlas determined by different convolutional neural networks corresponding to the same image, the first similar atlas of each image in the image set is obtained. Two homogeneous atlases; constructing the first connected graph according to the homogeneous relationship between images represented by the second homogeneous atlas of the images.
- each CNN After the computer equipment inputs the image set into each CNN, each CNN will extract the feature vector of each image in the image set; The similarity calculation is performed on the eigenvector of the image, and based on the size of the similarity, it is determined whether the other image is a similar image of the image, that is, the first similar atlas of the image determined by the CNN1 is obtained; similarly, the image can also be obtained First-of-its-kind atlas identified via CNN2 and CNN3. Then, the first class statistics determined by different CNNs of the same image are fused to obtain the second class atlas of the same image.
- the first similar atlases corresponding to image I determined based on CNN1 are J, A and B
- the first similar atlases corresponding to image I determined based on CNN2 are A, B and C
- the image I determined based on CNN3 The corresponding first similar atlases are J, A, and C. Therefore, the computer equipment fuses the first image atlases corresponding to the image I determined by each convolutional network, and obtains the second image atlases corresponding to the image I as J, A, and C. B and C.
- the computer device associates node i of image I with the nodes corresponding to images J, A, B, and C of the second image atlas Perform edge connections to obtain the corresponding node connection graph.
- a similarity threshold can be introduced to evaluate the similarity between two nodes.
- the step of determining whether the other image is a similar image of the image based on the size of the similarity it can be determined whether the similarity is greater than the similarity threshold. If so, it indicates that the other image has a high similarity with the image, so The nodes of these two images can be connected by edges to obtain the corresponding node connection graph.
- the formula for calculating the false recognition rate is (NIRA is the total number of inter-class tests, NFA is the number of times that the actual non-similar image is determined as the same image), and the calculation formula of the rejection rate is (NGRA is the total number of within-class tests, and NFR is the number of times actual class images are identified as non-class images).
- the computer device can obtain the first number of times when the actual non-similar image is determined as the same type image, and determine the false recognition rate of the test according to the first number; obtain the second time that the actual similar image is determined as the non-similar image, Determine the true rejection rate of the test according to the second number of times; based on the false rejection rate of the test and the false recognition rate of the test, adjust and determine the similarity threshold used by similar atlases; Can second similar atlas.
- the computer device After the computer device obtains the node connection graph corresponding to each image, it splices the node connection graph corresponding to each image to form a complete relationship graph (the complete relationship graph also belongs to the node connection graph). If the complete relationship graph is a connected graph, then The betweenness centrality measure of each connecting edge of the complete relationship graph can be directly calculated to obtain the key connecting edges i-j, k-l, m-n with large betweenness centrality measure. If the complete relation graph includes multiple connected graphs, the betweenness centrality measure of the connecting edges of each connected graph can be calculated separately.
- the computer device predicts whether each connected edge belongs to a wrongly connected edge according to the order of betweenness centrality measures of each connected edge included in the first connected graph from large to small (step S202);
- the betweenness centrality measure of this misconnected edge is larger than other connected edges contained in the connected graph, so when predicting whether each connected edge is a misconnected edge , which can preferentially predict the one with a larger betweenness centrality measure.
- the betweenness centrality measure of connecting edges can be determined by the following formula:
- V represents the set of nodes contained in the connected graph
- ⁇ (s, t) represents the number of shortest paths connecting node s and node t
- e) is the number of shortest paths connecting two nodes s and node t The number of paths passing through the connecting edge e among all the shortest paths.
- the computer device predicts the target connection edge belonging to the wrong connection edge, then obtain the reconstructed connected graph of the first connected graph; the reconstructed connected graph is obtained according to the reconstruction of the outer graph of the first connected graph ;
- the outer graph includes a graph other than the target connection edge in the first connected graph (step S203);
- the wrongly connected edge can be deleted from the first connected graph to obtain a graph other than the wrongly connected edge in the first connected graph, and this graph can be used as a reconstructed connected graph;
- the computer can also obtain a graph other than the wrongly connected edge in the first connected graph (that is, the outer graph), and use the outer graph as the criterion to reconstruct a graph that is consistent with the outer graph.
- step S204 When the computer device determines the second connected graph of the image set according to the first connected graph and the reconstructed connected graph (step S204), various processing methods may be included, and specific multiple processing methods will be introduced later.
- the prediction order of each connected edge is determined according to the size of the betweenness centrality measure of each connected edge of the first connected graph. If the betweenness centrality measure of the connected edge is larger, it indicates that the connected edge may be a connection Important connection edges of different clusters. Therefore, the connection edge with a large betweenness centrality measure is preferentially predicted to save computing resources and reduce time costs; if the connection edge is predicted to be a wrong connection edge, it means that the connection edge should not exist And the connected edges may connect different clusters, and then according to the graph reconstruction in the first connected graph except the wrong connected edges, the reconstructed connected graph is obtained, and the second connected graph of the image set is determined according to the first connected graph and the reconstructed connected graph. Connected graph, the image represented by each node in the second connected graph can be subsequently regarded as the same type of image, and an image clustering result with high accuracy can be obtained.
- the following steps may be performed: determine the to-be-predicted connecting edge among the said connecting edges according to the order; obtain the feature information of the to-be-predicted connecting edge;
- the feature information is input to a pre-built graph convolution network (Graph Convolution Network, GCN); the probability value of the connection edge to be predicted belonging to the wrong connection edge predicted and output by the graph convolution network based on the feature information is obtained ; Determine whether the connection side to be predicted is a wrong connection side based on the probability value.
- GCN Graph Convolution Network
- the feature information includes the connection relationship between nodes in the node set corresponding to the connection edge to be predicted, and the label value of each node in the node set; the node set includes the nodes connected by the connection edge to be predicted.
- the two target nodes, the first-order neighbor nodes and the second-order neighbor nodes of the two target nodes are the nodes in the second-order relationship graphs of the connection edges i-j, k-l, and m-n shown in FIG. 1 .
- the sum of the shortest paths from the neighbor nodes with a smaller label value to each target node is less than the sum of the shortest paths from the neighbor nodes with a larger label value to each target node; or, For any two neighbor nodes in the node set, the sum of the shortest paths from the neighbor nodes with a smaller label value to each target node is equal to the sum of the shortest paths from the neighbor nodes with a larger label value to each target node
- the shortest shortest path among the shortest paths from the neighbor node with a smaller label value to each target node is smaller than the shortest shortest path among the shortest paths from the neighbor node with a larger label value to each target node.
- the shortest path from neighbor node x to each target node is d(x,i), d(x,j), and the shortest path from neighbor node y to each target node is d(y,i), d(y,j) .
- d(x,i)+d(x,j) d(y,i)+d(y,j)
- a label value greater than that of the target node can be assigned to the neighbor node.
- the computer equipment inputs the above feature information corresponding to each connection edge i-j, k-l, m-n into the graph convolutional network in order of betweenness centrality measure from large to small, so as to determine that each key connection edge is a wrong connection A probability value of an edge; determining whether the to-be-predicted connection edge is a wrong connection edge based on the probability value.
- the optimized segmentation objective function established in the past is no longer applicable, and the graph convolutional network can extract features from the graph structure data and use other node knowledge to infer the target. Therefore, GCN is used to automatically learn the association information between node pairs, so as to solve the problem of the existence of key edges and improve the prediction accuracy.
- the computer device can perform the following steps:
- step S301 the computer device inputs the set of unlabeled sample images into multiple CNNs to obtain the feature vectors of each unlabeled sample image;
- Step S303 based on the feature vectors extracted by different CNNs, calculate whether the similarity between images is greater than the similarity threshold, obtain different neighbor structure representations of the same node, and perform local topology fusion on this basis to obtain the same type of image corresponding to each sample image set;
- Step S304 constructing a complete relational graph (Complete-Graph), the relational graph includes a plurality of connected graphs;
- Step S305 judging whether the set of connected graphs obtained from the complete relational graph is an empty set
- Step S306 selecting a connected graph from the set of connected graphs
- Step S307 calculate whether the number of nodes Nnode of the selected connected graph (denoted as CS-Graph(i)) is greater than or equal to the node number threshold Tnode; if not, CS-Graph(i) is put into the cluster set, and Carry out segmentation processing; if so, enter step S308;
- Step S308 calculate the betweenness centrality measure of all connection edges in CS-Graph(i), that is, the degree of importance, and sort all connection edges according to the betweenness centrality measure from large to small to form a set of connection edges to be predicted ( denoted as Edge(i));
- Step S309 judging whether Edge(i) is an empty set, if so, then proceed to the processing of the next connected graph, if not, then enter step S310;
- Step S310 sorting from large to small according to the betweenness centrality measure, extract the connection edge to be predicted with the largest betweenness centrality measure from Edge(i), and store the relevant information of the connection edge to be predicted (that is, the second-order relationship graph and The node information matrix) is input into the GCN for prediction, and the probability value that the to-be-predicted connection edge output by the GCN is a wrong connection edge is obtained;
- Step S311 judging whether the probability value that the connection side to be predicted is a wrong connection side is greater than the probability threshold (denoted as Pr);
- Step S312 if the probability value of the connection edge to be predicted is a wrong connection edge is greater than the probability threshold, delete the connection edge to be predicted from the selected connected graph;
- Step S313 judging whether the connectivity of the connected graph (equivalent to the reconstructed connected graph) that deletes the connection edge to be predicted has changed. If not, then return to step S309, when Edge(i) is not an empty set, perform the prediction of the next connected edge, and when Edge(i) is an empty set, perform the processing of the next connected graph until all connected graphs are processed .
- Step S314 if the connectivity changes, that is, the connectivity of the reconstructed connected graph is not connected, at this time, it means that the reconstructed connected graph includes multiple sub-connected graphs (that is, a new connected graph is obtained) and it is judged whether the sub-connected graph has a structure Sparse; wherein, whether the structure of the connected graph is sparse can be determined by the number of connected edges of the connected graph. If the number of connected edges of the connected graph is less than the threshold, the structure of the connected graph is considered sparse. If the number of connected edges of the connected graph is greater than or equal to the threshold, then The connected graph structure is considered to be dense.
- Step S315 if the structure of each sub-connected graph is dense, put the sub-connected graph into the connected graph set for segmentation processing;
- Step S316 if the structure of at least one sub-connected graph is sparse, restore the deleted current connection edge to be predicted, and perform the prediction process of the next connection edge.
- step S204 may include the following:
- the sub-connected graphs that have not been reconstructed in each sub-connected graph are A connected graph and/or a reconstructed sub-connected graph as the second connected graph of the image set; the reconstructed sub-connected graph does not contain connected edges belonging to wrongly connected edges or the number of nodes in the reconstructed connected graph is less than Node number threshold.
- the sub-connected graph is reconstructed.
- the connectivity of the reconstructed connected graph is disconnected, and the reconstructed connected graph contains at least one sub-connected graph with a sparse structure, and the target connected edge is the last connected edge predicted sequentially, then The first connected graph is used as the second connected graph of the image set.
- the connected graph is continuously segmented until a connected graph that satisfies the above conditions is obtained, and the connected graph is used as the second connected graph, that is, a set of clusters is obtained, and the same connected graph in the set of clusters is obtained.
- the images represented by the nodes are regarded as images of the same type, and the images of the same type are given the same label to realize intelligent image labeling.
- an image processing device comprising:
- a connected graph acquisition module 401 configured to obtain a first connected graph obtained by clustering the image set
- connection edge prediction module 402 configured to predict whether each connection edge is a wrong connection edge according to the order of betweenness centrality measures of each connection edge included in the first connected graph from large to small;
- the reconstructed connected graph acquisition module 403 is used to obtain the reconstructed connected graph of the first connected graph if the target connected edge belonging to the wrong connected graph is predicted; the reconstructed connected graph is based on the first connected graph Obtained by reconstructing the outer edge graph; the outer edge graph includes graphs other than the target connection edge in the first connected graph;
- a connected graph determining module 404 configured to determine a second connected graph of the image set according to the first connected graph and the reconstructed connected graph.
- the connected graph determining module 404 is further configured to: if the connectivity of the reconstructed connected graph is disconnected, and each sub-connected graph contained in the reconstructed connected graph is dense in structure, then Using the sub-connected graphs and/or reconstructed sub-connected graphs that have not been reconstructed in the sub-connected graphs as the second connected graph of the image set; the reconstructed sub-connected graphs do not contain The connection edge or the number of nodes of the reconstructed connected graph is lower than the node number threshold.
- the device further includes a connected graph reconstruction module, configured to if the number of nodes in the sub-connected graph is higher than the node number threshold and the sub-connected graph contains connection edges belonging to wrong connection edges, Then the sub-connected graph is reconstructed.
- a connected graph reconstruction module configured to if the number of nodes in the sub-connected graph is higher than the node number threshold and the sub-connected graph contains connection edges belonging to wrong connection edges, Then the sub-connected graph is reconstructed.
- the connected graph determining module 404 is further configured to: if the connectivity of the reconstructed connected graph is disconnected, and the reconstructed connected graph includes at least one sub-connected graph with a sparse structure, and the If the target connection edge is the last connection edge of sequential prediction, the first connected graph is used as the second connected graph of the image set.
- the connected graph determination module 404 is further configured to: if the connectivity of the reconstructed connected graph is connected, and the target connection edge is the last connection edge predicted sequentially, then Construct a connected graph as the second connected graph of the image set.
- connection edge prediction module 402 is further configured to determine the connection edges to be predicted in the connection edges according to the order; obtain the feature information of the connection edges to be predicted; the feature information includes The connection relationship between nodes in the node set corresponding to the connection edge to be predicted, and the label value of each node in the node set; the node set includes two target nodes connected by the connection edge to be predicted, two The respective first-order neighbor nodes and second-order neighbor nodes of the target node; for any two neighbor nodes in the node set, the sum of the shortest paths from the neighbor node with a smaller label value to each target node is less than that of the neighbor node with a larger label value The sum of the shortest paths from the neighbor node to each target node; or, for any two neighbor nodes in the node set, the sum of the shortest paths from the neighbor node with the smaller label value to each target node is equal to the sum of the shortest paths with the larger label value In the case of the sum of the shortest paths from neighbor no
- the connected graph acquisition module 401 is further configured to input the image set into different convolutional neural networks; obtain the first corresponding to each image in the image set determined based on each convolutional neural network. A similar atlas; according to the first similar atlas determined by different convolutional neural networks corresponding to the same image, a second similar atlas of each image in the image set is obtained; according to the representation of the second similar atlas of each image The homogeneous relationship among the images is used to construct the first connected graph.
- the device further includes a threshold adjustment module, which is used to obtain the first number of times when the actual non-similar image is determined as the same type image, and determine the false recognition rate of the test according to the first number; The second number of times the image is determined to be a non-similar image, and the test rejection rate is determined according to the second number; based on the test rejection rate and the test false recognition rate, the similarity threshold used to determine the similar atlas is adjusted.
- a threshold adjustment module which is used to obtain the first number of times when the actual non-similar image is determined as the same type image, and determine the false recognition rate of the test according to the first number; The second number of times the image is determined to be a non-similar image, and the test rejection rate is determined according to the second number; based on the test rejection rate and the test false recognition rate, the similarity threshold used to determine the similar atlas is adjusted.
- Each module in the above-mentioned image processing device may be fully or partially realized by software, hardware or a combination thereof.
- the above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, and can also be stored in the memory of the computer device in the form of software, so that the processor can call and execute the corresponding operations of the above modules.
- a computer device is provided, and the computer device may be a server, and its internal structure may be as shown in FIG. 5 .
- the computer device includes a processor, memory and a network interface connected by a system bus. Wherein, the processor of the computer device is used to provide calculation and control capabilities.
- the memory of the computer device includes a non-volatile storage medium and an internal memory.
- the non-volatile storage medium stores an operating system, computer programs and databases.
- the internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium.
- the database of the computer device is used to store image processing data.
- the network interface of the computer device is used to communicate with an external terminal via a network connection. When the computer program is executed by a processor, an image processing method is realized.
- FIG. 5 is only a block diagram of a part of the structure related to the solution of this application, and does not constitute a limitation to the computer equipment on which the solution of this application is applied.
- the specific computer equipment can be More or fewer components than shown in the figures may be included, or some components may be combined, or have a different arrangement of components.
- a computer device including a memory and a processor, the memory stores a computer program, and the processor implements the steps in the foregoing method embodiments when executing the computer program.
- a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, the steps in the foregoing method embodiments are implemented.
- any references to memory, storage, database or other media used in the various embodiments provided in the present application may include at least one of non-volatile memory and volatile memory.
- Non-volatile memory may include read-only memory (Read-Only Memory, ROM), magnetic tape, floppy disk, flash memory or optical memory, etc.
- Volatile memory can include random access memory (Random Access Memory, RAM) or external cache memory.
- RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM).
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Physics & Mathematics (AREA)
- Evolutionary Computation (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- General Engineering & Computer Science (AREA)
- Computing Systems (AREA)
- Software Systems (AREA)
- Molecular Biology (AREA)
- Computational Linguistics (AREA)
- Biophysics (AREA)
- Biomedical Technology (AREA)
- Mathematical Physics (AREA)
- General Health & Medical Sciences (AREA)
- Health & Medical Sciences (AREA)
- Discrete Mathematics (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Biology (AREA)
- Image Analysis (AREA)
Abstract
涉及计算机处理技术领域，一种图像处理方法、装置、设备和存储介质，主要包括：获取对图像集进行聚类得到的第一连通图（S201）；按照所述第一连通图包含的各连接边的介数中心性测度由大到小的顺序，预测所述各连接边是否属于错误连接边（S202）；若预测到属于错误连接边的目标连接边，则获取所述第一连通图的重构连通图；所述重构连通图是根据所述第一连通图的边外图重构得到的；所述边外图包括所述第一连通图中所述目标连接边以外的图（S203）；根据所述第一连通图和所述重构连通图，确定所述图像集的第二连通图（S204），在节省计算资源和降低时间成本的同时保证图像聚类准确性。
Description
本申请涉及计算机处理技术领域，特别是涉及一种图像处理方法、装置、计算机设备和存储介质。
随着计算机处理技术的发展，出现了通过图(gragh)处理实现图像聚类的方法。该方法构造的节点连接关系图中，将节点连接关系图所包含的连通图的各节点表征的各图像作为同一类图像。
如果连通图中存在较多错误连接边，将非同类的图像的节点连接在一起，则会影响图像聚类结果的准确性。为保证图像聚类结果的准确性，在得到连通图后可以进一步判断连通图各连接边是否属于错误连接边。在一些图像聚类场景中，连通图可能存在成千上万条连接边，所耗费的计算资源较大。
发明内容
基于此，有必要针对上述技术问题，提供一种图像处理方法、装置、计算机设备和存储介质。
一种图像处理方法，所述方法包括：
获取对图像集进行聚类得到的第一连通图；
按照所述第一连通图包含的各连接边的介数中心性测度由大到小的顺序，预测所述各连接边是否属于错误连接边；
若预测到属于错误连接边的目标连接边，则获取所述第一连通图的重构连通图；所述重构连通图是根据所述第一连通图的边外图重构得到的；所述边外图包括所述第一连通图中所述目标连接边以外的图；
根据所述第一连通图和所述重构连通图，确定所述图像集的第二连通图。
在其中一个实施例中，所述根据所述第一连通图和所述重构连通图，确定所述图像集的第二连通图，包括：
若所述重构连通图的连通性为不连通，且所述重构连通图包含的各子连通图均为结构稠密，则将所述各子连通图中未进行重构的子连通图和/或重构子连通图，作为所述图像集的第二连通图；所述重构子连通图不包含属于错误连接边的连接边或所述重构连通图的节点数低于节点数阈值。
在其中一个实施例中，所述方法还包括：
若所述子连通图的节点数高于所述节点数阈值且所述子连通图包含属于错误连接边的连接边，则对所述子连通图进行重构。
在其中一个实施例中，所述根据所述第一连通图和所述重构连通图，确定所述图像集的第二连通图，包括：
若所述重构连通图的连通性为不连通，且所述重构连通图包含至少一个结构稀疏的子连通图，且所述目标连接边为顺序预测的最后一条连接边，则将所述第一连通图作为所述图像集的第二连通图。
在其中一个实施例中，所述根据所述第一连通图和所述重构连通图，确定所述图像集的第二连通图，包括：
若所述重构连通图的连通性为连通，且所述目标连接边是顺序预测的最后一条连接边，则将所述重构连通图作为所述图像集的第二连通图。
在其中一个实施例中，所述按照所述第一连通图包含的各连接边的介数中心性测度由大到小的顺序，预测所述各连接边是否属于错误连接边，包括：
按照所述顺序确定所述各连接边中的待预测连接边；
获取所述待预测连接边的特征信息；所述特征信息包括与所述待预测连接边对应的节点集中的节点间的连接关系，以及所述节点集中各节点的标签值；所述节点集包括由所述待预测连接边连接的两个目标节点、两个目标节点各自的一阶邻居节点和二阶邻居节点；针对所述节点集中的任两个邻居节点，具有较小标签值的邻居节点至各目标节点的最短路径之和小于具有较大标签值的邻居节点至各目标节点的最短路径之和；或，针对所述节点集中的任两个邻居节点，在具有较小标签值的邻居节点至各目标节点的最短路径之和等于具有较大标签值的邻居节点至各目标节点的最短路径之和的情况下，具有较小标签值的邻居节点至各目标节点的最短路径中最短的最短路径小于具有较大标签值的邻 居节点至各目标节点的最短路径中最短的最短路径；所述邻居节点的标签值越小，所述邻居节点对图卷积网络预测所述待预测连接边是否属于错误连接边的贡献度越大；
将所述特征信息输入至预先构建的图卷积网络；
获取所述图卷积网络基于所述特征信息预测并输出的所述待预测连接边属于错误连接边的概率值；
基于所述概率值确定所述待预测连接边是否为错误连接边。
在其中一个实施例中，所述获取对图像集进行聚类得到的第一连通图，包括：
将所述图像集输入不同的卷积神经网络；
获取基于各卷积神经网络确定的所述图像集中每一图像对应的第一同类图集；
根据同一图像对应的由不同卷积神经网络确定的第一同类图集，得到所述图像集中各图像的第二同类图集；
根据所述各图像的第二同类图集表征的图像间的同类关系，构建所述第一连通图。
在其中一个实施例中，所述方法还包括：
获取将实际非同类图像确定为同类图像的第一次数，根据所述第一次数确定测试认假率；
获取将实际同类图像确定为非同类图像的第二次数，根据所述第二次数确定测试拒真率；
基于所述测试拒真率和测试认假率，调整确定同类图集所用的相似度阈值。
一种图像处理装置，所述装置包括：
连通图获取模块，用于获取对图像集进行聚类得到的第一连通图；
连接边预测模块，用于按照所述第一连通图包含的各连接边的介数中心性测度由大到小的顺序，预测所述各连接边是否属于错误连接边；
重构连通图获取模块，用于若预测到属于错误连接边的目标连接边，则获取所述第一连通图的重构连通图；所述重构连通图是根据所述第一连通图的边 外图重构得到的；所述边外图包括所述第一连通图中所述目标连接边以外的图；
连通图确定模块，用于根据所述第一连通图和所述重构连通图，确定所述图像集的第二连通图。
一种计算机设备，包括存储器和处理器，所述存储器存储有计算机程序，所述处理器执行所述计算机程序时实现上述的方法。
一种计算机可读存储介质，其上存储有计算机程序，所述计算机程序被处理器执行时实现上述的方法。
上述图像处理方法、装置、计算机设备和存储介质，获取对图像集进行聚类得到的第一连通图；按照所述第一连通图包含的各连接边的介数中心性测度由大到小的顺序，预测所述各连接边是否属于错误连接边；若预测到属于错误连接边的目标连接边，则获取所述第一连通图的重构连通图；所述重构连通图是根据所述第一连通图的边外图重构得到的；所述边外图包括所述第一连通图中所述目标连接边以外的图；根据所述第一连通图和所述重构连通图，确定所述图像集的第二连通图。
上述方法中，根据第一连通图的各连接边的介数中心性测度的大小确定各连接边预测顺序，若连接边的介数中心性测度较大，则表明该连接边可能是连接不同簇的重要连接边，因此，优先预测该介数中心性测度较大的连接边，节省计算资源，降低时间成本；若预测到该连接边为错误连接边，则说明该连接边不应存在且该连接边可能连接不同簇，接着根据该第一连通图中除错误连接边外的图重构，得到重构连通图，并根据第一连通图和重构连通图确定图像集的第二连通图，后续可以将该第二连通图中各节点表征的图像作为同一类图像，得到准确性较高的图像聚类结果。
图1为一个实施例中图像处理方法的流程示意图；
图2为一个实施例中图像处理方法的流程示意图；
图3为一个实施例中图像处理方法的流程示意图；
图4为一个实施例中图像处理装置的结构框图；
图5为一个实施例中计算机设备的内部结构图。
为了使本申请的目的、技术方案及优点更加清楚明白，以下结合附图及实施例，对本申请进行进一步详细说明。应当理解，此处描述的具体实施例仅仅用以解释本申请，并不用于限定本申请。
在本申请中提及“实施例”意味着，结合实施例描述的特定特征、结构或特性可以包含在本申请的至少一个实施例中。在说明书中的各个位置出现该短语并不一定均是指相同的实施例，也不是与其它实施例互斥的独立的或备选的实施例。本领域技术人员显式地和隐式地理解的是，本申请所描述的实施例可以与其它实施例相结合。
深度学习在人工智能领域得到许多突破，例如在计算机视觉、自然语言处理等众多人工智能方向上都取得了显著成果，通过深度学习得到的各类神经网络模型被用来完成分类、识别等任务。而为获得分类、识别等效果优异的神经网络模型(以下简称模型)需要经过一个有监督的训练过程，即利用带标签的样本图像不断调整神经网络模型参数，使其达到所要求的性能。
在实际应用场景中获得大批未标注样本图像是比较容易的，而完全依靠人工手动进行标注往往会耗费大量人力和时间，并且因为缺乏足够的先验知识，利用人工标注可能会使得这些标签基于主观判断而具有倾向性。如果仅使用少量有标签的样本图像参与后续神经网络模型训练过程，难以获得性能良好的神经网络模型，甚至会导致训练出来的神经网络模型出现过拟合现象。但如果忽略这些未标注样本图像，则会浪费极大数据资源，丢失神经网络模型学习任务所需信息。对于数量巨大的无标签样本图像，考虑到人工标注效率极低，标注质量通常参差不齐，因此，有必要在实现基本标注功能的同时，提升样本图像的标注效率和准确性，实现智能化的样本图像标注。
智能化的样本图像标注的其中一种方式可以是：对样本图像进行聚类，将属于不同类别的样本图像分开，其中类别也被称作簇。基于此，通过给同一簇中的样本图像标注相同的虚拟标签，能够为神经网络模型学习任务提供训练数 据样本。图聚类方法是一种基于图划分理论的聚类算法，与其它聚类方法相比，图聚类算法具有明显优势：这类方法对数据分布的适应性更强，可以识别任意形状的聚类。基于图划分的图像聚类算法的核心是将图像聚类问题看成图分割问题，其主要思想包含两个部分：第一部分是构造样本图像的节点连接关系图(该节点连接关系图可以是通过提取样本图像的特征向量进行构建的)；第二部分是确定优化分割目标并求解节点连接关系图划分问题。
在第一部分构造节点连接关系图时，可以通过先验知识来对节点间关联关系进行评估，生成的邻接矩阵的精度与超参数的取值有关。然而，在实际场景中，收集到的样本图像往往是样本数量不均衡的非理想数据集。若设置的超参数不合理，容易出现节点连接关系图中部分连接边被永久删除或者存在较多错误连接边，影响图像聚类表现。
考虑到单个卷积神经网络(Convolution Neural Network,CNN)模型提取到的特征可能存在偏差，如图1所示，本申请将未标注样本图像集分别输入到不同的CNN，获得同一节点的不同邻居结构表示，在此基础上进行局部拓扑融合，从多个不同的角度更加精确地描述样本图像，使其更具有鉴别性，特征表达能力更强。
具体来说，计算机设备在获取对图像集进行聚类得到的第一连通图(即图2的S201)时，可以通过如下步骤实现：将所述图像集输入不同的卷积神经网络；获取基于各卷积神经网络确定的所述图像集中每一图像对应的第一同类图集；根据同一图像对应的由不同卷积神经网络确定的第一同类图集，得到所述图像集中各图像的第二同类图集；根据所述各图像的第二同类图集表征的图像间的同类关系，构建所述第一连通图。
计算机设备将图像集输入各CNN后，各CNN会提取图像集的每一图像的特征向量；针对图像集中的任一图像，对该图像的由CNN1输出的特征向量与其他图像的由该CNN1输出的特征向量进行相似度计算，并基于相似度的大小确定该其他图像是否为该图像的同类图像，也即得到该图像经由该CNN1确定的第一同类图集；同样地，还可以得到该图像经由CNN2和CNN3确定的第一同类图集。然后，对同一图像的由不同CNN确定的第一同类统计进行融合，得 到该同一图像的第二同类图集。
其中，针对两张图像特征向量的计算公式为：
其中，图像A的D维特征向量为：A＝[a
1,a
2,…,a
D]，图像B的D维特征向量为B＝[b
1,b
2,…,b
D]。
针对图像I，基于CNN1确定的图像I对应的第一同类图集为J、A和B，基于CNN2确定的图像I对应的第一同类图集为A、B和C，基于CNN3确定的图像I对应的第一同类图集为J、A和C，因此，计算机设备融合各卷积网络确定的图像I对应的第一图像图集，得到图像I对应的第二图像图集为J、A、B和C。接着，计算机设备基于图像I的第二图像图集表征的图集中各图像与图像I的同类关系，将图像I的节点i与第二图像图集的图像J、A、B和C对应的节点进行边连接，得到对应的节点连接图。
进一步地，若仅使用相似度难以确定不同节点之间的连接边存在与否，那么可以引入相似度阈值，用于评估两节点的相似程度。
也即，上述基于相似度的大小确定该其他图像是否为该图像的同类图像的步骤中，可以确定相似度是否大于相似度阈值，若是，则说明该其他图像与该图像的相似程度高，因此可以用边连接这两张图像的节点，得到对应的节点连接关系图。
考虑到不合理的相似度阈值会遗漏正确连接边，同时错误连接边也可能出现在节点连接图中，因此，利用拒真率和认假率设置合适的阈值，以建立邻居节点过滤模型；其中，认假率的计算公式为
(NIRA是类间测试的总次数，NFA是将实际非同类图像确定为同类图像的次数)，拒真率的计算公式为
(NGRA是类内测试的总次数，NFR是将实际同类图像确定为非同类图像的次数)。
也即，计算机设备可以获取将实际非同类图像确定为同类图像的第一次数，根据所述第一次数确定测试认假率；获取将实际同类图像确定为非同类图像的第二次数，根据所述第二次数确定测试拒真率；基于所述测试拒真率和测试认假率，调整确定同类图集所用的相似度阈值；其中，该同类图集可以是第一同类图集也可以第二同类图集。
同样地，针对图像集的其他图像的处理方式可以参考对图像I的处理方式，这里不赘述。
计算机设备在得到各图像对应的节点连接图后，将各图像对应的节点连接图拼接起来形成完整关系图(该完整关系图也是属于节点连接图)，若该完整关系图是一个连通图，那么可以直接计算该完整关系图的各连接边的介数中心性测度，得到介数中心性测度大的关键连接边i-j、k-l、m-n。如果该完整关系图包括多个连通图，那么可以分别计算各连通图的连接边的介数中心性测度。
接着，计算机设备按照所述第一连通图包含的各连接边的介数中心性测度由大到小的顺序，预测所述各连接边是否属于错误连接边(步骤S202)；
如果连通图中存在不同簇的错误连接边，那么该错误连接边相较于连通图包含的其他连接边而言，介数中心性测度较大，因此在预测各连接边是否为错误连接边时，可以优先预测介数中心性测度较大的。其中，连接边的介数中心性测度可以通过如下公式确定：
其中，V表示连通图包含的节点集合，σ(s,t)表示连接节点s和节点t之间最短路径的数量，σ(s,t|e)是连接两节点s和节点t之间的所有最短路径中经过连接边e的路径数。
若计算机设备预测到属于错误连接边的目标连接边，则获取所述第一连通图的重构连通图；所述重构连通图是根据所述第一连通图的边外图重构得到的；所述边外图包括所述第一连通图中所述目标连接边以外的图(步骤S203)；
本步骤中，如果预测到错误连接边，那么可以从第一连通图中删除该错误连接边，得到第一连通图除该错误连接边以外的图，并将该图作为重构连通图； 又例如，如果预测到错误连接边，计算机还可以获取第一连通图中除该错误连接边以外的图(即边外图)，并以边外图为准，重构一个与边外图一致的图，得到重构连通图。
计算机设备根据所述第一连通图和所述重构连通图，确定所述图像集的第二连通图(步骤S204)时，可以包括多种处理方式，后续介绍具体的多种处理方式。
上述图像处理方法中，根据第一连通图的各连接边的介数中心性测度的大小确定各连接边预测顺序，若连接边的介数中心性测度较大，则表明该连接边可能是连接不同簇的重要连接边，因此，优先预测该介数中心性测度较大的连接边，节省计算资源，降低时间成本；若预测到该连接边为错误连接边，则说明该连接边不应存在且该连接边可能连接不同簇，接着根据该第一连通图中除错误连接边外的图重构，得到重构连通图，并根据第一连通图和重构连通图确定图像集的第二连通图，后续可以将该第二连通图中各节点表征的图像作为同一类图像，得到准确性较高的图像聚类结果。
其中，计算机设备在预测各连接边是否属于错误连接边时，可以执行以下步骤：按照所述顺序确定所述各连接边中的待预测连接边；获取所述待预测连接边的特征信息；将所述特征信息输入至预先构建的图卷积网络(Graph Convolution Network,GCN)；获取所述图卷积网络基于所述特征信息预测并输出的所述待预测连接边属于错误连接边的概率值；基于所述概率值确定所述待预测连接边是否为错误连接边。
其中，所述特征信息包括与所述待预测连接边对应的节点集中的节点间的连接关系，以及所述节点集中各节点的标签值；所述节点集包括由所述待预测连接边连接的两个目标节点、两个目标节点各自的一阶邻居节点和二阶邻居节点，如图1所示的连接边i-j、k-l、m-n各自的二阶关系图中的节点。
针对所述节点集中的任两个邻居节点，具有较小标签值的邻居节点至各目标节点的最短路径之和小于具有较大标签值的邻居节点至各目标节点的最短路径之和；或，针对所述节点集中的任两个邻居节点，在具有较小标签值的邻居节点至各目标节点的最短路径之和等于具有较大标签值的邻居节点至各目标节 点的最短路径之和的情况下，具有较小标签值的邻居节点至各目标节点的最短路径中最短的最短路径小于具有较大标签值的邻居节点至各目标节点的最短路径中最短的最短路径。
以节点集包括的目标节点为i和j，节点集包括的邻居节点为x和y为例介绍上述内容：
记邻居节点x至各目标节点的最短路径为d(x,i)、d(x,j)，邻居节点y至各目标节点的最短路径为d(y,i)、d(y,j)。
若d(x,i)+d(x,j)<d(y,i)+d(y,j)，则为邻居节点y赋予较大的标签值，为邻居节点x赋予较小的标签值。
若d(x,i)+d(x,j)＝d(y,i)+d(y,j)，则进一步确定邻居节点x至各目标节点的最短路径中最短的最短路径与邻居节点y至各目标节点的最短路径中最短的最短路径的相对大小，若邻居节点x至各目标节点的最短路径d(x,i)、d(x,j)中最短的最短路径为d(x,i)，邻居节点y至各目标节点的最短路径d(y,i)、d(y,j)中最短的最短路径为d(y,j)，且d(x,i)>d(y,j)，那么为邻居节点x赋予较大的标签值，为邻居节点y赋予较小的标签值。
若节点集中的邻居节点仅为一个，那么可以为该邻居节点赋予大于目标节点的标签值的标签值。
所述邻居节点的标签值越小，所述邻居节点对图卷积网络预测所述待预测连接边是否属于错误连接边的贡献度越大。
接着，计算机设备按照介数中心性测度由大到小的顺序，将各连接边i-j、k-l、m-n对应的上述特征信息输入至图卷积网络中进行预测，以确定各关键连接边为错误连接边的概率值；基于所述概率值确定所述待预测连接边是否为错误连接边。
上述方式，考虑到实际采集到的未标注样本图像的数据分布复杂，以往设立的优化分割目标函数不再适用，而图卷积网络能够从图结构数据中提取特征，利用其它节点知识来推理目标节点的特征，因此采用GCN自动学习节点对之间的关联信息，从而解决关键边的存在性问题，提高预测准确性。
在一个实施例中，如图3所示，计算机设备可以执行如下步骤：
步骤S301至步骤S302，计算机设备将未标注样本图像集输入至多个CNN中，得到各未标注样本图像的特征向量；
步骤S303，基于不同CNN提取的特征向量，计算图像间的相似度是否大于相似度阈值，得到同一节点的不同邻居结构表示，在此基础上进行局部拓扑融合，得到每一样本图像对应的同类图集；
步骤S304，构造完整关系图(Complete-Graph)，该关系图包括多个连通图；
步骤S305，判断从完整关系图中得到的连通图集合是否为空集合；
步骤S306，从连通图集合中任选一个连通图；
步骤S307，计算选取的连通图(记为CS-Graph(i))的节点数Nnode是否大于或等于节点数阈值Tnode；若否，则将CS-Graph(i)放入聚类簇集合，不进行分割处理；若是，则进入步骤S308；
步骤S308，计算CS-Graph(i)中所有连接边的介数中心性测度，即重要性程度，并将所有连接边按照介数中心性测度从大到小排序，形成待预测连接边集合(记为Edge(i))；
步骤S309，判断Edge(i)是否为空集合，若是，则进行下一连通图的处理，若否，则进入步骤S310；
步骤S310，按照介数中心性测度从大到小排序从Edge(i)中取出介数中心性测度最大的待预测连接边，并将该待预测连接边的相关信息(即二阶关系图和节点信息矩阵)输入GCN中进行预测，获取GCN输出的该待预测连接边为错误连接边的概率值；
步骤S311，判断该待预测连接边为错误连接边的概率值是否大于概率阈值(记为Pr)；
步骤S312，若该待预测连接边为错误连接边的概率值大于概率阈值，则从选取的连通图中删除该待预测连接边；
步骤S313，判断删除该待预测连接边的连通图(相当于重构连通图)的连通性是否改变。若否，则回到步骤S309，在Edge(i)不是空集合时进行下一条连接边的预测，在Edge(i)是空集合时进行下一连通图的处理，直至全部连通图都被处理。
步骤S314，若连通性发生改变，即重构连通图的连通性为不连通，此时，说明重构连通图包括多个子连通图(即得到了新的连通图)并判断子连通图是否结构稀疏；其中，连通图的结构是否稀疏可以通过连通图的连接边数量确定，如果连通图的连接边数量小于阈值，则认为连通图结构稀疏，如果连通图的连接边数量大于或等于阈值，则认为连通图结构稠密。
步骤S315，若各子连通图结构稠密，则将子连通图放入连通图集合中进行分割处理；
步骤S316，若至少一个子连通图的结构稀疏，则恢复已删除的当前待预测连接边，并进行下一连接边的预测处理。
通过图3和上述介绍可以明确，步骤S204的具体处理方式可以包括以下几种：
(1)若所述重构连通图的连通性为不连通，且所述重构连通图包含的各子连通图均为结构稠密，则将所述各子连通图中未进行重构的子连通图和/或重构子连通图，作为所述图像集的第二连通图；所述重构子连通图不包含属于错误连接边的连接边或所述重构连通图的节点数低于节点数阈值。
进一步地，如果所述子连通图的节点数高于所述节点数阈值且所述子连通图包含属于错误连接边的连接边，则对所述子连通图进行重构。
(2)若所述重构连通图的连通性为不连通，且所述重构连通图包含至少一个结构稀疏的子连通图，且所述目标连接边为顺序预测的最后一条连接边，则将所述第一连通图作为所述图像集的第二连通图。
(3)若所述重构连通图的连通性为连通，且所述目标连接边是顺序预测的最后一条连接边，则将所述重构连通图作为所述图像集的第二连通图。
通过上述方法，对连通图进行持续分割处理，直至得到满足上述几种情况的连通图，将该连通图作为第二连通图，也即得到聚类簇集合，将聚类簇集合中同一连通图的节点代表的图像作为同一类图像，并为同一类图像赋予相同的标注，实现智能化的图像标注。
应该理解的是，虽然图1至图3的流程图中的各个步骤按照箭头的指示依次显示，但是这些步骤并不是必然按照箭头指示的顺序依次执行。除非本文中 有明确的说明，这些步骤的执行并没有严格的顺序限制，这些步骤可以以其它的顺序执行。而且，图1至图3中的至少一部分步骤可以包括多个步骤或者多个阶段，这些步骤或者阶段并不必然是在同一时刻执行完成，而是可以在不同的时刻执行，这些步骤或者阶段的执行顺序也不必然是依次进行，而是可以与其它步骤或者其它步骤中的步骤或者阶段的至少一部分轮流或者交替地执行。
在一个实施例中，如图4所示，提供了一种图像处理装置，包括：
连通图获取模块401，用于获取对图像集进行聚类得到的第一连通图；
连接边预测模块402，用于按照所述第一连通图包含的各连接边的介数中心性测度由大到小的顺序，预测所述各连接边是否属于错误连接边；
重构连通图获取模块403，用于若预测到属于错误连接边的目标连接边，则获取所述第一连通图的重构连通图；所述重构连通图是根据所述第一连通图的边外图重构得到的；所述边外图包括所述第一连通图中所述目标连接边以外的图；
连通图确定模块404，用于根据所述第一连通图和所述重构连通图，确定所述图像集的第二连通图。
在一个实施例中，所述连通图确定模块404，还用于若所述重构连通图的连通性为不连通，且所述重构连通图包含的各子连通图均为结构稠密，则将所述各子连通图中未进行重构的子连通图和/或重构子连通图，作为所述图像集的第二连通图；所述重构子连通图不包含属于错误连接边的连接边或所述重构连通图的节点数低于节点数阈值。
在一个实施例中，所述装置还包括连通图重构模块，用于若所述子连通图的节点数高于所述节点数阈值且所述子连通图包含属于错误连接边的连接边，则对所述子连通图进行重构。
在一个实施例中，所述连通图确定模块404，还用于若所述重构连通图的连通性为不连通，且所述重构连通图包含至少一个结构稀疏的子连通图，且所述目标连接边为顺序预测的最后一条连接边，则将所述第一连通图作为所述图像集的第二连通图。
在一个实施例中，所述连通图确定模块404，还用于若所述重构连通图的连 通性为连通，且所述目标连接边是顺序预测的最后一条连接边，则将所述重构连通图作为所述图像集的第二连通图。
在一个实施例中，所述连接边预测模块402，还用于按照所述顺序确定所述各连接边中的待预测连接边；获取所述待预测连接边的特征信息；所述特征信息包括与所述待预测连接边对应的节点集中的节点间的连接关系，以及所述节点集中各节点的标签值；所述节点集包括由所述待预测连接边连接的两个目标节点、两个目标节点各自的一阶邻居节点和二阶邻居节点；针对所述节点集中的任两个邻居节点，具有较小标签值的邻居节点至各目标节点的最短路径之和小于具有较大标签值的邻居节点至各目标节点的最短路径之和；或，针对所述节点集中的任两个邻居节点，在具有较小标签值的邻居节点至各目标节点的最短路径之和等于具有较大标签值的邻居节点至各目标节点的最短路径之和的情况下，具有较小标签值的邻居节点至各目标节点的最短路径中最短的最短路径小于具有较大标签值的邻居节点至各目标节点的最短路径中最短的最短路径；所述邻居节点的标签值越小，所述邻居节点对图卷积网络预测所述待预测连接边是否属于错误连接边的贡献度越大；将所述特征信息输入至预先构建的图卷积网络；获取所述图卷积网络基于所述特征信息预测并输出的所述待预测连接边属于错误连接边的概率值；基于所述概率值确定所述待预测连接边是否为错误连接边。
在一个实施例中，所述连通图获取模块401，还用于将所述图像集输入不同的卷积神经网络；获取基于各卷积神经网络确定的所述图像集中每一图像对应的第一同类图集；根据同一图像对应的由不同卷积神经网络确定的第一同类图集，得到所述图像集中各图像的第二同类图集；根据所述各图像的第二同类图集表征的图像间的同类关系，构建所述第一连通图。
在一个实施例中，所述装置还包括阈值调整模块，用于获取将实际非同类图像确定为同类图像的第一次数，根据所述第一次数确定测试认假率；获取将实际同类图像确定为非同类图像的第二次数，根据所述第二次数确定测试拒真率；基于所述测试拒真率和测试认假率，调整确定同类图集所用的相似度阈值。
关于图像处理装置的具体限定可以参见上文中对于图像处理方法的限定， 在此不再赘述。上述图像处理装置中的各个模块可全部或部分通过软件、硬件及其组合来实现。上述各模块可以硬件形式内嵌于或独立于计算机设备中的处理器中，也可以以软件形式存储于计算机设备中的存储器中，以便于处理器调用执行以上各个模块对应的操作。
在一个实施例中，提供了一种计算机设备，该计算机设备可以是服务器，其内部结构图可以如图5所示。该计算机设备包括通过系统总线连接的处理器、存储器和网络接口。其中，该计算机设备的处理器用于提供计算和控制能力。该计算机设备的存储器包括非易失性存储介质、内存储器。该非易失性存储介质存储有操作系统、计算机程序和数据库。该内存储器为非易失性存储介质中的操作系统和计算机程序的运行提供环境。该计算机设备的数据库用于存储图像处理数据。该计算机设备的网络接口用于与外部的终端通过网络连接通信。该计算机程序被处理器执行时以实现一种图像处理方法。
本领域技术人员可以理解，图5中示出的结构，仅仅是与本申请方案相关的部分结构的框图，并不构成对本申请方案所应用于其上的计算机设备的限定，具体的计算机设备可以包括比图中所示更多或更少的部件，或者组合某些部件，或者具有不同的部件布置。
在一个实施例中，提供了一种计算机设备，包括存储器和处理器，存储器存储有计算机程序，处理器执行计算机程序时实现上述各个方法实施例中的步骤。
在一个实施例中，提供了一种计算机可读存储介质，其上存储有计算机程序，计算机程序被处理器执行时实现上述各个方法实施例中的步骤。
本领域普通技术人员可以理解实现上述实施例方法中的全部或部分流程，是可以通过计算机程序来指令相关的硬件来完成，上述的计算机程序可存储于一非易失性计算机可读取存储介质中，该计算机程序在执行时，可包括如上述各方法的实施例的流程。其中，本申请所提供的各实施例中所使用的对存储器、存储、数据库或其它介质的任何引用，均可包括非易失性和易失性存储器中的至少一种。非易失性存储器可包括只读存储器(Read-Only Memory，ROM)、磁带、软盘、闪存或光存储器等。易失性存储器可包括随机存取存储器(Random Access Memory，RAM)或外部高速缓冲存储器。作为说明而非局限，RAM可以是多种形式，比如静态随机存取存储器(Static Random Access Memory，SRAM)或动态随机存取存储器(Dynamic Random Access Memory，DRAM)等。
以上实施例的各技术特征可以进行任意的组合，为使描述简洁，未对上述实施例中的各个技术特征所有可能的组合都进行描述，然而，只要这些技术特征的组合不存在矛盾，都应当认为是本说明书记载的范围。
以上的实施例仅表达了本申请的几种实施方式，其描述较为具体和详细，但并不能因此而理解为对发明专利范围的限制。应当指出的是，对于本领域的普通技术人员来说，在不脱离本申请构思的前提下，还可以做出若干变形和改进，这些都属于本申请的保护范围。因此，本申请专利的保护范围应以所附权利要求为准。
Claims (11)
- 一种图像处理方法，其特征在于，所述方法包括：获取对图像集进行聚类得到的第一连通图；按照所述第一连通图包含的各连接边的介数中心性测度由大到小的顺序，预测所述各连接边是否属于错误连接边；若预测到属于错误连接边的目标连接边，则获取所述第一连通图的重构连通图；所述重构连通图是根据所述第一连通图的边外图重构得到的；所述边外图包括所述第一连通图中所述目标连接边以外的图；根据所述第一连通图和所述重构连通图，确定所述图像集的第二连通图。
- 根据权利要求1所述的方法，其特征在于，所述根据所述第一连通图和所述重构连通图，确定所述图像集的第二连通图，包括：若所述重构连通图的连通性为不连通，且所述重构连通图包含的各子连通图均为结构稠密，则将所述各子连通图中未进行重构的子连通图和/或重构子连通图，作为所述图像集的第二连通图；所述重构子连通图不包含属于错误连接边的连接边或所述重构连通图的节点数低于节点数阈值。
- 根据权利要求2所述的方法，其特征在于，所述方法还包括：若所述子连通图的节点数高于所述节点数阈值且所述子连通图包含属于错误连接边的连接边，则对所述子连通图进行重构。
- 根据权利要求1所述的方法，其特征在于，所述根据所述第一连通图和所述重构连通图，确定所述图像集的第二连通图，包括：若所述重构连通图的连通性为不连通，且所述重构连通图包含至少一个结构稀疏的子连通图，且所述目标连接边为顺序预测的最后一条连接边，则将所述第一连通图作为所述图像集的第二连通图。
- 根据权利要求1所述的方法，其特征在于，所述根据所述第一连通图和所述重构连通图，确定所述图像集的第二连通图，包括：若所述重构连通图的连通性为连通，且所述目标连接边是顺序预测的最后一条连接边，则将所述重构连通图作为所述图像集的第二连通图。
- 根据权利要求1所述的方法，其特征在于，所述按照所述第一连通图包含的各连接边的介数中心性测度由大到小的顺序，预测所述各连接边是否属于错 误连接边，包括：按照所述顺序确定所述各连接边中的待预测连接边；获取所述待预测连接边的特征信息；所述特征信息包括与所述待预测连接边对应的节点集中的节点间的连接关系，以及所述节点集中各节点的标签值；所述节点集包括由所述待预测连接边连接的两个目标节点、两个目标节点各自的一阶邻居节点和二阶邻居节点；针对所述节点集中的任两个邻居节点，具有较小标签值的邻居节点至各目标节点的最短路径之和小于具有较大标签值的邻居节点至各目标节点的最短路径之和；或，针对所述节点集中的任两个邻居节点，在具有较小标签值的邻居节点至各目标节点的最短路径之和等于具有较大标签值的邻居节点至各目标节点的最短路径之和的情况下，具有较小标签值的邻居节点至各目标节点的最短路径中最短的最短路径小于具有较大标签值的邻居节点至各目标节点的最短路径中最短的最短路径；所述邻居节点的标签值越小，所述邻居节点对图卷积网络预测所述待预测连接边是否属于错误连接边的贡献度越大；将所述特征信息输入至图卷积网络；获取所述图卷积网络基于所述特征信息预测并输出的所述待预测连接边属于错误连接边的概率值；基于所述概率值确定所述待预测连接边是否为错误连接边。
- 根据权利要求1至6任一项所述的方法，其特征在于，所述获取对图像集进行聚类得到的第一连通图，包括：将所述图像集输入不同的卷积神经网络；获取基于各卷积神经网络确定的所述图像集中每一图像对应的第一同类图集；根据同一图像对应的由不同卷积神经网络确定的第一同类图集，得到所述图像集中各图像的第二同类图集；根据所述各图像的第二同类图集表征的图像间的同类关系，构建所述第一连通图。
- 根据权利要求7所述的方法，其特征在于，所述方法还包括：获取将实际非同类图像确定为同类图像的第一次数，根据所述第一次数确定测试认假率；获取将实际同类图像确定为非同类图像的第二次数，根据所述第二次数确定测试拒真率；基于所述测试拒真率和测试认假率，调整确定同类图集所用的相似度阈值。
- 一种图像处理装置，其特征在于，所述装置包括：连通图获取模块，用于获取对图像集进行聚类得到的第一连通图；连接边预测模块，用于按照所述第一连通图包含的各连接边的介数中心性测度由大到小的顺序，预测所述各连接边是否属于错误连接边；重构连通图获取模块，用于若预测到属于错误连接边的目标连接边，则获取所述第一连通图的重构连通图；所述重构连通图是根据所述第一连通图的边外图重构得到的；所述边外图包括所述第一连通图中所述目标连接边以外的图；连通图确定模块，用于根据所述第一连通图和所述重构连通图，确定所述图像集的第二连通图。
- 一种计算机设备，包括存储器和处理器，所述存储器存储有计算机程序，其特征在于，所述处理器执行所述计算机程序时实现权利要求1至8中任一项所述的方法。
- 一种计算机可读存储介质，其上存储有计算机程序，其特征在于，所述计算机程序被处理器执行时实现权利要求1至8中任一项所述的方法。
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110718310.7 | 2021-06-28 | ||
| CN202110718310.7A CN113627464B (zh) | 2021-06-28 | 2021-06-28 | 图像处理方法、装置、设备和存储介质 |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| WO2023273171A1 true WO2023273171A1 (zh) | 2023-01-05 |
Family
ID=78378455
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| PCT/CN2021/136446 WO2023273171A1 (zh) | 2021-06-28 | 2021-12-08 | 图像处理方法、装置、设备和存储介质 |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN113627464B (zh) |
| WO (1) | WO2023273171A1 (zh) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113627464B (zh) * | 2021-06-28 | 2024-02-27 | 广州广电运通金融电子股份有限公司 | 图像处理方法、装置、设备和存储介质 |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20040013305A1 (en) * | 2001-11-14 | 2004-01-22 | Achi Brandt | Method and apparatus for data clustering including segmentation and boundary detection |
| CN106991132A (zh) * | 2017-03-08 | 2017-07-28 | 南京信息工程大学 | 一种基于图集重构与图核降维的图分类方法 |
| CN110097112A (zh) * | 2019-04-26 | 2019-08-06 | 大连理工大学 | 一种基于重构图的图学习模型 |
| CN110633734A (zh) * | 2019-08-22 | 2019-12-31 | 成都信息工程大学 | 一种基于图论相关理论进行异常检测的方法 |
| CN112115996A (zh) * | 2020-09-11 | 2020-12-22 | 北京达佳互联信息技术有限公司 | 图像数据的处理方法、装置、设备及存储介质 |
| CN113627464A (zh) * | 2021-06-28 | 2021-11-09 | 广州广电运通金融电子股份有限公司 | 图像处理方法、装置、设备和存储介质 |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110727836B (zh) * | 2019-12-17 | 2020-04-07 | 南京华飞数据技术有限公司 | 基于Spark GraphX的社交网络分析系统及其实现方法 |
-
2021
- 2021-06-28 CN CN202110718310.7A patent/CN113627464B/zh active Active
- 2021-12-08 WO PCT/CN2021/136446 patent/WO2023273171A1/zh unknown
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20040013305A1 (en) * | 2001-11-14 | 2004-01-22 | Achi Brandt | Method and apparatus for data clustering including segmentation and boundary detection |
| CN106991132A (zh) * | 2017-03-08 | 2017-07-28 | 南京信息工程大学 | 一种基于图集重构与图核降维的图分类方法 |
| CN110097112A (zh) * | 2019-04-26 | 2019-08-06 | 大连理工大学 | 一种基于重构图的图学习模型 |
| CN110633734A (zh) * | 2019-08-22 | 2019-12-31 | 成都信息工程大学 | 一种基于图论相关理论进行异常检测的方法 |
| CN112115996A (zh) * | 2020-09-11 | 2020-12-22 | 北京达佳互联信息技术有限公司 | 图像数据的处理方法、装置、设备及存储介质 |
| CN113627464A (zh) * | 2021-06-28 | 2021-11-09 | 广州广电运通金融电子股份有限公司 | 图像处理方法、装置、设备和存储介质 |
Non-Patent Citations (1)
| Title |
|---|
| TONG ZONGHE, YUAN LINING; WANG YANG: "Theory and application of Convolutional Neural Network", INFORMATION TECHNOLOGY AND INFORMATIZATION - XINXI JISHU YU XINXIHUA, SHANDONG DIANZI XUEHUI, CN, no. 2, 28 February 2020 (2020-02-28), CN , pages 187 - 192, XP093018769, ISSN: 1672-9528, DOI: 10.3969/j.issn.1672-9528.2020.02.061 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113627464A (zh) | 2021-11-09 |
| CN113627464B (zh) | 2024-02-27 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| WO2020259582A1 (zh) | 神经网络模型的训练方法、装置和电子设备 | |
| WO2022068196A1 (zh) | 跨模态的数据处理方法、装置、存储介质以及电子装置 | |
| CN111914644B (zh) | 一种基于双模态协同的弱监督时序动作定位方法及系统 | |
| CN109388565B (zh) | 基于生成式对抗网络的软件系统性能优化方法 | |
| CN112685504B (zh) | 一种面向生产过程的分布式迁移图学习方法 | |
| CN111047563B (zh) | 一种应用于医学超声图像的神经网络构建方法 | |
| CN112819023A (zh) | 样本集的获取方法、装置、计算机设备和存储介质 | |
| CN113378913A (zh) | 一种基于自监督学习的半监督节点分类方法 | |
| Fan et al. | Federated few-shot learning with adversarial learning | |
| CN112528845B (zh) | 一种基于深度学习的物理电路图识别方法及其应用 | |
| WO2022062419A1 (zh) | 基于非督导金字塔相似性学习的目标重识别方法及系统 | |
| US20230056760A1 (en) | Method and apparatus for processing graph data, device, storage medium, and program product | |
| CN110751191A (zh) | 一种图像的分类方法及系统 | |
| CN113568954A (zh) | 网络流量预测数据预处理阶段的参数最优化方法及系统 | |
| WO2023273171A1 (zh) | 图像处理方法、装置、设备和存储介质 | |
| CN115114409A (zh) | 一种基于软参数共享的民航不安全事件联合抽取方法 | |
| CN117034100A (zh) | 基于分层池化架构自适应图分类方法、系统、设备和介质 | |
| CN110830291A (zh) | 一种基于元路径的异质信息网络的节点分类方法 | |
| CN114897085A (zh) | 一种基于封闭子图链路预测的聚类方法及计算机设备 | |
| CN113392867A (zh) | 一种图像识别方法、装置、计算机设备及存储介质 | |
| CN114610758A (zh) | 基于数据仓库的数据处理方法、装置、可读介质及设备 | |
| CN111160077A (zh) | 一种大规模人脸动态聚类方法 | |
| CN113590720A (zh) | 数据分类方法、装置、计算机设备和存储介质 | |
| CN113811897B (zh) | 神经网络模型的推理方法、装置、计算机设备和存储介质 | |
| CN111091198B (zh) | 一种数据处理方法及装置 |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| 121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 21948097 Country of ref document: EP Kind code of ref document: A1 |
|
| NENP | Non-entry into the national phase |
Ref country code: DE |