WO2024098742A1 - Image processing method, apparatus, electronic device, and storage medium - Google Patents

Image processing method, apparatus, electronic device, and storage medium Download PDF

Info

Publication number
WO2024098742A1
WO2024098742A1 PCT/CN2023/099726 CN2023099726W WO2024098742A1 WO 2024098742 A1 WO2024098742 A1 WO 2024098742A1 CN 2023099726 W CN2023099726 W CN 2023099726W WO 2024098742 A1 WO2024098742 A1 WO 2024098742A1
Authority
WO
WIPO (PCT)
Prior art keywords
tubular structure
edge
node
connecting edge
connection
Prior art date
Application number
PCT/CN2023/099726
Other languages
French (fr)
Chinese (zh)
Inventor
丁廉
龙云飞
朱森华
涂丹丹
Original Assignee
华为云计算技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 华为云计算技术有限公司 filed Critical 华为云计算技术有限公司
Publication of WO2024098742A1 publication Critical patent/WO2024098742A1/en

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/0464—Convolutional networks [CNN, ConvNet]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/11—Region-based segmentation

Definitions

  • the present application relates to the field of image processing technology, and in particular to an image processing method, device, electronic device and storage medium.
  • tubular structure tracking refers to the method of obtaining the topological structure of tubular structures (such as blood vessels, trachea, nerve axons and other tubular objects).
  • Tubular structure tracking is an important issue in the biomedical field, and has important applications in bronchoscopy, arterial interventional surgery, neuron tracking, etc.
  • the existing tubular structure tracking methods mainly include semi-automatic tracking methods based on the shortest path algorithm and fully automatic tracking methods based on the fast marching algorithm.
  • the semi-automatic tracking method based on the shortest path algorithm requires a lot of manual operation, which is time-consuming and costly; and the semi-automatic tracking method based on the shortest path algorithm and the fully automatic tracking method based on the fast marching algorithm will have disconnection problems when the signal is low or there is an interference signal.
  • an embodiment of the present application provides an image processing method, the method comprising: acquiring an image to be processed, the image to be processed comprising a tubular structure; determining a plurality of nodes in the tubular structure; connecting a first node among the plurality of nodes with at least one second node to obtain an adjusted tubular structure, wherein the second node is a node among the plurality of nodes other than the first node; and constructing at least one topological graph corresponding to the tubular structure according to the adjusted tubular structure.
  • an adjusted tubular structure is obtained by performing an over-connection operation on the nodes of the tubular structure in the processed image; at least one topological map corresponding to the tubular structure is constructed according to the adjusted tubular structure; automatic tracking of the tubular structure can be achieved without a large amount of manual operation, thereby reducing tracking time and tracking costs and improving tracking efficiency; and, by performing an over-connection operation on multiple nodes of the tubular structure, it can be ensured that the multiple nodes of the tubular structure can be connected.
  • an over-connection operation can be performed on each node in the tubular structure, thereby ensuring that each node of the tubular structure can be connected, solving the problem of disconnection in tubular structure tracking.
  • the method further includes: when there are multiple topological maps, screening the at least one topological map.
  • the multiple topological maps obtained are screened, and the screened topological maps can be used as the tracking results of the tubular structure, which can further improve the accuracy of the tracking results of the tubular structure.
  • the first node among the multiple nodes is connected to at least one second node to obtain an adjusted
  • the tubular structure comprises: connecting the first node with at least one of the second nodes within a preset range thereof to obtain the adjusted tubular structure.
  • constructing at least one topological graph corresponding to the tubular structure based on the adjusted tubular structure includes: constructing the at least one topological graph based on at least one node in the adjusted tubular structure, at least one connecting edge in the adjusted tubular structure and a feature corresponding to the at least one connecting edge.
  • the characteristics of the connecting edges can be characterized by the features corresponding to the connecting edges. According to the nodes, connecting edges and the features corresponding to the connecting edges in the adjusted tubular structure, more accurate connecting edges can be screened out, thereby constructing a more accurate topological map corresponding to the tubular structure.
  • constructing the at least one topological graph according to at least one node in the adjusted tubular structure, at least one connecting edge in the adjusted tubular structure, and the features corresponding to the at least one connecting edge includes: obtaining the weight of the at least one connecting edge according to the features corresponding to the at least one connecting edge; constructing the at least one topological graph according to the at least one node, the at least one connecting edge, and the weight of the at least one connecting edge.
  • the weight of the connecting edge is obtained according to the characteristics corresponding to the connecting edge in the adjusted tubular structure.
  • the larger the weight of the connecting edge the more likely it is that the connecting edge belongs to the topological graph corresponding to the tubular structure.
  • the smaller the weight the lower the possibility that the connecting edge belongs to the topological graph corresponding to the tubular structure. Therefore, a more accurate topological graph corresponding to the tubular structure can be constructed based on the nodes, connecting edges and the weights of the connecting edges in the adjusted tubular structure.
  • obtaining the weight of the at least one connecting edge according to the features corresponding to the at least one connecting edge includes: inputting the features corresponding to the at least one connecting edge into a preset model to obtain the probability that the at least one connecting edge belongs to the at least one topological graph; obtaining the weight of the at least one connecting edge according to the probability that the at least one connecting edge belongs to the at least one topological graph; wherein the preset model is trained based on the features corresponding to each connecting edge in the topological graph training sample.
  • the probability that each connecting edge belongs to the topological map corresponding to the tubular structure in the image to be processed is obtained, thereby obtaining the weight of each connecting edge, and converting the tubular structure tracking problem into the connecting edge prediction problem of the tubular structure.
  • the traditional iterative numerical calculation is converted into a neural network matrix calculation, which can greatly reduce the amount of calculation, reduce memory usage, shorten the tracking time, and thus improve the tracking efficiency of the tubular structure.
  • the feature corresponding to the at least one connecting edge indicates the geometric properties of the at least one connecting edge, and/or the geometric properties of at least one end node in the at least one connecting edge.
  • the features corresponding to the at least one connecting edge include: at least one item of the coordinates of at least one end node of the at least one connecting edge, the length of the at least one connecting edge, and the slope of the at least one connecting edge.
  • the feature corresponding to the connecting edge indicates the geometric characteristics of the connecting edge, and/or at least one end of the connecting edge
  • the geometric characteristics of the nodes can be used to more accurately predict the probability that the connecting edges belong to the topological graph corresponding to the tubular structure, so that a more accurate topological graph corresponding to the tubular structure can be constructed.
  • the preset model is a graph neural network model.
  • the graph neural network model can be used to more accurately learn the features corresponding to the connection edges.
  • the probability that the connection edge belongs to the topological map corresponding to the tubular structure can be more accurately predicted, thereby constructing a more accurate topological map corresponding to the tubular structure;
  • the graph neural network model can be used to transform the tubular structure tracking problem into the tubular structure connection edge prediction problem.
  • the traditional iterative numerical calculation is transformed into a neural network matrix calculation, which can greatly reduce the amount of calculation, reduce memory usage, shorten the tracking time, and thus improve the tracking efficiency of the tubular structure.
  • constructing the at least one topological graph according to the at least one node, the at least one connecting edge and the weight of the at least one connecting edge includes: constructing the at least one topological graph based on a minimum spanning tree algorithm according to the at least one node, the at least one connecting edge and the weight of the at least one connecting edge.
  • the ring structure in the adjusted tubular structure is removed based on the minimum spanning tree algorithm, and a tree topology graph corresponding to the tubular structure is constructed, which can reduce the computational complexity, shorten the tracking time, and improve the tracking efficiency.
  • the topology graph is a tree topology graph.
  • the topological map corresponding to the tubular structure in the biomedical field and other fields is usually a tree topological map, so the tree topological map corresponding to the tubular structure can be constructed as the tracking result of the tubular structure in the biomedical field and other fields.
  • constructing at least one topological graph corresponding to the tubular structure according to the at least one node, the at least one connecting edge and the weight of the at least one connecting edge includes: determining any connecting edge with the largest weight in the first connecting edge set according to the weight of the at least one connecting edge; wherein the initial state of the first connecting edge set includes the at least one connecting edge; when any connecting edge with the largest weight and the connecting edges in the second connecting edge set do not form a ring structure, adding any connecting edge with the largest weight to the second connecting edge set; wherein the initial state of the second connecting edge set is an empty set; removing any connecting edge with the largest weight from the first connecting edge set to update the first connecting edge set; and based on the updated first connecting edge set, repeatedly performing the above-mentioned determination of any connecting edge with the largest weight in the first connecting edge set and subsequent operations until the number of connecting edges in the second connecting edge set is N-1, where N is the number of the number of the
  • the topological graph corresponding to the constructed tubular structure is a tree topological graph; since the number of nodes of the tubular structure is N, when the second connecting edge set contains N-1 connecting edges, it can be ensured that each node of the tubular structure can be connected, thereby solving the problem of disconnection in tubular structure tracking; each time, any connecting edge with the largest weight in the first connecting edge set is selected to determine whether to add it to the second connecting edge set. Since the weight of the connecting edge is positively correlated with the probability that the connecting edge belongs to the topological graph corresponding to the tubular structure, this can ensure that the topological graph corresponding to the constructed tubular structure is more accurate.
  • determining a plurality of nodes in the tubular structure includes: extracting a skeleton line of the tubular structure; According to the skeleton line of the tubular structure, a plurality of nodes in the tubular structure are determined.
  • the nodes of the tubular structure can be determined by extracting the skeleton line of the tubular structure, providing a basis for subsequent tracking of the tubular structure.
  • an embodiment of the present application provides an image processing device, comprising: an acquisition module, used to acquire an image to be processed, wherein the image to be processed includes a tubular structure; a node determination module, used to determine multiple nodes in the tubular structure; a connection module, used to connect a first node among the multiple nodes with at least one second node to obtain an adjusted tubular structure, wherein the second node is a node among the multiple nodes other than the first node; and a construction module, used to construct at least one topological graph corresponding to the tubular structure according to the adjusted tubular structure.
  • an adjusted tubular structure is obtained by performing an over-connection operation on the nodes of the tubular structure in the processed image; at least one topological map corresponding to the tubular structure is constructed according to the adjusted tubular structure; automatic tracking of the tubular structure can be achieved without a large amount of manual operation, thereby reducing tracking time and tracking costs and improving tracking efficiency; and, by performing an over-connection operation on multiple nodes of the tubular structure, it can be ensured that the multiple nodes of the tubular structure can be connected.
  • an over-connection operation can be performed on each node in the tubular structure, thereby ensuring that each node of the tubular structure can be connected, solving the problem of disconnection in tubular structure tracking.
  • the device further includes: a screening module, configured to screen the at least one topology map when there are multiple at least one topology maps.
  • the multiple topological maps obtained are screened, and the screened topological maps can be used as the tracking results of the tubular structure, which can further improve the accuracy of the tracking results of the tubular structure.
  • connection module is further used to: connect the first node with at least one second node within a preset range thereof to obtain the adjusted tubular structure.
  • the construction module is also used to: construct the at least one topological graph based on at least one node in the adjusted tubular structure, at least one connecting edge in the adjusted tubular structure and features corresponding to the at least one connecting edge.
  • the characteristics of the connecting edges can be characterized by the features corresponding to the connecting edges. According to the nodes, connecting edges and the features corresponding to the connecting edges in the adjusted tubular structure, more accurate connecting edges can be screened out, thereby constructing a more accurate topological map corresponding to the tubular structure.
  • the construction module is further used to: obtain the weight of the at least one connection edge according to the characteristics corresponding to the at least one connection edge; and construct the at least one topological graph according to the at least one node, the at least one connection edge and the weight of the at least one connection edge.
  • the weight of the connection edge is obtained according to the features corresponding to the connection edge in the adjusted tubular structure.
  • the larger the weight of the connection edge the more likely it is that the connection edge belongs to the topological graph corresponding to the tubular structure.
  • the smaller the weight the more likely it is that the connection edge belongs to the topological graph corresponding to the tubular structure.
  • the possibility that the connection edge belongs to the topological graph corresponding to the tubular structure is lower; thus, according to the nodes, connection edges and connection edge weights in the adjusted tubular structure, a more accurate topological graph corresponding to the tubular structure can be constructed.
  • the construction module is further used to: input the features corresponding to the at least one connection edge into a preset model to obtain the probability that the at least one connection edge belongs to the at least one topological graph; obtain the weight of the at least one connection edge based on the probability that the at least one connection edge belongs to the at least one topological graph; wherein the preset model is trained based on the features corresponding to each connection edge in the topological graph training sample.
  • the probability that each connecting edge belongs to the topological map corresponding to the tubular structure in the image to be processed is obtained, thereby obtaining the weight of each connecting edge, and converting the tubular structure tracking problem into the connecting edge prediction problem of the tubular structure.
  • the traditional iterative numerical calculation is converted into a neural network matrix calculation, which can greatly reduce the amount of calculation, reduce memory usage, shorten the tracking time, and thus improve the tracking efficiency of the tubular structure.
  • the feature corresponding to the at least one connecting edge indicates the geometric characteristics of the at least one connecting edge, and/or the geometric characteristics of at least one end node in the at least one connecting edge.
  • the features corresponding to the at least one connecting edge include: at least one item of the coordinates of at least one end node of the at least one connecting edge, the length of the at least one connecting edge, and the slope of the at least one connecting edge.
  • the features corresponding to the connecting edge indicate the geometric properties of the connecting edge and/or the geometric properties of at least one end node in the connecting edge. These features can more accurately predict the probability that the connecting edge belongs to the topological graph corresponding to the tubular structure, thereby constructing a more accurate topological graph corresponding to the tubular structure.
  • the preset model is a graph neural network model.
  • the graph neural network model can be used to more accurately learn the features corresponding to the connection edges.
  • the probability that the connection edge belongs to the topological map corresponding to the tubular structure can be more accurately predicted, thereby constructing a more accurate topological map corresponding to the tubular structure;
  • the graph neural network model can be used to transform the tubular structure tracking problem into the tubular structure connection edge prediction problem.
  • the traditional iterative numerical calculation is transformed into a neural network matrix calculation, which can greatly reduce the amount of calculation, reduce memory usage, shorten the tracking time, and thus improve the tracking efficiency of the tubular structure.
  • the construction module is further used to: construct the at least one topological graph based on the minimum spanning tree algorithm according to the at least one node, the at least one connecting edge and the weight of the at least one connecting edge.
  • the ring structure in the over-connected topology graph is removed based on the minimum spanning tree algorithm, and a tree topology graph corresponding to the tubular structure is constructed, which can reduce the computational complexity, shorten the tracking time, and improve the tracking efficiency.
  • the topology graph is a tree topology graph.
  • the topological map corresponding to the tubular structure in the biomedical field and other fields is usually a tree topological map, so the tree topological map corresponding to the tubular structure can be constructed as the tracking result of the tubular structure in the biomedical field and other fields.
  • the construction module is further used to: determine the edge with the largest weight in the first connection edge set according to the weight of the at least one connection edge any connecting edge of; wherein, the initial state of the first connecting edge set includes the at least one connecting edge; when any connecting edge with the largest weight and the connecting edges in the second connecting edge set do not form a ring structure, any connecting edge with the largest weight is added to the second connecting edge set; wherein, the initial state of the second connecting edge set is an empty set; remove any connecting edge with the largest weight from the first connecting edge set to update the first connecting edge set; and based on the updated first connecting edge set, repeat the above-mentioned determination of any connecting edge with the largest weight in the first connecting edge set and subsequent operations until the number of connecting edges in the second connecting edge set is N-1, N is the number of the at least one node; based on the second connecting edge set and the at least one node, construct a topological graph corresponding to the
  • the topological graph corresponding to the constructed tubular structure is a tree topological graph; since the number of nodes of the tubular structure is N, when the second connecting edge set contains N-1 connecting edges, it can be ensured that each node of the tubular structure can be connected, thereby solving the problem of disconnection in tubular structure tracking; each time, any connecting edge with the largest weight in the first connecting edge set is selected to determine whether to add it to the second connecting edge set. Since the weight of the connecting edge is positively correlated with the probability that the connecting edge belongs to the topological graph corresponding to the tubular structure, this can ensure that the topological graph corresponding to the constructed tubular structure is more accurate.
  • the node determination module is further used to: extract the skeleton line of the tubular structure; and determine a plurality of nodes in the tubular structure according to the skeleton line of the tubular structure.
  • the nodes of the tubular structure can be determined by extracting the skeleton line of the tubular structure, providing a basis for subsequent tracking of the tubular structure.
  • an embodiment of the present application provides an electronic device, comprising: a processor; a memory for storing processor executable instructions; wherein the processor is configured to implement the first aspect or one or more image processing methods of the first aspect when executing the instructions.
  • an embodiment of the present application provides a computer-readable storage medium having computer program instructions stored thereon, which, when executed by a processor, implement the first aspect or one or more image processing methods of the first aspect.
  • an embodiment of the present application provides a computer program product, which, when executed on a computer, enables the computer to execute the above-mentioned first aspect or one or more of the image processing methods of the first aspect.
  • FIG1 is a schematic diagram showing a fully automatic tracking method based on a fast marching algorithm according to an embodiment of the present application.
  • FIG. 2 is a schematic diagram showing an application scenario of an image processing method according to an embodiment of the present application.
  • FIG3 shows a flow chart of an image processing method according to an embodiment of the present application.
  • FIG. 4 is a schematic diagram showing a method of constructing a connection edge according to an embodiment of the present application.
  • FIG5 shows a flow chart of an image processing method according to an embodiment of the present application.
  • FIG6 shows a flow chart of an image processing method according to an embodiment of the present application.
  • FIG. 7 is a schematic diagram showing a plurality of topological graphs corresponding to a tubular structure constructed according to an embodiment of the present application.
  • FIG8( a )-( b ) are schematic diagrams showing skeleton lines of extracted rat brain nerves according to an embodiment of the present application.
  • FIG. 9 shows a block diagram of an image processing apparatus according to an embodiment of the present application.
  • FIG10 is a schematic diagram showing the structure of an electronic device according to an embodiment of the present application.
  • Tubular structure tracking has important applications in the biomedical field.
  • FIG1 shows a schematic diagram of a fully automatic tracking method based on a fast marching algorithm according to an embodiment of the present application.
  • this method needs to traverse the entire image. For a tubular structure with N nodes, N-1 iterations are required, which requires a huge amount of calculation, large memory consumption, and a long time. For example, it takes about 1 hour to track a single zebrafish brain.
  • this method may mistakenly judge areas with noise or low signals in the image as nerve endings, resulting in disconnection problems and inaccurate tracking results. For example, if there is noise or low neural signal in the area where the connecting edge S7 in Figure 1 is located, the connecting edge S7 may be mistakenly judged as a nerve ending, so that the nerve after the connecting edge S7 is not tracked, resulting in disconnection problems.
  • an embodiment of the present application provides an image processing method.
  • the image processing method provided in the embodiments of the present application can be applied to tubular structure tracking scenarios in the biomedical field, including but not limited to the medical scenario of segmenting the pulmonary trachea, cerebral arteries, carotid arteries, peripheral arteries, etc.; and the brain science scenario of neural atlas tracking of mesoscopic or microscopic imaging data of various animal models (such as zebrafish, mice, monkeys).
  • FIG2 is a schematic diagram of an application scenario of an image processing method according to an embodiment of the present application.
  • an image 201 to be processed is input into an image processing device 202.
  • the image 201 to be processed may be a three-dimensional (3D) medical image or a two-dimensional (2D) medical image, without limitation.
  • the image 201 to be processed includes a tubular structure.
  • the image 201 to be processed is a 3D brain image of a mouse brain scanned by a microscope.
  • the image processing method provided by the embodiment of the present application is executed by the image processing device 202 (for detailed description, see below).
  • the tubular structure in the image 201 to be processed can be automatically tracked to obtain an image processing result 203, i.e., the tracking result of the tubular structure in the image 201 to be processed.
  • the image processing result 203 is a mouse brain nerve tracking result obtained by the image processing device 202 processing the mouse brain 3D brain image.
  • the embodiment of the present application does not limit the type of the image processing device 202 .
  • the image processing device 202 may be independently configured, integrated in other devices, or implemented through software or a combination of software and hardware.
  • the image processing device 202 may also be a device or system with data processing capabilities, or a component or chip set in these devices or systems.
  • the image processing device 202 may be a cloud server, a desktop computer, a portable computer, a network server, a personal digital assistant (PDA), a mobile phone, a tablet computer, a wireless terminal device, an embedded device, a medical device, or other device with data processing capabilities, or a component or chip in these devices.
  • PDA personal digital assistant
  • the image processing device 202 may also be a chip or a processor with processing functions, and the image processing device 202 may include multiple processors.
  • the processor may be a single-CPU processor or a multi-CPU processor.
  • FIG3 shows a flow chart of an image processing method according to an embodiment of the present application.
  • the method may be performed by the image processing device 202 in FIG2 .
  • the method may include the following steps:
  • the image to be processed may be a two-dimensional image, a three-dimensional image, etc.
  • the image to be processed may be a medical image, for example, a 3D lung CT scan image, a 3D brain image, etc.
  • the tubular structure may include blood vessels, trachea, nerve axons, etc.
  • the image to be processed may be preprocessed, and the preprocessing may include one or more operations such as image channel adjustment, image scaling, size adjustment, cropping, denoising, rotation transformation, image enhancement, non-target area exclusion or normalization.
  • the area corresponding to the tubular structure in the image to be processed can be extracted.
  • the image to be processed can be subjected to image segmentation processing to obtain the area corresponding to the tubular structure, so as to facilitate the subsequent tracking of the tubular structure.
  • the image to be processed can be input into a trained 3D U-Net convolutional neural network for image segmentation processing to obtain the area corresponding to the tubular structure, wherein the convolutional neural network can be trained in an existing manner.
  • the outer contour of the tubular structure in the image to be processed can be segmented by binarization and other methods, and the signal of the tubular structure in the image to be processed can be enhanced by distance transformation and other methods, thereby obtaining the area corresponding to the tubular structure.
  • S302 Determine a plurality of nodes in the tubular structure.
  • determining a plurality of nodes in the tubular structure may include:
  • the skeleton line of the tubular structure can be extracted by a skeleton extraction algorithm in the related art.
  • the skeleton line of the tubular structure can be extracted by using a skeleton extraction algorithm based on distance transformation, a skeleton extraction algorithm based on the largest disk, and the like.
  • a skeleton extraction algorithm based on the largest disk is used to generate multiple inscribed disks in the tubular structure, and the centers of the multiple inscribed disks are connected to obtain the skeleton line of the tubular structure.
  • the nodes in the tubular structure may include one or more of the endpoints, intersections, and inflection points of the skeleton line of the tubular structure.
  • all the endpoints, intersections, and inflection points in the skeleton line of the tubular structure may be determined as nodes in the tubular structure.
  • S303 Connect a first node among the multiple nodes with at least one second node to obtain an adjusted tubular structure, wherein the second node is a node among the multiple nodes except the first node.
  • At least one second node in the tubular structure can be selected to connect with the first node to construct at least one connection edge, wherein the first node is connected to a second node to construct a connection edge; the operation of connecting nodes in the tubular structure in this way can be called an over-connection operation.
  • an over-connection operation is performed on each of the multiple nodes, thereby constructing multiple connection edges, ensuring that multiple nodes can be connected, and the multiple connection edges and the multiple nodes can constitute the adjusted tubular structure.
  • FIG4 shows a schematic diagram of constructing connection edges according to an embodiment of the present application.
  • a connection operation is performed on each node of the tubular structure, and at least one connection edge can be constructed, and these connection edges and nodes can constitute the adjusted tubular structure.
  • it can be ensured that each node of the tubular structure has a connection relationship with at least one other node other than itself, and each node can be connected, thereby solving the problem of disconnection in tubular structure tracking.
  • a first node among multiple nodes can be connected to at least one second node within a preset range to obtain an adjusted tubular structure; in this way, for each of the multiple nodes, a connection edge corresponding to each node can be constructed, and these connection edges and the multiple nodes can constitute the adjusted tubular structure; wherein, the preset range can be set by a person skilled in the art as needed. For example, for each node in the tubular structure, nodes less than P pixels away from it can be connected to it respectively, and connection edges corresponding to each node can be constructed, and these connection edges and all nodes in the tubular structure can constitute the adjusted tubular structure.
  • a first node among multiple nodes can be connected to the M second nodes closest to it to obtain an adjusted tubular structure.
  • a connection edge corresponding to each node can be constructed, and these connection edges and the multiple nodes can constitute the adjusted tubular structure; wherein the value of M can be set by a person skilled in the art as needed.
  • the distance between other nodes and the node can be calculated, and the 5 second nodes closest to it can be selected to connect to them respectively, and the connection edge corresponding to each node can be constructed.
  • S304 Construct at least one topological graph corresponding to the tubular structure according to the adjusted tubular structure.
  • the topological graph includes the multiple nodes in the adjusted tubular structure, and may also include one or more connection edges corresponding to each of the nodes constructed above.
  • the topological graph may include each node of the tubular structure, and may also include one or more connection edges corresponding to each of the nodes.
  • the topological map is a tree topological map; in the fields of biomedicine and the like, the topological map corresponding to the tubular structure is usually a tree topological map, so a tree topological map corresponding to the tubular structure can be constructed as a tracking result of the tubular structure in the fields of biomedicine and the like.
  • constructing at least one topological graph corresponding to the tubular structure based on the adjusted tubular structure may include: constructing the at least one topological graph based on at least one node in the adjusted tubular structure, at least one connecting edge in the adjusted tubular structure, and a feature corresponding to the at least one connecting edge.
  • the feature corresponding to the at least one connecting edge indicates a geometric property of the at least one connecting edge and/or a geometric property of at least one end node in the at least one connecting edge.
  • the feature corresponding to the connecting edge may include the geometric feature of the connecting edge, or the geometric feature of at least one end node of the connecting edge, or both.
  • the feature corresponding to each connecting edge may include at least one of the coordinates of at least one end node of each connecting edge, the length of each connecting edge, and the slope of each connecting edge. The length of each connecting edge and the slope of each connecting edge may be determined based on the coordinates of the two end nodes of the connecting edge.
  • the characteristics of the connecting edges can be characterized by the features corresponding to the connecting edges. According to the nodes, connecting edges and the features corresponding to the connecting edges in the adjusted tubular structure, more accurate connecting edges can be screened out, thereby constructing a more accurate topological graph corresponding to the tubular structure.
  • constructing the at least one topological graph based on at least one node in the adjusted tubular structure, at least one connecting edge in the adjusted tubular structure, and a feature corresponding to the at least one connecting edge may include: obtaining a weight of the at least one connecting edge based on a feature corresponding to the at least one connecting edge; and constructing the at least one topological graph based on the at least one node, the at least one connecting edge, and the weight of the at least one connecting edge.
  • the weight of the connection edge is positively correlated with the probability that the connection edge belongs to the topological graph corresponding to the tubular structure.
  • the probability that the connection edge belongs to the topological graph corresponding to the tubular structure can be obtained based on the features corresponding to the connection edge, thereby obtaining the weight of the connection edge; examples of possible implementations of this process are shown below.
  • the weight of the connection edge is obtained based on the features corresponding to the connection edge in the adjusted tubular structure.
  • the larger the weight of the connection edge the more likely it is that the connection edge belongs to the topological graph corresponding to the tubular structure.
  • the smaller the weight the lower the probability that the connection edge belongs to the topological graph corresponding to the tubular structure.
  • constructing the at least one topological graph according to the at least one node, the at least one connecting edge and the weight of the at least one connecting edge may include: constructing the at least one topological graph based on a minimum spanning tree (MST) algorithm according to the at least one node, the at least one connecting edge and the weight of the at least one connecting edge.
  • MST minimum spanning tree
  • the adjusted tubular structure obtained in the above step S303 may include a ring structure
  • the ring structure in the adjusted tubular structure can be removed based on the minimum spanning tree algorithm to construct a tree topology graph corresponding to the tubular structure; thus, constructing a tree topology graph corresponding to the tubular structure based on the minimum spanning tree algorithm can reduce the computational complexity, shorten the tracking time, and improve the tracking efficiency.
  • a tree topology graph corresponding to the tubular structure can be constructed based on the Kruskal minimum spanning tree algorithm. At least one topological graph corresponding to the tubular structure.
  • the topological map may be used as the tracking result of the tubular structure.
  • the at least one topological map when there are multiple topological maps corresponding to the constructed tubular structure, the at least one topological map can be screened.
  • the multiple topological maps obtained can be screened, and the screened topological map can be used as the tracking result of the tubular structure, thereby further improving the accuracy of the tracking result of the tubular structure.
  • the weights of all the connecting edges in the topology can be added together to calculate the sum of the weights of the connecting edges of the topology; then the topology with the largest sum of the weights of the connecting edges can be selected as the tracking result of the tubular structure. In this way, by calculating the sum of the weights of the connecting edges of each topology, the topology with the highest sum of the weights of the connecting edges can be automatically screened.
  • the topology with the largest sum of the weights of the connecting edges can be selected as the tracking result of the tubular structure, which can improve the accuracy of the tracking result of the tubular structure.
  • the multiple topological maps obtained can be manually screened according to experience or actual needs, and a topological map obtained by screening is used as the tracking result of the tubular structure.
  • the user manually selects the multiple topological maps obtained and uses the selected topological map as the tracking result of the tubular structure, which can improve the accuracy and usability of the tubular structure tracking result.
  • the nodes of the tubular structure in the processed image are over-connected to obtain an adjusted tubular structure; based on the adjusted tubular structure, at least one topological map corresponding to the tubular structure is constructed; the tubular structure can be automatically tracked without a lot of manual operation, thus reducing the tracking time and tracking cost and improving the tracking efficiency; and, by over-connecting the multiple nodes of the tubular structure, it can be ensured that the multiple nodes of the tubular structure can be connected.
  • an over-connection operation can be performed on each node in the tubular structure, thereby ensuring that each node of the tubular structure can be connected, solving the problem of disconnection in tubular structure tracking.
  • the following is an exemplary description of a possible implementation method of obtaining the weight of at least one connecting edge according to the feature corresponding to at least one connecting edge.
  • FIG5 shows a flow chart of an image processing method according to an embodiment of the present application.
  • the method may be performed by the image processing device 202 in FIG2 .
  • the method may include the following steps:
  • the topological map training sample may include the connecting edges corresponding to each node of a tubular structure; the features corresponding to each connecting edge in the topological map training sample may be consistent with the feature type corresponding to at least one connecting edge in step S304 of Figure 3, for example, may include the coordinates of at least one end node of each connecting edge, the length of each connecting edge, and the slope of each connecting edge; the label corresponding to each connecting edge in the topological map training sample may be whether the connecting edge belongs to the topological map corresponding to the tubular structure. If the connecting edge belongs to the topological map corresponding to the tubular structure, the label corresponding to the connecting edge may be 1; if the connecting edge does not belong to the topological map corresponding to the tubular structure, the label corresponding to the connecting edge may be 0.
  • the preset model can be a trained graph neural network model, wherein the graph neural network refers to using a neural network to learn topological map data, extract and discover features in the topological map data, and meet the needs of clustering, classification, edge prediction, segmentation, A general term for algorithms that generate graph learning task requirements; for example, it can be a trained graph convolutional neural network model.
  • the graph neural network model can be trained by conventional training methods; the initial parameters of each model parameter in the graph neural network model can be default parameters.
  • the input can be the features corresponding to each connection edge in the topological map training sample, and the output can be the probability that the connection edge belongs to the topological map corresponding to the tubular structure;
  • the trained graph neural network model i.e., the preset model
  • the training end condition can be set by those skilled in the art according to actual needs; for example, the loss function value can be calculated based on the probability that the output connection edge belongs to the topological map corresponding to the tubular structure and the true probability that the connection edge belongs to the topological map corresponding to the tubular structure, and the loss function value can be used to adjust the parameters in the graph neural network model until the loss function converges, the training is completed, and the trained graph neural network model is obtained.
  • the features corresponding to the connection edges can be learned more accurately, and the features of each connection edge constructed above are input into the trained graph neural network model, which can more accurately predict the probability that each connection edge constructed above belongs to the topological map corresponding to the tubular structure.
  • the coordinates of at least one end node of each connection edge, the length of each connection edge, and the slope of each connection edge can be input into a trained graph neural network model to obtain the probability that each connection edge belongs to the topological graph corresponding to the tubular structure in the image to be processed. Since the coordinates of at least one end node of the connection edge, the length of the connection edge, and the slope of the connection edge can reflect the position information and geometric characteristics of the connection edge, by inputting these features into the trained graph neural network model, the probability that the connection edge belongs to the tubular structure can be more accurately predicted.
  • S502 Obtain a weight of at least one connection edge according to a probability that the at least one connection edge belongs to at least one topological graph corresponding to the tubular structure.
  • the weight may be positively correlated with the probability.
  • the probability that the connection edge belongs to the topological map corresponding to the tubular structure in the image to be processed may be used as the weight of the connection edge; as another example, the probability that the connection edge belongs to the topological map corresponding to the tubular structure in the image to be processed may be multiplied by a preset coefficient as the weight of the connection edge, and the preset coefficient may be set by a person skilled in the art.
  • the image processing method provided in an embodiment of the present application can compress the time for tracking a single zebrafish cranial nerve from 1 hour to 30 seconds compared with the tracking method based on the fast marching algorithm.
  • FIG6 shows a flow chart of an image processing method according to an embodiment of the present application.
  • the method may be performed by the image processing device 202 in FIG2 .
  • the method may include the following steps:
  • the connecting edges in the first connecting edge set can be sorted from large to small according to the weights of each connecting edge, so as to determine the connecting edge with the largest weight in the first connecting edge set; if there are multiple connecting edges with the largest weight in the first connecting edge set, any one connecting edge can be selected from them.
  • S604 Construct a topological graph corresponding to the tubular structure according to the second connection edge set and the at least one node.
  • a topological graph corresponding to the tubular structure can be constructed by all the connection edges in the second connection edge set and the nodes of the tubular structure.
  • the topological graph corresponding to the constructed tubular structure is a tree topological graph; since the number of nodes of the tubular structure is N, when the second connecting edge set contains N-1 connecting edges, it can be ensured that each node of the tubular structure can be connected, thereby solving the problem of disconnection in tubular structure tracking; each time, any connecting edge with the largest weight in the first connecting edge set is selected to determine whether to add it to the second connecting edge set. Since the weight of the connecting edge is positively correlated with the probability that the connecting edge belongs to the topological graph corresponding to the tubular structure, this can ensure that the topological graph corresponding to the constructed tubular structure is more accurate.
  • steps S601 to S604 may be performed multiple times. Since there may be multiple connection edges with the largest weight in the first connection edge set in step S601, multiple topological graphs corresponding to the tubular structure may be constructed.
  • Figure 7 shows a schematic diagram of constructing multiple topological graphs corresponding to a tubular structure according to an implementation of the present application.
  • the weight of each connection edge is the probability that the connection edge belongs to the topological graph corresponding to the tubular structure.
  • the topological graph corresponding to the tubular structure includes a node set and a connection edge set.
  • the node set is composed of all nodes of the tubular structure.
  • the initial state of the connection edge set i.e., the example of the second connection edge set mentioned above
  • the initial state of the connection edge set is an empty set; the connection edges can be sorted from large to small according to the weight of each connection edge, and judgment is performed in sequence according to the sorting of the connection edges.
  • connection edge and the connection edge included in the connection edge set of the topological graph corresponding to the tubular structure do not form a ring structure, the connection edge is added to the connection edge set until the connection edge set contains N-1 connection edges, and the judgment is stopped, where N is the number of nodes of the tubular structure. number; a topological graph corresponding to the tubular structure can be constructed according to the node set and the edge set at this time; when judging the edges with the same weight, the judgment can be made in different orders. For example, there are 4 edges with a weight of 0.9, namely edge a, edge b, edge c, and edge d.
  • edge a, edge b, edge c, and edge d whether each edge can be added to the edge set, or it can be judged in the order of edge d, edge c, edge b, and edge a whether each edge can be added to the edge set; the edges with the same weight can be judged in different orders, and finally different edge sets can be obtained, so that multiple different topological graphs corresponding to the tubular structure can be constructed, such as topological graphs 1 and 2 in Figure 7.
  • the above-mentioned image processing method provided in the embodiment of the present application can be used to automatically track rat brain nerves.
  • the image to be processed can be a 3D rat brain image scanned by a microscope, and the tubular structure in the image to be processed is the rat brain nerve.
  • the image to be processed is input into a trained segmentation model to obtain a rat brain nerve segmentation result;
  • the segmentation model can be a 3D U-Net network model, and the 3D U-Net network model can be trained based on a labeled rat brain nerve data set to obtain a trained segmentation model.
  • a skeleton extraction algorithm based on the maximum disk can be used to generate a series of inscribed disks in the rat brain nerve segmentation result, and the centers of the circles are connected to obtain the skeleton lines of the rat brain nerves.
  • Figure 8(a)-(b) shows a schematic diagram of extracting the skeleton lines of rat brain nerves according to an embodiment of the present application
  • Figure 8(a) shows a schematic diagram of extracting the skeleton lines of rat brain nerves according to an embodiment of the present application.
  • the cutting result is shown in FIG8(b) of the skeleton line of the rat brain nerve according to an embodiment of the present application. According to the skeleton line of the rat brain nerve, each node of the rat brain nerve can be determined.
  • the features corresponding to each connection edge in the over-connected neural topology map are input into the trained graph neural network model, and the probability that each connection edge belongs to the topology map corresponding to the rat brain nerve can be obtained; the features corresponding to the connection edge may include the coordinates of at least one end node of the connection edge, the length of the connection edge and the slope of the connection edge.
  • the probability that each connection edge belongs to the topology map corresponding to the rat brain nerve can be used as the weight of each connection edge, and at least one tree topology map corresponding to the rat brain nerve can be constructed based on the minimum spanning tree algorithm; the method for constructing a tree topology map based on the minimum spanning tree algorithm can refer to steps S601 to S604 in FIG6.
  • the tree topology graph can be used as the tracing result of the mouse brain nerves; when the number of tree topology graphs corresponding to mouse brain nerves is multiple, for each tree topology graph corresponding to the mouse brain nerves, the weights of all its connecting edges can be added up, and the sum of the connecting edge weights of each tree topology graph can be calculated. By comparing the sums of the connecting edge weights of each tree topology graph, the tree topology graph with the largest sum of the connecting edge weights can be used as the tracing result of the mouse brain nerves, thereby realizing end-to-end automatic tracking of the tubular structure.
  • the image processing method provided in the embodiment of the present application can ensure that each node of the tubular structure is connected by constructing an adjusted tubular structure, thereby avoiding the disconnection problem caused by weak signal or interference signal of the tubular structure; based on the graph neural network model, the probability of each connection edge belonging to the topological graph corresponding to the tubular structure is predicted, and the tubular structure tracking problem is converted into a tubular structure connection edge prediction problem.
  • the traditional iterative numerical calculation is converted into a neural network matrix calculation, which can greatly reduce the amount of calculation, reduce memory usage, shorten the tracking time, and improve the tracking efficiency of the tubular structure.
  • the embodiments of the present application further provide an image processing device, which can be used to execute the technical solution described in the above method embodiments.
  • the steps of the image processing method shown in FIG. 3, FIG. 5 or FIG. 6 can be executed.
  • Figure 9 shows a block diagram of an image processing device according to an embodiment of the present application.
  • the device may include: an acquisition module 901, used to acquire an image to be processed, wherein the image to be processed includes a tubular structure; a node determination module 902, used to determine multiple nodes in the tubular structure; a connection module 903, used to connect a first node among the multiple nodes with at least one second node to obtain an adjusted tubular structure, wherein the second node is a node among the multiple nodes other than the first node; a construction module 904, used to construct at least one topological graph corresponding to the tubular structure according to the adjusted tubular structure.
  • an adjusted tubular structure is obtained by performing an over-connection operation on the nodes of the tubular structure in the processed image; at least one topological map corresponding to the tubular structure is constructed based on the adjusted tubular structure; automatic tracking of the tubular structure can be achieved without a large amount of manual operation, thus reducing tracking time and tracking costs and improving tracking efficiency; and, by performing an over-connection operation on multiple nodes of the tubular structure, it can be ensured that the multiple nodes of the tubular structure can be connected.
  • an over-connection operation can be performed on each node in the tubular structure, thereby ensuring that each node of the tubular structure can be connected, thereby solving the problem of disconnection in tubular structure tracking.
  • the image processing device may further include: a screening module, configured to screen the at least one topological map when there are multiple at least one topological maps.
  • connection module 903 is further used to connect the first node with at least one second node within a preset range thereof to obtain the adjusted tubular structure.
  • the construction module 904 is further configured to: The at least one topological graph is constructed based on at least one node of the adjusted tubular structure, at least one connecting edge in the adjusted tubular structure, and a feature corresponding to the at least one connecting edge.
  • the construction module 904 is further used to: obtain the weight of the at least one connection edge according to the characteristics corresponding to the at least one connection edge; and construct the at least one topological graph according to the at least one node, the at least one connection edge and the weight of the at least one connection edge.
  • the construction module 904 is also used to: input the features corresponding to the at least one connection edge into a preset model to obtain the probability that the at least one connection edge belongs to the at least one topological graph; obtain the weight of the at least one connection edge based on the probability that the at least one connection edge belongs to the at least one topological graph; wherein the preset model is trained based on the features corresponding to each connection edge in the topological graph training sample.
  • the feature corresponding to the at least one connecting edge indicates a geometric property of the at least one connecting edge and/or a geometric property of at least one end node in the at least one connecting edge.
  • the preset model is a graph neural network model.
  • the construction module 904 is further used to: construct the at least one topological graph based on a minimum spanning tree algorithm according to the at least one node, the at least one connecting edge and the weight of the at least one connecting edge.
  • the topology map is a tree topology map.
  • the construction module 904 is also used to: determine any connecting edge with the largest weight in the first connecting edge set based on the weight of the at least one connecting edge; wherein the initial state of the first connecting edge set includes the at least one connecting edge; when any connecting edge with the largest weight and the connecting edges in the second connecting edge set do not form a ring structure, add any connecting edge with the largest weight to the second connecting edge set; wherein the initial state of the second connecting edge set is an empty set; remove any connecting edge with the largest weight from the first connecting edge set to update the first connecting edge set; and based on the updated first connecting edge set, repeat the above-mentioned determination of any connecting edge with the largest weight in the first connecting edge set and subsequent operations until the number of connecting edges in the second connecting edge set is N-1, where N is the number of the at least one node; and construct a topological graph corresponding to the tubular structure based on the second connecting edge set and the at least one node.
  • the node determination module 902 is further configured to: extract a skeleton line of the tubular structure; and determine a plurality of nodes in the tubular structure according to the skeleton line of the tubular structure.
  • the division of the modules in the above device is only a division of logical functions. In actual implementation, they can be fully or partially integrated into one physical entity, or they can be physically separated.
  • the modules in the device can be implemented in the form of a processor calling software; for example, the device includes a processor, the processor is connected to a memory, the memory stores instructions, and the processor calls the instructions stored in the memory to implement any of the above methods or to implement the functions of the modules of the device, wherein the processor is, for example, a general-purpose processor, such as a central processing unit (CPU) or a microprocessor, and the memory is a memory inside the device or a memory outside the device.
  • CPU central processing unit
  • microprocessor a microprocessor
  • the modules in the device can be implemented in the form of hardware circuits, and the functions of some or all of the modules can be implemented by designing the hardware circuits, and the hardware circuits can be understood as one or more processors; for example, in one implementation, the hardware circuit is an application-specific integrated circuit (ASIC), and the functions of some or all of the above modules are implemented by designing the logical relationship of the components in the circuit; for example, in another implementation, the hardware circuit can be implemented by a programmable logic device (PLD) to realize the present Taking a Field Programmable Gate Array (FPGA) as an example, it can include a large number of logic gate circuits, and the connection relationship between the logic gate circuits is configured through a configuration file to realize the functions of some or all of the above modules. All modules of the above device can be implemented in the form of a processor calling software, or in the form of hardware circuits, or in part in the form of a processor calling software, and the rest in the form of hardware circuits.
  • PLD programmable logic device
  • the processor is a circuit with the ability to process signals.
  • the processor may be a circuit with the ability to read and run instructions, such as a CPU, a microprocessor, a graphics processing unit (GPU), a digital signal processor (DSP), a neural-network processing unit (NPU), a tensor processing unit (TPU), etc.; in another implementation, the processor may implement certain functions through the logical relationship of a hardware circuit, and the logical relationship of the hardware circuit is fixed or reconfigurable, such as a hardware circuit implemented by an ASIC or PLD, such as an FPGA.
  • the process of the processor loading a configuration document to implement the hardware circuit configuration can be understood as the process of the processor loading instructions to implement the functions of some or all of the above modules.
  • each module in the above device can be one or more processors (or processing circuits) configured to implement the above embodiment method, such as: CPU, GPU, NPU, TPU, microprocessor, DSP, ASIC, FPGA, or a combination of at least two of these processor forms.
  • processors or processing circuits
  • each module in the above device can be fully or partially integrated together, or can be implemented independently, which is not limited.
  • the embodiment of the present application also provides an image processing device, comprising: a processor; a memory for storing instructions executable by the processor; wherein the processor is configured to implement the method of the above embodiment when executing the instructions.
  • an image processing device comprising: a processor; a memory for storing instructions executable by the processor; wherein the processor is configured to implement the method of the above embodiment when executing the instructions. Exemplarily, each step of the image processing method shown in FIG. 3, FIG. 5 or FIG. 6 can be executed.
  • FIG10 shows a schematic diagram of the structure of an electronic device according to an embodiment of the present application.
  • the electronic device may include: at least one processor 1001 , a communication line 1002 , a memory 1003 and at least one communication interface 1004 .
  • Processor 1001 can be a general-purpose central processing unit, a microprocessor, a specific application integrated circuit, or one or more integrated circuits for controlling the execution of the program of the present application; processor 1001 can also include a heterogeneous computing architecture of multiple general-purpose processors, for example, it can be a combination of at least two of CPU, GPU, microprocessor, DSP, ASIC, FPGA; as an example, processor 1001 can be CPU+GPU or CPU+ASIC or CPU+FPGA.
  • the communication link 1002 may include a pathway to transmit information between the above-mentioned components.
  • the communication interface 1004 uses any transceiver-like device for communicating with other devices or communication networks, such as Ethernet, RAN, wireless local area networks (WLAN), etc.
  • Ethernet Ethernet
  • RAN wireless local area networks
  • WLAN wireless local area networks
  • the memory 1003 can be a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (RAM) or other types of dynamic storage devices that can store information and instructions, or an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical disc, laser disc, optical disc, digital versatile disc, Blu-ray disc, etc.), a disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of an instruction or data structure and can be accessed by a computer, but is not limited to this.
  • the memory can be independent and connected to the processor through a communication line 1002.
  • the memory can also be integrated with the processor.
  • the memory provided in the embodiment of the present application can generally have non-volatility.
  • the memory 1003 is used to store the computer execution instructions for executing the scheme of the present application, and is controlled by the processor 1001 to execute.
  • the processor 1001 is used to execute the computer-executable instructions stored in the memory 1003, so as to implement the method provided in the above embodiments of the present application; illustratively, the above FIG. 3, The steps of the image processing method shown in FIG. 5 or FIG. 6 .
  • the computer-executable instructions in the embodiments of the present application may also be referred to as application code, which is not specifically limited in the embodiments of the present application.
  • the processor 1001 may include one or more CPUs, for example, CPU0 in FIG. 10 ; the processor 1001 may also include a CPU and any one of a GPU, an ASIC, and an FPGA, for example, CPU0+GPU0 or CPU 0+ASIC0 or CPU0+FPGA0 in FIG. 10 .
  • the electronic device may include multiple processors, such as processor 1001 and processor 1007 in FIG. 10.
  • processors may be a single-core (single-CPU) processor, a multi-core (multi-CPU) processor, or a heterogeneous computing architecture including multiple general-purpose processors.
  • the processor here may refer to one or more devices, circuits, and/or processing cores for processing data (e.g., computer program instructions).
  • the electronic device may further include an output device 1005 and an input device 1006.
  • the output device 1005 communicates with the processor 1001 and may display information in a variety of ways.
  • the output device 1005 may be a liquid crystal display (LCD), a light emitting diode (LED) display device, a cathode ray tube (CRT) display device, or a projector, etc.
  • LCD liquid crystal display
  • LED light emitting diode
  • CRT cathode ray tube
  • a projector etc.
  • it may be a display device such as a vehicle-mounted HUD, an AR-HUD, a display, etc.
  • the input device 1006 communicates with the processor 1001 and may receive user input in a variety of ways.
  • the input device 1006 may be a mouse, a keyboard, a touch screen device, or a sensor device, etc.
  • the embodiments of the present application provide a computer-readable storage medium on which computer program instructions are stored, and when the computer program instructions are executed by a processor, the method in the above embodiments is implemented. Exemplarily, each step of the image processing method shown in Figure 3, Figure 5 or Figure 6 can be implemented.
  • the embodiments of the present application provide a computer program product, which may include, for example, a computer-readable code or a non-volatile computer-readable storage medium carrying the computer-readable code; when the computer program product is run on a computer, the computer executes the method in the above embodiment. Exemplarily, each step of the image processing method shown in FIG. 3, FIG. 5 or FIG. 6 may be executed.
  • a computer-readable storage medium may be a tangible device that can hold and store instructions used by an instruction execution device.
  • a computer-readable storage medium may be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing.
  • Computer-readable storage media include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disk read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device, such as a punch card or a raised structure in a groove on which instructions are stored, and any suitable combination of the foregoing.
  • RAM random access memory
  • ROM read-only memory
  • EPROM or flash memory erasable programmable read-only memory
  • SRAM static random access memory
  • CD-ROM compact disk read-only memory
  • DVD digital versatile disk
  • memory stick a floppy disk
  • mechanical encoding device such as a punch card or a raised structure in a groove on which instructions are stored, and any suitable combination of the foregoing.
  • a computer-readable storage medium is not to be interpreted as a transient signal per se, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagating through a waveguide or other transmission medium (e.g., a light pulse through a fiber optic cable), or an electrical signal transmitted through a wire.
  • the computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing/processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and/or a wireless network.
  • the network can include copper transmission cables, optical fiber transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, and/or edge servers.
  • the network adapter card or network interface in each computing/processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in the computer-readable storage medium in each computing/processing device.
  • the computer program instructions for performing the operation of the present application can be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-related instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages, such as Smalltalk, C++, etc., and conventional procedural programming languages, such as "C" language or similar programming languages.
  • Computer-readable program instructions can be executed completely on the user's computer, partially on the user's computer, as an independent software package, partially on the user's computer, partially on the remote computer, or completely on the remote computer or server.
  • the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (for example, using an Internet service provider to connect through the Internet).
  • LAN local area network
  • WAN wide area network
  • an Internet service provider for example, using an Internet service provider to connect through the Internet.
  • the electronic circuits can execute computer-readable program instructions, thereby realizing various aspects of the present application.
  • These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the processor of the computer or other programmable data processing device, a device that implements the functions/actions specified in one or more boxes in the flowchart and/or block diagram is generated.
  • These computer-readable program instructions can also be stored in a computer-readable storage medium, and these instructions cause the computer, programmable data processing device, and/or other equipment to work in a specific manner, so that the computer-readable medium storing the instructions includes a manufactured product, which includes instructions for implementing various aspects of the functions/actions specified in one or more boxes in the flowchart and/or block diagram.
  • Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operating steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions/actions specified in one or more boxes in the flowchart and/or block diagram.
  • each square box in the flow chart or block diagram can represent a part of a module, program segment or instruction, and a part of the module, program segment or instruction includes one or more executable instructions for realizing the logical function of the specification.
  • the function marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two continuous square boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the function involved.
  • each square box in the block diagram and/or flow chart, and the combination of the square boxes in the block diagram and/or flow chart can be realized by a dedicated hardware-based system that performs the function or action of the specification, or can be realized by a combination of special-purpose hardware and computer instructions.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Computational Linguistics (AREA)
  • Data Mining & Analysis (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Health & Medical Sciences (AREA)
  • Evolutionary Computation (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Image Analysis (AREA)

Abstract

The present application relates to an image processing method, an apparatus, an electronic device and a storage medium. The image processing method may comprise: acquiring an image to be processed, the image to be processed comprising a tubular structure; determining a plurality of nodes in the tubular structure; connecting a first node among the plurality of nodes to at least one second node so as to obtain an adjusted tubular structure; and according to the adjusted tubular structure, constructing at least one topological graph corresponding to the tubular structure. Thus, automatically tracking a tubular structure can be achieved without a large amount of manual operations, thereby shortening the tracking time, lowering the tracking cost and improving the tracking efficiency. In addition, over-connection operation for the plurality of nodes in the tubular structure can ensure connection of all of the plurality of nodes in the tubular structure, thereby solving the disconnection problem during tubular structure tracking.

Description

一种图像处理方法、装置、电子设备和存储介质Image processing method, device, electronic device and storage medium
本申请要求于2022年11月7日提交中国专利局、申请号为202211386400.1、发明名称为“一种图像处理方法、装置、电子设备和存储介质”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。This application claims priority to the Chinese patent application filed with the China Patent Office on November 7, 2022, with application number 202211386400.1 and invention name “An image processing method, device, electronic device and storage medium”, the entire contents of which are incorporated by reference in this application.
技术领域Technical Field
本申请涉及图像处理技术领域，尤其涉及一种图像处理方法、装置、电子设备和存储介质。The present application relates to the field of image processing technology, and in particular to an image processing method, device, electronic device and storage medium.
背景技术Background technique
在生物医学领域，管状结构追踪指获得管状结构(如血管、气管、神经轴突等管状形态的物体)的拓扑结构的方式。管状结构追踪是生物医学领域的重要问题，在支气管镜检查、动脉介入手术、神经元追踪等方面有着重要应用。现有的管状结构追踪方法主要有基于最短路径算法的半自动追踪方法和基于快速行进(Fast Marching)算法的全自动追踪方法。但基于最短路径算法的半自动追踪方法需要大量人工操作，耗时久，成本高；并且基于最短路径算法的半自动追踪方法和基于快速行进算法的全自动追踪方法，在信号低或存在干扰信号的情况下，都会产生断线问题。In the biomedical field, tubular structure tracking refers to the method of obtaining the topological structure of tubular structures (such as blood vessels, trachea, nerve axons and other tubular objects). Tubular structure tracking is an important issue in the biomedical field, and has important applications in bronchoscopy, arterial interventional surgery, neuron tracking, etc. The existing tubular structure tracking methods mainly include semi-automatic tracking methods based on the shortest path algorithm and fully automatic tracking methods based on the fast marching algorithm. However, the semi-automatic tracking method based on the shortest path algorithm requires a lot of manual operation, which is time-consuming and costly; and the semi-automatic tracking method based on the shortest path algorithm and the fully automatic tracking method based on the fast marching algorithm will have disconnection problems when the signal is low or there is an interference signal.
发明内容Summary of the invention
有鉴于此，提出了一种图像处理方法、装置、电子设备、存储介质及计算机程序产品。In view of this, an image processing method, an apparatus, an electronic device, a storage medium and a computer program product are proposed.
第一方面，本申请的实施例提供了一种图像处理方法，所述方法包括：获取待处理图像，所述待处理图像包括管状结构；确定所述管状结构中的多个节点；将所述多个节点中的第一节点与至少一个第二节点进行连接，得到调整后的管状结构，其中，所述第二节点为所述多个节点中除所述第一节点外的节点；根据调整后的管状结构，构建所述管状结构对应的至少一个拓扑图。In a first aspect, an embodiment of the present application provides an image processing method, the method comprising: acquiring an image to be processed, the image to be processed comprising a tubular structure; determining a plurality of nodes in the tubular structure; connecting a first node among the plurality of nodes with at least one second node to obtain an adjusted tubular structure, wherein the second node is a node among the plurality of nodes other than the first node; and constructing at least one topological graph corresponding to the tubular structure according to the adjusted tubular structure.
基于上述技术方案，通过对待处理图像中管状结构的节点进行过连接操作，得到调整后的管状结构；根据调整后的管状结构，构建管状结构对应的至少一个拓扑图；可以实现对管状结构进行自动追踪，无需大量人工操作，减少了追踪时间和追踪成本，提高了追踪效率；并且，通过对管状结构的多个节点进行过连接操作，可以保证管状结构的该多个节点都能够被连接，作为一个示例，可以对管状结构中的每个节点进行过连接操作，从而保证管状结构的每个节点都能够被连接，解决了管状结构追踪中的断线问题。Based on the above technical solution, an adjusted tubular structure is obtained by performing an over-connection operation on the nodes of the tubular structure in the processed image; at least one topological map corresponding to the tubular structure is constructed according to the adjusted tubular structure; automatic tracking of the tubular structure can be achieved without a large amount of manual operation, thereby reducing tracking time and tracking costs and improving tracking efficiency; and, by performing an over-connection operation on multiple nodes of the tubular structure, it can be ensured that the multiple nodes of the tubular structure can be connected. As an example, an over-connection operation can be performed on each node in the tubular structure, thereby ensuring that each node of the tubular structure can be connected, solving the problem of disconnection in tubular structure tracking.
根据第一方面，在所述第一方面的第一种可能的实现方式中，所述方法还包括：在所述至少一个拓扑图的数量为多个的情况下，对所述至少一个拓扑图进行筛选。According to the first aspect, in a first possible implementation manner of the first aspect, the method further includes: when there are multiple topological maps, screening the at least one topological map.
基于上述技术方案，对得到的多个拓扑图进行筛选，可以将筛选得到的拓扑图作为管状结构的追踪结果，可以进一步提高管状结构追踪结果的准确性。Based on the above technical solution, the multiple topological maps obtained are screened, and the screened topological maps can be used as the tracking results of the tubular structure, which can further improve the accuracy of the tracking results of the tubular structure.
根据第一方面或第一方面的第一种可能的实现方式，在所述第一方面的第二种可能的实现方式中，所述将所述多个节点中的第一节点与至少一个第二节点进行连接，得到调整后的 管状结构，包括：将所述第一节点与距其预设范围内的至少一个所述第二节点进行连接，得到所述调整后的管状结构。According to the first aspect or the first possible implementation of the first aspect, in a second possible implementation of the first aspect, the first node among the multiple nodes is connected to at least one second node to obtain an adjusted The tubular structure comprises: connecting the first node with at least one of the second nodes within a preset range thereof to obtain the adjusted tubular structure.
基于上述技术方案，由于管状结构中距离较近的节点之间存在连接关系的可能性较大，对于每个节点，将距其预设范围内的其他节点分别与其进行连接，可以将管状结构中可能存在连接关系的节点连接起来，这样通过对管状结构的节点进行过连接操作，所构建得到的过连接边属于管状结构对应的拓扑图的概率更大，从而可以构建更准确的管状结构对应的拓扑图。Based on the above technical solution, since there is a high possibility that there is a connection relationship between nodes that are close to each other in the tubular structure, for each node, other nodes within a preset range are connected to it respectively, so that the nodes that may have a connection relationship in the tubular structure can be connected. In this way, by performing an over-connection operation on the nodes of the tubular structure, the probability that the constructed over-connected edges belong to the topological graph corresponding to the tubular structure is higher, thereby constructing a more accurate topological graph corresponding to the tubular structure.
根据第一方面或第一方面上述各种可能的实现方式，在所述第一方面的第三种可能的实现方式中，所述根据调整后的管状结构，构建所述管状结构对应的至少一个拓扑图，包括：根据所述调整后的管状结构中的至少一个节点、所述调整后的管状结构中的至少一个连接边和所述至少一个连接边对应的特征，构建所述至少一个拓扑图。According to the first aspect or the above-mentioned various possible implementation methods of the first aspect, in a third possible implementation method of the first aspect, constructing at least one topological graph corresponding to the tubular structure based on the adjusted tubular structure includes: constructing the at least one topological graph based on at least one node in the adjusted tubular structure, at least one connecting edge in the adjusted tubular structure and a feature corresponding to the at least one connecting edge.
基于上述技术方案，通过连接边对应的特征可以表征连接边的特性，根据调整后的管状结构中的节点、连接边和连接边对应的特征，可以筛选出更加准确的连接边，从而可以构建出更准确的管状结构对应的拓扑图。Based on the above technical solution, the characteristics of the connecting edges can be characterized by the features corresponding to the connecting edges. According to the nodes, connecting edges and the features corresponding to the connecting edges in the adjusted tubular structure, more accurate connecting edges can be screened out, thereby constructing a more accurate topological map corresponding to the tubular structure.
根据第一方面的第三种可能的实现方式，在所述第一方面的第四种可能的实现方式中，所述根据所述调整后的管状结构中的至少一个节点、所述调整后的管状结构中的至少一个连接边和所述至少一个连接边对应的特征，构建所述至少一个拓扑图，包括：根据所述至少一个连接边对应的特征，得到所述至少一个连接边的权值；根据所述至少一个节点、所述至少一个连接边和所述至少一个连接边的权值，构建所述至少一个拓扑图。According to the third possible implementation manner of the first aspect, in the fourth possible implementation manner of the first aspect, constructing the at least one topological graph according to at least one node in the adjusted tubular structure, at least one connecting edge in the adjusted tubular structure, and the features corresponding to the at least one connecting edge, includes: obtaining the weight of the at least one connecting edge according to the features corresponding to the at least one connecting edge; constructing the at least one topological graph according to the at least one node, the at least one connecting edge, and the weight of the at least one connecting edge.
基于上述技术方案，根据调整后的管状结构中的连接边对应的特征得到连接边的权值，连接边的权值越大则表示该连接边越有可能属于管状结构对应的拓扑图，反之，权值越小则表示该连接边属于管状结构对应的拓扑图的可能性越低；从而根据调整后的管状结构中的节点、连接边和连接边的权值，可以构建出更准确的管状结构对应的拓扑图。Based on the above technical solution, the weight of the connecting edge is obtained according to the characteristics corresponding to the connecting edge in the adjusted tubular structure. The larger the weight of the connecting edge, the more likely it is that the connecting edge belongs to the topological graph corresponding to the tubular structure. Conversely, the smaller the weight, the lower the possibility that the connecting edge belongs to the topological graph corresponding to the tubular structure. Therefore, a more accurate topological graph corresponding to the tubular structure can be constructed based on the nodes, connecting edges and the weights of the connecting edges in the adjusted tubular structure.
根据第一方面的第四种可能的实现方式，在所述第一方面的第五种可能的实现方式中，所述根据所述至少一个连接边对应的特征，得到所述至少一个连接边的权值，包括：将所述至少一个连接边对应的特征输入预设模型，得到所述至少一个连接边属于所述至少一个拓扑图的概率；根据所述至少一个连接边属于所述至少一个拓扑图的概率，得到所述至少一个连接边的权值；其中，所述预设模型基于拓扑图训练样本中每一连接边对应的特征训练得到。According to the fourth possible implementation manner of the first aspect, in the fifth possible implementation manner of the first aspect, obtaining the weight of the at least one connecting edge according to the features corresponding to the at least one connecting edge includes: inputting the features corresponding to the at least one connecting edge into a preset model to obtain the probability that the at least one connecting edge belongs to the at least one topological graph; obtaining the weight of the at least one connecting edge according to the probability that the at least one connecting edge belongs to the at least one topological graph; wherein the preset model is trained based on the features corresponding to each connecting edge in the topological graph training sample.
基于上述技术方案，通过将各连接边对应的特征输入至预设模型中，得到各连接边属于待处理图像中的管状结构对应的拓扑图的概率，从而得到各连接边的权值，将管状结构追踪问题转化为管状结构的连接边预测问题，与现有的管状结构追踪方法相比，将传统的迭代数值计算转化为一次神经网络矩阵计算，可以极大地减少计算量，减少内存占用，缩短追踪时间，从而提升管状结构的追踪效率。Based on the above technical scheme, by inputting the features corresponding to each connecting edge into the preset model, the probability that each connecting edge belongs to the topological map corresponding to the tubular structure in the image to be processed is obtained, thereby obtaining the weight of each connecting edge, and converting the tubular structure tracking problem into the connecting edge prediction problem of the tubular structure. Compared with the existing tubular structure tracking method, the traditional iterative numerical calculation is converted into a neural network matrix calculation, which can greatly reduce the amount of calculation, reduce memory usage, shorten the tracking time, and thus improve the tracking efficiency of the tubular structure.
根据第一方面的第三、四或五种可能的实现方式，在所述第一方面的第六种可能的实现方式中，所述至少一个连接边对应的特征指示所述至少一个连接边的几何特性，和/或所述至少一个连接边中至少一端节点的几何特性。According to the third, fourth or fifth possible implementation manner of the first aspect, in the sixth possible implementation manner of the first aspect, the feature corresponding to the at least one connecting edge indicates the geometric properties of the at least one connecting edge, and/or the geometric properties of at least one end node in the at least one connecting edge.
作为一个示例，所述至少一个连接边对应的特征，包括：所述至少一个连接边至少一端节点的坐标、所述至少一个连接边的长度、所述至少一个连接边的斜率中的至少一项。As an example, the features corresponding to the at least one connecting edge include: at least one item of the coordinates of at least one end node of the at least one connecting edge, the length of the at least one connecting edge, and the slope of the at least one connecting edge.
基于上述技术方案，连接边对应的特征指示连接边的几何特性，和/或连接边中至少一端 节点的几何特性，通过这些特征可以更准确地预测连接边属于管状结构对应的拓扑图的概率，从而可以构建更准确的管状结构对应的拓扑图。Based on the above technical solution, the feature corresponding to the connecting edge indicates the geometric characteristics of the connecting edge, and/or at least one end of the connecting edge The geometric characteristics of the nodes can be used to more accurately predict the probability that the connecting edges belong to the topological graph corresponding to the tubular structure, so that a more accurate topological graph corresponding to the tubular structure can be constructed.
根据第一方面的第五种可能的实现方式，在所述第一方面的第七种可能的实现方式中，所述预设模型为图神经网络模型。According to the fifth possible implementation manner of the first aspect, in the seventh possible implementation manner of the first aspect, the preset model is a graph neural network model.
基于上述技术方案，使用图神经网络模型可以更准确地学习到连接边对应的特征，将各连接边对应的特征输入至训练好的图神经网络模型中，可以更准确地预测连接边属于管状结构对应的拓扑图的概率，从而可以构建更准确的管状结构对应的拓扑图；利用图神经网络模型可以将管状结构追踪问题转化为管状结构的连接边预测问题，与现有的管状结构追踪方法相比，将传统的迭代数值计算转化为一次神经网络矩阵计算，可以极大地减少计算量，减少内存占用，缩短追踪时间，从而提升管状结构的追踪效率。Based on the above technical solution, the graph neural network model can be used to more accurately learn the features corresponding to the connection edges. By inputting the features corresponding to each connection edge into the trained graph neural network model, the probability that the connection edge belongs to the topological map corresponding to the tubular structure can be more accurately predicted, thereby constructing a more accurate topological map corresponding to the tubular structure; the graph neural network model can be used to transform the tubular structure tracking problem into the tubular structure connection edge prediction problem. Compared with the existing tubular structure tracking method, the traditional iterative numerical calculation is transformed into a neural network matrix calculation, which can greatly reduce the amount of calculation, reduce memory usage, shorten the tracking time, and thus improve the tracking efficiency of the tubular structure.
根据第一方面的第四、五、六或七种可能的实现方式，在所述第一方面的第八种可能的实现方式中，所述根据所述至少一个节点、所述至少一个连接边和所述至少一个连接边的权值，构建所述至少一个拓扑图，包括：根据所述至少一个节点、所述至少一个连接边和所述至少一个连接边的权值，基于最小生成树算法，构建所述至少一个拓扑图。According to the fourth, fifth, sixth or seventh possible implementation manner of the first aspect, in the eighth possible implementation manner of the first aspect, constructing the at least one topological graph according to the at least one node, the at least one connecting edge and the weight of the at least one connecting edge includes: constructing the at least one topological graph based on a minimum spanning tree algorithm according to the at least one node, the at least one connecting edge and the weight of the at least one connecting edge.
基于上述技术方案，基于最小生成树算法去除得到的调整后的管状结构中的环形结构，构建管状结构对应的树形拓扑图，可以降低计算复杂度，缩短追踪时间，提升追踪效率。Based on the above technical solution, the ring structure in the adjusted tubular structure is removed based on the minimum spanning tree algorithm, and a tree topology graph corresponding to the tubular structure is constructed, which can reduce the computational complexity, shorten the tracking time, and improve the tracking efficiency.
根据第一方面或第一方面上述各种可能的实现方式，在所述第一方面的第九种可能的实现方式中，所述拓扑图为树形拓扑图。According to the first aspect or the various possible implementation manners of the first aspect, in a ninth possible implementation manner of the first aspect, the topology graph is a tree topology graph.
基于上述技术方案，在生物医学等领域管状结构对应的拓扑图通常为树形拓扑图，因此可以构建管状结构对应的树形拓扑图作为生物医学等领域管状结构的追踪结果。Based on the above technical solution, the topological map corresponding to the tubular structure in the biomedical field and other fields is usually a tree topological map, so the tree topological map corresponding to the tubular structure can be constructed as the tracking result of the tubular structure in the biomedical field and other fields.
根据第一方面的第四种可能的实现方式，在所述第一方面的第十种可能的实现方式中，所述根据所述至少一个节点、所述至少一个连接边和所述至少一个连接边的权值，构建所述管状结构对应的至少一个拓扑图，包括：根据所述至少一个连接边的权值，确定第一连接边集合中权值最大的任一连接边；其中，所述第一连接边集合的初始状态包括所述至少一个连接边；在所述权值最大的任一连接边与第二连接边集合中的连接边不构成环形结构的情况下，将所述权值最大的任一连接边添加到所述第二连接边集合中；其中，所述第二连接边集合的初始状态为空集；从所述第一连接边集合中移除所述权值最大的任一连接边，以更新所述第一连接边集合；并基于更新后的所述第一连接边集合，重复执行上述确定第一连接边集合中权值最大的任一连接边及之后的操作，直到所述第二连接边集合中的连接边的数量为N-1，N为所述至少一个节点的数量；根据所述第二连接边集合及所述至少一个节点，构建所述管状结构对应的一个拓扑图。According to the fourth possible implementation manner of the first aspect, in the tenth possible implementation manner of the first aspect, constructing at least one topological graph corresponding to the tubular structure according to the at least one node, the at least one connecting edge and the weight of the at least one connecting edge includes: determining any connecting edge with the largest weight in the first connecting edge set according to the weight of the at least one connecting edge; wherein the initial state of the first connecting edge set includes the at least one connecting edge; when any connecting edge with the largest weight and the connecting edges in the second connecting edge set do not form a ring structure, adding any connecting edge with the largest weight to the second connecting edge set; wherein the initial state of the second connecting edge set is an empty set; removing any connecting edge with the largest weight from the first connecting edge set to update the first connecting edge set; and based on the updated first connecting edge set, repeatedly performing the above-mentioned determination of any connecting edge with the largest weight in the first connecting edge set and subsequent operations until the number of connecting edges in the second connecting edge set is N-1, where N is the number of the at least one node; constructing a topological graph corresponding to the tubular structure according to the second connecting edge set and the at least one node.
基于上述技术方案，由于第二连接边集合中所有的连接边都不构成环形结构，可以保证构建得到的管状结构对应的拓扑图为树形拓扑图；由于管状结构的节点的数量为N个，在第二连接边集合中包含N-1个连接边时，可以保证管状结构的每个节点都能被连接，从而可以解决管状结构追踪中的断线问题；每次选择第一连接边集合中权值最大的任一连接边判断是否添加进第二连接边集合，由于连接边的权值与该连接边属于管状结构对应的拓扑图的概率正相关，这样可以保证构建得到的管状结构对应的拓扑图的准确性更高。Based on the above technical solution, since all the connecting edges in the second connecting edge set do not form a ring structure, it can be ensured that the topological graph corresponding to the constructed tubular structure is a tree topological graph; since the number of nodes of the tubular structure is N, when the second connecting edge set contains N-1 connecting edges, it can be ensured that each node of the tubular structure can be connected, thereby solving the problem of disconnection in tubular structure tracking; each time, any connecting edge with the largest weight in the first connecting edge set is selected to determine whether to add it to the second connecting edge set. Since the weight of the connecting edge is positively correlated with the probability that the connecting edge belongs to the topological graph corresponding to the tubular structure, this can ensure that the topological graph corresponding to the constructed tubular structure is more accurate.
根据第一方面或第一方面上述各种可能的实现方式，在所述第一方面的第十一种可能的实现方式中，所述确定所述管状结构中的多个节点，包括：提取所述管状结构的骨架线；根 据所述管状结构的骨架线，确定所述管状结构中的多个节点。According to the first aspect or the various possible implementations of the first aspect, in an eleventh possible implementation of the first aspect, determining a plurality of nodes in the tubular structure includes: extracting a skeleton line of the tubular structure; According to the skeleton line of the tubular structure, a plurality of nodes in the tubular structure are determined.
基于上述技术方案，可以通过提取管状结构的骨架线确定管状结构的节点，为后续进行管状结构的追踪提供基础。Based on the above technical solution, the nodes of the tubular structure can be determined by extracting the skeleton line of the tubular structure, providing a basis for subsequent tracking of the tubular structure.
第二方面，本申请的实施例提供了一种图像处理装置，所述装置包括：获取模块，用于获取待处理图像，所述待处理图像包括管状结构；节点确定模块，用于确定所述管状结构中的多个节点；连接模块，用于将所述多个节点中的第一节点与至少一个第二节点进行连接，得到调整后的管状结构，其中，所述第二节点为所述多个节点中除所述第一节点外的节点；构建模块，用于根据调整后的管状结构，构建所述管状结构对应的至少一个拓扑图。In a second aspect, an embodiment of the present application provides an image processing device, comprising: an acquisition module, used to acquire an image to be processed, wherein the image to be processed includes a tubular structure; a node determination module, used to determine multiple nodes in the tubular structure; a connection module, used to connect a first node among the multiple nodes with at least one second node to obtain an adjusted tubular structure, wherein the second node is a node among the multiple nodes other than the first node; and a construction module, used to construct at least one topological graph corresponding to the tubular structure according to the adjusted tubular structure.
基于上述技术方案，通过对待处理图像中管状结构的节点进行过连接操作，得到调整后的管状结构；根据调整后的管状结构，构建管状结构对应的至少一个拓扑图；可以实现对管状结构进行自动追踪，无需大量人工操作，减少了追踪时间和追踪成本，提高了追踪效率；并且，通过对管状结构的多个节点进行过连接操作，可以保证管状结构的该多个节点都能够被连接，作为一个示例，可以对管状结构中的每个节点进行过连接操作，从而保证管状结构的每个节点都能够被连接，解决了管状结构追踪中的断线问题。Based on the above technical solution, an adjusted tubular structure is obtained by performing an over-connection operation on the nodes of the tubular structure in the processed image; at least one topological map corresponding to the tubular structure is constructed according to the adjusted tubular structure; automatic tracking of the tubular structure can be achieved without a large amount of manual operation, thereby reducing tracking time and tracking costs and improving tracking efficiency; and, by performing an over-connection operation on multiple nodes of the tubular structure, it can be ensured that the multiple nodes of the tubular structure can be connected. As an example, an over-connection operation can be performed on each node in the tubular structure, thereby ensuring that each node of the tubular structure can be connected, solving the problem of disconnection in tubular structure tracking.
根据第二方面，在所述第二方面的第一种可能的实现方式中，所述装置还包括：筛选模块，用于在所述至少一个拓扑图的数量为多个的情况下，对所述至少一个拓扑图进行筛选。According to the second aspect, in a first possible implementation manner of the second aspect, the device further includes: a screening module, configured to screen the at least one topology map when there are multiple at least one topology maps.
基于上述技术方案，对得到的多个拓扑图进行筛选，可以将筛选得到的拓扑图作为管状结构的追踪结果，可以进一步提高管状结构追踪结果的准确性。Based on the above technical solution, the multiple topological maps obtained are screened, and the screened topological maps can be used as the tracking results of the tubular structure, which can further improve the accuracy of the tracking results of the tubular structure.
根据第二方面或第二方面的第一种可能的实现方式，在所述第二方面的第二种可能的实现方式中，所述连接模块，还用于：将所述第一节点与距其预设范围内的至少一个所述第二节点进行连接，得到所述调整后的管状结构。According to the second aspect or the first possible implementation of the second aspect, in the second possible implementation of the second aspect, the connection module is further used to: connect the first node with at least one second node within a preset range thereof to obtain the adjusted tubular structure.
基于上述技术方案，由于管状结构中距离较近的节点之间存在连接关系的可能性较大，对于每个节点，将距其预设范围内的其他节点分别与其进行连接，可以将管状结构中可能存在连接关系的节点连接起来，这样通过对管状结构的节点进行过连接操作，所构建得到的过连接边属于管状结构对应的拓扑图的概率更大，从而可以构建更准确的管状结构对应的拓扑图。Based on the above technical solution, since there is a high possibility that there is a connection relationship between nodes that are close to each other in the tubular structure, for each node, other nodes within a preset range are connected to it respectively, so that the nodes that may have a connection relationship in the tubular structure can be connected. In this way, by performing an over-connection operation on the nodes of the tubular structure, the probability that the constructed over-connected edges belong to the topological graph corresponding to the tubular structure is higher, thereby constructing a more accurate topological graph corresponding to the tubular structure.
根据第二方面或第二方面上述各种可能的实现方式，在所述第二方面的第三种可能的实现方式中，所述构建模块，还用于：根据所述调整后的管状结构中的至少一个节点、所述调整后的管状结构中的至少一个连接边和所述至少一个连接边对应的特征，构建所述至少一个拓扑图。According to the second aspect or the above-mentioned various possible implementation methods of the second aspect, in a third possible implementation method of the second aspect, the construction module is also used to: construct the at least one topological graph based on at least one node in the adjusted tubular structure, at least one connecting edge in the adjusted tubular structure and features corresponding to the at least one connecting edge.
基于上述技术方案，通过连接边对应的特征可以表征连接边的特性，根据调整后的管状结构中的节点、连接边和连接边对应的特征，可以筛选出更加准确的连接边，从而可以构建出更准确的管状结构对应的拓扑图。Based on the above technical solution, the characteristics of the connecting edges can be characterized by the features corresponding to the connecting edges. According to the nodes, connecting edges and the features corresponding to the connecting edges in the adjusted tubular structure, more accurate connecting edges can be screened out, thereby constructing a more accurate topological map corresponding to the tubular structure.
根据第二方面的第三种可能的实现方式，在所述第二方面的第四种可能的实现方式中，所述构建模块，还用于：根据所述至少一个连接边对应的特征，得到所述至少一个连接边的权值；根据所述至少一个节点、所述至少一个连接边和所述至少一个连接边的权值，构建所述至少一个拓扑图。According to the third possible implementation manner of the second aspect, in the fourth possible implementation manner of the second aspect, the construction module is further used to: obtain the weight of the at least one connection edge according to the characteristics corresponding to the at least one connection edge; and construct the at least one topological graph according to the at least one node, the at least one connection edge and the weight of the at least one connection edge.
基于上述技术方案，根据调整后的管状结构中的连接边对应的特征得到连接边的权值，连接边的权值越大则表示该连接边越有可能属于管状结构对应的拓扑图，反之，权值越小则 表示该连接边属于管状结构对应的拓扑图的可能性越低；从而根据调整后的管状结构中的节点、连接边和连接边的权值，可以构建出更准确的管状结构对应的拓扑图。Based on the above technical solution, the weight of the connection edge is obtained according to the features corresponding to the connection edge in the adjusted tubular structure. The larger the weight of the connection edge, the more likely it is that the connection edge belongs to the topological graph corresponding to the tubular structure. Conversely, the smaller the weight, the more likely it is that the connection edge belongs to the topological graph corresponding to the tubular structure. The possibility that the connection edge belongs to the topological graph corresponding to the tubular structure is lower; thus, according to the nodes, connection edges and connection edge weights in the adjusted tubular structure, a more accurate topological graph corresponding to the tubular structure can be constructed.
根据第二方面的第四种可能的实现方式，在所述第二方面的第五种可能的实现方式中，所述构建模块，还用于：将所述至少一个连接边对应的特征输入预设模型，得到所述至少一个连接边属于所述至少一个拓扑图的概率；根据所述至少一个连接边属于所述至少一个拓扑图的概率，得到所述至少一个连接边的权值；其中，所述预设模型基于拓扑图训练样本中每一连接边对应的特征训练得到。According to the fourth possible implementation manner of the second aspect, in the fifth possible implementation manner of the second aspect, the construction module is further used to: input the features corresponding to the at least one connection edge into a preset model to obtain the probability that the at least one connection edge belongs to the at least one topological graph; obtain the weight of the at least one connection edge based on the probability that the at least one connection edge belongs to the at least one topological graph; wherein the preset model is trained based on the features corresponding to each connection edge in the topological graph training sample.
基于上述技术方案，通过将各连接边对应的特征输入至预设模型中，得到各连接边属于待处理图像中的管状结构对应的拓扑图的概率，从而得到各连接边的权值，将管状结构追踪问题转化为管状结构的连接边预测问题，与现有的管状结构追踪方法相比，将传统的迭代数值计算转化为一次神经网络矩阵计算，可以极大地减少计算量，减少内存占用，缩短追踪时间，从而提升管状结构的追踪效率。Based on the above technical scheme, by inputting the features corresponding to each connecting edge into the preset model, the probability that each connecting edge belongs to the topological map corresponding to the tubular structure in the image to be processed is obtained, thereby obtaining the weight of each connecting edge, and converting the tubular structure tracking problem into the connecting edge prediction problem of the tubular structure. Compared with the existing tubular structure tracking method, the traditional iterative numerical calculation is converted into a neural network matrix calculation, which can greatly reduce the amount of calculation, reduce memory usage, shorten the tracking time, and thus improve the tracking efficiency of the tubular structure.
根据第二方面的第三、四或五种可能的实现方式，在所述第二方面的第六种可能的实现方式中，所述至少一个连接边对应的特征指示所述至少一个连接边的几何特性，和/或所述至少一个连接边中至少一端节点的几何特性。According to the third, fourth or fifth possible implementation manner of the second aspect, in the sixth possible implementation manner of the second aspect, the feature corresponding to the at least one connecting edge indicates the geometric characteristics of the at least one connecting edge, and/or the geometric characteristics of at least one end node in the at least one connecting edge.
作为一个示例，所述至少一个连接边对应的特征，包括：所述至少一个连接边至少一端节点的坐标、所述至少一个连接边的长度、所述至少一个连接边的斜率中的至少一项。As an example, the features corresponding to the at least one connecting edge include: at least one item of the coordinates of at least one end node of the at least one connecting edge, the length of the at least one connecting edge, and the slope of the at least one connecting edge.
基于上述技术方案，连接边对应的特征指示连接边的几何特性，和/或连接边中至少一端节点的几何特性，通过这些特征可以更准确地预测连接边属于管状结构对应的拓扑图的概率，从而可以构建更准确的管状结构对应的拓扑图。Based on the above technical solution, the features corresponding to the connecting edge indicate the geometric properties of the connecting edge and/or the geometric properties of at least one end node in the connecting edge. These features can more accurately predict the probability that the connecting edge belongs to the topological graph corresponding to the tubular structure, thereby constructing a more accurate topological graph corresponding to the tubular structure.
根据第二方面的第五种可能的实现方式，在所述第二方面的第七种可能的实现方式中，所述预设模型为图神经网络模型。According to the fifth possible implementation manner of the second aspect, in the seventh possible implementation manner of the second aspect, the preset model is a graph neural network model.
基于上述技术方案，使用图神经网络模型可以更准确地学习到连接边对应的特征，将各连接边对应的特征输入至训练好的图神经网络模型中，可以更准确地预测连接边属于管状结构对应的拓扑图的概率，从而可以构建更准确的管状结构对应的拓扑图；利用图神经网络模型可以将管状结构追踪问题转化为管状结构的连接边预测问题，与现有的管状结构追踪方法相比，将传统的迭代数值计算转化为一次神经网络矩阵计算，可以极大地减少计算量，减少内存占用，缩短追踪时间，从而提升管状结构的追踪效率。Based on the above technical solution, the graph neural network model can be used to more accurately learn the features corresponding to the connection edges. By inputting the features corresponding to each connection edge into the trained graph neural network model, the probability that the connection edge belongs to the topological map corresponding to the tubular structure can be more accurately predicted, thereby constructing a more accurate topological map corresponding to the tubular structure; the graph neural network model can be used to transform the tubular structure tracking problem into the tubular structure connection edge prediction problem. Compared with the existing tubular structure tracking method, the traditional iterative numerical calculation is transformed into a neural network matrix calculation, which can greatly reduce the amount of calculation, reduce memory usage, shorten the tracking time, and thus improve the tracking efficiency of the tubular structure.
根据第二方面的第四、五、六或七种可能的实现方式，在所述第二方面的第八种可能的实现方式中，所述构建模块，还用于：根据所述至少一个节点、所述至少一个连接边和所述至少一个连接边的权值，基于最小生成树算法，构建所述至少一个拓扑图。According to the fourth, fifth, sixth or seventh possible implementation manner of the second aspect, in the eighth possible implementation manner of the second aspect, the construction module is further used to: construct the at least one topological graph based on the minimum spanning tree algorithm according to the at least one node, the at least one connecting edge and the weight of the at least one connecting edge.
基于上述技术方案，基于最小生成树算法去除过连接的拓扑图中的环形结构，构建管状结构对应的树形拓扑图，可以降低计算复杂度，缩短追踪时间，提升追踪效率。Based on the above technical solution, the ring structure in the over-connected topology graph is removed based on the minimum spanning tree algorithm, and a tree topology graph corresponding to the tubular structure is constructed, which can reduce the computational complexity, shorten the tracking time, and improve the tracking efficiency.
根据第二方面或第二方面上述各种可能的实现方式，在所述第二方面的第九种可能的实现方式中，所述拓扑图为树形拓扑图。According to the second aspect or the various possible implementations of the second aspect, in a ninth possible implementation of the second aspect, the topology graph is a tree topology graph.
基于上述技术方案，在生物医学等领域管状结构对应的拓扑图通常为树形拓扑图，因此可以构建管状结构对应的树形拓扑图作为生物医学等领域管状结构的追踪结果。Based on the above technical solution, the topological map corresponding to the tubular structure in the biomedical field and other fields is usually a tree topological map, so the tree topological map corresponding to the tubular structure can be constructed as the tracking result of the tubular structure in the biomedical field and other fields.
根据第二方面的第四种可能的实现方式，在所述第二方面的第十种可能的实现方式中，所述构建模块，还用于：根据所述至少一个连接边的权值，确定第一连接边集合中权值最大 的任一连接边；其中，所述第一连接边集合的初始状态包括所述至少一个连接边；在所述权值最大的任一连接边与第二连接边集合中的连接边不构成环形结构的情况下，将所述权值最大的任一连接边添加到所述第二连接边集合中；其中，所述第二连接边集合的初始状态为空集；从所述第一连接边集合中移除所述权值最大的任一连接边，以更新所述第一连接边集合；并基于更新后的所述第一连接边集合，重复执行上述确定第一连接边集合中权值最大的任一连接边及之后的操作，直到所述第二连接边集合中的连接边的数量为N-1，N为所述至少一个节点的数量；根据所述第二连接边集合及所述至少一个节点，构建所述管状结构对应的一个拓扑图。According to the fourth possible implementation manner of the second aspect, in the tenth possible implementation manner of the second aspect, the construction module is further used to: determine the edge with the largest weight in the first connection edge set according to the weight of the at least one connection edge any connecting edge of; wherein, the initial state of the first connecting edge set includes the at least one connecting edge; when any connecting edge with the largest weight and the connecting edges in the second connecting edge set do not form a ring structure, any connecting edge with the largest weight is added to the second connecting edge set; wherein, the initial state of the second connecting edge set is an empty set; remove any connecting edge with the largest weight from the first connecting edge set to update the first connecting edge set; and based on the updated first connecting edge set, repeat the above-mentioned determination of any connecting edge with the largest weight in the first connecting edge set and subsequent operations until the number of connecting edges in the second connecting edge set is N-1, N is the number of the at least one node; based on the second connecting edge set and the at least one node, construct a topological graph corresponding to the tubular structure.
基于上述技术方案，由于第二连接边集合中所有的连接边都不构成环形结构，可以保证构建得到的管状结构对应的拓扑图为树形拓扑图；由于管状结构的节点的数量为N个，在第二连接边集合中包含N-1个连接边时，可以保证管状结构的每个节点都能被连接，从而可以解决管状结构追踪中的断线问题；每次选择第一连接边集合中权值最大的任一连接边判断是否添加进第二连接边集合，由于连接边的权值与该连接边属于管状结构对应的拓扑图的概率正相关，这样可以保证构建得到的管状结构对应的拓扑图的准确性更高。Based on the above technical solution, since all the connecting edges in the second connecting edge set do not form a ring structure, it can be ensured that the topological graph corresponding to the constructed tubular structure is a tree topological graph; since the number of nodes of the tubular structure is N, when the second connecting edge set contains N-1 connecting edges, it can be ensured that each node of the tubular structure can be connected, thereby solving the problem of disconnection in tubular structure tracking; each time, any connecting edge with the largest weight in the first connecting edge set is selected to determine whether to add it to the second connecting edge set. Since the weight of the connecting edge is positively correlated with the probability that the connecting edge belongs to the topological graph corresponding to the tubular structure, this can ensure that the topological graph corresponding to the constructed tubular structure is more accurate.
根据第二方面或第二方面上述各种可能的实现方式，在所述第二方面的第十一种可能的实现方式中，所述节点确定模块，还用于：提取所述管状结构的骨架线；根据所述管状结构的骨架线，确定所述管状结构中的多个节点。According to the second aspect or the above-mentioned various possible implementation methods of the second aspect, in an eleventh possible implementation method of the second aspect, the node determination module is further used to: extract the skeleton line of the tubular structure; and determine a plurality of nodes in the tubular structure according to the skeleton line of the tubular structure.
基于上述技术方案，可以通过提取管状结构的骨架线确定管状结构的节点，为后续进行管状结构的追踪提供基础。Based on the above technical solution, the nodes of the tubular structure can be determined by extracting the skeleton line of the tubular structure, providing a basis for subsequent tracking of the tubular structure.
第三方面，本申请的实施例提供了一种电子设备，包括：处理器；用于存储处理器可执行指令的存储器；其中，所述处理器被配置为执行所述指令时实现第一方面或第一方面的一种或几种的图像处理方法。In a third aspect, an embodiment of the present application provides an electronic device, comprising: a processor; a memory for storing processor executable instructions; wherein the processor is configured to implement the first aspect or one or more image processing methods of the first aspect when executing the instructions.
第四方面，本申请的实施例提供了一种计算机可读存储介质，其上存储有计算机程序指令，所述计算机程序指令被处理器执行时实现第一方面或第一方面的一种或几种的图像处理方法。In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium having computer program instructions stored thereon, which, when executed by a processor, implement the first aspect or one or more image processing methods of the first aspect.
第五方面，本申请的实施例提供了一种计算机程序产品，当所述计算机程序产品在计算机上运行时，使得所述计算机执行上述第一方面或第一方面的一种或几种的图像处理方法。In a fifth aspect, an embodiment of the present application provides a computer program product, which, when executed on a computer, enables the computer to execute the above-mentioned first aspect or one or more of the image processing methods of the first aspect.
上述第三方面至第五方面的技术效果，参见上述第一方面或第二方面。For the technical effects of the third to fifth aspects, please refer to the first or second aspect.
根据下面参考附图对示例性实施例的详细说明，本申请的其它特征及方面将变得清楚。Other features and aspects of the present application will become apparent from the following detailed description of exemplary embodiments with reference to the attached drawings.
附图说明BRIEF DESCRIPTION OF THE DRAWINGS
包含在说明书中并且构成说明书的一部分的附图与说明书一起示出了本申请的示例性实施例、特征和方面，并且用于解释本申请的原理。The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate exemplary embodiments, features, and aspects of the present application and, together with the description, serve to explain the principles of the present application.
图1示出根据本申请一实施例的基于快速行进算法的全自动追踪方法的示意图。FIG1 is a schematic diagram showing a fully automatic tracking method based on a fast marching algorithm according to an embodiment of the present application.
图2示出根据本申请一实施例的一种图像处理方法的应用场景的示意图。FIG. 2 is a schematic diagram showing an application scenario of an image processing method according to an embodiment of the present application.
图3示出根据本申请一实施例的一种图像处理方法的流程图。FIG3 shows a flow chart of an image processing method according to an embodiment of the present application.
图4示出根据本申请一实施例的构建连接边的示意图。FIG. 4 is a schematic diagram showing a method of constructing a connection edge according to an embodiment of the present application.
图5示出根据本申请一实施例的一种图像处理方法的流程图。FIG5 shows a flow chart of an image processing method according to an embodiment of the present application.
图6示出根据本申请一实施例的一种图像处理方法的流程图。 FIG6 shows a flow chart of an image processing method according to an embodiment of the present application.
图7示出根据本申请一实施的构建管状结构对应的多个拓扑图的示意图。FIG. 7 is a schematic diagram showing a plurality of topological graphs corresponding to a tubular structure constructed according to an embodiment of the present application.
图8(a)-(b)示出根据本申请一实施例的提取鼠脑神经的骨架线的示意图。FIG8( a )-( b ) are schematic diagrams showing skeleton lines of extracted rat brain nerves according to an embodiment of the present application.
图9示出根据本申请一实施例的图像处理装置的框图。FIG. 9 shows a block diagram of an image processing apparatus according to an embodiment of the present application.
图10示出根据本申请一实施例的一种电子设备的结构示意图，FIG10 is a schematic diagram showing the structure of an electronic device according to an embodiment of the present application.
具体实施方式Detailed ways
以下将参考附图详细说明本申请的各种示例性实施例、特征和方面。附图中相同的附图标记表示功能相同或相似的元件。尽管在附图中示出了实施例的各种方面，但是除非特别指出，不必按比例绘制附图。Various exemplary embodiments, features and aspects of the present application will be described in detail below with reference to the accompanying drawings. The same reference numerals in the accompanying drawings represent elements with the same or similar functions. Although various aspects of the embodiments are shown in the accompanying drawings, the drawings are not necessarily drawn to scale unless otherwise specified.
在这里专用的词“示例性”意为“用作例子、实施例或说明性”。这里作为“示例性”所说明的任何实施例不必解释为优于或好于其它实施例。The word “exemplary” is used exclusively herein to mean “serving as an example, example, or illustration.” Any embodiment described herein as “exemplary” is not necessarily to be construed as preferred or advantageous over other embodiments.
另外，为了更好的说明本申请，在下文的具体实施方式中给出了众多的具体细节。本领域技术人员应当理解，没有某些具体细节，本申请同样可以实施。在一些实例中，对于本领域技术人员熟知的方法、手段、元件和电路未作详细描述，以便于凸显本申请的主旨。In addition, in order to better illustrate the present application, numerous specific details are provided in the following specific embodiments. It should be understood by those skilled in the art that the present application can also be implemented without certain specific details. In some examples, methods, means, components and circuits well known to those skilled in the art are not described in detail in order to highlight the subject matter of the present application.
管状结构追踪在生物医学领域有着重要应用。相关技术中，管状结构追踪方法主要有以下两种：Tubular structure tracking has important applications in the biomedical field. In related technologies, there are mainly two methods for tracking tubular structures:
(1)基于最短路径算法的半自动追踪方法：该方法一般集成到管状结构标注工具中，每次操作在管状结构的信号上手动选择两点，该方法可以自动寻找这两点之间的最短路径并连接。该方法存在以下缺点：一是需要大量人工操作，耗时久，成本高；二是如果选择的两点距离过长，可能存在干扰信号(如图像噪声)或信号低的情况，将导致错误连接或断线的问题。(1) Semi-automatic tracking method based on the shortest path algorithm: This method is generally integrated into the tubular structure annotation tool. Each time two points are manually selected on the signal of the tubular structure, this method can automatically find the shortest path between the two points and connect them. This method has the following disadvantages: first, it requires a lot of manual operation, which is time-consuming and costly; second, if the distance between the two selected points is too long, there may be interference signals (such as image noise) or low signals, which will lead to incorrect connections or disconnection problems.
(2)基于快速行进算法的全自动追踪方法：(2) Fully automatic tracking method based on fast marching algorithm:
采用快速行进算法，寻找图像中距当前像素点A一定范围内(例如5×5像素范围内)管状结构信号最强(如灰度值最大)的像素点B，之后再以像素点B为当前点持续迭代，直到遍历完全整个图像。图1示出根据本申请一实施例的基于快速行进算法的全自动追踪方法的示意图，如图1所示，从管状结构的当前节点开始寻找距当前节点一定范围内信号最强的节点，之后再以该节点为当前节点持续迭代，直到遍历完全整个图像，得到各节点之间的连接边S1～S9，从而得到该管状结构对应的拓扑图。该方法存在以下缺点：一是该方法需要遍历整个图像，对于有N个节点的管状结构，需要进行N-1次迭代，计算量极大，内存消耗大，耗时久，例如，对于单例斑马鱼脑追踪需要1小时左右；二是该方法会把图像中存在噪声或信号低的区域误判断为神经末梢，从而导致断线问题，使得追踪结果不准确，例如，若图1中连接边S7所在的区域存在噪声或神经信号低，连接边S7可能被误判断为神经末梢，使得连接边S7之后的神经没有被追踪到，导致断线问题。A fast marching algorithm is used to find the pixel point B with the strongest signal (such as the largest gray value) of the tubular structure within a certain range (for example, within a range of 5×5 pixels) from the current pixel point A in the image, and then iterates continuously with pixel point B as the current point until the entire image is traversed. FIG1 shows a schematic diagram of a fully automatic tracking method based on a fast marching algorithm according to an embodiment of the present application. As shown in FIG1 , starting from the current node of the tubular structure, a node with the strongest signal within a certain range from the current node is found, and then iterates continuously with the node as the current node until the entire image is traversed, and the connecting edges S1 to S9 between the nodes are obtained, thereby obtaining a topological map corresponding to the tubular structure. This method has the following disadvantages: First, this method needs to traverse the entire image. For a tubular structure with N nodes, N-1 iterations are required, which requires a huge amount of calculation, large memory consumption, and a long time. For example, it takes about 1 hour to track a single zebrafish brain. Second, this method may mistakenly judge areas with noise or low signals in the image as nerve endings, resulting in disconnection problems and inaccurate tracking results. For example, if there is noise or low neural signal in the area where the connecting edge S7 in Figure 1 is located, the connecting edge S7 may be mistakenly judged as a nerve ending, so that the nerve after the connecting edge S7 is not tracked, resulting in disconnection problems.
为了解决相关技术中管状结构追踪方法计算量大、追踪效率低及存在断线的问题，本申请实施例提供了一种图像处理方法。In order to solve the problems of large amount of calculation, low tracking efficiency and disconnection in the tubular structure tracking method in the related art, an embodiment of the present application provides an image processing method.
本申请实施例提供的图像处理方法可以应用于生物医学领域的管状结构追踪场景，包括但不限于医学场景下针对肺气管、脑动脉、颈动脉、外周动脉等进行分段的场景；以及脑科学场景下针对各种模式的动物(例如斑马鱼、鼠、猴)的介观或微观成像数据进行神经图谱追踪的场景。 The image processing method provided in the embodiments of the present application can be applied to tubular structure tracking scenarios in the biomedical field, including but not limited to the medical scenario of segmenting the pulmonary trachea, cerebral arteries, carotid arteries, peripheral arteries, etc.; and the brain science scenario of neural atlas tracking of mesoscopic or microscopic imaging data of various animal models (such as zebrafish, mice, monkeys).
图2示出根据本申请一实施例的一种图像处理方法的应用场景的示意图，如图2所示，将待处理图像201输入图像处理装置202中，示例性地，待处理图像201可以是三维(Three Dimensions，3D)医学图像，也可以是二维(Two Dimensions，2D)医学图像，对此不作限定；该待处理图像201包括管状结构；例如，如图所示待处理图像201为显微镜扫描的鼠脑3D脑图像；通过图像处理装置202执行本申请实施例提供的图像处理方法(详细描述参见下文)，可以对待处理图像201中的管状结构进行自动追踪，得到图像处理结果203，即待处理图像201中管状结构的追踪结果；例如，如图所示图像处理结果203为图像处理装置202对鼠脑3D脑图像进行处理得到的鼠脑神经追踪结果。FIG2 is a schematic diagram of an application scenario of an image processing method according to an embodiment of the present application. As shown in FIG2 , an image 201 to be processed is input into an image processing device 202. Exemplarily, the image 201 to be processed may be a three-dimensional (3D) medical image or a two-dimensional (2D) medical image, without limitation. The image 201 to be processed includes a tubular structure. For example, as shown in the figure, the image 201 to be processed is a 3D brain image of a mouse brain scanned by a microscope. The image processing method provided by the embodiment of the present application is executed by the image processing device 202 (for detailed description, see below). The tubular structure in the image 201 to be processed can be automatically tracked to obtain an image processing result 203, i.e., the tracking result of the tubular structure in the image 201 to be processed. For example, as shown in the figure, the image processing result 203 is a mouse brain nerve tracking result obtained by the image processing device 202 processing the mouse brain 3D brain image.
本申请实施例不限定该图像处理装置202的类型。The embodiment of the present application does not limit the type of the image processing device 202 .
示例性地，该图像处理装置202可以是独立设置，也可以集成在其他装置中，还可以是通过软件或者软件与硬件结合实现。Exemplarily, the image processing device 202 may be independently configured, integrated in other devices, or implemented through software or a combination of software and hardware.
示例性地，该图像处理装置202还可以为具有数据处理能力的设备或系统，或设置在这些设备或系统中的部件或者芯片。例如，该图像处理装置202可以是云端服务器、台式机、便携式电脑、网络服务器、掌上电脑(personal digital assistant，PDA)、移动手机、平板电脑、无线终端设备、嵌入式设备、医疗设备或其他具有数据处理功能的设备，或者为这些设备内的部件或者芯片。Exemplarily, the image processing device 202 may also be a device or system with data processing capabilities, or a component or chip set in these devices or systems. For example, the image processing device 202 may be a cloud server, a desktop computer, a portable computer, a network server, a personal digital assistant (PDA), a mobile phone, a tablet computer, a wireless terminal device, an embedded device, a medical device, or other device with data processing capabilities, or a component or chip in these devices.
示例性地，该图像处理装置202还可以是具有处理功能的芯片或处理器，该图像处理装置202可以包括多个处理器。处理器可以是一个单核(single-CPU)处理器，也可以是一个多核(multi-CPU)处理器。Exemplarily, the image processing device 202 may also be a chip or a processor with processing functions, and the image processing device 202 may include multiple processors. The processor may be a single-CPU processor or a multi-CPU processor.
需要说明的是，本申请实施例描述的上述应用场景是为了更加清楚的说明本申请实施例的技术方案，并不构成对于本申请实施例提供的技术方案的限定，本领域普通技术人员可知，针对其他相似的或新的场景的出现，本申请实施例提供的技术方案对于类似的技术问题，同样适用。It should be noted that the above-mentioned application scenarios described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application, and do not constitute a limitation on the technical solutions provided in the embodiments of the present application. Ordinary technicians in this field can know that the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems in response to the emergence of other similar or new scenarios.
下面对本申请实施例提供的图像处理方法进行详细说明。The image processing method provided in the embodiments of the present application is described in detail below.
图3示出根据本申请一实施例的一种图像处理方法的流程图，示例性地，该方法可以由上述图2中图像处理装置202执行，如图3所示，该方法可以包括以下步骤：FIG3 shows a flow chart of an image processing method according to an embodiment of the present application. Exemplarily, the method may be performed by the image processing device 202 in FIG2 . As shown in FIG3 , the method may include the following steps:
S301、获取待处理图像，该待处理图像包括管状结构。S301, obtaining an image to be processed, where the image to be processed includes a tubular structure.
示例性地，待处理图像可以是二维图像、三维图像等。作为一个示例，待处理图像可以是医学图像，例如，可以是3D肺部CT扫描图像、3D脑图像等。Exemplarily, the image to be processed may be a two-dimensional image, a three-dimensional image, etc. As an example, the image to be processed may be a medical image, for example, a 3D lung CT scan image, a 3D brain image, etc.
示例性地，管状结构可以包括血管、气管、神经轴突等。Illustratively, the tubular structure may include blood vessels, trachea, nerve axons, etc.
示例性地，可以对待处理图像进行预处理，预处理可以包括图像通道调整、图像缩放、尺寸调整、裁剪、去噪、旋转变换、图像增强、非目标区域排除或归一化等一项或多项操作。Exemplarily, the image to be processed may be preprocessed, and the preprocessing may include one or more operations such as image channel adjustment, image scaling, size adjustment, cropping, denoising, rotation transformation, image enhancement, non-target area exclusion or normalization.
示例性地，获取待处理图像后，可以提取待处理图像中管状结构对应的区域。作为一个示例，可以对待处理图像进行图像分割处理，得到管状结构对应的区域，方便后续对管状结构进行追踪。例如，可以将待处理图像输入训练好的3D U-Net卷积神经网络进行图像分割处理，得到管状结构对应的区域，其中，卷积神经网络可采用现有方式进行训练。再例如，可以通过二值化等方法分割待处理图像中管状结构的外轮廓，还可以通过距离变换等方法强化待处理图像中管状结构的信号，从而得到管状结构对应的区域。 Exemplarily, after obtaining the image to be processed, the area corresponding to the tubular structure in the image to be processed can be extracted. As an example, the image to be processed can be subjected to image segmentation processing to obtain the area corresponding to the tubular structure, so as to facilitate the subsequent tracking of the tubular structure. For example, the image to be processed can be input into a trained 3D U-Net convolutional neural network for image segmentation processing to obtain the area corresponding to the tubular structure, wherein the convolutional neural network can be trained in an existing manner. For another example, the outer contour of the tubular structure in the image to be processed can be segmented by binarization and other methods, and the signal of the tubular structure in the image to be processed can be enhanced by distance transformation and other methods, thereby obtaining the area corresponding to the tubular structure.
S302、确定管状结构中的多个节点。S302: Determine a plurality of nodes in the tubular structure.
在一种可能的实现方式中，所述确定管状结构中的多个节点，可以包括：In a possible implementation manner, determining a plurality of nodes in the tubular structure may include:
(1)提取管状结构的骨架线。(1) Extract the skeleton line of the tubular structure.
示例性地，可以通过相关技术中的骨架提取算法提取管状结构的骨架线。例如，可以使用基于距离变换的骨架提取算法、基于最大圆盘的骨架提取算法等方法提取管状结构的骨架线。作为一个示例，采用基于最大圆盘的骨架提取算法，在管状结构中生成多个内切圆盘，连接这个多个内切圆盘的圆心，从而得到该管状结构的骨架线。Exemplarily, the skeleton line of the tubular structure can be extracted by a skeleton extraction algorithm in the related art. For example, the skeleton line of the tubular structure can be extracted by using a skeleton extraction algorithm based on distance transformation, a skeleton extraction algorithm based on the largest disk, and the like. As an example, a skeleton extraction algorithm based on the largest disk is used to generate multiple inscribed disks in the tubular structure, and the centers of the multiple inscribed disks are connected to obtain the skeleton line of the tubular structure.
(2)根据所提取的管状结构的骨架线，确定该管状结构中的多个节点。(2) Determine a plurality of nodes in the tubular structure based on the extracted skeleton line of the tubular structure.
示例性地，管状结构中的节点可以包括管状结构的骨架线的端点、交叉点、折点中的一种或多种。作为一个示例，可以将管状结构的骨架线中所有的端点、交叉点和折点确定为管状结构中的节点。Exemplarily, the nodes in the tubular structure may include one or more of the endpoints, intersections, and inflection points of the skeleton line of the tubular structure. As an example, all the endpoints, intersections, and inflection points in the skeleton line of the tubular structure may be determined as nodes in the tubular structure.
S303、将所述多个节点中的第一节点与至少一个第二节点进行连接，得到调整后的管状结构，其中，所述第二节点为所述多个节点中除所述第一节点外的节点。S303: Connect a first node among the multiple nodes with at least one second node to obtain an adjusted tubular structure, wherein the second node is a node among the multiple nodes except the first node.
该步骤中，对于多个节点中的第一节点，可以选择管状结构中的至少一个第二节点与其进行连接，构建至少一个连接边，其中，第一节点与一个第二节点进行连接，可以构建一个连接边；通过这种方式对管状结构中节点进行连接的操作可以被称为过连接操作。这样，对多个节点中的每一节点均进行过连接操作，从而构建出多条连接边，保证多个节点都能够被连接，该多条连接边和该多个节点即可构成调整后的管状结构。In this step, for a first node among the multiple nodes, at least one second node in the tubular structure can be selected to connect with the first node to construct at least one connection edge, wherein the first node is connected to a second node to construct a connection edge; the operation of connecting nodes in the tubular structure in this way can be called an over-connection operation. In this way, an over-connection operation is performed on each of the multiple nodes, thereby constructing multiple connection edges, ensuring that multiple nodes can be connected, and the multiple connection edges and the multiple nodes can constitute the adjusted tubular structure.
作为一个示例，可以对管状结构中的每个节点进行过连接操作，得到调整后的管状结构。图4示出根据本申请一实施例的构建连接边的示意图。如图4所示，对管状结构的每一节点均进行过连接操作，可以构建至少一个连接边，由这些连接边及节点可以构成调整后的管状结构。这样，通过对管状结构的每个节点均进行过连接操作，可以保证管状结构的每个节点都与除自身以外的至少一个其他节点有连接关系，保证每个节点都能够被连接，从而可以解决管状结构追踪中的断线问题。As an example, a connection operation can be performed on each node in the tubular structure to obtain an adjusted tubular structure. FIG4 shows a schematic diagram of constructing connection edges according to an embodiment of the present application. As shown in FIG4 , a connection operation is performed on each node of the tubular structure, and at least one connection edge can be constructed, and these connection edges and nodes can constitute the adjusted tubular structure. In this way, by performing a connection operation on each node of the tubular structure, it can be ensured that each node of the tubular structure has a connection relationship with at least one other node other than itself, and each node can be connected, thereby solving the problem of disconnection in tubular structure tracking.
作为一个示例，可以将多个节点中的第一节点，与距其预设范围内的至少一个第二节点进行连接，得到调整后的管状结构；这样，针对多个节点中的每一节点，可以构建各节点对应的连接边，由这些连接边及该多个节点可以构成调整后的管状结构；其中，预设范围可以由本领域技术人员根据需要进行设定。例如，对于管状结构中的每个节点，可以将与其相距小于P个像素点的节点分别与其进行连接，构建每一节点对应的连接边，这些连接边和管状结构中的所有节点即可构成调整后的管状结构。由于管状结构中距离较近的节点之间存在连接关系的可能性较大，对于多个节点中的每个节点，将距其预设范围内的其他节点分别与其进行连接，可以将管状结构中可能存在连接关系的节点连接起来，这样通过对管状结构的节点进行过连接操作，所构建得到的过连接边属于管状结构对应的拓扑图的概率更大。As an example, a first node among multiple nodes can be connected to at least one second node within a preset range to obtain an adjusted tubular structure; in this way, for each of the multiple nodes, a connection edge corresponding to each node can be constructed, and these connection edges and the multiple nodes can constitute the adjusted tubular structure; wherein, the preset range can be set by a person skilled in the art as needed. For example, for each node in the tubular structure, nodes less than P pixels away from it can be connected to it respectively, and connection edges corresponding to each node can be constructed, and these connection edges and all nodes in the tubular structure can constitute the adjusted tubular structure. Since there is a greater possibility of a connection relationship between nodes that are closer in the tubular structure, for each of the multiple nodes, other nodes within a preset range can be connected to it respectively, and nodes that may have a connection relationship in the tubular structure can be connected, so that by performing an over-connection operation on the nodes of the tubular structure, the probability that the over-connection edge constructed belongs to the topological graph corresponding to the tubular structure is greater.
作为另一个示例，可以将多个节点中的第一节点，与其距离最近的M个第二节点进行连接，得到调整后的管状结构，这样，针对多个节点中的每一节点，可以构建各节点对应的连接边，由这些连接边及该多个节点可以构成调整后的管状结构；其中，M的数值可以由本领域技术人员根据需要进行设定。例如，对于管状结构中的每个节点，可以计算其他节点与该节点的距离，并可以选择距离最近的5个第二节点分别与其进行连接，构建每一节点对应的连接边，这些连接边和管状结构中的所有节点即可构成调整后的管状结构。 As another example, a first node among multiple nodes can be connected to the M second nodes closest to it to obtain an adjusted tubular structure. In this way, for each of the multiple nodes, a connection edge corresponding to each node can be constructed, and these connection edges and the multiple nodes can constitute the adjusted tubular structure; wherein the value of M can be set by a person skilled in the art as needed. For example, for each node in the tubular structure, the distance between other nodes and the node can be calculated, and the 5 second nodes closest to it can be selected to connect to them respectively, and the connection edge corresponding to each node can be constructed. These connection edges and all nodes in the tubular structure can constitute the adjusted tubular structure.
S304、根据调整后的管状结构，构建管状结构对应的至少一个拓扑图。S304: Construct at least one topological graph corresponding to the tubular structure according to the adjusted tubular structure.
其中，该拓扑图包括上述调整后的管状结构中的多个节点，还可以包括上述构建的各节点对应的一个或多个连接边。作为一个示例，该拓扑图中可以包括管状结构的每一节点，还可以包括各节点对应的一个或多个连接边。The topological graph includes the multiple nodes in the adjusted tubular structure, and may also include one or more connection edges corresponding to each of the nodes constructed above. As an example, the topological graph may include each node of the tubular structure, and may also include one or more connection edges corresponding to each of the nodes.
示例性地，该拓扑图为树形拓扑图；在生物医学等领域管状结构对应的拓扑图通常为树形拓扑图，因此可以构建管状结构对应的树形拓扑图作为生物医学等领域管状结构的追踪结果。Exemplarily, the topological map is a tree topological map; in the fields of biomedicine and the like, the topological map corresponding to the tubular structure is usually a tree topological map, so a tree topological map corresponding to the tubular structure can be constructed as a tracking result of the tubular structure in the fields of biomedicine and the like.
在一种可能的实现方式中，所述根据调整后的管状结构，构建所述管状结构对应的至少一个拓扑图，可以包括：根据所述调整后的管状结构中的至少一个节点、所述调整后的管状结构中的至少一个连接边和所述至少一个连接边对应的特征，构建所述至少一个拓扑图。In one possible implementation, constructing at least one topological graph corresponding to the tubular structure based on the adjusted tubular structure may include: constructing the at least one topological graph based on at least one node in the adjusted tubular structure, at least one connecting edge in the adjusted tubular structure, and a feature corresponding to the at least one connecting edge.
示例性地，所述至少一个连接边对应的特征指示所述至少一个连接边的几何特性，和/或所述至少一个连接边中至少一端节点的几何特性。Exemplarily, the feature corresponding to the at least one connecting edge indicates a geometric property of the at least one connecting edge and/or a geometric property of at least one end node in the at least one connecting edge.
作为一个示例，连接边对应的特征，可以包括连接边的几何特征，或者连接边至少一端节点的几何特征，或者包括这两者。例如，每一连接边对应的特征可以包括每一连接边至少一端节点的坐标、每一连接边的长度、每一连接边的斜率中的至少一项。每一连接边的长度及每一连接边的斜率可以根据该连接边两端节点的坐标确定。As an example, the feature corresponding to the connecting edge may include the geometric feature of the connecting edge, or the geometric feature of at least one end node of the connecting edge, or both. For example, the feature corresponding to each connecting edge may include at least one of the coordinates of at least one end node of each connecting edge, the length of each connecting edge, and the slope of each connecting edge. The length of each connecting edge and the slope of each connecting edge may be determined based on the coordinates of the two end nodes of the connecting edge.
这样，通过连接边对应的特征可以表征连接边的特性，根据调整后的管状结构中的节点、连接边和连接边对应的特征，可以筛选出更加准确的连接边，从而可以构建出更准确的管状结构对应的拓扑图。In this way, the characteristics of the connecting edges can be characterized by the features corresponding to the connecting edges. According to the nodes, connecting edges and the features corresponding to the connecting edges in the adjusted tubular structure, more accurate connecting edges can be screened out, thereby constructing a more accurate topological graph corresponding to the tubular structure.
示例性地，所述根据所述调整后的管状结构中的至少一个节点、所述调整后的管状结构中的至少一个连接边和所述至少一个连接边对应的特征，构建所述至少一个拓扑图，可以包括：根据所述至少一个连接边对应的特征，得到所述至少一个连接边的权值；根据所述至少一个节点、所述至少一个连接边和所述至少一个连接边的权值，构建所述至少一个拓扑图。Exemplarily, constructing the at least one topological graph based on at least one node in the adjusted tubular structure, at least one connecting edge in the adjusted tubular structure, and a feature corresponding to the at least one connecting edge may include: obtaining a weight of the at least one connecting edge based on a feature corresponding to the at least one connecting edge; and constructing the at least one topological graph based on the at least one node, the at least one connecting edge, and the weight of the at least one connecting edge.
示例性地，连接边的权值与该连接边属于管状结构对应的拓扑图的概率正相关。示例性地，可以根据连接边对应的特征，得到该连接边属于管状结构对应的拓扑图的概率，从而得到该连接边的权值；该过程可能实现方式的示例参见下文。这样，根据调整后的管状结构中的连接边对应的特征得到连接边的权值，连接边的权值越大则表示该连接边越有可能属于管状结构对应的拓扑图，反之，权值越小则表示该连接边属于管状结构对应的拓扑图的可能性越低；从而根据调整后的管状结构中的节点、连接边和连接边的权值，可以构建出更准确的管状结构对应的拓扑图。Exemplarily, the weight of the connection edge is positively correlated with the probability that the connection edge belongs to the topological graph corresponding to the tubular structure. Exemplarily, the probability that the connection edge belongs to the topological graph corresponding to the tubular structure can be obtained based on the features corresponding to the connection edge, thereby obtaining the weight of the connection edge; examples of possible implementations of this process are shown below. In this way, the weight of the connection edge is obtained based on the features corresponding to the connection edge in the adjusted tubular structure. The larger the weight of the connection edge, the more likely it is that the connection edge belongs to the topological graph corresponding to the tubular structure. Conversely, the smaller the weight, the lower the probability that the connection edge belongs to the topological graph corresponding to the tubular structure. Thus, based on the nodes, connection edges and connection edge weights in the adjusted tubular structure, a more accurate topological graph corresponding to the tubular structure can be constructed.
作为一个示例，所述根据所述至少一个节点、所述至少一个连接边和所述至少一个连接边的权值，构建所述至少一个拓扑图，可以包括：根据所述至少一个节点、所述至少一个连接边和所述至少一个连接边的权值，基于最小生成树(Minimum Spanning Tree，MST)算法，构建所述至少一个拓扑图。As an example, constructing the at least one topological graph according to the at least one node, the at least one connecting edge and the weight of the at least one connecting edge may include: constructing the at least one topological graph based on a minimum spanning tree (MST) algorithm according to the at least one node, the at least one connecting edge and the weight of the at least one connecting edge.
由于管状结构的追踪结果一般为树形拓扑图，而上述步骤S303中得到的调整后的管状结构可能包括环形结构，因此，可以基于最小生成树算法去除调整后的管状结构中的环形结构，构建管状结构对应的树形拓扑图；这样，基于最小生成树算法构建管状结构对应的树形拓扑图，可以降低计算复杂度，缩短追踪时间，提升追踪效率。例如，可以根据所述至少一个节点、所述至少一个连接边和所述至少一个连接边的权值，基于kruskal最小生成树算法，构建 管状结构对应的至少一个拓扑图。Since the tracking result of the tubular structure is generally a tree topology graph, and the adjusted tubular structure obtained in the above step S303 may include a ring structure, the ring structure in the adjusted tubular structure can be removed based on the minimum spanning tree algorithm to construct a tree topology graph corresponding to the tubular structure; thus, constructing a tree topology graph corresponding to the tubular structure based on the minimum spanning tree algorithm can reduce the computational complexity, shorten the tracking time, and improve the tracking efficiency. For example, according to the at least one node, the at least one connecting edge, and the weight of the at least one connecting edge, a tree topology graph corresponding to the tubular structure can be constructed based on the Kruskal minimum spanning tree algorithm. At least one topological graph corresponding to the tubular structure.
在一种可能的实现方式中，在构建的管状结构对应的至少一个拓扑图的数量为一个的情况下，可以将该拓扑图作为管状结构的追踪结果。In a possible implementation, when the number of at least one topological map corresponding to the constructed tubular structure is one, the topological map may be used as the tracking result of the tubular structure.
在一种可能的实现方式中，在构建的管状结构对应的至少一个拓扑图的数量为多个的情况下，可以对该至少一个拓扑图进行筛选。示例性地，可以对得到的多个拓扑图进行筛选，并可以将筛选出的一个拓扑图作为管状结构的追踪结果，从而可以进一步提高管状结构追踪结果的准确性。In a possible implementation, when there are multiple topological maps corresponding to the constructed tubular structure, the at least one topological map can be screened. For example, the multiple topological maps obtained can be screened, and the screened topological map can be used as the tracking result of the tubular structure, thereby further improving the accuracy of the tracking result of the tubular structure.
作为一个示例，对于管状结构对应的一个拓扑图，可以将该拓扑图中所有连接边的权值相加，计算该拓扑图的连接边的权值总和；进而可以选择连接边的权值总和最大的一个拓扑图作为管状结构的追踪结果。这样，通过计算各拓扑图的连接边的权值总和，自动筛选得到连接边的权值总和最高的拓扑图，由于连接边的权值与该连接边属于管状结构对应的拓扑图的概率正相关，选择连接边的权值总和最大的拓扑图作为管状结构的追踪结果，可以提高管状结构追踪结果的准确性。As an example, for a topology corresponding to a tubular structure, the weights of all the connecting edges in the topology can be added together to calculate the sum of the weights of the connecting edges of the topology; then the topology with the largest sum of the weights of the connecting edges can be selected as the tracking result of the tubular structure. In this way, by calculating the sum of the weights of the connecting edges of each topology, the topology with the highest sum of the weights of the connecting edges can be automatically screened. Since the weight of the connecting edge is positively correlated with the probability that the connecting edge belongs to the topology corresponding to the tubular structure, the topology with the largest sum of the weights of the connecting edges can be selected as the tracking result of the tubular structure, which can improve the accuracy of the tracking result of the tubular structure.
作为另一个示例，可以按照经验或根据实际需要对得到的多个拓扑图进行人工筛选，将筛选得到的一个拓扑图作为管状结构的追踪结果。这样，由用户对得到的多个拓扑图进行人工选择，并将选择的拓扑图作为管状结构的追踪结果，可以提高管状结构追踪结果的准确性和可用性。As another example, the multiple topological maps obtained can be manually screened according to experience or actual needs, and a topological map obtained by screening is used as the tracking result of the tubular structure. In this way, the user manually selects the multiple topological maps obtained and uses the selected topological map as the tracking result of the tubular structure, which can improve the accuracy and usability of the tubular structure tracking result.
这样，通过上述步骤S301-S304，对待处理图像中管状结构的节点进行过连接操作，得到调整后的管状结构；根据调整后的管状结构，构建管状结构对应的至少一个拓扑图；可以实现对管状结构进行自动追踪，无需大量人工操作，减少了追踪时间和追踪成本，提高了追踪效率；并且，通过对管状结构的多个节点进行过连接操作，可以保证管状结构的该多个节点都能够被连接，作为一个示例，可以对管状结构中的每个节点进行过连接操作，从而保证管状结构的每个节点都能够被连接，解决了管状结构追踪中的断线问题。In this way, through the above steps S301-S304, the nodes of the tubular structure in the processed image are over-connected to obtain an adjusted tubular structure; based on the adjusted tubular structure, at least one topological map corresponding to the tubular structure is constructed; the tubular structure can be automatically tracked without a lot of manual operation, thus reducing the tracking time and tracking cost and improving the tracking efficiency; and, by over-connecting the multiple nodes of the tubular structure, it can be ensured that the multiple nodes of the tubular structure can be connected. As an example, an over-connection operation can be performed on each node in the tubular structure, thereby ensuring that each node of the tubular structure can be connected, solving the problem of disconnection in tubular structure tracking.
下面对上述根据至少一个连接边对应的特征，得到至少一个连接边的权值的可能实现方式进行示例性地说明。The following is an exemplary description of a possible implementation method of obtaining the weight of at least one connecting edge according to the feature corresponding to at least one connecting edge.
图5示出根据本申请一实施例的一种图像处理方法的流程图，示例性地，该方法可以由上述图2中图像处理装置202执行，如图5所示，可以包括以下步骤：FIG5 shows a flow chart of an image processing method according to an embodiment of the present application. Exemplarily, the method may be performed by the image processing device 202 in FIG2 . As shown in FIG5 , the method may include the following steps:
S501、将至少一个连接边对应的特征输入预设模型，得到至少一个连接边属于管状结构对应的至少一个拓扑图的概率；其中，预设模型基于拓扑图训练样本中每一连接边对应的特征训练得到。S501. Input the feature corresponding to at least one connection edge into a preset model to obtain the probability that at least one connection edge belongs to at least one topological graph corresponding to the tubular structure; wherein the preset model is trained based on the feature corresponding to each connection edge in the topological graph training sample.
示例性地，拓扑图训练样本可以包括某一管状结构各节点对应的过连接边；拓扑图训练样本中每一连接边对应的特征可与图3的步骤S304中至少一个连接边对应的特征类型一致，例如可以包括每一连接边至少一端节点的坐标、每一连接边的长度、每一连接边的斜率；拓扑图训练样本中每一连接边对应的标签可以是该连接边是否属于该管状结构对应的拓扑图，若该连接边属于该管状结构对应的拓扑图，该连接边对应的标签可以为1；若该连接边不属于该管状结构对应的拓扑图，该连接边对应的标签可以为0。Exemplarily, the topological map training sample may include the connecting edges corresponding to each node of a tubular structure; the features corresponding to each connecting edge in the topological map training sample may be consistent with the feature type corresponding to at least one connecting edge in step S304 of Figure 3, for example, may include the coordinates of at least one end node of each connecting edge, the length of each connecting edge, and the slope of each connecting edge; the label corresponding to each connecting edge in the topological map training sample may be whether the connecting edge belongs to the topological map corresponding to the tubular structure. If the connecting edge belongs to the topological map corresponding to the tubular structure, the label corresponding to the connecting edge may be 1; if the connecting edge does not belong to the topological map corresponding to the tubular structure, the label corresponding to the connecting edge may be 0.
示例性地，预设模型可以是训练好的图神经网络模型，其中，图神经网络是指使用神经网络来学习拓扑图数据，提取和发掘拓扑图数据中的特征，满足聚类、分类、边预测、分割、 生成等图学习任务需求的算法总称；例如，可以为训练好的图卷积神经网络模型。示例性地，可以通过常规训练方式对图神经网络模型进行训练；图神经网络模型内的各个模型参数的初始参数可以是默认参数，在对图神经网络模型进行训练时，输入可以是拓扑图训练样本中每一连接边对应的特征，输出可以是该连接边属于管状结构对应的拓扑图的概率；可以在满足训练结束条件时，得到训练好的图神经网络模型(即预设模型)，训练结束条件可以由本领域技术人员根据实际需要进行设定；例如，可以根据输出的连接边属于管状结构对应的拓扑图的概率与该连接边属于管状结构对应的拓扑图的真实概率计算损失函数值，并利用该损失函数值调整图神经网络模型中的参数，直至损失函数收敛，完成训练，得到训练好的图神经网络模型。这样，使用图神经网络模型进行训练，可以更准确地学习到连接边对应的特征，将上述所构建的各连接边的特征输入搭配训练好的图神经网络模型中，可以更准确地预测上述所构建的各连接边属于管状结构对应的拓扑图的概率。Exemplarily, the preset model can be a trained graph neural network model, wherein the graph neural network refers to using a neural network to learn topological map data, extract and discover features in the topological map data, and meet the needs of clustering, classification, edge prediction, segmentation, A general term for algorithms that generate graph learning task requirements; for example, it can be a trained graph convolutional neural network model. Exemplarily, the graph neural network model can be trained by conventional training methods; the initial parameters of each model parameter in the graph neural network model can be default parameters. When the graph neural network model is trained, the input can be the features corresponding to each connection edge in the topological map training sample, and the output can be the probability that the connection edge belongs to the topological map corresponding to the tubular structure; when the training end condition is met, the trained graph neural network model (i.e., the preset model) can be obtained, and the training end condition can be set by those skilled in the art according to actual needs; for example, the loss function value can be calculated based on the probability that the output connection edge belongs to the topological map corresponding to the tubular structure and the true probability that the connection edge belongs to the topological map corresponding to the tubular structure, and the loss function value can be used to adjust the parameters in the graph neural network model until the loss function converges, the training is completed, and the trained graph neural network model is obtained. In this way, using the graph neural network model for training, the features corresponding to the connection edges can be learned more accurately, and the features of each connection edge constructed above are input into the trained graph neural network model, which can more accurately predict the probability that each connection edge constructed above belongs to the topological map corresponding to the tubular structure.
作为一个示例，可以将每一连接边至少一端节点的坐标、每一连接边的长度、每一连接边的斜率输入至训练好的图神经网络模型中，得到每一连接边属于待处理图像中的管状结构对应的拓扑图的概率。由于连接边至少一端节点的坐标、连接边的长度、连接边的斜率这些特征可以反映连接边的位置信息和几何特性，通过将这些特征输入至训练好的图神经网络模型中，可以更准确地预测连接边属于管状结构的概率。As an example, the coordinates of at least one end node of each connection edge, the length of each connection edge, and the slope of each connection edge can be input into a trained graph neural network model to obtain the probability that each connection edge belongs to the topological graph corresponding to the tubular structure in the image to be processed. Since the coordinates of at least one end node of the connection edge, the length of the connection edge, and the slope of the connection edge can reflect the position information and geometric characteristics of the connection edge, by inputting these features into the trained graph neural network model, the probability that the connection edge belongs to the tubular structure can be more accurately predicted.
S502、根据所述至少一个连接边属于所述管状结构对应的至少一个拓扑图的概率，得到至少一个连接边的权值。S502: Obtain a weight of at least one connection edge according to a probability that the at least one connection edge belongs to at least one topological graph corresponding to the tubular structure.
示例性地，权值可与概率成正相关。作为一个示例，可以将连接边属于待处理图像中的管状结构对应的拓扑图的概率作为该连接边的权值；作为另一个示例，可以将连接边属于待处理图像中的管状结构对应的拓扑图的概率乘上预设系数，作为该连接边的权值，预设系数可以由本领域技术人员进行设定。Exemplarily, the weight may be positively correlated with the probability. As an example, the probability that the connection edge belongs to the topological map corresponding to the tubular structure in the image to be processed may be used as the weight of the connection edge; as another example, the probability that the connection edge belongs to the topological map corresponding to the tubular structure in the image to be processed may be multiplied by a preset coefficient as the weight of the connection edge, and the preset coefficient may be set by a person skilled in the art.
本申请实施例中，通过将各连接边对应的特征输入至预设模型中，得到各连接边属于待处理图像中的管状结构对应的拓扑图的概率，从而得到各连接边的权值，从而将管状结构追踪问题转化为管状结构的连接边预测问题，与现有的管状结构追踪方法相比，将传统的迭代数值计算转化为图神经网络模型的矩阵计算，可以极大地减少计算量，减少内存占用，缩短追踪时间，从而提升管状结构的追踪效率。作为一个示例，本申请实施例提供的图像处理方法，与基于快速行进算法的追踪方法相比，可以将单例斑马鱼脑神经追踪的时间从1小时压缩到30秒。In an embodiment of the present application, by inputting the features corresponding to each connecting edge into a preset model, the probability that each connecting edge belongs to the topological graph corresponding to the tubular structure in the image to be processed is obtained, thereby obtaining the weight of each connecting edge, thereby converting the tubular structure tracking problem into the tubular structure connecting edge prediction problem. Compared with the existing tubular structure tracking method, converting the traditional iterative numerical calculation into the matrix calculation of the graph neural network model can greatly reduce the amount of calculation, reduce memory usage, shorten the tracking time, and thus improve the tracking efficiency of the tubular structure. As an example, the image processing method provided in an embodiment of the present application can compress the time for tracking a single zebrafish cranial nerve from 1 hour to 30 seconds compared with the tracking method based on the fast marching algorithm.
下面对上述根据所述至少一个节点、所述至少一个连接边和所述至少一个连接边的权值，构建所述管状结构对应的至少一个拓扑图的可能实现方式进行示例性地说明。The following is an exemplary description of possible implementations of constructing at least one topological graph corresponding to the tubular structure based on the at least one node, the at least one connecting edge and the weight of the at least one connecting edge.
图6示出根据本申请一实施例的一种图像处理方法的流程图，示例性地，该方法可以由上述图2中图像处理装置202执行，如图6所示，可以包括以下步骤：FIG6 shows a flow chart of an image processing method according to an embodiment of the present application. Exemplarily, the method may be performed by the image processing device 202 in FIG2 . As shown in FIG6 , the method may include the following steps:
S601、根据所构建的至少一个连接边的权值，确定第一连接边集合中权值最大的任一连接边；其中，第一连接边集合的初始状态包括该至少一个连接边。S601. Determine any connection edge with the largest weight in a first connection edge set according to the weight of at least one connection edge constructed; wherein the initial state of the first connection edge set includes the at least one connection edge.
示例性地，对于第一连接边集合中的连接边，可以按照各连接边的权值由大到小进行排序，从而确定第一连接边集合中权值最大的一条连接边；若第一连接边集合中权值最大的连接边有多条，可以从中选择任一连接边。 Exemplarily, the connecting edges in the first connecting edge set can be sorted from large to small according to the weights of each connecting edge, so as to determine the connecting edge with the largest weight in the first connecting edge set; if there are multiple connecting edges with the largest weight in the first connecting edge set, any one connecting edge can be selected from them.
S602、在所述权值最大的任一连接边与第二连接边集合中的连接边不构成环形结构的情况下，将所述权值最大的任一连接边添加到该第二连接边集合中；其中，第二连接边集合的初始状态为空集。S602. When any connecting edge with the largest weight does not form a ring structure with the connecting edges in the second connecting edge set, add any connecting edge with the largest weight to the second connecting edge set; wherein the initial state of the second connecting edge set is an empty set.
S603、从所述第一连接边集合中移除所述权值最大的任一连接边，以更新所述第一连接边集合；并基于更新后的所述第一连接边集合，重复执行上述确定第一连接边集合中权值最大的任一连接边及之后的操作，直到第二连接边集合中的连接边的数量为N-1，N为所述至少一个节点的数量。S603. Remove any connecting edge with the largest weight from the first connecting edge set to update the first connecting edge set; and based on the updated first connecting edge set, repeat the above-mentioned operations of determining any connecting edge with the largest weight in the first connecting edge set and subsequent operations until the number of connecting edges in the second connecting edge set is N-1, where N is the number of the at least one node.
S604、根据第二连接边集合及所述至少一个节点，构建管状结构对应的一个拓扑图。S604: Construct a topological graph corresponding to the tubular structure according to the second connection edge set and the at least one node.
示例性地，在第二连接边集合中包含N-1个连接边时，可以由第二连接边集合中的所有连接边及管状结构的节点构建出管状结构对应的一个拓扑图。Exemplarily, when the second connection edge set includes N-1 connection edges, a topological graph corresponding to the tubular structure can be constructed by all the connection edges in the second connection edge set and the nodes of the tubular structure.
本申请实施例中，由于第二连接边集合中所有的连接边都不构成环形结构，可以保证构建得到的管状结构对应的拓扑图为树形拓扑图；由于管状结构的节点的数量为N个，在第二连接边集合中包含N-1个连接边时，可以保证管状结构的每个节点都能被连接，从而可以解决管状结构追踪中的断线问题；每次选择第一连接边集合中权值最大的任一连接边判断是否添加进第二连接边集合，由于连接边的权值与该连接边属于管状结构对应的拓扑图的概率正相关，这样可以保证构建得到的管状结构对应的拓扑图的准确性更高。In the embodiment of the present application, since all the connecting edges in the second connecting edge set do not form a ring structure, it can be ensured that the topological graph corresponding to the constructed tubular structure is a tree topological graph; since the number of nodes of the tubular structure is N, when the second connecting edge set contains N-1 connecting edges, it can be ensured that each node of the tubular structure can be connected, thereby solving the problem of disconnection in tubular structure tracking; each time, any connecting edge with the largest weight in the first connecting edge set is selected to determine whether to add it to the second connecting edge set. Since the weight of the connecting edge is positively correlated with the probability that the connecting edge belongs to the topological graph corresponding to the tubular structure, this can ensure that the topological graph corresponding to the constructed tubular structure is more accurate.
示例性地，可以多次执行步骤S601～S604，由于步骤S601中第一连接边集合中权值最大的连接边可能有多条，可以构建出管状结构对应的多个拓扑图。Exemplarily, steps S601 to S604 may be performed multiple times. Since there may be multiple connection edges with the largest weight in the first connection edge set in step S601, multiple topological graphs corresponding to the tubular structure may be constructed.
图7示出根据本申请一实施的构建管状结构对应的多个拓扑图的示意图，如图7所示，对于上述图4所示的各连接边，各连接边的权值为该连接边属于管状结构对应的拓扑图的概率，管状结构对应的拓扑图包括节点集合和连接边集合，节点集合由管状结构的所有节点构成，连接边集合(即上述第二连接边集合的示例)的初始状态为空集；可以按照各连接边的权值由大到小对连接边进行排序，按照连接边的排序依次进行判断，若连接边与管状结构对应的拓扑图的连接边集合中包含的连接边不构成环形结构，则将该连接边添加进连接边集合中，直到连接边集合中包含N-1个连接边时，停止判断，N为管状结构的节点的数量；根据此时的节点集合和连接边集合可以构建管状结构对应的一个拓扑图；在对权值相同的连接边进行判断时，可以按照不同的顺序进行判断，例如，权值为0.9的连接边有4个，分别为连接边a、连接边b、连接边c、连接边d，可以按照连接边a、连接边b、连接边c、连接边d的顺序依次判断各连接边是否可以添加进连接边集合中，也可以按照连接边d、连接边c、连接边b、连接边a的顺序依次判断各连接边是否可以添加进连接边集合中；按照不同的顺序对权值相同的连接边进行判断，最后可以得到不同的连接边集合，从而可以构建管状结构对应的多个不同的拓扑图，例如图7中的拓扑图1、2等。Figure 7 shows a schematic diagram of constructing multiple topological graphs corresponding to a tubular structure according to an implementation of the present application. As shown in Figure 7, for each connection edge shown in Figure 4 above, the weight of each connection edge is the probability that the connection edge belongs to the topological graph corresponding to the tubular structure. The topological graph corresponding to the tubular structure includes a node set and a connection edge set. The node set is composed of all nodes of the tubular structure. The initial state of the connection edge set (i.e., the example of the second connection edge set mentioned above) is an empty set; the connection edges can be sorted from large to small according to the weight of each connection edge, and judgment is performed in sequence according to the sorting of the connection edges. If the connection edge and the connection edge included in the connection edge set of the topological graph corresponding to the tubular structure do not form a ring structure, the connection edge is added to the connection edge set until the connection edge set contains N-1 connection edges, and the judgment is stopped, where N is the number of nodes of the tubular structure. number; a topological graph corresponding to the tubular structure can be constructed according to the node set and the edge set at this time; when judging the edges with the same weight, the judgment can be made in different orders. For example, there are 4 edges with a weight of 0.9, namely edge a, edge b, edge c, and edge d. It can be judged in the order of edge a, edge b, edge c, and edge d whether each edge can be added to the edge set, or it can be judged in the order of edge d, edge c, edge b, and edge a whether each edge can be added to the edge set; the edges with the same weight can be judged in different orders, and finally different edge sets can be obtained, so that multiple different topological graphs corresponding to the tubular structure can be constructed, such as topological graphs 1 and 2 in Figure 7.
举例来说，可以应用本申请实施例提供的上述图像处理方法对鼠脑神经进行自动追踪。待处理图像可以是显微镜扫描的3D鼠脑图像，待处理图像中的管状结构为鼠脑神经。将待处理图像输入至训练好的分割模型中，可以得到鼠脑神经分割结果；例如，该分割模型可以是3D U-Net网络模型，可以基于已标注的鼠脑神经数据集对3D U-Net网络模型训练，得到训练好的分割模型。可以使用基于最大圆盘的骨架提取算法，在鼠脑神经分割结果中生成一系列的内切圆盘，连接其圆心，得到鼠脑神经的骨架线。图8(a)-(b)示出根据本申请一实施例的提取鼠脑神经的骨架线的示意图，图8(a)示出根据本申请一实施例的鼠脑神经分 割结果，图8(b)示出根据本申请一实施例的鼠脑神经的骨架线。根据鼠脑神经的骨架线，可以确定鼠脑神经的各个节点。对于鼠脑神经的每一个节点，可以将其周围距离小于P个像素的其它节点分别与其进行连接，从而构建过连接的神经拓扑图。将过连接的神经拓扑图中各连接边对应的特征输入至训练好的图神经网络模型中，可以得到各连接边属于鼠脑神经对应的拓扑图的概率；连接边对应的特征可以包括连接边至少一端节点的坐标、连接边的长度和连接边的斜率。可以将各连接边属于鼠脑神经对应的拓扑图的概率作为各连接边的权值，基于最小生成树算法，构建鼠脑神经对应的至少一个树形拓扑图；基于最小生成树算法构建树形拓扑图的方法可以参照图6中步骤S601～S604。在鼠脑神经对应的树形拓扑图的数量为一个的情况下，可以将该树形拓扑图作为鼠脑神经的追踪结果；在鼠脑神经对应的树形拓扑图的数量为多个的情况下，对于鼠脑神经对应的每个树形拓扑图，可以将其所有连接边的权值相加，计算得到每个树形拓扑图的连接边权值总和，比较各树形拓扑图的连接边权值总和，可以将连接边权值总和最大的树形拓扑图作为鼠脑神经的追踪结果，从而实现端到端的管状结构自动追踪。For example, the above-mentioned image processing method provided in the embodiment of the present application can be used to automatically track rat brain nerves. The image to be processed can be a 3D rat brain image scanned by a microscope, and the tubular structure in the image to be processed is the rat brain nerve. The image to be processed is input into a trained segmentation model to obtain a rat brain nerve segmentation result; for example, the segmentation model can be a 3D U-Net network model, and the 3D U-Net network model can be trained based on a labeled rat brain nerve data set to obtain a trained segmentation model. A skeleton extraction algorithm based on the maximum disk can be used to generate a series of inscribed disks in the rat brain nerve segmentation result, and the centers of the circles are connected to obtain the skeleton lines of the rat brain nerves. Figure 8(a)-(b) shows a schematic diagram of extracting the skeleton lines of rat brain nerves according to an embodiment of the present application, and Figure 8(a) shows a schematic diagram of extracting the skeleton lines of rat brain nerves according to an embodiment of the present application. The cutting result is shown in FIG8(b) of the skeleton line of the rat brain nerve according to an embodiment of the present application. According to the skeleton line of the rat brain nerve, each node of the rat brain nerve can be determined. For each node of the rat brain nerve, other nodes with a distance of less than P pixels around it can be connected to it respectively, so as to construct a connected neural topology map. The features corresponding to each connection edge in the over-connected neural topology map are input into the trained graph neural network model, and the probability that each connection edge belongs to the topology map corresponding to the rat brain nerve can be obtained; the features corresponding to the connection edge may include the coordinates of at least one end node of the connection edge, the length of the connection edge and the slope of the connection edge. The probability that each connection edge belongs to the topology map corresponding to the rat brain nerve can be used as the weight of each connection edge, and at least one tree topology map corresponding to the rat brain nerve can be constructed based on the minimum spanning tree algorithm; the method for constructing a tree topology map based on the minimum spanning tree algorithm can refer to steps S601 to S604 in FIG6. When the number of tree topology graphs corresponding to mouse brain nerves is one, the tree topology graph can be used as the tracing result of the mouse brain nerves; when the number of tree topology graphs corresponding to mouse brain nerves is multiple, for each tree topology graph corresponding to the mouse brain nerves, the weights of all its connecting edges can be added up, and the sum of the connecting edge weights of each tree topology graph can be calculated. By comparing the sums of the connecting edge weights of each tree topology graph, the tree topology graph with the largest sum of the connecting edge weights can be used as the tracing result of the mouse brain nerves, thereby realizing end-to-end automatic tracking of the tubular structure.
本申请实施例提供的图像处理方法，通过构建调整后的管状结构，可以保证管状结构的每个节点都被连接，从而可以避免由于管状结构的信号弱或存在干扰信号而产生的断线问题；基于图神经网络模型预测每个连接边属于管状结构对应的拓扑图的概率，将管状结构追踪问题转化为管状结构的连接边预测问题，与现有的管状结构追踪方法相比，将传统的迭代数值计算转化为一次神经网络矩阵计算，可以极大地减少计算量，减少内存占用，缩短追踪时间，提升管状结构的追踪效率。The image processing method provided in the embodiment of the present application can ensure that each node of the tubular structure is connected by constructing an adjusted tubular structure, thereby avoiding the disconnection problem caused by weak signal or interference signal of the tubular structure; based on the graph neural network model, the probability of each connection edge belonging to the topological graph corresponding to the tubular structure is predicted, and the tubular structure tracking problem is converted into a tubular structure connection edge prediction problem. Compared with the existing tubular structure tracking method, the traditional iterative numerical calculation is converted into a neural network matrix calculation, which can greatly reduce the amount of calculation, reduce memory usage, shorten the tracking time, and improve the tracking efficiency of the tubular structure.
基于上述方法实施例的同一发明构思，本申请的实施例还提供了一种图像处理装置，该图像处理装置可以用于执行上述方法实施例所描述的技术方案。例如，可以执行上述图3、图5或图6中所示图像处理方法的各步骤。Based on the same inventive concept of the above method embodiments, the embodiments of the present application further provide an image processing device, which can be used to execute the technical solution described in the above method embodiments. For example, the steps of the image processing method shown in FIG. 3, FIG. 5 or FIG. 6 can be executed.
图9示出根据本申请一实施例的图像处理装置的框图，如图9所示，该装置可以包括：获取模块901，用于获取待处理图像，所述待处理图像包括管状结构；节点确定模块902，用于确定所述管状结构中的多个节点；连接模块903，用于将所述多个节点中的第一节点与至少一个第二节点进行连接，得到调整后的管状结构，其中，所述第二节点为所述多个节点中除所述第一节点外的节点；构建模块904，用于根据调整后的管状结构，构建所述管状结构对应的至少一个拓扑图。Figure 9 shows a block diagram of an image processing device according to an embodiment of the present application. As shown in Figure 9, the device may include: an acquisition module 901, used to acquire an image to be processed, wherein the image to be processed includes a tubular structure; a node determination module 902, used to determine multiple nodes in the tubular structure; a connection module 903, used to connect a first node among the multiple nodes with at least one second node to obtain an adjusted tubular structure, wherein the second node is a node among the multiple nodes other than the first node; a construction module 904, used to construct at least one topological graph corresponding to the tubular structure according to the adjusted tubular structure.
本申请实施例，通过对待处理图像中管状结构的节点进行过连接操作，得到调整后的管状结构；根据调整后的管状结构，构建管状结构对应的至少一个拓扑图；可以实现对管状结构进行自动追踪，无需大量人工操作，减少了追踪时间和追踪成本，提高了追踪效率；并且，通过对管状结构的多个节点进行过连接操作，可以保证管状结构的该多个节点都能够被连接，作为一个示例，可以对管状结构中的每个节点进行过连接操作，从而保证管状结构的每个节点都能够被连接，解决了管状结构追踪中的断线问题。In the embodiment of the present application, an adjusted tubular structure is obtained by performing an over-connection operation on the nodes of the tubular structure in the processed image; at least one topological map corresponding to the tubular structure is constructed based on the adjusted tubular structure; automatic tracking of the tubular structure can be achieved without a large amount of manual operation, thus reducing tracking time and tracking costs and improving tracking efficiency; and, by performing an over-connection operation on multiple nodes of the tubular structure, it can be ensured that the multiple nodes of the tubular structure can be connected. As an example, an over-connection operation can be performed on each node in the tubular structure, thereby ensuring that each node of the tubular structure can be connected, thereby solving the problem of disconnection in tubular structure tracking.
在一种可能的实现方式中，该图像处理装置还可以包括：筛选模块，用于在所述至少一个拓扑图的数量为多个的情况下，对所述至少一个拓扑图进行筛选。In a possible implementation manner, the image processing device may further include: a screening module, configured to screen the at least one topological map when there are multiple at least one topological maps.
在一种可能的实现方式中，所述连接模块903，还用于：将所述第一节点，与距其预设范围内的至少一个所述第二节点进行连接，得到所述调整后的管状结构。In a possible implementation, the connection module 903 is further used to connect the first node with at least one second node within a preset range thereof to obtain the adjusted tubular structure.
在一种可能的实现方式中，所述构建模块904，还用于：根据所述调整后的管状结构中 的至少一个节点、所述调整后的管状结构中的至少一个连接边和所述至少一个连接边对应的特征，构建所述至少一个拓扑图。In a possible implementation, the construction module 904 is further configured to: The at least one topological graph is constructed based on at least one node of the adjusted tubular structure, at least one connecting edge in the adjusted tubular structure, and a feature corresponding to the at least one connecting edge.
在一种可能的实现方式中，所述构建模块904，还用于：根据所述至少一个连接边对应的特征，得到所述至少一个连接边的权值；根据所述至少一个节点、所述至少一个连接边和所述至少一个连接边的权值，构建所述至少一个拓扑图。In a possible implementation, the construction module 904 is further used to: obtain the weight of the at least one connection edge according to the characteristics corresponding to the at least one connection edge; and construct the at least one topological graph according to the at least one node, the at least one connection edge and the weight of the at least one connection edge.
在一种可能的实现方式中，所述构建模块904，还用于：将所述至少一个连接边对应的特征输入预设模型，得到所述至少一个连接边属于所述至少一个拓扑图的概率；根据所述至少一个连接边属于所述至少一个拓扑图的概率，得到所述至少一个连接边的权值；其中，所述预设模型基于拓扑图训练样本中每一连接边对应的特征训练得到。In one possible implementation, the construction module 904 is also used to: input the features corresponding to the at least one connection edge into a preset model to obtain the probability that the at least one connection edge belongs to the at least one topological graph; obtain the weight of the at least one connection edge based on the probability that the at least one connection edge belongs to the at least one topological graph; wherein the preset model is trained based on the features corresponding to each connection edge in the topological graph training sample.
在一种可能的实现方式中，所述至少一个连接边对应的特征指示所述至少一个连接边的几何特性，和/或所述至少一个连接边中至少一端节点的几何特性。In a possible implementation manner, the feature corresponding to the at least one connecting edge indicates a geometric property of the at least one connecting edge and/or a geometric property of at least one end node in the at least one connecting edge.
在一种可能的实现方式中，所述预设模型为图神经网络模型。In a possible implementation, the preset model is a graph neural network model.
在一种可能的实现方式中，所述构建模块904，还用于：根据所述至少一个节点、所述至少一个连接边和所述至少一个连接边的权值，基于最小生成树算法，构建所述至少一个拓扑图。In a possible implementation, the construction module 904 is further used to: construct the at least one topological graph based on a minimum spanning tree algorithm according to the at least one node, the at least one connecting edge and the weight of the at least one connecting edge.
在一种可能的实现方式中，所述拓扑图为树形拓扑图。In a possible implementation manner, the topology map is a tree topology map.
在一种可能的实现方式中，所述构建模块904，还用于：根据所述至少一个连接边的权值，确定第一连接边集合中权值最大的任一连接边；其中，所述第一连接边集合的初始状态包括所述至少一个连接边；在所述权值最大的任一连接边与所述第二连接边集合中的连接边不构成环形结构的情况下，将所述权值最大的任一连接边添加到所述第二连接边集合中；其中，所述第二连接边集合的初始状态为空集；从所述第一连接边集合中移除所述权值最大的任一连接边，以更新所述第一连接边集合；并基于更新后的所述第一连接边集合，重复执行上述确定第一连接边集合中权值最大的任一连接边及之后的操作，直到所述第二连接边集合中的连接边的数量为N-1，N为所述至少一个节点的数量；根据所述第二连接边集合及所述至少一个节点，构建所述管状结构对应的一个拓扑图。In one possible implementation, the construction module 904 is also used to: determine any connecting edge with the largest weight in the first connecting edge set based on the weight of the at least one connecting edge; wherein the initial state of the first connecting edge set includes the at least one connecting edge; when any connecting edge with the largest weight and the connecting edges in the second connecting edge set do not form a ring structure, add any connecting edge with the largest weight to the second connecting edge set; wherein the initial state of the second connecting edge set is an empty set; remove any connecting edge with the largest weight from the first connecting edge set to update the first connecting edge set; and based on the updated first connecting edge set, repeat the above-mentioned determination of any connecting edge with the largest weight in the first connecting edge set and subsequent operations until the number of connecting edges in the second connecting edge set is N-1, where N is the number of the at least one node; and construct a topological graph corresponding to the tubular structure based on the second connecting edge set and the at least one node.
在一种可能的实现方式中，所述节点确定模块902，还用于：提取所述管状结构的骨架线；根据所述管状结构的骨架线，确定所述管状结构中的多个节点。In a possible implementation, the node determination module 902 is further configured to: extract a skeleton line of the tubular structure; and determine a plurality of nodes in the tubular structure according to the skeleton line of the tubular structure.
上述图9所示的图像处理装置及其各种可能的实现方式的技术效果及具体描述可参见上述图像处理方法，此处不再赘述。The technical effects and specific descriptions of the image processing device shown in FIG. 9 and its various possible implementations can be found in the above-mentioned image processing method, which will not be repeated here.
应理解以上装置中各模块的划分仅是一种逻辑功能的划分，实际实现时可以全部或部分集成到一个物理实体上，也可以物理上分开。此外，装置中的模块可以以处理器调用软件的形式实现；例如装置包括处理器，处理器与存储器连接，存储器中存储有指令，处理器调用存储器中存储的指令，以实现以上任一种方法或实现该装置各模块的功能，其中处理器例如为通用处理器，例如中央处理单元(Central Processing Unit，CPU)或微处理器，存储器为装置内的存储器或装置外的存储器。或者，装置中的模块可以以硬件电路的形式实现，可以通过对硬件电路的设计实现部分或全部模块的功能，该硬件电路可以理解为一个或多个处理器；例如，在一种实现中，该硬件电路为专用集成电路(application-specific integrated circuit，ASIC)，通过对电路内元件逻辑关系的设计，实现以上部分或全部模块的功能；再如，在另一种实现中，该硬件电路为可以通过可编程逻辑器件(programmable logic device，PLD)实现，以现 场可编程门阵列(Field Programmable Gate Array，FPGA)为例，其可以包括大量逻辑门电路，通过配置文件来配置逻辑门电路之间的连接关系，从而实现以上部分或全部模块的功能。以上装置的所有模块可以全部通过处理器调用软件的形式实现，或全部通过硬件电路的形式实现，或部分通过处理器调用软件的形式实现，剩余部分通过硬件电路的形式实现。It should be understood that the division of the modules in the above device is only a division of logical functions. In actual implementation, they can be fully or partially integrated into one physical entity, or they can be physically separated. In addition, the modules in the device can be implemented in the form of a processor calling software; for example, the device includes a processor, the processor is connected to a memory, the memory stores instructions, and the processor calls the instructions stored in the memory to implement any of the above methods or to implement the functions of the modules of the device, wherein the processor is, for example, a general-purpose processor, such as a central processing unit (CPU) or a microprocessor, and the memory is a memory inside the device or a memory outside the device. Alternatively, the modules in the device can be implemented in the form of hardware circuits, and the functions of some or all of the modules can be implemented by designing the hardware circuits, and the hardware circuits can be understood as one or more processors; for example, in one implementation, the hardware circuit is an application-specific integrated circuit (ASIC), and the functions of some or all of the above modules are implemented by designing the logical relationship of the components in the circuit; for example, in another implementation, the hardware circuit can be implemented by a programmable logic device (PLD) to realize the present Taking a Field Programmable Gate Array (FPGA) as an example, it can include a large number of logic gate circuits, and the connection relationship between the logic gate circuits is configured through a configuration file to realize the functions of some or all of the above modules. All modules of the above device can be implemented in the form of a processor calling software, or in the form of hardware circuits, or in part in the form of a processor calling software, and the rest in the form of hardware circuits.
在本申请实施例中，处理器是一种具有信号的处理能力的电路，在一种实现中，处理器可以是具有指令读取与运行能力的电路，例如CPU、微处理器、图形处理器(graphics processing unit，GPU)、数字信号处理器(digital signal processor，DSP)、神经网络处理器(neural-network processing unit，NPU)、张量处理器(tensor processing unit，TPU)等；在另一种实现中，处理器可以通过硬件电路的逻辑关系实现一定功能，该硬件电路的逻辑关系是固定的或可以重构的，例如处理器为ASIC或PLD实现的硬件电路，例如FPGA。在可重构的硬件电路中，处理器加载配置文档，实现硬件电路配置的过程，可以理解为处理器加载指令，以实现以上部分或全部模块的功能的过程。In the embodiments of the present application, the processor is a circuit with the ability to process signals. In one implementation, the processor may be a circuit with the ability to read and run instructions, such as a CPU, a microprocessor, a graphics processing unit (GPU), a digital signal processor (DSP), a neural-network processing unit (NPU), a tensor processing unit (TPU), etc.; in another implementation, the processor may implement certain functions through the logical relationship of a hardware circuit, and the logical relationship of the hardware circuit is fixed or reconfigurable, such as a hardware circuit implemented by an ASIC or PLD, such as an FPGA. In a reconfigurable hardware circuit, the process of the processor loading a configuration document to implement the hardware circuit configuration can be understood as the process of the processor loading instructions to implement the functions of some or all of the above modules.
可见，以上装置中的各模块可以是被配置成实施以上实施例方法的一个或多个处理器(或处理电路)，例如：CPU、GPU、NPU、TPU、微处理器、DSP、ASIC、FPGA，或这些处理器形式中至少两种的组合。此外，以上装置中的各模块可以全部或部分可以集成在一起，或者可以独立实现，对此不作限定。It can be seen that each module in the above device can be one or more processors (or processing circuits) configured to implement the above embodiment method, such as: CPU, GPU, NPU, TPU, microprocessor, DSP, ASIC, FPGA, or a combination of at least two of these processor forms. In addition, each module in the above device can be fully or partially integrated together, or can be implemented independently, which is not limited.
本申请的实施例还提供了一种图像处理装置，包括：处理器；用于存储处理器可执行指令的存储器；其中，所述处理器被配置为执行所述指令时实现上述实施例的方法。示例性地，可以执行上述图3、图5或图6中所示图像处理方法的各步骤。The embodiment of the present application also provides an image processing device, comprising: a processor; a memory for storing instructions executable by the processor; wherein the processor is configured to implement the method of the above embodiment when executing the instructions. Exemplarily, each step of the image processing method shown in FIG. 3, FIG. 5 or FIG. 6 can be executed.
图10示出根据本申请一实施例的一种电子设备的结构示意图，如图10所示，该电子设备可以包括：至少一个处理器1001，通信线路1002，存储器1003以及至少一个通信接口1004。FIG10 shows a schematic diagram of the structure of an electronic device according to an embodiment of the present application. As shown in FIG10 , the electronic device may include: at least one processor 1001 , a communication line 1002 , a memory 1003 and at least one communication interface 1004 .
处理器1001可以是一个通用中央处理器，微处理器，特定应用集成电路，或一个或多个用于控制本申请方案程序执行的集成电路；处理器1001也可以包括多个通用处理器的异构运算架构，例如，可以是CPU、GPU、微处理器、DSP、ASIC、FPGA中至少两种的组合；作为一个示例，处理器1001可以是CPU+GPU或者CPU+ASIC或者CPU+FPGA。Processor 1001 can be a general-purpose central processing unit, a microprocessor, a specific application integrated circuit, or one or more integrated circuits for controlling the execution of the program of the present application; processor 1001 can also include a heterogeneous computing architecture of multiple general-purpose processors, for example, it can be a combination of at least two of CPU, GPU, microprocessor, DSP, ASIC, FPGA; as an example, processor 1001 can be CPU+GPU or CPU+ASIC or CPU+FPGA.
通信线路1002可包括一通路，在上述组件之间传送信息。The communication link 1002 may include a pathway to transmit information between the above-mentioned components.
通信接口1004，使用任何收发器一类的装置，用于与其他设备或通信网络通信，如以太网，RAN，无线局域网(wireless local area networks，WLAN)等。The communication interface 1004 uses any transceiver-like device for communicating with other devices or communication networks, such as Ethernet, RAN, wireless local area networks (WLAN), etc.
存储器1003可以是只读存储器(read-only memory，ROM)或可存储静态信息和指令的其他类型的静态存储设备，随机存取存储器(random access memory，RAM)或者可存储信息和指令的其他类型的动态存储设备，也可以是电可擦可编程只读存储器(electrically erasable programmable read-only memory，EEPROM)、只读光盘(compact disc read-only memory，CD-ROM)或其他光盘存储、光碟存储(包括压缩光碟、激光碟、光碟、数字通用光碟、蓝光光碟等)、磁盘存储介质或者其他磁存储设备、或者能够用于携带或存储具有指令或数据结构形式的期望的程序代码并能够由计算机存取的任何其他介质，但不限于此。存储器可以是独立存在，通过通信线路1002与处理器相连接。存储器也可以和处理器集成在一起。本申请实施例提供的存储器通常可以具有非易失性。其中，存储器1003用于存储执行本申请方案的计算机执行指令，并由处理器1001来控制执行。处理器1001用于执行存储器1003中存储的计算机执行指令，从而实现本申请上述实施例中提供的方法；示例性地，可以实现上述图3、 图5或图6中所示图像处理方法的各步骤。The memory 1003 can be a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (RAM) or other types of dynamic storage devices that can store information and instructions, or an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical disc, laser disc, optical disc, digital versatile disc, Blu-ray disc, etc.), a disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of an instruction or data structure and can be accessed by a computer, but is not limited to this. The memory can be independent and connected to the processor through a communication line 1002. The memory can also be integrated with the processor. The memory provided in the embodiment of the present application can generally have non-volatility. Among them, the memory 1003 is used to store the computer execution instructions for executing the scheme of the present application, and is controlled by the processor 1001 to execute. The processor 1001 is used to execute the computer-executable instructions stored in the memory 1003, so as to implement the method provided in the above embodiments of the present application; illustratively, the above FIG. 3, The steps of the image processing method shown in FIG. 5 or FIG. 6 .
可选的，本申请实施例中的计算机执行指令也可以称之为应用程序代码，本申请实施例对此不作具体限定。Optionally, the computer-executable instructions in the embodiments of the present application may also be referred to as application code, which is not specifically limited in the embodiments of the present application.
示例性地，处理器1001可以包括一个或多个CPU，例如，图10中的CPU0；处理器1001也可以包括一个CPU，及GPU、ASIC、FPGA中任一个，例如，图10中的CPU0+GPU0或者CPU 0+ASIC0或者CPU0+FPGA0。Exemplarily, the processor 1001 may include one or more CPUs, for example, CPU0 in FIG. 10 ; the processor 1001 may also include a CPU and any one of a GPU, an ASIC, and an FPGA, for example, CPU0+GPU0 or CPU 0+ASIC0 or CPU0+FPGA0 in FIG. 10 .
示例性地，该电子设备可以包括多个处理器，例如图10中的处理器1001和处理器1007。这些处理器中的每一个可以是一个单核(single-CPU)处理器，也可以是一个多核(multi-CPU)处理器，或者是包括多个通用处理器的异构运算架构。这里的处理器可以指一个或多个设备、电路、和/或用于处理数据(例如计算机程序指令)的处理核。Exemplarily, the electronic device may include multiple processors, such as processor 1001 and processor 1007 in FIG. 10. Each of these processors may be a single-core (single-CPU) processor, a multi-core (multi-CPU) processor, or a heterogeneous computing architecture including multiple general-purpose processors. The processor here may refer to one or more devices, circuits, and/or processing cores for processing data (e.g., computer program instructions).
在具体实现中，作为一种实施例，该电子设备还可以包括输出设备1005和输入设备1006。输出设备1005和处理器1001通信，可以以多种方式来显示信息。例如，输出设备1005可以是液晶显示器(liquid crystal display，LCD)，发光二级管(light emitting diode，LED)显示设备，阴极射线管(cathode ray tube，CRT)显示设备，或投影仪(projector)等，例如，可以为车载HUD、AR-HUD、显示器等显示设备。输入设备1006和处理器1001通信，可以以多种方式接收用户的输入。例如，输入设备1006可以是鼠标、键盘、触摸屏设备或传感设备等。In a specific implementation, as an embodiment, the electronic device may further include an output device 1005 and an input device 1006. The output device 1005 communicates with the processor 1001 and may display information in a variety of ways. For example, the output device 1005 may be a liquid crystal display (LCD), a light emitting diode (LED) display device, a cathode ray tube (CRT) display device, or a projector, etc. For example, it may be a display device such as a vehicle-mounted HUD, an AR-HUD, a display, etc. The input device 1006 communicates with the processor 1001 and may receive user input in a variety of ways. For example, the input device 1006 may be a mouse, a keyboard, a touch screen device, or a sensor device, etc.
本申请的实施例提供了一种计算机可读存储介质，其上存储有计算机程序指令，所述计算机程序指令被处理器执行时实现上述实施例中的方法。示例性地，可以实现上述图3、图5或图6中所示图像处理方法的各步骤。The embodiments of the present application provide a computer-readable storage medium on which computer program instructions are stored, and when the computer program instructions are executed by a processor, the method in the above embodiments is implemented. Exemplarily, each step of the image processing method shown in Figure 3, Figure 5 or Figure 6 can be implemented.
本申请的实施例提供了一种计算机程序产品，例如，可以包括计算机可读代码，或者承载有计算机可读代码的非易失性计算机可读存储介质；当所述计算机程序产品在计算机上运行时，使得所述计算机执行上述实施例中的方法。示例性地，可以执行上述图3、图5或图6中所示图像处理方法的各步骤。The embodiments of the present application provide a computer program product, which may include, for example, a computer-readable code or a non-volatile computer-readable storage medium carrying the computer-readable code; when the computer program product is run on a computer, the computer executes the method in the above embodiment. Exemplarily, each step of the image processing method shown in FIG. 3, FIG. 5 or FIG. 6 may be executed.
计算机可读存储介质可以是可以保持和存储由指令执行设备使用的指令的有形设备。计算机可读存储介质例如可以是――但不限于――电存储设备、磁存储设备、光存储设备、电磁存储设备、半导体存储设备或者上述的任意合适的组合。计算机可读存储介质的更具体的例子(非穷举的列表)包括：便携式计算机盘、硬盘、随机存取存储器(RAM)、只读存储器(ROM)、可擦式可编程只读存储器(EPROM或闪存)、静态随机存取存储器(SRAM)、便携式压缩盘只读存储器(CD-ROM)、数字多功能盘(DVD)、记忆棒、软盘、机械编码设备、例如其上存储有指令的打孔卡或凹槽内凸起结构、以及上述的任意合适的组合。这里所使用的计算机可读存储介质不被解释为瞬时信号本身，诸如无线电波或者其他自由传播的电磁波、通过波导或其他传输媒介传播的电磁波(例如，通过光纤电缆的光脉冲)、或者通过电线传输的电信号。A computer-readable storage medium may be a tangible device that can hold and store instructions used by an instruction execution device. A computer-readable storage medium may be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. More specific examples of computer-readable storage media (a non-exhaustive list) include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disk read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device, such as a punch card or a raised structure in a groove on which instructions are stored, and any suitable combination of the foregoing. As used herein, a computer-readable storage medium is not to be interpreted as a transient signal per se, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagating through a waveguide or other transmission medium (e.g., a light pulse through a fiber optic cable), or an electrical signal transmitted through a wire.
这里所描述的计算机可读程序指令可以从计算机可读存储介质下载到各个计算/处理设备，或者通过网络、例如因特网、局域网、广域网和/或无线网下载到外部计算机或外部存储设备。网络可以包括铜传输电缆、光纤传输、无线传输、路由器、防火墙、交换机、网关计算机和/或边缘服务器。每个计算/处理设备中的网络适配卡或者网络接口从网络接收计算机可读程序指令，并转发该计算机可读程序指令，以供存储在各个计算/处理设备中的计算机可读存储介质中。 The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing/processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and/or a wireless network. The network can include copper transmission cables, optical fiber transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, and/or edge servers. The network adapter card or network interface in each computing/processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in the computer-readable storage medium in each computing/processing device.
用于执行本申请操作的计算机程序指令可以是汇编指令、指令集架构(ISA)指令、机器指令、机器相关指令、微代码、固件指令、状态设置数据、或者以一种或多种编程语言的任意组合编写的源代码或目标代码，所述编程语言包括面向对象的编程语言—诸如Smalltalk、C++等，以及常规的过程式编程语言—诸如“C”语言或类似的编程语言。计算机可读程序指令可以完全地在用户计算机上执行、部分地在用户计算机上执行、作为一个独立的软件包执行、部分在用户计算机上部分在远程计算机上执行、或者完全在远程计算机或服务器上执行。在涉及远程计算机的情形中，远程计算机可以通过任意种类的网络—包括局域网(LAN)或广域网(WAN)—连接到用户计算机，或者，可以连接到外部计算机(例如利用因特网服务提供商来通过因特网连接)。在一些实施例中，通过利用计算机可读程序指令的状态信息来个性化定制电子电路，例如可编程逻辑电路、现场可编程门阵列(FPGA)或可编程逻辑阵列(PLA)，该电子电路可以执行计算机可读程序指令，从而实现本申请的各个方面。The computer program instructions for performing the operation of the present application can be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-related instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages, such as Smalltalk, C++, etc., and conventional procedural programming languages, such as "C" language or similar programming languages. Computer-readable program instructions can be executed completely on the user's computer, partially on the user's computer, as an independent software package, partially on the user's computer, partially on the remote computer, or completely on the remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (for example, using an Internet service provider to connect through the Internet). In some embodiments, by using the state information of computer-readable program instructions to personalize electronic circuits, such as programmable logic circuits, field programmable gate arrays (FPGAs) or programmable logic arrays (PLAs), the electronic circuits can execute computer-readable program instructions, thereby realizing various aspects of the present application.
这里参照根据本申请实施例的方法、装置(系统)和计算机程序产品的流程图和/或框图描述了本申请的各个方面。应当理解，流程图和/或框图的每个方框以及流程图和/或框图中各方框的组合，都可以由计算机可读程序指令实现。Various aspects of the present application are described herein with reference to the flowcharts and/or block diagrams of the methods, devices (systems) and computer program products according to the embodiments of the present application. It should be understood that each box in the flowchart and/or block diagram and the combination of each box in the flowchart and/or block diagram can be implemented by computer-readable program instructions.
这些计算机可读程序指令可以提供给通用计算机、专用计算机或其它可编程数据处理装置的处理器，从而生产出一种机器，使得这些指令在通过计算机或其它可编程数据处理装置的处理器执行时，产生了实现流程图和/或框图中的一个或多个方框中规定的功能/动作的装置。也可以把这些计算机可读程序指令存储在计算机可读存储介质中，这些指令使得计算机、可编程数据处理装置和/或其他设备以特定方式工作，从而，存储有指令的计算机可读介质则包括一个制造品，其包括实现流程图和/或框图中的一个或多个方框中规定的功能/动作的各个方面的指令。These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the processor of the computer or other programmable data processing device, a device that implements the functions/actions specified in one or more boxes in the flowchart and/or block diagram is generated. These computer-readable program instructions can also be stored in a computer-readable storage medium, and these instructions cause the computer, programmable data processing device, and/or other equipment to work in a specific manner, so that the computer-readable medium storing the instructions includes a manufactured product, which includes instructions for implementing various aspects of the functions/actions specified in one or more boxes in the flowchart and/or block diagram.
也可以把计算机可读程序指令加载到计算机、其它可编程数据处理装置、或其它设备上，使得在计算机、其它可编程数据处理装置或其它设备上执行一系列操作步骤，以产生计算机实现的过程，从而使得在计算机、其它可编程数据处理装置、或其它设备上执行的指令实现流程图和/或框图中的一个或多个方框中规定的功能/动作。Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operating steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions/actions specified in one or more boxes in the flowchart and/or block diagram.
附图中的流程图和框图显示了根据本申请的多个实施例的系统、方法和计算机程序产品的可能实现的体系架构、功能和操作。在这点上，流程图或框图中的每个方框可以代表一个模块、程序段或指令的一部分，所述模块、程序段或指令的一部分包含一个或多个用于实现规定的逻辑功能的可执行指令。在有些作为替换的实现中，方框中所标注的功能也可以以不同于附图中所标注的顺序发生。例如，两个连续的方框实际上可以基本并行地执行，它们有时也可以按相反的顺序执行，这依所涉及的功能而定。也要注意的是，框图和/或流程图中的每个方框、以及框图和/或流程图中的方框的组合，可以用执行规定的功能或动作的专用的基于硬件的系统来实现，或者可以用专用硬件与计算机指令的组合来实现。The flow chart and block diagram in the accompanying drawings show the possible architecture, function and operation of the system, method and computer program product according to multiple embodiments of the present application. In this regard, each square box in the flow chart or block diagram can represent a part of a module, program segment or instruction, and a part of the module, program segment or instruction includes one or more executable instructions for realizing the logical function of the specification. In some alternative implementations, the function marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two continuous square boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the function involved. It should also be noted that each square box in the block diagram and/or flow chart, and the combination of the square boxes in the block diagram and/or flow chart can be realized by a dedicated hardware-based system that performs the function or action of the specification, or can be realized by a combination of special-purpose hardware and computer instructions.
以上已经描述了本申请的各实施例，上述说明是示例性的，并非穷尽性的，并且也不限于所披露的各实施例。在不偏离所说明的各实施例的范围和精神的情况下，对于本技术领域的普通技术人员来说许多修改和变更都是显而易见的。本文中所用术语的选择，旨在最好地解释各实施例的原理、实际应用或对市场中的技术改进，或者使本技术领域的其它普通技术人员能理解本文披露的各实施例。 The embodiments of the present application have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and changes will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or technical improvements in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.

Claims (26)

  1. 一种图像处理方法，其特征在于，包括：An image processing method, characterized by comprising:
    获取待处理图像，所述待处理图像包括管状结构；Acquiring an image to be processed, wherein the image to be processed includes a tubular structure;
    确定所述管状结构中的多个节点；determining a plurality of nodes in the tubular structure;
    将所述多个节点中的第一节点与至少一个第二节点进行连接，得到调整后的管状结构，其中，所述第二节点为所述多个节点中除所述第一节点外的节点；Connecting a first node among the plurality of nodes with at least one second node to obtain an adjusted tubular structure, wherein the second node is a node among the plurality of nodes other than the first node;
    根据调整后的管状结构，构建所述管状结构对应的至少一个拓扑图。According to the adjusted tubular structure, at least one topological graph corresponding to the tubular structure is constructed.
  2. 根据权利要求1所述的方法，其特征在于，所述方法还包括：The method according to claim 1, characterized in that the method further comprises:
    在所述至少一个拓扑图的数量为多个的情况下，对所述至少一个拓扑图进行筛选。When there are multiple topological maps, the at least one topological map is screened.
  3. 根据权利要求1或2所述的方法，其特征在于，所述将所述多个节点中的第一节点与至少一个第二节点进行连接，得到调整后的管状结构，包括：The method according to claim 1 or 2, characterized in that the step of connecting a first node among the plurality of nodes with at least one second node to obtain an adjusted tubular structure comprises:
    将所述第一节点与距其预设范围内的至少一个所述第二节点进行连接，得到所述调整后的管状结构。The first node is connected to at least one of the second nodes within a preset range thereof to obtain the adjusted tubular structure.
  4. 根据权利要求1-3中任一项所述的方法，其特征在于，所述根据调整后的管状结构，构建所述管状结构对应的至少一个拓扑图，包括：The method according to any one of claims 1 to 3, characterized in that constructing at least one topological map corresponding to the tubular structure according to the adjusted tubular structure comprises:
    根据所述调整后的管状结构中的至少一个节点、所述调整后的管状结构中的至少一个连接边和所述至少一个连接边对应的特征，构建所述至少一个拓扑图。The at least one topological graph is constructed according to at least one node in the adjusted tubular structure, at least one connecting edge in the adjusted tubular structure, and a feature corresponding to the at least one connecting edge.
  5. 根据权利要求4所述的方法，其特征在于，所述根据所述调整后的管状结构中的至少一个节点、所述调整后的管状结构中的至少一个连接边和所述至少一个连接边对应的特征，构建所述至少一个拓扑图，包括：The method according to claim 4, characterized in that constructing the at least one topological graph according to at least one node in the adjusted tubular structure, at least one connecting edge in the adjusted tubular structure, and a feature corresponding to the at least one connecting edge comprises:
    根据所述至少一个连接边对应的特征，得到所述至少一个连接边的权值；Obtaining a weight of the at least one connecting edge according to a feature corresponding to the at least one connecting edge;
    根据所述至少一个节点、所述至少一个连接边和所述至少一个连接边的权值，构建所述至少一个拓扑图。The at least one topological graph is constructed according to the at least one node, the at least one connecting edge and the weight of the at least one connecting edge.
  6. 根据权利要求5所述的方法，其特征在于，所述根据所述至少一个连接边对应的特征，得到所述至少一个连接边的权值，包括：The method according to claim 5, characterized in that obtaining the weight of the at least one connecting edge according to the feature corresponding to the at least one connecting edge comprises:
    将所述至少一个连接边对应的特征输入预设模型，得到所述至少一个连接边属于所述至少一个拓扑图的概率；Inputting the feature corresponding to the at least one connection edge into a preset model to obtain a probability that the at least one connection edge belongs to the at least one topological graph;
    根据所述至少一个连接边属于所述至少一个拓扑图的概率，得到所述至少一个连接边的权值；Obtaining a weight of the at least one connection edge according to a probability that the at least one connection edge belongs to the at least one topological graph;
    其中，所述预设模型基于拓扑图训练样本中每一连接边对应的特征训练得到。The preset model is obtained based on feature training corresponding to each connection edge in the topology graph training sample.
  7. 根据权利要求4-6中任一项所述的方法，其特征在于，所述至少一个连接边对应的特征指示所述至少一个连接边的几何特性，和/或所述至少一个连接边中至少一端节点的几何特性。 The method according to any one of claims 4-6 is characterized in that the feature corresponding to the at least one connecting edge indicates the geometric characteristics of the at least one connecting edge and/or the geometric characteristics of at least one end node in the at least one connecting edge.
  8. 根据权利要求6所述的方法，其特征在于，所述预设模型为图神经网络模型。The method according to claim 6 is characterized in that the preset model is a graph neural network model.
  9. 根据权利要求5-8中任一项所述的方法，其特征在于，所述根据所述至少一个节点、所述至少一个连接边和所述至少一个连接边的权值，构建所述至少一个拓扑图，包括：The method according to any one of claims 5 to 8, characterized in that constructing the at least one topological graph according to the at least one node, the at least one connecting edge and the weight of the at least one connecting edge comprises:
    根据所述至少一个节点、所述至少一个连接边和所述至少一个连接边的权值，基于最小生成树算法，构建所述至少一个拓扑图。According to the at least one node, the at least one connecting edge and the weight of the at least one connecting edge, based on a minimum spanning tree algorithm, the at least one topological graph is constructed.
  10. 根据权利要求1-9中任一项所述的方法，其特征在于，所述拓扑图为树形拓扑图。The method according to any one of claims 1-9 is characterized in that the topology graph is a tree topology graph.
  11. 根据权利要求5所述的方法，其特征在于，所述根据所述至少一个节点、所述至少一个连接边和所述至少一个连接边的权值，构建所述至少一个拓扑图，包括：The method according to claim 5, characterized in that constructing the at least one topological graph according to the at least one node, the at least one connecting edge and the weight of the at least one connecting edge comprises:
    根据所述至少一个连接边的权值，确定第一连接边集合中权值最大的任一连接边；其中，所述第一连接边集合的初始状态包括所述至少一个连接边；Determine any one connecting edge with the largest weight in the first connecting edge set according to the weight of the at least one connecting edge; wherein the initial state of the first connecting edge set includes the at least one connecting edge;
    在所述权值最大的任一连接边与第二连接边集合中的连接边不构成环形结构的情况下，将所述权值最大的任一连接边添加到所述第二连接边集合中；其中，所述第二连接边集合的初始状态为空集；When any connecting edge with the largest weight does not form a ring structure with the connecting edges in the second connecting edge set, adding any connecting edge with the largest weight to the second connecting edge set; wherein the initial state of the second connecting edge set is an empty set;
    从所述第一连接边集合中移除所述权值最大的任一连接边，以更新所述第一连接边集合；并基于更新后的所述第一连接边集合，重复执行上述确定第一连接边集合中权值最大的任一连接边及之后的操作，直到所述第二连接边集合中的连接边的数量为N-1，N为所述至少一个节点的数量；Remove any one of the connected edges with the largest weight from the first connected edge set to update the first connected edge set; and based on the updated first connected edge set, repeatedly perform the above-mentioned operations of determining any one of the connected edges with the largest weight in the first connected edge set and subsequent operations until the number of connected edges in the second connected edge set is N-1, where N is the number of the at least one node;
    根据所述第二连接边集合及所述至少一个节点，构建所述管状结构对应的一个拓扑图。A topological graph corresponding to the tubular structure is constructed according to the second connection edge set and the at least one node.
  12. 根据权利要求1-11中任一项所述的方法，其特征在于，所述确定所述管状结构中的多个节点，包括：The method according to any one of claims 1 to 11, characterized in that the determining of a plurality of nodes in the tubular structure comprises:
    提取所述管状结构的骨架线；Extracting the skeleton line of the tubular structure;
    根据所述管状结构的骨架线，确定所述管状结构中的多个节点。A plurality of nodes in the tubular structure are determined according to the skeleton line of the tubular structure.
  13. 一种图像处理装置，其特征在于，包括：获取模块，用于获取待处理图像，所述待处理图像包括管状结构；节点确定模块，用于确定所述管状结构中的多个节点；连接模块，用于将所述多个节点中的第一节点与至少一个第二节点进行连接，得到调整后的管状结构，其中，所述第二节点为所述多个节点中除所述第一节点外的节点；构建模块，用于根据调整后的管状结构，构建所述管状结构对应的至少一个拓扑图。An image processing device, characterized in that it includes: an acquisition module, used to acquire an image to be processed, wherein the image to be processed includes a tubular structure; a node determination module, used to determine multiple nodes in the tubular structure; a connection module, used to connect a first node among the multiple nodes with at least one second node to obtain an adjusted tubular structure, wherein the second node is a node among the multiple nodes other than the first node; and a construction module, used to construct at least one topological graph corresponding to the tubular structure according to the adjusted tubular structure.
  14. 根据权利要求13所述的装置，其特征在于，所述装置还包括：筛选模块，用于在所述至少一个拓扑图的数量为多个的情况下，对所述至少一个拓扑图进行筛选。The device according to claim 13 is characterized in that the device further comprises: a screening module, used to screen the at least one topological map when the number of the at least one topological map is multiple.
  15. 根据权利要求13或14所述的装置，其特征在于，所述连接模块，还用于：将所述第一节点与距其预设范围内的至少一个所述第二节点进行连接，得到所述调整后的管状结构。 The device according to claim 13 or 14 is characterized in that the connection module is also used to: connect the first node with at least one of the second nodes within a preset range thereof to obtain the adjusted tubular structure.
  16. 根据权利要求13-15中任一项所述的装置，其特征在于，所述构建模块，还用于：根据所述调整后的管状结构中的至少一个节点、所述调整后的管状结构中的至少一个连接边和所述至少一个连接边对应的特征，构建所述至少一个拓扑图。The device according to any one of claims 13-15 is characterized in that the construction module is also used to: construct the at least one topological graph based on at least one node in the adjusted tubular structure, at least one connecting edge in the adjusted tubular structure and the characteristics corresponding to the at least one connecting edge.
  17. 根据权利要求16所述的装置，其特征在于，所述构建模块，还用于：根据所述至少一个连接边对应的特征，得到所述至少一个连接边的权值；根据所述至少一个节点、所述至少一个连接边和所述至少一个连接边的权值，构建所述至少一个拓扑图。The device according to claim 16 is characterized in that the construction module is also used to: obtain the weight of the at least one connection edge according to the characteristics corresponding to the at least one connection edge; and construct the at least one topological graph according to the at least one node, the at least one connection edge and the weight of the at least one connection edge.
  18. 根据权利要求17所述的装置，其特征在于，所述构建模块，还用于：将所述至少一个连接边对应的特征输入预设模型，得到所述至少一个连接边属于所述至少一个拓扑图的概率；根据所述至少一个连接边属于所述至少一个拓扑图的概率，得到所述至少一个连接边的权值；其中，所述预设模型基于拓扑图训练样本中每一连接边对应的特征训练得到。The device according to claim 17 is characterized in that the construction module is also used to: input the features corresponding to the at least one connection edge into a preset model to obtain the probability that the at least one connection edge belongs to the at least one topological graph; obtain the weight of the at least one connection edge based on the probability that the at least one connection edge belongs to the at least one topological graph; wherein the preset model is trained based on the features corresponding to each connection edge in the topological graph training sample.
  19. 根据权利要求16-18中任一项所述的装置，其特征在于，所述至少一个连接边对应的特征指示所述至少一个连接边的几何特性，和/或所述至少一个连接边中至少一端节点的几何特性。The device according to any one of claims 16-18 is characterized in that the feature corresponding to the at least one connecting edge indicates the geometric characteristics of the at least one connecting edge and/or the geometric characteristics of at least one end node in the at least one connecting edge.
  20. 根据权利要求18所述的装置，其特征在于，所述预设模型为图神经网络模型。The device according to claim 18 is characterized in that the preset model is a graph neural network model.
  21. 根据权利要求17-20中任一项所述的装置，其特征在于，所述构建模块，还用于：根据所述至少一个节点、所述至少一个连接边和所述至少一个连接边的权值，基于最小生成树算法，构建所述至少一个拓扑图。The device according to any one of claims 17-20 is characterized in that the construction module is also used to: construct the at least one topological graph based on a minimum spanning tree algorithm according to the at least one node, the at least one connecting edge and the weight of the at least one connecting edge.
  22. 根据权利要求13-21中任一项所述的装置，其特征在于，所述拓扑图为树形拓扑图。The device according to any one of claims 13-21 is characterized in that the topology diagram is a tree topology diagram.
  23. 根据权利要求17所述的装置，其特征在于，所述构建模块，还用于：根据所述至少一个连接边的权值，确定第一连接边集合中权值最大的任一连接边；其中，所述第一连接边集合的初始状态包括所述至少一个连接边；在所述权值最大的任一连接边与第二连接边集合中的连接边不构成环形结构的情况下，将所述权值最大的任一连接边添加到所述第二连接边集合中；其中，所述第二连接边集合的初始状态为空集；从所述第一连接边集合中移除所述权值最大的任一连接边，以更新所述第一连接边集合；并基于更新后的所述第一连接边集合，重复执行上述确定第一连接边集合中权值最大的任一连接边及之后的操作，直到所述第二连接边集合中的连接边的数量为N-1，N为所述至少一个节点的数量；根据所述第二连接边集合及所述至少一个节点，构建所述管状结构对应的一个拓扑图。The device according to claim 17 is characterized in that the construction module is also used to: determine any connection edge with the largest weight in the first connection edge set according to the weight of the at least one connection edge; wherein the initial state of the first connection edge set includes the at least one connection edge; when any connection edge with the largest weight and the connection edges in the second connection edge set do not form a ring structure, add any connection edge with the largest weight to the second connection edge set; wherein the initial state of the second connection edge set is an empty set; remove any connection edge with the largest weight from the first connection edge set to update the first connection edge set; and based on the updated first connection edge set, repeat the above-mentioned determination of any connection edge with the largest weight in the first connection edge set and subsequent operations until the number of connection edges in the second connection edge set is N-1, N is the number of the at least one node; and construct a topological graph corresponding to the tubular structure based on the second connection edge set and the at least one node.
  24. 根据权利要求13-23中任一项所述的装置，其特征在于，所述节点确定模块，还用于：提取所述管状结构的骨架线；根据所述管状结构的骨架线，确定所述管状结构中的多个节点。 The device according to any one of claims 13-23 is characterized in that the node determination module is also used to: extract the skeleton line of the tubular structure; and determine a plurality of nodes in the tubular structure based on the skeleton line of the tubular structure.
  25. 一种电子设备，其特征在于，包括：An electronic device, comprising:
    处理器；processor;
    用于存储处理器可执行指令的存储器；a memory for storing processor-executable instructions;
    其中，所述处理器被配置为执行所述指令时实现权利要求1-12中任意一项所述的方法。Wherein, the processor is configured to implement the method described in any one of claims 1-12 when executing the instructions.
  26. 一种非易失性计算机可读存储介质，其上存储有计算机程序指令，其特征在于，所述计算机程序指令被处理器执行时实现权利要求1-12中任意一项所述的方法。 A non-volatile computer-readable storage medium having computer program instructions stored thereon, characterized in that when the computer program instructions are executed by a processor, the method described in any one of claims 1 to 12 is implemented.
PCT/CN2023/099726 2022-11-07 2023-06-12 Image processing method, apparatus, electronic device, and storage medium WO2024098742A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202211386400.1A CN118037622A (en) 2022-11-07 2022-11-07 Image processing method, device, electronic equipment and storage medium
CN202211386400.1 2022-11-07

Publications (1)

Publication Number Publication Date
WO2024098742A1 true WO2024098742A1 (en) 2024-05-16

Family

ID=91002916

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2023/099726 WO2024098742A1 (en) 2022-11-07 2023-06-12 Image processing method, apparatus, electronic device, and storage medium

Country Status (2)

Country Link
CN (1) CN118037622A (en)
WO (1) WO2024098742A1 (en)

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109635876A (en) * 2017-12-21 2019-04-16 北京昆仑医云科技有限公司 The computer implemented method, apparatus and medium of dissection label are generated for physiology tree construction
WO2020211284A1 (en) * 2019-04-18 2020-10-22 北京市商汤科技开发有限公司 Image processing method and apparatus, electronic device, and storage medium
CN113658104A (en) * 2021-07-21 2021-11-16 南方科技大学 Blood vessel image processing method, electronic device and computer-readable storage medium
CN113902763A (en) * 2021-11-18 2022-01-07 重庆文理学院 Retinal vessel reconstruction method based on oriented vessel tree
CN115131390A (en) * 2022-06-29 2022-09-30 深圳睿心智能医疗科技有限公司 Image segmentation method, image segmentation device, electronic equipment and storage medium

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109635876A (en) * 2017-12-21 2019-04-16 北京昆仑医云科技有限公司 The computer implemented method, apparatus and medium of dissection label are generated for physiology tree construction
WO2020211284A1 (en) * 2019-04-18 2020-10-22 北京市商汤科技开发有限公司 Image processing method and apparatus, electronic device, and storage medium
CN113658104A (en) * 2021-07-21 2021-11-16 南方科技大学 Blood vessel image processing method, electronic device and computer-readable storage medium
CN113902763A (en) * 2021-11-18 2022-01-07 重庆文理学院 Retinal vessel reconstruction method based on oriented vessel tree
CN115131390A (en) * 2022-06-29 2022-09-30 深圳睿心智能医疗科技有限公司 Image segmentation method, image segmentation device, electronic equipment and storage medium

Also Published As

Publication number Publication date
CN118037622A (en) 2024-05-14

Similar Documents

Publication Publication Date Title
Yang et al. Automatic pixel‐level crack detection and measurement using fully convolutional network
JP2022540634A (en) 3D Point Cloud Object Detection and Instance Segmentation Based on Deep Learning
CN111488921B (en) Intelligent analysis system and method for panoramic digital pathological image
CN112801164A (en) Training method, device and equipment of target detection model and storage medium
CN113361578B (en) Training method and device for image processing model, electronic equipment and storage medium
WO2022236824A1 (en) Target detection network construction optimization method, apparatus and device, and medium and product
US11688061B2 (en) Interpretation of whole-slide images in digital pathology
Miao et al. Label3DMaize: toolkit for 3D point cloud data annotation of maize shoots
JP7387964B2 (en) Training method, sorting method, apparatus, device and medium for sorting learning model
JP2022168167A (en) Image processing method, device, electronic apparatus, and storage medium
WO2022100607A1 (en) Method for determining neural network structure and apparatus thereof
García-Aguilar et al. Optimized instance segmentation by super-resolution and maximal clique generation
CN115359308A (en) Model training method, apparatus, device, storage medium, and program for identifying difficult cases
CN113962737A (en) Target recognition model training method and device, and target recognition method and device
WO2024060839A1 (en) Object operation method and apparatus, computer device, and computer storage medium
CN111179276A (en) Image processing method and device
WO2024098742A1 (en) Image processing method, apparatus, electronic device, and storage medium
CN115482436B (en) Training method and device for image screening model and image screening method
CN115861255A (en) Model training method, device, equipment, medium and product for image processing
US20230062313A1 (en) Generating 2d mapping using 3d data
CN115170599A (en) Method and device for vessel segmentation through link prediction of graph neural network
CN114707638A (en) Model training method, model training device, object recognition method, object recognition device, object recognition medium and product
CN112613544A (en) Target detection method, device, electronic equipment and computer readable medium
CN117649683B (en) Acne grading method, device, equipment and storage medium
CN116824609B (en) Document format detection method and device and electronic equipment