CN111242915A - Blood vessel 3D/2D registration method and device based on Monte Carlo tree search - Google Patents

Blood vessel 3D/2D registration method and device based on Monte Carlo tree search Download PDF

Info

Publication number
CN111242915A
CN111242915A CN202010022660.5A CN202010022660A CN111242915A CN 111242915 A CN111242915 A CN 111242915A CN 202010022660 A CN202010022660 A CN 202010022660A CN 111242915 A CN111242915 A CN 111242915A
Authority
CN
China
Prior art keywords
vessel
matching
node
tree
search
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202010022660.5A
Other languages
Chinese (zh)
Other versions
CN111242915B (en
Inventor
杨健
艾丹妮
范敬凡
朱建军
王涌天
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Institute of Technology BIT
Original Assignee
Beijing Institute of Technology BIT
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Institute of Technology BIT filed Critical Beijing Institute of Technology BIT
Priority to CN202010022660.5A priority Critical patent/CN111242915B/en
Publication of CN111242915A publication Critical patent/CN111242915A/en
Application granted granted Critical
Publication of CN111242915B publication Critical patent/CN111242915B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0012—Biomedical image inspection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/30—Determination of transform parameters for the alignment of images, i.e. image registration
    • G06T7/33—Determination of transform parameters for the alignment of images, i.e. image registration using feature-based methods
    • G06T7/344—Determination of transform parameters for the alignment of images, i.e. image registration using feature-based methods involving models
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30004—Biomedical image processing
    • G06T2207/30101—Blood vessel; Artery; Vein; Vascular
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30172—Centreline of tubular or elongated structure
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
    • Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
    • Y02P90/30—Computing systems specially adapted for manufacturing

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Medical Informatics (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Radiology & Medical Imaging (AREA)
  • Quality & Reliability (AREA)
  • Apparatus For Radiation Diagnosis (AREA)

Abstract

A blood vessel 3D/2D registration method and device based on Monte Carlo tree search are disclosed, and the blood vessel map matching accuracy of a 3D image and a 2D image is high. The method comprises the following steps: (1) realizing vessel map matching by using the topological consistency of the 3D and 2D vessels, and representing the matching of the 3D and 2D vessel maps as a set of vertex pairs and a set of edge pairs; (2) the vessel matching process is regarded as solving the dense matching of the 3D and 2D vessel points, the sparse matching between the vertexes is estimated firstly, and then the edges between the two vertexes are matched and connected, so that the dense matching of the two vessel graphs is obtained; (3) constructing a search tree according to the characteristic that the vessel edge matching can be decomposed into continuous states; (4) calculating a registration result by using a closed solution on each node of the search tree, and designing an evaluation score; (5) variants of MCTS are used to traverse the search tree, evaluating scores as reward values for each node in the tree, the MCTS aiming to find the node in the search tree space that has the highest reward.

Description

Blood vessel 3D/2D registration method and device based on Monte Carlo tree search
Technical Field
The invention relates to the technical field of medical image processing, in particular to a blood vessel 3D/2D registration method based on Monte Carlo tree search and a blood vessel 3D/2D registration device based on Monte Carlo tree search.
Background
At present, minimally invasive interventional therapy is the main treatment means of vascular diseases, and the operation of surgical instruments is guided by X-ray angiography (XRA) images during the interventional therapy. Contrast agent is injected through the catheter into the artery of interest and imaged in a manner that is satisfactory for visualizing the lumen of the vessel. In XRA, the surgical instruments for endovascular navigation are also clearly shown. However, due to the lack of spatial information in XRA, accurate interventional procedures under guidance of single-view two-dimensional projections are difficult for the interventionalist. Therefore, in the interventional operation, the doctor often uses multi-view angiographic images obtained by rotating the C-arm, which, however, increases the injection of the contrast agent and puts a burden on the patient. To address this issue, preoperative Computed Tomography Angiography (CTA) images may be used in conjunction with intraoperative XRA imaging. Both physicians and patients may benefit from multi-modal data fusion and visualization by overlaying three-dimensional vessel projections on two-dimensional real-time images to enhance the interventional images. To achieve this goal, 3D/2D registration techniques are key in which good alignment and correspondence is obtained.
The 3D/2D registration method typically uses one preoperative 3D image and multiple intraoperative 2D X radiographs as the registration source to achieve registration. The 3D and 2D image data and the imaging geometry of the C-arm are required as input to the registration model. For a sequence of X-ray images, the 3D/2D registration is generally considered to be 3D/2D + t. Rotational imaging is a multi-planar X-ray imaging technique commonly used in clinics, in which case multi-planar 3D/2D registration can be considered as 3D/2.5D. Since 3D/2D registration is the basis for 3D/2D + t and 3D/2.5D, only registration of 3D images to a single frame monoplane 2D image is discussed herein. The 3D/2D registration methods are classified into grayscale-based and feature-based registration methods according to the nature of the registration technique.
Grayscale-based methods are typically implemented by optimizing a similarity measure of 3D preoperative images and 2D intraoperative image projections. Digital Reconstruction Radiography (DRR) and Maximum Intensity Projection (MIP) are two common methods of generating simulated X-ray projections from Computed Tomography (CT) images. Hipwell analyzed six similarity indicators for DRR/MIP and X-ray images, with modal intensity and gradient differences performing best. Considering that grayscale-based methods use whole-image grayscale information for registration, they are very sensitive to background outliers. Furthermore, optimization-based methods may yield smaller capture ranges when registering data with large scale transformations.
Feature-based 3D/2D registration relies on consistent features extracted from images of both modalities. The centerline is the most commonly used feature representation in vessel registration. The Iterative Closest Point (ICP) method decomposes point cloud registration into an alternating continuous process that contains matching and registration stages. For the matching phase, point correspondences may be assigned by finding the minimum euclidean distance of the points. Baka introduces a 3D/2D registration method that backprojects two-dimensional points into three-dimensional space and then performs the classical ICP process. Rivest-Henault uses the center of the blood vessel to calculate the distance transformation in advance and construct the objective function, so that the registration process can be accelerated. Benseghir proposes an iterative nearest neighbor curve (ICC) method that uses vessel branches as nearest neighbor based pairing elements and then estimates a transformation that minimizes the sum of distances between the paired branches. These ICP-like methods are very sensitive to noise and outliers because they limit the correspondence to one-to-one matches. Nearest neighbor based matching also results in these methods relying heavily on the initial pose.
Due to the sensitivity of hard allocation strategies to noise and outliers, in the probabilistic allocation framework, soft allocation strategies relax the one-to-one correspondence into one-to-many. On the basis of a Gaussian Mixture Model (GMM) and expectation-maximization (EM) algorithm, myrnenko proposes a Coherent Point Drift (CPD) method that forces coherent movement of the GMM centrosome to preserve the global topology of the point set. Kang uses the same framework as CPD for 3D/2D point cloud registration. And solving the optimal estimation of the registration parameters by adopting a particle swarm optimization algorithm when the nonlinear characteristic of perspective projection is considered. Baka proposed a method from Jianan aThe direction constraint OGMM method expanded by the ndVemuri method estimates the L of two point sets by using the direction and the position2And optimizing the distance to realize 3D/2D coronary artery registration. The OGMM has high accuracy and robustness to noisy data since it utilizes the location of the centerline.
Since vessel topology is an invariant property across morphology and dimensions, graph matching becomes an effective vessel registration method. The matching of the vessel maps can be described as first estimating the correspondence of vessel bifurcation points, and then matching the bifurcation points as vertices to connect the curves between them. Serragell describes vessel registration as a search process to find the most likely correspondence, and uses an a priori search to speed up the process.
Pinheiro describes the vessel matching as a tree search method based on the topological consistency of the graph to be matched, and adopts Monte Carlo tree search to solve the problem. Moriconi defines the affinity function of graph matching by setting node and edge attributes, and then maximizes the function of the secondary distribution problem to obtain node matching.
Several of the above methods are related to 3D/3D or 2D/2D vasculature matching/registration. For 3D/2D vessel matching, the overlap problem in the projections determines that the method needs to emphasize outliers and noise. The ICC method uses vessel topology by a pairing curve between two nodes. For noisy two-dimensional maps, the candidates may be constrained using neighborhood relationships. By ensuring the connectivity of the bifurcation point, Benseghir provides a divide-and-conquer tree-shaped centerline matching method on the basis of an ICC method. Liu treats 3D and 2D vessels as tree topologies, which are represented as sequences. And then, matching the sequences through the sequences to realize the blood vessel matching of the nodes. By maintaining the context, the sequence is extracted using a topological ordering algorithm and then traversed in sequence.
Due to the nonlinear nature of perspective projection, 3D/2D registration based on numerical optimization is prone to fall into local extrema, making these methods sensitive to the initial registration pose. The initialization operation is the key to 3D/2D registration. Generally, the preoperative and intraoperative images come from different devices. The capture range of most registration methods is not sufficient to cover the transformation between the coordinate systems of the two acquisition devices. Markelj describes some initialization methods such as alignment of patient position and orientation, registration of corresponding marker pairs, and manual initialization. However, an automatic initialization method using intrinsic features may be more suitable for intraoperative 3D/2D registration. Varnavas precomputes a two-dimensional projection template for a large range of 3D poses and uses the intra-operative image to evaluate the similarity between the template and the generalized hough transform of the two-dimensional perspective to obtain initial alignment. Miao establishes a shape context coding library of a two-dimensional profile extracted from a metal implant profile, and adopts a Jensen-Shannon divergence algorithm as matching measurement of rapid library matching. Gouveria proposes a regression-based CTA and XRA initial registration method that relates two-dimensional projection image features to rigid transformation parameters.
In order to obtain a larger capture range or a registration method insensitive to the initial pose, a frame based on matching and then transforming is more suitable than an optimization-based method. Assuming that the correspondence of 3D and 2D points is known, the goal of 3D/2D registration is similar to the PnP (periodic-n-point) problem in the field of computer vision. Its purpose is to determine the pose of the camera by simplifying the estimation problem of the point transform to that of the coordinates of the four control points.
Disclosure of Invention
In order to overcome the defects of the prior art, the invention provides a vessel 3D/2D registration method based on Monte Carlo tree search, which can be independent of the initial pose of registration and has high vessel map matching accuracy of a 3D image and a 2D image.
The technical scheme of the invention is as follows: the vessel 3D/2D registration method based on Monte Carlo tree search comprises the following steps:
(1) realizing vessel map matching by using the topological consistency of the 3D vessel and the 2D vessel, and representing the matching of the 3D vessel map and the 2D vessel map as a set of vertex pairs and a set of edge pairs;
(2) the vessel matching process is regarded as solving the dense matching of the 3D and 2D vessel points, the sparse matching between the vertexes is estimated firstly, and then the edges between the two vertexes are matched and connected, so that the dense matching of the two vessel graphs is obtained;
(3) constructing a search tree according to the characteristic that the vessel edge matching can be decomposed into continuous states;
(4) calculating a registration result by using a closed solution on each node of the search tree, and designing an evaluation score;
(5) a variation of MCTS is searched using a monte carlo tree for traversing the search tree, evaluating scores as the prize value for each node in the tree, and the MCTS aims to find the node in the search tree space that has the highest prize.
The method realizes the matching of the vessel maps by utilizing the topological consistency of the 3D vessel and the 2D vessel, and the matching of the 3D vessel map and the 2D vessel map is expressed as a set of vertex pairs and a set of edge pairs; the vessel matching process is regarded as solving the dense matching of the 3D and 2D vessel points, the sparse matching between the vertexes is estimated firstly, and then the edges between the two vertexes are matched and connected, so that the dense matching of the two vessel graphs is obtained; constructing a search tree according to the characteristic that the vessel edge matching can be decomposed into continuous states; calculating a registration result by using a closed solution on each node of the search tree, and designing an evaluation score; the MCTS aims to find the node with the highest reward in a search tree space, so that the initial pose of registration is not depended on, and the vascular map matching accuracy of the 3D image and the 2D image is high.
There is also provided a vessel 3D/2D registration apparatus based on monte carlo tree search, comprising:
a matching result representing module, which realizes the vessel map matching by using the topological consistency of the 3D vessel and the 2D vessel, and represents the matching of the 3D vessel map and the 2D vessel map as a set of vertex pairs and a set of edge pairs;
the dense matching module considers the blood vessel matching process as solving dense matching of 3D and 2D blood vessel points, firstly estimates the sparse matching between the vertexes, and then matches and connects edges between the two vertexes, thereby obtaining the dense matching of the two blood vessel images;
the search tree construction module constructs a search tree according to the characteristic that the vessel edge matching can be decomposed into continuous states;
a calculation result and node scoring module which calculates a registration result using a closed solution at each node of the search tree and designs an evaluation score;
a best results finding module that searches variants of the MCTS using a monte carlo tree for traversing the search tree, evaluating scores as reward values for each node in the tree, the MCTS targeting the node in the search tree space that has the highest reward.
Drawings
Fig. 1a is a 3D vessel model to be registered; FIG. 1b is a 3D vessel centerline; FIG. 1c is a 2D contrast image; figure 1D is the 2D vessel centerline.
Figure 2 is an iteration of MCTS-based vessel matching. A pair of vessel vertexes is selected on the root node, and then the tree is expanded through four steps of selection, expansion, simulation and back propagation.
Fig. 3 shows a flow chart of a vessel 3D/2D registration method based on monte carlo tree search according to the present invention.
Detailed Description
As shown in fig. 3, the vessel 3D/2D registration method based on monte carlo tree search includes the following steps:
(1) realizing vessel map matching by using the topological consistency of the 3D vessel and the 2D vessel, and representing the matching of the 3D vessel map and the 2D vessel map as a set of vertex pairs and a set of edge pairs;
(2) the vessel matching process is regarded as solving the dense matching of the 3D and 2D vessel points, the sparse matching between the vertexes is estimated firstly, and then the edges between the two vertexes are matched and connected, so that the dense matching of the two vessel graphs is obtained;
(3) constructing a search tree according to the characteristic that the vessel edge matching can be decomposed into continuous states;
(4) calculating a registration result by using a closed solution on each node of the search tree, and designing an evaluation score;
(5) a variation of MCTS is searched using a monte carlo tree for traversing the search tree, evaluating scores as the prize value for each node in the tree, and the MCTS aims to find the node in the search tree space that has the highest prize.
The method realizes the matching of the vessel maps by utilizing the topological consistency of the 3D vessel and the 2D vessel, and the matching of the 3D vessel map and the 2D vessel map is expressed as a set of vertex pairs and a set of edge pairs; the vessel matching process is regarded as solving the dense matching of the 3D and 2D vessel points, the sparse matching between the vertexes is estimated firstly, and then the edges between the two vertexes are matched and connected, so that the dense matching of the two vessel graphs is obtained; constructing a search tree according to the characteristic that the vessel edge matching can be decomposed into continuous states; calculating a registration result by using a closed solution on each node of the search tree, and designing an evaluation score; the MCTS aims to find the node with the highest reward in a search tree space, so that the initial pose of registration is not depended on, and the vascular map matching accuracy of the 3D image and the 2D image is high.
Preferably, in the step (1), the vascular map is shown as
Figure BDA0002361359460000071
Wherein
Figure BDA0002361359460000072
In the form of a set of vertices,
Figure BDA0002361359460000073
is a set of edges that are to be considered,
Figure BDA0002361359460000074
including the end points and bifurcation points of the vessel, the edges being represented as curves connecting two adjacent vertices, a 3D vessel map
Figure BDA0002361359460000075
And 2D vessel maps
Figure BDA0002361359460000076
Is represented as a set of vertex pairs
Figure BDA0002361359460000077
Set of sum-edge pairs
Figure BDA0002361359460000078
Figure BDA0002361359460000079
By passing
Figure BDA00023613594600000710
And (4) directly exporting.
Preferably, in step (1), the registration of the 3D and 2D vessel structures is expressed as finding the optimal transformation T of the 3D vessel according to equation (1), the projection of the 3D vessel after the transformation T is optimally aligned with the 2D vessel,
Figure BDA00023613594600000711
wherein
Figure BDA00023613594600000712
Representing the quantitative distance of two vessel maps, P is the perspective projection operation, which is determined by the X-ray imaging device and is invariant, T is the 3D rigid transformation to be solved, omegaTIs the solution space of T.
3D point set
Figure BDA00023613594600000713
And 2D point set
Figure BDA00023613594600000714
Can be used
Figure BDA00023613594600000715
Figure BDA00023613594600000716
And (4) showing. Assuming that π is known, the formulaQuasi-problems are similar to the PnP problem in the field of computer vision. The PnP converts the transformation problem estimated as the point to the coordinate problem estimated as the four control points, thereby obtaining the pose of the camera. A closed solution strategy is used to solve this problem, and a description of the specific method is set forth below.
In step (2), the 3D blood vessel is generally considered to be an acyclic graph, and the 2D blood vessel is represented by a cyclic graph, because a plurality of projected three-dimensional branches are overlapped on a 2D plane, thereby forming a pseudo-bifurcation and a connecting edge. Thus, the vessel topology in 3D and 2D is usually different
Figure BDA0002361359460000081
The goal of the vessel matching is to find
Figure BDA0002361359460000082
And
Figure BDA0002361359460000083
maximum topological consistency, similar to the sub-graph isomorphism problem. Isomorphic subgraph
Figure BDA0002361359460000084
And
Figure BDA0002361359460000085
can be expressed as a match of vertices and edges of
Figure BDA0002361359460000086
Figure BDA0002361359460000087
And
Figure BDA0002361359460000088
due to the fact that
Figure BDA0002361359460000089
Containing all vertices of the matched edge and no other vertices, so matching of edges
Figure BDA00023613594600000810
Vertex matching can be uniquely determined
Figure BDA00023613594600000811
Preferably, in step (2), dense matching of points on two edges is achieved by uniform interpolation, which is described as follows
Figure BDA00023613594600000812
Full reflection to pi
Figure BDA00023613594600000813
Wherein
Figure BDA00023613594600000814
Is composed of
Figure BDA00023613594600000815
The vessel registration problem is further translated into a matching problem of the edges of the graph.
Preferably, in said step (3), according to
Figure BDA00023613594600000816
Given the initial vertex matching pairs, new matching edges are added step by step along the vessel map
Figure BDA00023613594600000817
Figure BDA00023613594600000818
Selecting a pair of non-vessel end edges from the current matching set
Figure BDA00023613594600000819
Figure BDA00023613594600000820
So that
Figure BDA00023613594600000821
And
Figure BDA00023613594600000822
end vertex of as a new matching edge pair
Figure BDA00023613594600000823
And
Figure BDA00023613594600000824
then traverse the topology of the vessel to find a range of vertices
Figure BDA00023613594600000825
And
Figure BDA00023613594600000826
the search range is defined by a super edge, a long edge is formed by K continuous edges, and K generally takes a value of 2 or 3; given current vessel edge matching
Figure BDA00023613594600000827
Obtaining a plurality of edge pairs to be matched through the process
Figure BDA00023613594600000828
Preferably, in the step (4), the evaluation score Q is formula (2)
Figure BDA00023613594600000829
For a given
Figure BDA0002361359460000091
The rigid transformation being obtained by two fill shots
Figure BDA0002361359460000092
The first term of Q is related to the degree of overlap of the 3D and 2D point sets, the projection of the 3D points
Figure BDA0002361359460000093
And 2D points
Figure BDA0002361359460000094
Is achieved by a distance transformation, σ is a scale parameter for normalizing the distance, the second term of Q is used to penalize the scale change that occurs during the 3D to 2D projection, s3DAnd s2DRespectively representing post-projection 3D vertices
Figure BDA0002361359460000095
And 2D vertices
Figure BDA0002361359460000096
Distribution scale of (1), order matrix
Figure BDA0002361359460000097
Represents
Figure BDA0002361359460000098
Matrix array
Figure BDA0002361359460000099
Represents
Figure BDA00023613594600000910
Figure BDA00023613594600000911
The method uses a variation of a Monte Carlo Tree Search (MCTS) for traversing the search tree, with the score defined in equation (2) as the reward value for each node in the tree. The goal of MCTS is to search the tree space
Figure BDA00023613594600000912
The node with the highest reward is found, wherein the most critical step is to select the child node exactly at the current state. This process is accomplished by iteratively constructing a partial search tree, which the MCTS expands by iteratively performing the following four steps, as shown in figure 1.
Preferably, the step (5) comprises the following substeps:
(5.1) selecting: selecting the most urgent expandable node in the current search tree, starting from the root node to the top and the bottom, and realizing the expansion by a greedy strategy at each layer; the expandable node means that at least one pair of new matching edges is found according to the matching state of the current node; the urgency degree of the node is calculated by formula (3)
Figure BDA00023613594600000913
Wherein QsimRepresenting the possible maximum reward value of a subtree which takes the current node as a root node, N representing the number of co-iteration, and N representing the number of access times of the current node; therefore, the second term is used to penalize the current node with too high number of accesses, so as to encourage the exploration of the non-accessed or less-accessed branches; γ is used to balance the development of known nodes and the exploration of unaccessed nodes;
(5.2) expansion: adding at most N to the selected nodeexpA child node; when present, is greater than NexpCalculates their reward and selects the higher NexpA child node;
(5.3) simulation: taking each newly expanded child node as a root node, and accessing the nodes downwards in a depth-first mode; randomly selecting child nodes in each layer until reaching the deepest leaf node; this process repeats for NsimNext, the rewards for these visited leaf nodes are then computed, with the highest reward being Q for the extended nodesim；
(5.4) counter-propagating: updating Q along the path from the selected node to the root nodesimA value and a node visit number n;
these four steps are iteratively performed until a maximum calculated amount N is reachedmaxOr the theoretically highest reward QmaxAfter the iteration is over, the node with the highest reward Q is obtained from the search, and the corresponding match among the nodes
Figure BDA0002361359460000101
And the transformation T is the final result of the method.
It will be understood by those skilled in the art that all or part of the steps in the method of the above embodiments may be implemented by hardware instructions related to a program, the program may be stored in a computer-readable storage medium, and when executed, the program includes the steps of the method of the above embodiments, and the storage medium may be: ROM/RAM, magnetic disks, optical disks, memory cards, and the like. Therefore, corresponding to the method of the present invention, the present invention also includes a blood vessel 3D/2D registration apparatus based on monte carlo tree search, which is generally expressed in the form of functional modules corresponding to the steps of the method. The device includes:
a matching result representing module, which realizes the vessel map matching by using the topological consistency of the 3D vessel and the 2D vessel, and represents the matching of the 3D vessel map and the 2D vessel map as a set of vertex pairs and a set of edge pairs;
the dense matching module considers the blood vessel matching process as solving dense matching of 3D and 2D blood vessel points, firstly estimates the sparse matching between the vertexes, and then matches and connects edges between the two vertexes, thereby obtaining the dense matching of the two blood vessel images;
the search tree construction module constructs a search tree according to the characteristic that the vessel edge matching can be decomposed into continuous states;
a calculation result and node scoring module which calculates a registration result using a closed solution at each node of the search tree and designs an evaluation score;
a best results finding module that searches variants of the MCTS using a monte carlo tree for traversing the search tree, evaluating scores as reward values for each node in the tree, the MCTS targeting the node in the search tree space that has the highest reward.
The above description is only a preferred embodiment of the present invention, and is not intended to limit the present invention in any way, and all simple modifications, equivalent variations and modifications made to the above embodiment according to the technical spirit of the present invention still belong to the protection scope of the technical solution of the present invention.

Claims (8)

1. The blood vessel 3D/2D registration method based on Monte Carlo tree search is characterized in that: which comprises the following steps:
(1) realizing vessel map matching by using the topological consistency of the 3D vessel and the 2D vessel, and representing the matching of the 3D vessel map and the 2D vessel map as a set of vertex pairs and a set of edge pairs;
(2) the vessel matching process is regarded as solving the dense matching of the 3D and 2D vessel points, the sparse matching between the vertexes is estimated firstly, and then the edges between the two vertexes are matched and connected, so that the dense matching of the two vessel graphs is obtained;
(3) constructing a search tree according to the characteristic that the vessel edge matching can be decomposed into continuous states;
(4) calculating a registration result by using a closed solution on each node of the search tree, and designing an evaluation score;
(5) a variation of MCTS is searched using a monte carlo tree for traversing the search tree, evaluating scores as the prize value for each node in the tree, and the MCTS aims to find the node in the search tree space that has the highest prize.
2. The monte carlo tree search based vessel 3D/2D registration method according to claim 1, characterized by: in the step (1), the vascular map is shown as
Figure FDA0002361359450000011
Wherein
Figure FDA0002361359450000012
Is a set of vertices, epsilon is a set of edges,
Figure FDA0002361359450000013
including the end points and bifurcation points of the vessel, the edges being represented as curves connecting two adjacent vertices, a 3D vessel map
Figure FDA0002361359450000014
And 2D vessel maps
Figure FDA0002361359450000015
Is represented as a set of vertex pairs
Figure FDA0002361359450000016
Set of sum-edge pairs
Figure FDA0002361359450000017
Figure FDA0002361359450000018
πεBy passing
Figure FDA0002361359450000019
And (4) directly exporting.
3. The monte carlo tree search based vessel 3D/2D registration method according to claim 2, wherein: in said step (1), the registration of the 3D and 2D vessel structures is expressed as finding the optimal transformation T of the 3D vessel, the projection of the 3D vessel after the transformation T is applied to the 2D vessel achieving the optimal alignment according to equation (1),
Figure FDA0002361359450000021
wherein
Figure FDA0002361359450000022
Representing the quantitative distance of two vessel maps, P is the perspective projection operation, which is determined by the X-ray imaging device and is invariant, T is the 3D rigid transformation to be solved, omegaTIs the solution space of T.
4. The Monte Carlo tree search based vessel 3D/2D registration method according to claim 3, wherein: in step (2), dense matching of points on two edges is achieved by uniform interpolation, and the process is described as being from piεFull reflection to pi
Figure FDA0002361359450000023
Figure FDA0002361359450000024
Wherein
Figure FDA0002361359450000025
Is piεThe vessel registration problem is further translated into a matching problem of the edges of the graph.
5. The Monte Carlo tree search based vessel 3D/2D registration method according to claim 4, wherein: in the step (3), according to piεGiven the initial vertex matching pairs, new matching edges are added step by step along the vessel map
Figure FDA0002361359450000026
Selecting a pair of non-vessel end edges from the current matching set
Figure FDA0002361359450000027
So that
Figure FDA0002361359450000028
And
Figure FDA0002361359450000029
end vertex of as a new matching edge pair
Figure FDA00023613594500000210
And
Figure FDA00023613594500000211
then traverse the topology of the vessel to find a range of vertices
Figure FDA00023613594500000212
And
Figure FDA00023613594500000213
the search range is defined by a super edge, a long edge is formed by K continuous edges, and K generally takes a value of 2 or 3; given current vessel edge matching
Figure FDA00023613594500000214
Obtaining a plurality of edge pairs to be matched through the process
Figure FDA00023613594500000215
6. The Monte Carlo tree search based vessel 3D/2D registration method according to claim 5, wherein: in the step (4), the evaluation score Q is a formula (2)
Figure FDA00023613594500000216
For a given piεThe rigid transformation is subjected to two fill shots to obtain T ═ f1(f2(πε) The first term of Q is related to the degree of overlap of the 3D and 2D point sets, the projection of the 3D points
Figure FDA00023613594500000217
And 2D points
Figure FDA0002361359450000031
Is achieved by a distance transformation, σ is a scale parameter for normalizing the distance, the second term of Q is used to penalize the scale change that occurs during the 3D to 2D projection, s3DAnd s2DRespectively representing post-projection 3D vertices
Figure FDA0002361359450000032
And 2D vertices
Figure FDA0002361359450000033
Distribution of (2)Dimension, order matrix
Figure FDA0002361359450000034
Represents
Figure FDA0002361359450000035
Matrix array
Figure FDA0002361359450000036
Represents
Figure FDA0002361359450000037
Figure FDA0002361359450000038
7. The Monte Carlo tree search based vessel 3D/2D registration method according to claim 6, wherein: the step (5) comprises the following sub-steps:
(5.1) selecting: selecting the most urgent expandable node in the current search tree, starting from the root node to the top and the bottom, and realizing the expansion by a greedy strategy at each layer; the expandable node means that at least one pair of new matching edges is found according to the matching state of the current node; the urgency degree of the node is calculated by formula (3)
Figure FDA0002361359450000039
Wherein QsimRepresenting the possible maximum reward value of a subtree which takes the current node as a root node, N representing the number of co-iteration, and N representing the number of access times of the current node; therefore, the second term is used to penalize the current node with too high number of accesses, so as to encourage the exploration of the non-accessed or less-accessed branches; γ is used to balance the development of known nodes and the exploration of unaccessed nodes;
(5.2) expansion: adding at most N to the selected nodeexpA child node; when present, is greater than NexpCompute their child nodesAwarding and selecting a higher NexpA child node;
(5.3) simulation: taking each newly expanded child node as a root node, and accessing the nodes downwards in a depth-first mode; randomly selecting child nodes in each layer until reaching the deepest leaf node; this process repeats for NsimNext, the rewards for these visited leaf nodes are then computed, with the highest reward being Q for the extended nodesim；
(5.4) counter-propagating: updating Q along the path from the selected node to the root nodesimA value and a node visit number n;
these four steps are iteratively performed until a maximum calculated amount N is reachedmaxOr the theoretically highest reward QmaxAfter the iteration is finished, the node with the highest reward Q is obtained from the search, and the corresponding matching pi in the nodeεAnd the transformation T is the final result of the method.
8. Vessel 3D/2D registration device based on Monte Carlo tree search, its characterized in that: it includes:
a matching result representing module, which realizes the vessel map matching by using the topological consistency of the 3D vessel and the 2D vessel, and represents the matching of the 3D vessel map and the 2D vessel map as a set of vertex pairs and a set of edge pairs;
the dense matching module considers the blood vessel matching process as solving dense matching of 3D and 2D blood vessel points, firstly estimates the sparse matching between the vertexes, and then matches and connects edges between the two vertexes, thereby obtaining the dense matching of the two blood vessel images;
the search tree construction module constructs a search tree according to the characteristic that the vessel edge matching can be decomposed into continuous states;
a calculation result and node scoring module which calculates a registration result using a closed solution at each node of the search tree and designs an evaluation score;
a best results finding module that searches variants of the MCTS using a monte carlo tree for traversing the search tree, evaluating scores as reward values for each node in the tree, the MCTS targeting the node in the search tree space that has the highest reward.
CN202010022660.5A 2020-01-09 2020-01-09 Vascular 3D/2D registration method and device based on Monte Carlo tree search Active CN111242915B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202010022660.5A CN111242915B (en) 2020-01-09 2020-01-09 Vascular 3D/2D registration method and device based on Monte Carlo tree search

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010022660.5A CN111242915B (en) 2020-01-09 2020-01-09 Vascular 3D/2D registration method and device based on Monte Carlo tree search

Publications (2)

Publication Number Publication Date
CN111242915A true CN111242915A (en) 2020-06-05
CN111242915B CN111242915B (en) 2023-08-04

Family

ID=70873031

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010022660.5A Active CN111242915B (en) 2020-01-09 2020-01-09 Vascular 3D/2D registration method and device based on Monte Carlo tree search

Country Status (1)

Country Link
CN (1) CN111242915B (en)

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109993730A (en) * 2019-03-20 2019-07-09 北京理工大学 3D/2D blood vessel method for registering and device
CN110604616A (en) * 2019-09-10 2019-12-24 中国科学院深圳先进技术研究院 Interventional operation path planning method and system based on graph search and electronic equipment

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109993730A (en) * 2019-03-20 2019-07-09 北京理工大学 3D/2D blood vessel method for registering and device
CN110604616A (en) * 2019-09-10 2019-12-24 中国科学院深圳先进技术研究院 Interventional operation path planning method and system based on graph search and electronic equipment

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
李致勋;张英涛;唐降龙;: "基于节点拓扑一致性的2D/3D冠脉血管非刚性配准", 智能计算机与应用, no. 06, pages 20 - 24 *

Also Published As

Publication number Publication date
CN111242915B (en) 2023-08-04

Similar Documents

Publication Publication Date Title
US11816837B2 (en) Vascular characteristic determination with correspondence modeling of a vascular tree
US10776935B2 (en) System and method for correcting data for deformations during image-guided procedures
CN109074639B (en) Image registration system and method in medical imaging system
EP1880673B1 (en) Method of quantifying organ dynamic behavior, device, method of predicting organ position, device, radiation irradiating method, device and organ abnormality detecting device
AU2007221876B2 (en) Registration of images of an organ using anatomical features outside the organ
CN109589170B (en) Left atrial appendage closure guidance in medical imaging
JP5129480B2 (en) System for performing three-dimensional reconstruction of tubular organ and method for operating blood vessel imaging device
JP6283672B2 (en) An improved method of correcting data for deformation during image guidance procedures
US11730395B2 (en) Reconstruction of an anatomical structure from intrabody measurements
US20080123927A1 (en) Apparatus and methods of compensating for organ deformation, registration of internal structures to images, and applications of same
CN103229210B (en) Image registration device
Özgür et al. Preoperative liver registration for augmented monocular laparoscopy using backward–forward biomechanical simulation
US20180189966A1 (en) System and method for guidance of laparoscopic surgical procedures through anatomical model augmentation
CN110432986B (en) System and method for constructing virtual radial ultrasound images from CT data
US11017531B2 (en) Shell-constrained localization of vasculature
Vásquez Osorio et al. Accurate CT/MR vessel‐guided nonrigid registration of largely deformed livers
CN111260704B (en) Vascular structure 3D/2D rigid registration method and device based on heuristic tree search
Yoon et al. Topological recovery for non-rigid 2D/3D registration of coronary artery models
CN111784751B (en) 3D/2D registration-based guide wire 3D simulation tracking method and device
CN111242915A (en) Blood vessel 3D/2D registration method and device based on Monte Carlo tree search
Zhu et al. Monte Carlo tree search for 3D/2D registration of vessel graphs
KR20210112436A (en) Method of non-rigid registration for multi-modality based on features of blood vessel, recording medium and device for performing the method
Laura Graph-matching and FEM-based registration of computed tomographies for outcome validation of liver interventions
Ainchil Cayuela 3D vessel reconstruction based on intra-operative intravascular ultrasound for robotic autonomous catheter navigation
JP2024504025A (en) container shape

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant