KR20210112436A - Method of non-rigid registration for multi-modality based on features of blood vessel, recording medium and device for performing the method - Google Patents

Abstract

A multi-modality non-rigid body registration method based on blood vessel feature information comprises the steps of: performing rigid body registration in order to match a space of a three-dimensional CTA image taken before surgery with a space of a two-dimensional XA image taken during surgery; generating a three-dimensional blood vessel graph and a two-dimensional blood vessel graph for a blood vessel structure represented by vertices, edges, and roots from each blood vessel centerline of the three-dimensional CTA image and the two-dimensional XA image; reconstructing candidate segments of the two-dimensional blood vessel graph by searching for a path of the two-dimensional blood vessel graph corresponding to the segment connected between parent and child nodes in the three-dimensional blood vessel graph; matching the vertices of the three-dimensional blood vessel graph with the vertices of the two-dimensional blood vessel graph, respectively, based on a vertical inclination of one vertex of the three-dimensional CTA image and the distance between the one vertex and an adjacent vertex within a preset range; selecting candidate line of a two-dimensional blood vessel by comparing the similarity between the segments of the three-dimensional blood vessel graph and the candidate segments of the two-dimensional blood vessel graph; and performing non-rigid body transformation of moving the position of the vertex of the three-dimensional blood vessel graph to the vertex of the candidate line of the selected two-dimensional blood vessel by using a gradient descent method such that a predefined energy function has a minimum value. Accordingly, it is possible to provide accurate blood vessel information by providing a fast registration technology of the three-dimensional CTA image and the two-dimensional XA image.

Description

Multimodality non-rigid body registration method based on blood vessel characteristic information, recording medium and apparatus for performing the same

The present invention relates to a method for registering a multimodality non-rigid body based on blood vessel characteristic information, a recording medium and an apparatus for performing the same, and more particularly, to a 3D CTA image taken before the procedure of the same patient and to provide useful guiding information to an operator. It relates to a technology for fast and accurate registration of 2D XA images taken during the procedure.

Medical imaging technologies used in medical procedures or surgeries are typically X-ray (X-ray imaging technology, radiography) and CT (Computed Tomography).

X-ray images the internal structure of the living body on a flat fluorescent plate or film by using the attenuation characteristics of the transmitted X-rays of each tissue, whereas CT projects X-rays while traveling around the cross section of the human body, and X-rays The internal structure is imaged by measuring the amount that decreases as it passes.

Since 3D CT images are mainly taken before surgery and 2D X-ray images are often used during surgery, accurate matching between two heterogeneous images is very important in the medical field. Previously performed 2D and 3D non-rigid body registration studies of coronary arteries are as follows.

Rivest-Henault et al. proposed a non-rigid body registration technique using an energy function by extracting a vessel centerline from a CTA image and a biplane XA image. To align the two blood vessels, registration was performed to the point where the distance was minimized through affine transformation. For regional transformation, slope-based matching point search was performed, and the matching point search error for geometrical relationships was minimized by applying Epipolar Constraint between biplane images.

In addition, for natural deformation, an energy term was designed and applied in consideration of the stiffness of blood vessels and the effect of myocardium. However, although the accuracy is improved by utilizing geometric relationships in 3D space, there is a disadvantage that it cannot be performed on a single XA image. In addition, there is a disadvantage that the execution time is very long.

Baka et al. proposed a non-rigid registration technique using the Oriented Gaussian Mixture Model (OGMM) between the vessel centerline extracted from the CTA image and the XA image. A 4D GMM was formed by adding a direction vector to the blood vessel centerline, and matching was performed so that the distance value between the two blood vessels was minimized using SSM (Statistical Shape Model) and a modified L2 distance measurement method.

Robustness is improved by adding direction information, but computation complexity increases along with dimensionality increase by including direction vectors, which has a disadvantage in that it takes a long time to perform.

Gatta et al. proposed a non-rigid body matching technique using SIFT (Scale Invariant Feature Transform) flow. The XA image was used by removing an unnecessary background using a vessel enhancement filter, and the CTA image was used for registration by dividing the blood vessel and generating a DRR image based on the Dicom information of the C-arm.

The two vessels were matched by minimizing the SIFT flow energy function. However, misalignment may occur due to the limitation of background removal, and there are disadvantages in that it takes a long time to generate the DRR image.

Kim et al. extracted the centerline of the blood vessel from the XA image and the CTA image, projected the 3D blood vessel into the 2D blood vessel, and then performed registration. For accurate registration, rotation, movement, and size transformation were repeatedly performed on the projected blood vessels to optimize the distance between the blood vessels.

In addition, TPS-RPM-based non-rigid registration proposed by Chui et al. was performed to improve accuracy through partial deformation of blood vessels. However, after the centerline of the 3D CTA blood vessel is projected onto the 2D image once, the registration process is performed only in 2D, so there is a problem in that the 3D CTA blood vessel topology information cannot be preserved because the transformation of the 3D object is not considered. In addition, due to the lack of depth information, there is a limit in that it may converge to a local minimum in a region where a local error exists, resulting in an erroneous matching result.

US 2014/0301618 A1 US 7,450,743 B2 KR 10-2014-0120145 A

High-speed rigid body registration technique of 2D XA images taken during surgery and 3D CTA images taken before surgery, Taeyong Park, Shin Yongbin, Lim Seonhye, Jeongjin Lee, Journal of the Korean Society for Multimedia, Vol. 16, No. 12 1454p ~ 1464p, 2013

Accordingly, the technical problem of the present invention has been conceived in this regard, and an object of the present invention is to provide a multimodality non-rigid body registration method based on vascular characteristic information of a 3D CTA image taken before the procedure and a 2D XA image taken during the procedure of the same patient. will do

Another object of the present invention is to provide a recording medium in which a computer program for performing the multimodality non-rigid body matching method based on blood vessel characteristic information is recorded.

Another object of the present invention is to provide an apparatus for performing the multimodality non-rigid body matching method based on the blood vessel characteristic information.

The multi-modality non-rigid body registration method based on blood vessel characteristic information according to an embodiment for realizing the object of the present invention is rigid body registration for matching the space of the 3D CTA image taken before the procedure and the 2D XA image taken during the procedure. performing the steps; generating a 3D vascular graph and a 2D vascular graph for vascular structures represented by vertices, edges, and roots from each vascular centerline of the 3D CTA image and the 2D XA image; reconstructing candidate segments of the 2D blood vessel graph by searching a path of the 2D blood vessel graph corresponding to a segment connected to a parent and child node in the 3D blood vessel graph; matching the vertices of the 3D blood vessel graph and the vertices of the 2D blood vessel graph, respectively, based on a vertical inclination of one vertex of the 3D CTA image and a distance between the one point and an adjacent vertex within a preset range; selecting a candidate line of a 2D blood vessel by comparing the similarity between the slices of the 3D blood vessel graph and the candidate slices of the 2D blood vessel graph; and a non-rigid body transformation step of moving the position of the vertex of the 3D blood vessel graph to the vertex of the candidate line of the selected 2D blood vessel using the gradient descent method so that the predefined energy function has a minimum value. .

In an embodiment of the present invention, performing the rigid body registration may include: acquiring Digital Imaging and Communications in Medicine (DICOM) information when capturing the 2D XA image; generating a virtual environment when the 2D XA image is acquired by applying the DICOM information to a 3D blood vessel centerline extracted from the 3D CTA image; and projecting the 3D blood vessel center line and the 2D blood vessel center line to the virtual environment to perform precise registration to match the registration space.

In an embodiment of the present invention, performing the precise registration includes: performing a thinning process of the 3D blood vessel center line and the 2D blood vessel center line; generating a distance map between the thinned 3D blood vessel center line and the 2D blood vessel center line; performing a similarity comparison based on the distance map; and performing precise matching using a transformation function to which the Powell optimization technique is applied.

In an embodiment of the present invention, the performing of the precise registration includes: moving the center point of the 3D blood vessel center line projected on the virtual environment to the origin; sequentially converting a rotation vector and a movement vector of the 3D blood vessel center line based on the origin; and moving the center point of the 3D blood vessel center line to the center point of the 2D blood vessel center line.

In an embodiment of the present invention, the step of reconstructing the candidate segments of the 2D blood vessel graph may be performed on the assumption that the shape properties of the 3D blood vessel graph and the 2D blood vessel graph are similar and exist within a certain range.

In an embodiment of the present invention, the step of reconstructing the candidate fragments of the 2D blood vessel graph includes extracting the candidate fragments of the 2D blood vessel graph in which the distance between the 3D blood vessel graph and the shape attribute of the 2D blood vessel graph is lower than a preset threshold. to do; selecting one candidate segment most similar to the shape attribute of the 3D blood vessel graph from among the candidate segments of the 2D blood vessel graph; generating a connectable structure by searching all paths of the selected candidate fragment; and performing a connection relationship analysis on the candidate line of the 2D blood vessel of the selected candidate fragment to remove the child candidate line that is not connected to the parent candidate line.

In an embodiment of the present invention, the connection relationship analysis may be sequentially performed from the lowest level including the leaf node to the highest level including the root node.

In an embodiment of the present invention, the step of selecting a candidate line of a 2D blood vessel by comparing the similarity includes using a difference in distance, slope, length, and thickness between the intercept of the 3D blood vessel graph and the candidate line of the 2D blood vessel to compare the similarity. can be compared.

In an embodiment of the present invention, in the non-rigid deformation step of moving the position of the vertex of the 3D vessel graph to the vertex of the candidate line of the selected 2D vessel, the energy function is an energy term for calculating continuity, curvature It may include an energy term to calculate and an energy term to change the position and shape of the 3D vessel to match the 2D vessel.

In a computer-readable storage medium according to an embodiment for realizing another object of the present invention, a computer program for performing the multimodality non-rigid body matching method based on the blood vessel characteristic information is recorded.

A multimodality non-rigid body matching device based on blood vessel characteristic information according to an embodiment for realizing another object of the present invention is to match the space of the 3D CTA image taken before the procedure and the 2D XA image taken during the procedure. a rigid body matching unit that performs rigid body registration; Graph generation for generating 3D blood vessel graphs and 2D blood vessel graphs for blood vessel structures represented by vertices, edges, and roots from each blood vessel centerline of the 3D CTA image and the 2D XA image wealth; a blood vessel analyzer reconstructing candidate segments of the 2D blood vessel graph by searching a path of the 2D blood vessel graph corresponding to a segment connected to a parent and a child node in the 3D blood vessel graph; A matching unit that matches the vertices of the 3D blood vessel graph and the vertices of the 2D blood vessel graph, respectively, based on the vertical inclination of one vertex of the 3D CTA image and the distance between the one point and an adjacent vertex within a preset range ; a similarity comparison unit for selecting a candidate line of a 2D blood vessel by comparing the similarity between the slices of the 3D blood vessel graph and the candidate slices of the 2D blood vessel graph; and a non-rigid deformation unit that moves the position of the vertex of the 3D blood vessel graph to the vertex of the candidate line of the selected 2D blood vessel using the gradient descent method so that the predefined energy function has a minimum value. .

According to this multimodality non-rigid body registration method based on vessel characteristic information, it is possible to provide more accurate vessel information by overcoming the limitation of rigid body registration by allowing changes in the position and shape of blood vessels according to external factors such as heartbeat and respiration to be reflected. have.

In addition, the present invention can be used as an auxiliary image for diagnosis, treatment confirmation, automatic blood vessel labeling, follow-up examination and research as well as coronary intervention. In addition, it can be applied not only to the cardiovascular system, but also to other blood vessels or an application field that divides a procedure/surgical tool similar to a blood vessel.

1 is a block diagram of a multimodality non-rigid body matching apparatus based on blood vessel characteristic information according to an embodiment of the present invention.
FIG. 2 is a view for explaining a process of generating a graph for a blood vessel structure in the graph generating unit of FIG. 1 .
FIG. 3 is a diagram for explaining a process of extracting a segment of a 3D blood vessel in the blood vessel analysis unit of FIG. 1 .
FIG. 4 is a view for explaining a process of generating all possible connection structures of the 2D blood vessel corresponding to the fragment of the 3D blood vessel of FIG. 3 .
5 is a view comparing the vertex matching using the distance and the slope of the present invention with that of the prior art.
6 is a flowchart of a method for matching multiple modality non-rigid bodies based on blood vessel feature information according to an embodiment of the present invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS [0010] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS [0010] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS [0023] Reference is made to the accompanying drawings, which show by way of illustration specific embodiments in which the present invention may be practiced. These embodiments are described in sufficient detail to enable those skilled in the art to practice the present invention. It should be understood that the various embodiments of the present invention are different but need not be mutually exclusive. For example, certain shapes, structures, and characteristics described herein with respect to one embodiment may be embodied in other embodiments without departing from the spirit and scope of the invention. In addition, it should be understood that the location or arrangement of individual components within each disclosed embodiment may be changed without departing from the spirit and scope of the present invention. Accordingly, the detailed description set forth below is not intended to be taken in a limiting sense, and the scope of the invention, if properly described, is limited only by the appended claims, along with all scope equivalents to those claimed. Like reference numerals in the drawings refer to the same or similar functions throughout the various aspects.

Hereinafter, preferred embodiments of the present invention will be described in more detail with reference to the drawings.

1 is a block diagram of a multimodality non-rigid body matching apparatus based on blood vessel characteristic information according to an embodiment of the present invention.

The multimodality non-rigid body matching device (10, hereinafter device) based on blood vessel characteristic information according to the present invention provides fast and rapid 3D CTA images and 2D XA images taken during the procedure of the same patient in order to provide useful guiding information to the operator. It provides an accurate matching technique.

Referring to FIG. 1 , the apparatus 10 according to the present invention includes a rigid body matching unit 100 , a graph generating unit 200 , a blood vessel analyzing unit 300 , a matching unit 400 , a similarity comparing unit 500 and a ratio It includes a rigid deformable part 600 .

In the apparatus 10 of the present invention, software (application) for performing multimodality non-rigid registration based on blood vessel characteristic information may be installed and executed, and the rigid body matching unit 100, the graph generating unit 200, and the The configuration of the blood vessel analysis unit 300 , the matching unit 400 , the similarity comparison unit 500 , and the non-rigid body deforming unit 600 is a multi-modality non-rigid body based on the vessel characteristic information executed in the device 10 . It can be controlled by software to perform the registration.

The device 10 may be a separate terminal or a module of the terminal. In addition, the rigid body matching unit 100 , the graph generating unit 200 , the blood vessel analyzing unit 300 , the matching unit 400 , the similarity comparing unit 500 , and the non-rigid body deforming unit 600 . A configuration may be formed of an integrated module, or may consist of one or more modules. However, on the contrary, each configuration may be formed of a separate module.

The device 10 may be movable or stationary. The apparatus 10 may be in the form of a server or an engine, and may be a device, an application, a terminal, a user equipment (UE), a mobile station (MS), or a wireless device. (wireless device), may be called other terms such as a handheld device (handheld device).

The device 10 may execute or manufacture various software based on an operating system (OS), that is, the system. The operating system is a system program for software to use the hardware of the device, and is a mobile computer operating system such as Android OS, iOS, Windows Mobile OS, Bada OS, Symbian OS, Blackberry OS, and Windows series, Linux series, Unix series, It can include all computer operating systems such as MAC, AIX, and HP-UX.

The rigid body matching unit 100 performs rigid body registration to match the space of a 3D computed tomography angiography (CTA) image taken before the procedure and a 2D X-ray angiogram (XA) image taken during the procedure.

The registration of the multi-modality image precedes the registration spatial matching of the 2D XA image and the 3D CTA image. This is based on DICOM (Digital Imaging and Communications in Medicine) information acquired together with 2D XA imaging.

DICOM information is applied to the 3D blood vessel centerline extracted from the 3D CTA image, creates a virtual environment at the time the 2D XA image is acquired, and applies and projects it to Equation 1 below to match the registration space.

[Equation 1]

However, since there is no movement variable t in the DICOM information and the location of blood vessels is affected by other external factors such as the patient's posture, heart rate and respiration, precise matching is performed to minimize errors.

After performing the thinning process for quick and accurate distance comparison, a distance map is generated and similarity comparison is performed. In addition, precise matching is performed using a transformation function to which the Powell optimization technique is applied.

,

,

)와 x축, y축, z축을 중심으로 하는 회전 벡터(

,

,

)로 이루어진다. 이를 위해 투영된 3D 혈관 Centerline의 중심점(

)을 원점으로 이동시킨 뒤 회전 벡터와 이동 벡터를 순차적으로 변환한다. The transformation function is a movement vector (

,

,

) and the rotation vector (

,

,

) is made of For this purpose, the center point of the projected 3D vessel Centerline (

) to the origin, and then the rotation vector and the movement vector are sequentially transformed.

)으로 3D 혈관 중심선의 중심점(

)을 이동하여 아래의 수학식 2와 같이 변환한다.Then, the center point of the 2D vessel centerline (

) as the center point of the 3D vessel centerline (

) to transform it as shown in Equation 2 below.

[Equation 2]

However, the detailed process of the rigid body registration is only an example, and other rigid body registration techniques between the 2D XA image and the 3D CTA image may be used instead.

The graph generating unit 200 includes a 3D blood vessel graph for a blood vessel structure represented by a vertex, an edge, and a root from each blood vessel centerline of the 3D CTA image and the 2D XA image, and Create a 2D blood vessel graph.

FIG. 2 is a view for explaining a process of generating a graph for a blood vessel structure in the graph generating unit of FIG. 1 .

와 형태 속성

로 구성된

로 표현한다. Referring to FIG. 2 , in the present invention, a graph structure of a blood vessel structure is generated in order to efficiently compare 2D blood vessels and 3D blood vessels by reducing the complexity of the blood vessel structure. This is a graph structure

and shape properties

composed of

expressed as

는 정점

, Edge

, Root

로 구성되며

로 표현한다. 형태 속성

는 Edge당 n개의 혈관 중심선의 중점들(Centerline Points)로 구성되어 있고, 이는

와

로 표현한다.graph structure

is the vertex

, Edge

, Root

is composed of

expressed as shape properties

is composed of n centerline points per edge, which is

Wow

expressed as

는 분지점이 명확하게 구분되어 분할되어 있기 때문에 트리 구조가 생성된다. 반면, 2D 혈관 구조는 배경의 혼란으로 인해 잘못된 분지점이 발생될 수 있다. 또한, 깊이(Depth) 정보 부족으로 인해 혈관이 겹쳐지거나 교차하는 영역이 발생할 수 있다. 이러한 문제를 개선하기 위해 도 2와 같이 가장 작은 단위로 생성한다.

A tree structure is created because the branch points are clearly divided and divided. On the other hand, the 2D vasculature may cause false branching points due to background confusion. Also, a region where blood vessels overlap or intersect may occur due to lack of depth information. In order to improve this problem, it is generated in the smallest unit as shown in FIG. 2 .

The blood vessel analyzer 300 reconstructs candidate segments of the 2D blood vessel graph by searching the path of the 2D blood vessel graph corresponding to the segment connected to the parent and child nodes in the 3D blood vessel graph.

FIG. 3 is a diagram for explaining a process of extracting a segment of a 3D blood vessel in the blood vessel analysis unit of FIG. 1 . FIG. 4 is a view for explaining a process of generating all possible connection structures of the 2D blood vessel corresponding to the fragment of the 3D blood vessel of FIG. 3 .

에 대응하는

에 대하여 혈관구조 재구성을 위해 다음과 같은 2가지 사실을 전제로 수행한다. 1) 동일 환자의 2D, 3D 혈관 데이터를 사용하기 때문에 정확하게 분할된

와

는 유사하다. 2) 강체정합을 통해 동일한

와

는 일정 범위 내에 존재한다.In the present invention, the extracted

corresponding to

In order to reconstruct the vasculature, the following two facts are premised. 1) Because 2D and 3D blood vessel data of the same patient are used,

Wow

is similar 2) the same through rigid body registration

Wow

is within a certain range.

에 대하여

를 만족하는

를

에 대한 후보 절편(Candidate Segment)의 집합을

라고 정의한다. 이때,

은 2D 후보 절편 개수를 의미한다. Based on these assumptions, each

about

to satisfy

cast

A set of candidate segments for

define it as At this time,

is the number of 2D candidate fragments.

을 이용하여

와 유사한 하나의 연결된 구조를 생성한다.

에 대하여 생성된 연결된 구조를

라 정의한다.

은 모든 경로를 탐색하여 후보 절편 사이의 연결 가능한 모든 구조를 도 4와 같이 생성한다.next,

using

Creates a single linked structure similar to

The linked structure created for

define it as

searches all paths to generate all connectable structures between candidate fragments as shown in FIG. 4 .

Finally, an unnecessary candidate line is removed by performing a connection relationship analysis on a candidate line (Candi-Line) constituting the candidate fragments. The connection relationship analysis may be performed in order from the lowest level including the leaf node to the highest level including the root node.

Since the 3D blood vessel structure is a single, uninterrupted structure, the connection elements of the 2D candidate line are analyzed based on the 3D segment connected by the adjacent parent-child nodes. In this case, the child candidate line not connected to the parent candidate line may be regarded as an unnecessary candidate line and removed.

The matching unit 400 compares the vertex of the 3D blood vessel graph and the 2D blood vessel graph based on the vertical inclination of one vertex of the 3D CTA image and the distance between the one point and an adjacent vertex within a preset range. Match each vertex.

5 is a view comparing the vertex matching using the distance and the slope of the present invention with that of the prior art.

FIG. 5(a) is a distance-based matching method, and FIG. 5(a) illustrates a gradient-based matching method. The curvature information of the vessel center line is used to find the optimal 2D vessel center line vertex that matches each point of the 3D vessel center line.

Referring to FIG. 5 , when point matching using only simple curvature is performed, erroneous point matching may be performed in some areas due to deformations such as errors, beats, and respirations occurring in the division or matching process. Also, due to the lack of depth information of the 2D XA image, multiple points may be matched.

In order to perform robust and accurate matching in consideration of such an error, in the present invention, Equation 3 below is used by using information of neighboring adjacent points together.

[Equation 3]

는 3D 혈관 중심선의

번째 Point에 대한 수직 기울기를 나타내고,

는

와 Matching된 2D Point를 나타낸다.

은

의 인접 Point 개수를 나타내고,

는 두 점 사이의 유클리드 거리를 나타낸다.

는

를 기울기로 갖고

를 지나는 직선의

절편을 나타낸다.

는 자유 파라미터이다.도 5(c)는 본 발명의 방식으로 수행된 Point Matching 결과를 보여준다. 도 5(c)를 참조하면, 곡률만을 이용하거나 한 점만을 이용한 Point Matching 보다 인접 Point의 정보를 활용한 Point Matching 기법이 균등하고 정확하게 이루어진다.

is the 3D vessel centerline

Represents the vertical inclination to the th Point,

Is

2D Point matched with

silver

represents the number of adjacent Points of

is the Euclidean distance between two points.

Is

have a slope of

of a straight line passing through

indicates the intercept.

is a free parameter. FIG. 5(c) shows the result of point matching performed by the method of the present invention. Referring to FIG. 5( c ), a point matching technique using information of adjacent points is performed equally and accurately rather than point matching using only a curvature or using only one point.

The similarity comparison unit 500 compares the similarity between the slices of the 3D blood vessel graph and the candidate slices of the 2D blood vessel graph to select a candidate line for the 2D blood vessel.

In the present invention, an optimal candidate line is selected through Equation 4 below by using the difference between the distance, slope, length, and thickness between the 3D intercept and the candidate line (Candi-Line) for accurate similarity comparison.

[Equation 4]

,

,

는 3D 절편과 후보 라인 사이의 기울기, 두께, 길이 차이를 의미한다.

는 0이상의 값을 가지며, 각 항에 대해 끼치는 영향의 정도를 의미한다. 이는 실험적으로 결정될 수 있으며, 각각의 항들은 아래의 수학식 5 내지 수학식 7과 같이 정의된다.in Equation 4

,

,

denotes the difference in slope, thickness, and length between the 3D intercept and the candidate line.

has a value of 0 or more, indicating the degree of influence on each term. This may be determined experimentally, and each term is defined as in Equations 5 to 7 below.

[Equation 5]

[Equation 6]

[Equation 7]

와

는 혈관의 두께를 나타내며,

와

는 투영된 3D 절편과 2D 후보 라인의 길이를 나타낸다. 3D 혈관의 두께는 혈관의 기울기에 대하여 수직인 벡터의 길이를 통해 측정한다. 길이는 두 혈관이 Matching된 시작점과 끝점

,

사이의 경로

에 대하여

에서

를 조건으로 아래의 수학식 8과 같이 구성된다.

Wow

represents the thickness of the blood vessel,

Wow

denotes the projected 3D intercept and the length of the 2D candidate line. The thickness of the 3D blood vessel is measured through the length of the vector perpendicular to the slope of the vessel. The length is the starting point and the ending point where the two blood vessels are matched.

,

path between

about

at

It is constructed as in Equation 8 below.

[Equation 8]

의 길이는 아래의 수학식 9와 같이 Matching된 Point들의 유클리드 거리의 합으로 정의된다.

The length of is defined as the sum of the Euclidean distances of the Matched Points as in Equation 9 below.

[Equation 9]

은 점

의 차원을 나타낸다.상기 비강체 변형부(600)는 미리 정의된 에너지 함수가 최소값을 갖도록 경사 하강법(Gradient Decent Method)을 이용하여 상기 3D 혈관 그래프의 정점의 위치를 선정된 2D 혈관의 후보 라인의 정점으로 이동시킨다.here

silver point

The non-rigid body deforming unit 600 determines the position of the vertex of the 3D blood vessel graph by using the gradient descent method so that the predefined energy function has a minimum value as a candidate line of the selected 2D blood vessel. move to the vertex of

To change the position and shape of the vessel centerline, an energy function is defined and the positions of the 3D vessel centerline points are changed using the gradient descent method so that the energy function has a minimum value. The energy function is defined by three terms as shown in Equation 10 below.

[Equation 10]

는 혈관 중심선의 연속성(Continuous)을 나타낸다. 혈관 중심선

가

개의 점

으로 구성될 때,

번째 Point에서의

는 아래의 수학식 11과 같다.Each energy term serves to induce the shape and position to approximate the 2D vessel centerline while preserving the topological information of the 3D vessel centerline.

represents the continuity of the vessel center line. blood vessel center line

go

dog dots

When composed of

at the second point

is the same as in Equation 11 below.

[Equation 11]

는 인접 점간의 평균거리를 나타내며, 혈관 중심선의 인접점 간격이 일정하게 유지되도록 한다.

는 혈관 중심선이 진동하거나 급격한 굴곡 변화가 생기지 않고, 완만하며 자연스러운 형태를 만드는 역할을 한다.

는 곡률을 계산하는 에너지 항을 의미하며, 아래의 수학식 12와 같이 표현된다.

represents the average distance between adjacent points, and ensures that the distance between adjacent points on the center line of blood vessels is kept constant.

It plays a role in creating a smooth and natural shape without causing the blood vessel centerline to vibrate or abrupt change in curvature.

denotes an energy term for calculating the curvature, and is expressed as in Equation 12 below.

[Equation 12]

와

는 혈관 중심선의 기본 구조를 결정하는데 사용되는 에너지 항인 반면,

는 혈관 중심선의 특징을 반영한다. 즉, 3D 혈관을 2D 혈관에 맞추어 위치와 형태가 변화하도록 만들어주는 에너지 항으로 아래의 수학식 13과 같이 정의된다.

Wow

is the energy term used to determine the basic structure of the vessel centerline, whereas

reflects the characteristics of the vessel centerline. That is, it is an energy term that makes the position and shape of the 3D blood vessel change according to the 2D blood vessel, and is defined as in Equation 13 below.

[Equation 13]

는

을 나타낸다.

일 경우는 1) 매칭 Point가 없거나 2) 두 점 사이의 거리가

를 벗어났을 경우이며, 나머지는 모두

로 정의한다.

는

에 매칭된 Point를 나타낸다.이에 따라, 본 발명은 혈관의 심장 박동과 호흡 등 기타 외부 요인에 따른 위치와 형태 변화를 반영 가능하도록 함으로써 강체 정합의 한계를 극복하여 보다 정확한 혈관 정보를 제공할 수 있다.

Is

indicates

If 1) there is no matching point or 2) the distance between the two points is

out of , and all others are

to be defined as

Is

The point matched to .

In addition, the present invention can be used as an auxiliary image for diagnosis, treatment confirmation, automatic blood vessel labeling, follow-up examination and research as well as coronary intervention. In addition, it can be applied not only to the cardiovascular system, but also to other blood vessels or an application field that divides a procedure/surgical tool similar to a blood vessel.

6 is a flowchart of a method for matching multiple modality non-rigid bodies based on blood vessel feature information according to an embodiment of the present invention.

The multimodality non-rigid body matching method based on blood vessel characteristic information according to the present embodiment may be performed in substantially the same configuration as the apparatus 10 of FIG. 1 . Accordingly, the same components as those of the device 10 of FIG. 1 are given the same reference numerals, and repeated descriptions are omitted.

In addition, the multimodality non-rigid body registration method based on blood vessel characteristic information according to the present embodiment may be executed by software (application) for performing multi-modality non-rigid body registration based on blood vessel characteristic information.

The multimodality non-rigid body registration method based on blood vessel characteristic information according to the present invention provides a fast and accurate registration technique of a 3D CTA image taken before the procedure and a 2D XA image taken during the procedure of the same patient in order to provide useful guiding information to the operator. .

Referring to FIG. 6 , the multimodality non-rigid body registration method based on blood vessel characteristic information according to the present embodiment performs rigid body registration to match the space of a 3D CTA image taken before the procedure and a 2D XA image taken during the procedure ( step S10).

The step of performing the rigid body registration (step S10) includes obtaining DICOM (Digital Imaging and Communications in Medicine) information when the 2D XA image is taken, and applying the DICOM information to the 3D blood vessel centerline extracted from the 3D CTA image. Thus, the steps of generating a virtual environment at the time the 2D XA image is acquired and projecting the 3D blood vessel center line and the 2D blood vessel center line onto the virtual environment to perform precise registration to match the registration space may be performed.

The performing of the precise registration includes performing a thinning process of the 3D blood vessel center line and the 2D blood vessel center line, generating a distance map between the thinned 3D blood vessel center line and the 2D blood vessel center line, and based on the distance map A step of performing similarity comparison and a step of performing precise matching using a transformation function to which the Powell optimization technique is applied may be performed.

Here, the performing of the precise registration includes moving the center point of the 3D blood vessel center line projected on the virtual environment to the origin, and sequentially converting the rotation vector and the movement vector of the 3D blood vessel center line based on the origin. and moving the center point of the 3D blood vessel center line to the center point of the 2D blood vessel center line.

A 3D blood vessel graph and a 2D blood vessel graph are generated for a blood vessel structure represented by a point, an edge, and a root from each blood vessel centerline of the 3D CTA image and the 2D XA image (step S20).

와 형태 속성

로 구성된

로 표현한다. In the present invention, a graph structure of the blood vessel structure is generated in order to efficiently compare 2D blood vessels and 3D blood vessels by reducing the complexity of the blood vessel structure. This is a graph structure

and shape properties

composed of

expressed as

는 정점

, Edge

, Root

로 구성되며

로 표현한다. 형태 속성

는 Edge당 n개의 혈관 중심선의 중점들(Centerline Points)로 구성되어 있고, 이는

와

로 표현한다.graph structure

is the vertex

, Edge

, Root

is composed of

expressed as shape properties

is composed of n centerline points per edge, which is

Wow

expressed as

는 분지점이 명확하게 구분되어 분할되어있기 때문에 트리 구조가 생성된다. 반면, 2D 혈관 구조는 배경의 혼란으로 인해 잘못된 분지점이 발생될 수 있다. 또한, 깊이(Depth) 정보 부족으로 인해 혈관이 겹쳐지거나 교차하는 영역이 발생할 수 있다. 이러한 문제를 개선하기 위해 도 2와 같이 가장 작은 단위로 생성한다.

A tree structure is created because the branch points are clearly divided and divided. On the other hand, the 2D vasculature may cause false branching points due to background confusion. Also, a region where blood vessels overlap or intersect may occur due to lack of depth information. In order to improve this problem, it is generated in the smallest unit as shown in FIG. 2 .

The path of the 2D blood vessel graph corresponding to the segment connected to the parent and child nodes in the 3D blood vessel graph is searched, and candidate segments of the 2D blood vessel graph are reconstructed (step S30).

The step of reconstructing the candidate segments of the 2D blood vessel graph (step S30) is performed on the assumption that the shape properties of the 3D blood vessel graph and the 2D blood vessel graph are similar and exist within a certain range.

Specifically, the 3D blood vessel graph and the 2D blood vessel graph have a distance of a shape attribute lower than a preset threshold, and candidate fragments of the 2D blood vessel graph are extracted, and the shape of the 3D blood vessel graph among the candidate fragments of the 2D blood vessel graph. One candidate fragment with the most similar properties is selected.

The process of creating a connectable structure by exploring all paths of the selected candidate fragment, and removing the child candidate lines that are not connected to the parent candidate line by performing a connection relationship analysis on the candidate line of the 2D blood vessel of the selected candidate fragment. rough

The connection relationship analysis may be sequentially performed from the lowest level including the leaf node to the highest level including the root node.

Based on the vertical inclination of one vertex of the 3D CTA image and the distance between the one point and an adjacent vertex within a preset range, the vertices of the 3D blood vessel graph and the vertices of the 2D blood vessel graph are respectively matched (step S40).

In case of performing point matching using only simple curvature, incorrect point matching may occur in some areas due to deformations such as errors, beats, and respirations occurring in the division or matching process. In addition, due to the lack of depth information of the 2D XA image, multiple points may be matched.

In order to perform robust and accurate matching in consideration of such an error, in the present invention, the distance and slope information of the neighboring points are used together.

A candidate line of the 2D blood vessel is selected by comparing the similarity between the slices of the 3D blood vessel graph and the candidate slices of the 2D blood vessel graph (step S50).

In order to select an optimal candidate line, the similarity is compared using the difference in distance, slope, length, and thickness between the intercept of the 3D blood vessel graph and the candidate line of the 2D blood vessel.

A non-rigid transformation is performed by moving the position of the vertex of the 3D blood vessel graph to the vertex of the candidate line of the selected 2D vessel using the gradient descent method so that the predefined energy function has a minimum value (step S60).

The energy function may include an energy term for calculating continuity, an energy term for calculating curvature, and an energy term for changing the position and shape of a 3D blood vessel to a 2D blood vessel.

Accordingly, the present invention can provide more accurate blood vessel information by overcoming the limitation of rigid body registration by allowing changes in the position and shape of blood vessels according to other external factors, such as heartbeat and respiration, to be reflected.

Such a blood vessel characteristic information-based multimodality non-rigid body matching method may be implemented as an application or implemented in the form of program instructions that may be executed through various computer components and recorded in a computer-readable recording medium. The computer-readable recording medium may include program instructions, data files, data structures, etc. alone or in combination.

The program instructions recorded on the computer-readable recording medium are specially designed and configured for the present invention, and may be known and available to those skilled in the computer software field.

Examples of the computer-readable recording medium include a hard disk, a magnetic medium such as a floppy disk and a magnetic tape, an optical recording medium such as a CD-ROM, a DVD, and a magneto-optical medium such as a floppy disk. media), and hardware devices specially configured to store and execute program instructions such as ROM, RAM, flash memory, and the like.

Examples of program instructions include not only machine language codes such as those generated by a compiler, but also high-level language codes that can be executed by a computer using an interpreter or the like. The hardware device may be configured to operate as one or more software modules to perform processing according to the present invention, and vice versa.

Although described above with reference to the embodiments, those skilled in the art can variously modify and change the present invention within the scope without departing from the spirit and scope of the present invention described in the claims below You will understand.

The present invention provides a fast and accurate registration technique for visualizing the fusion with the 2D XA image taken during the procedure using the 3D CTA image taken before the procedure of the same patient. It can help to make it easier to understand the structure. Accordingly, it can be efficiently utilized in the medical field by providing navigation information to the clinician and shortening the operation and operation time.

10: Multimodality Non-Rigid Body Matching Device Based on Vessel Characteristic Information
100: rigid body registration part
200: graph generating unit
300: blood vessel analysis unit
400: matching unit
500: similarity comparison unit
600: non-rigid deformation part

Claims (11)

performing rigid body registration to match the space between the 3D CTA image taken before the procedure and the 2D XA image taken during the procedure;
generating a 3D vascular graph and a 2D vascular graph for vascular structures represented by vertices, edges, and roots from each vascular centerline of the 3D CTA image and the 2D XA image;
reconstructing candidate segments of the 2D blood vessel graph by searching a path of the 2D blood vessel graph corresponding to a segment connected to parent and child nodes in the 3D blood vessel graph;
matching the vertices of the 3D blood vessel graph and the vertices of the 2D blood vessel graph, respectively, based on a vertical inclination of one vertex of the 3D CTA image and a distance between the one point and an adjacent vertex within a preset range;
selecting a candidate line of a 2D blood vessel by comparing the similarity between the slices of the 3D blood vessel graph and the candidate slices of the 2D blood vessel graph; and
A non-rigid body transformation step of moving the position of the vertex of the 3D blood vessel graph to the vertex of the candidate line of the selected 2D vessel using the gradient descent method so that the predefined energy function has a minimum value; A multimodality non-rigid registration method based on vascular feature information.
The method of claim 1, wherein performing the rigid body registration comprises:
acquiring DICOM (Digital Imaging and Communications in Medicine) information when taking the 2D XA image;
generating a virtual environment when the 2D XA image is acquired by applying the DICOM information to a 3D blood vessel centerline extracted from the 3D CTA image; and
Projecting the 3D blood vessel center line and the 2D blood vessel center line to the virtual environment to perform precise registration to match the registration space;
The method of claim 2, wherein performing the precise matching comprises:
performing a thinning process of the 3D blood vessel center line and the 2D blood vessel center line;
generating a distance map between the thinned 3D blood vessel center line and the 2D blood vessel center line;
performing a similarity comparison based on the distance map; and
Performing precise registration using a transformation function to which Powell's optimization technique is applied; A multimodality non-rigid body registration method based on blood vessel feature information, comprising a.
The method of claim 3, wherein performing the precise matching comprises:
moving the center point of the 3D blood vessel center line projected on the virtual environment to the origin;
sequentially converting a rotation vector and a movement vector of the 3D blood vessel centerline based on the origin; and
Moving the center point of the 3D blood vessel center line to the center point of the 2D blood vessel center line; including, a multi-modality non-rigid body matching method based on blood vessel characteristic information.
The method of claim 1, wherein reconstructing the candidate segments of the 2D blood vessel graph comprises:
The 3D blood vessel graph and the 2D blood vessel graph have similar shape properties and are performed on the assumption that they exist within a certain range.
The method of claim 5, wherein reconstructing the candidate segments of the 2D blood vessel graph comprises:
extracting candidate fragments of the 2D blood vessel graph whose distance between the shape attribute of the 3D blood vessel graph and the 2D blood vessel graph is lower than a preset threshold;
selecting one candidate segment most similar to the shape attribute of the 3D blood vessel graph from among the candidate segments of the 2D blood vessel graph;
generating a connectable structure by searching all paths of the selected candidate fragment; and
A method for matching multiple modality non-rigid bodies based on blood vessel characteristic information, comprising: performing a connection relationship analysis on candidate lines of 2D blood vessels of the selected candidate fragment to remove child candidate lines that are not connected to parent candidate lines.
7. The method of claim 6,
and wherein the connection relationship analysis is sequentially performed from the lowest level including the leaf node to the highest level including the root node.
The method of claim 1, wherein the selecting a candidate line for a 2D blood vessel by comparing the similarity comprises:
A multimodality non-rigid registration method based on blood vessel characteristic information, which compares similarities using differences in distance, slope, length, and thickness between the intercept of the 3D blood vessel graph and the candidate line of the 2D blood vessel.
According to claim 1, wherein the non-rigid deformation step of moving the position of the vertex of the 3D blood vessel graph to the vertex of the candidate line of the selected 2D blood vessel,
The energy function includes an energy term for calculating continuity, an energy term for calculating curvature, and an energy term for changing the position and shape of a 3D vessel to a 2D vessel. matching method.
According to claim 1,
A computer-readable storage medium in which a computer program for performing the multimodality non-rigid body matching method based on the blood vessel characteristic information is recorded.
a rigid body matching unit that performs rigid body registration to match the space between the 3D CTA image taken before the procedure and the 2D XA image taken during the procedure;
Graph generation for generating 3D blood vessel graphs and 2D blood vessel graphs for blood vessel structures represented by vertices, edges, and roots from each blood vessel centerline of the 3D CTA image and the 2D XA image wealth;
a blood vessel analyzer configured to reconstruct candidate segments of the 2D blood vessel graph by searching a path of the 2D blood vessel graph corresponding to a segment connected to a parent and a child node in the 3D blood vessel graph;
A matching unit that matches the vertices of the 3D blood vessel graph and the vertices of the 2D blood vessel graph, respectively, based on the vertical inclination of one vertex of the 3D CTA image and the distance between the one point and an adjacent vertex within a preset range ;
a similarity comparison unit for selecting a candidate line of a 2D blood vessel by comparing the similarity between the slices of the 3D blood vessel graph and the candidate slices of the 2D blood vessel graph; and
A non-rigid deformation unit that moves the position of the vertex of the 3D blood vessel graph to the vertex of the candidate line of the selected 2D blood vessel using the gradient descent method so that the predefined energy function has a minimum value; A multimodality non-rigid body matching device based on vessel characteristic information.

Images