KR102627917B1 - Method for determining fiducial points for anatomical segmentation of the heart - Google Patents

Abstract

The method for determining a reference point for anatomical segmentation of a heart image of the present invention includes determining a reference point where the endocardium of the right ventricle (RV) and the left ventricle (LV) are connected in a plurality of heart images according to a predetermined rule. and when the reference point of the heart image is not determined according to the rule, determining the reference point of the heart image for which the reference point is not determined based on the heart image adjacent to the heart image for which the reference point is not determined.

Description

{Method for determining fiducial points for anatomical segmentation of the heart}

The present invention relates to a method for determining reference points for anatomical segmentation of cardiac images. This invention includes results carried out with support from the Ministry of SMEs and Startups' Startup and Commercialization Support Project [10448495, Development of an automated model for heart disease diagnosis through comprehensive cardiac magnetic resonance imaging (MRI) analysis].

Among various molecular imaging techniques, magnetic resonance imaging (MRI) provides excellent anatomical imaging based on the interaction between molecules surrounding the tissue (lattice) and hydrogen atoms (protons). Because it can provide images, it is considered one of the most powerful and non-invasive diagnostic tools.

Recently, magnetic resonance imaging imaging technologies are being developed to quantitatively measure biophysical quantities in addition to existing anatomical tomography images. Representative biophysical quantities that can be measured with MRI include the self-recovery time of hydrogen atoms due to high-frequency pulses in human tissue, blood flow rate, diffusion, and perfusion, and can be measured non-invasively, safely, and accurately.

In particular, through cardiac magnetic resonance imaging, the movements of the heart's systole and diastole can be seen and the overall heart function as well as local heart function can be evaluated.

Specifically, in order to evaluate the movement of the left and right heart, the heart is measured based on the point where the endocardium of the right ventricle (RV) and the left ventricle (LV) are connected (Right Ventricle insertion points; below RVIP, reference point). Heart movement can be evaluated by anatomically dividing the heart and observing the movement of each segment.

Meanwhile, the location or shape of the heart may have various characteristics for each individual, and the heart displayed as an image may have various shapes depending on the imaging device or imaging angle, so it is important to accurately determine the reference point for accurate anatomical segmentation.

Conventionally, reference points for anatomical segmentation in heart images were determined through deep learning-based image processing, but there is a problem in which reference points cannot be determined in some heart images among multiple heart images.
(Patent Document 1) KR 10-2017-0084435 A
(Patent Document 2) KR 10-2019-0084100 A

In order to solve the above-mentioned problem, the purpose of the present invention is to propose a method for determining reference points for anatomical segmentation of the heart when the reference point of the heart image is not determined through deep learning-based image processing.

A method of determining a reference point for anatomical segmentation of a heart image according to an embodiment of the present invention to solve the above technical problem includes a reference point where the endocardium of the right ventricle (RV) and the left ventricle (LV) are connected in a plurality of heart images. determining the reference point of the heart image according to a predetermined rule, and if the reference point of the heart image is not determined according to the rule, the reference point of the heart image for which the reference point is not determined based on the heart image adjacent to the heart image for which the reference point is not determined It may include a step of determining.

In addition, it further includes calculating a guide line for the reference point by curve fitting in a three-dimensional space based on the reference point of the adjacent heart images, and the step of determining the reference point includes calculating the calculated guide line. Based on this, the reference point of the heart image for which the reference point has not been determined can be determined.

Additionally, in determining the reference point, the reference point may be determined by considering an axial gap between the adjacent heart images and a heart image for which the reference point has not been determined.

Additionally, the adjacent heart image may be a heart image for which a reference point has been determined through the image processing, and may be a heart image parallel to the same axis as a heart image for which the reference point has not been determined.

Additionally, the step of determining the reference point may determine the reference point of a heart image for which the reference point has not been determined at the location of the determined reference point of the adjacent heart image.

In addition, the step of determining according to the predetermined rule includes segmenting the acquired cardiac image using a learned neural network into a plurality of regions according to a predetermined standard on orthogonal coordinates, and dividing the left ventricle region among the divided regions. It may further include determining a center point, converting the heart image into a polar coordinate system using the determined center point, and determining a reference point for segmentation in the heart image converted to the polar coordinate system.

Meanwhile, a cardiac image segmentation device for anatomical segmentation of heart images is a device that determines the reference point where the endocardium of the right ventricle (RV) and the left ventricle (LV) are connected in multiple heart images according to a predetermined rule. 1 A reference point determination unit and, when the reference point of the heart image is not determined according to the rule, a second reference point determination unit that determines the reference point of the heart image for which the reference point has not been determined based on the heart image adjacent to the heart image for which the reference point has not been determined. It can be included.

In addition, it further includes a guide line calculation unit that calculates a guide line for the reference point by curve fitting in a three-dimensional space based on the reference points of the adjacent heart images, and the second reference point determination unit calculates the guide line for the reference point. Based on this, the reference point of the heart image for which the reference point has not been determined can be determined.

Additionally, the second reference point determination unit may determine the reference point by considering an axial gap between the adjacent heart images and a heart image for which the reference point has not been determined.

Additionally, the adjacent heart image may be a heart image for which a reference point has been determined through the image processing, and may be a heart image parallel to the same axis as a heart image for which the reference point has not been determined.

Additionally, the second reference point determination unit may determine a reference point of a heart image for which the reference point has not been determined at the location of the determined reference point of the adjacent heart image.

In addition, the first reference point determination unit segments the heart image acquired using the learned neural network into a plurality of regions according to predetermined standards on orthogonal coordinates, and determines the center point of the left ventricular region among the divided regions, The heart image can be converted to a polar coordinate system using the determined center point, and a reference point for segmentation can be determined from the heart image converted to the polar coordinate system.

According to the present invention, even when the reference point is not determined according to a predetermined rule, the reference point can be determined using an adjacent heart image.

Additionally, by using adjacent heart images, the reference point of a heart image for which the reference point has not been determined can be accurately determined.

Additionally, by determining the reference point using adjacent heart images, the reference point of all heart images can be accurately determined, and through this, accurate anatomical segmentation can be performed.

1 is a flowchart showing a method for determining a reference point for anatomical segmentation of a heart image according to an embodiment of the present invention.
Figure 2 is an exemplary diagram showing a method for segmenting a heart image according to an embodiment of the present invention.
Figures 3 and 4 are flowcharts showing a method for determining a reference point of a heart image according to an embodiment of the present invention.
Figure 5 is an exemplary diagram showing a neural network according to an embodiment of the present invention.
Figure 6 is an exemplary diagram showing a heart image segmented according to standardized criteria in one embodiment of the present invention.
Figures 7 and 8 are exemplary diagrams showing a method for determining a reference point according to an embodiment of the present invention.
Figure 9 is a block diagram showing the configuration of a cardiac image segmentation device according to an embodiment of the present invention.

The following merely illustrates the principles of the invention. Therefore, a person skilled in the art can invent various devices that embody the principles of the invention and are included in the concept and scope of the invention, although not clearly described or shown herein. In addition, all conditional terms and embodiments listed in this specification are, in principle, clearly intended only for the purpose of ensuring that the inventive concept is understood, and should be understood as not limiting to the specifically listed embodiments and states. .

The above-mentioned purpose, features and advantages will become clearer through the following detailed description in relation to the attached drawings, and accordingly, those skilled in the art in the technical field to which the invention pertains will be able to easily implement the technical idea of the invention. .

Additionally, when describing the invention, if it is determined that a detailed description of known technology related to the invention may unnecessarily obscure the gist of the invention, the detailed description will be omitted. Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the attached drawings.

Magnetic resonance imaging (MRI), or MRI, divides the components of the recovery process in which atoms longitudinally magnetized by a magnetic field are inverted using an inversion recovery RF pulse and then recovers in the form of an exponential curve, and the recovery time according to the components is determined. can be calculated. An MRI signal is obtained by scanning the signal generated from the subject in k space (k - space), and the obtained MRI signal is converted to obtain an MRI image.

To segment a heart image, which is a cardiac MRI image, according to the American Heart Association's (AHA) standardized heart segmentation criteria (e.g., 17 segments), the endocardium of the Right Ventricle (RV) and the Left Ventricle (LV) ) can be determined at the connected point (Right Ventricle insertion points (RVIP)), and heart segmentation can be performed based on the determined point. Hereinafter, this specification will be described by naming the above-mentioned point (RVIP) as a reference point.

1 is a flowchart showing a method for determining a reference point for anatomical segmentation of a cardiac image according to an embodiment of the present invention.

Referring to FIG. 1, the cardiac image segmentation apparatus 100 acquires a plurality of heart images from an imaging device, and selects a reference point at which the endocardium of the right ventricle (RV) and the left ventricle (LV) are connected in the acquired plurality of heart images. can be determined according to a predetermined rule (S100). Here, the imaging device may be a device that acquires magnetic resonance images, such as an MRI device. Additionally, the predetermined rule may refer to a method of determining a reference point through deep learning-based image processing. This will be described in more detail later with reference to FIG. 4 and below.

Next, when the reference point of the heart image is not determined according to a predetermined rule, the heart image segmentation device 100 determines the reference point of the heart image for which the reference point is not determined based on the heart image adjacent to the heart image for which the reference point is not determined. (S200). Here, the adjacent heart image is a heart image whose reference point has been determined according to a predetermined rule among the plurality of heart images in step (S100), and is located on the same axis as the heart image for which the reference point has not been determined, and is located at a predetermined interval based on the axis. Accordingly, it may be an image of the heart on a plane perpendicular to the axis.

Preferably, in this embodiment, a reference point can be determined using a plurality of heart images based on the central axis of the left ventricle.

In this regard, referring to FIG. 2, the heart image may be divided into base, mid, and top (apex) at predetermined reference height intervals in the height direction of the left ventricle of the heart, and the heart image may be divided into the corresponding divided It can mean each image of a heart cross-section in position.

That is, the plurality of heart images may be images of consecutive heart cross-sections with different heights along the axis in the height direction of the heart.

Referring again to FIG. 1 , in the step of determining the reference point ( S200 ), the cardiac image segmentation apparatus 100 may determine the reference point of a heart image for which the reference point has not been determined at the location of the determined reference point of the adjacent heart image. At this time, the heart image segmentation apparatus 100 may use the heart image located closest to the heart image for which the reference point has not been determined among adjacent heart images.

That is, the heart image segmentation apparatus 100 can be used to estimate the reference point as the reference point of an undetermined heart image based on the determined reference point of the adjacent heart image.

In addition, the cardiac image segmentation apparatus 100 may further include a process of calculating a guide line to determine a reference point of a heart image for which a reference point has not been determined, and this will be described with additional reference to FIG. 3 .

Referring to FIG. 3, the cardiac image segmentation apparatus 100 may calculate a guide line for the reference point by performing curve fitting in a three-dimensional space based on the reference point of a plurality of adjacent heart images (S11).

Specifically, the heart image segmentation apparatus 100 may calculate a function representing a trend in 3D space of reference points of adjacent heart images, and calculate a guide line in 3D space based on the function. Here, the guide line may be a trend line indicating the trend of reference points of heart images.

Next, in the step of determining the reference point (S200), the cardiac image segmentation apparatus 100 may determine the reference point of the heart image for which the reference point has not been determined based on the calculated guide line.

At this time, the heart image segmentation apparatus 100 may use the axial gap between adjacent heart images and the heart image for which the reference point is not determined to determine the reference point of the heart image for which the reference point is not determined within the guide line in the three-dimensional space.

The cardiac image segmentation apparatus 100 may determine a point on the guide line as the reference point by considering the axial gap between adjacent heart images and the heart image for which the reference point has not been determined.

Hereinafter, the decision step (S100) according to a predetermined rule will be described in more detail with reference to FIG. 4.

Referring to FIG. 4, the heart image segmentation apparatus 100 may segment a heart image acquired using a learned neural network into a plurality of regions according to a predetermined standard on orthogonal coordinates (S21). Here, the plurality of regions refers to regions divided into the right ventricular endocardium, left ventricular endocardium, and left ventricular myocardium (myocardium) of the heart.

In addition, the above-described neural network is learned to distinguish and segment a plurality of regions in a heart image, and the neural network may be composed of a CNN (Convolution Neural Network) model composed of layers that perform a plurality of convolution operations. In this regard, description will be made with additional reference to FIG. 5.

Figure 5 is an exemplary diagram showing a neural network according to an embodiment of the present invention.

Referring to FIG. 5, when the heart image 31 is input to the neural network 30 that has been learned to divide the heart image into right ventricular endocardium, left ventricular endocardium, and left ventricular myocardium regions, the features are divided through the layers inside the neural network 30. Values can be emphasized through convolution.

In addition, various feature values are output in the form of a new feature map through calculation with the filter determined for each convolution layer, and the final feature map generated through repeated calculation for each layer is again through an up-sampling process. By outputting object location information in pixel units according to the size of the input heart image 31, the heart image 33 divided into a plurality of regions can be output.

Specifically, in this embodiment, the neural network 30 consists of a first network that performs convolution and a second network that performs up-sampling in order to output the segmentation result at higher resolution, but the stages of each network are structured in a mutually symmetrical structure. It can be configured.

The first network of the neural network 30 gradually outputs a feature map in which characteristics of the myocardial region are emphasized through a convolution operation for the input heart image 31. Additionally, the feature map can be reduced in size by Max-pooling and the number of channels can be expanded depending on the filter being calculated.

Finally, the feature map generated in the first network can be re-extended again through the second network having a mutually symmetrical structure.

The second network can expand the size of the image through de(up)convolution instead of reducing the number of channels.

At this time, each stage of the second network uses the image of an expanded size through deconvolution and the output value of the corresponding stage of the first network to perform a convolution operation and generate a feature map for each stage.

That is, the second network concatenates the deconvolution result and the corresponding output value of the convolution step of the first network, and performs a convolution operation on the synthesized image to generate a feature map.

The above process can also be repeated for each step corresponding to the first network, and preferably can be performed until the size of the input heart image 31 and the size of the feature map output from the second network become the same. The heart image 33 divided into a plurality of regions through the second network may be composed of a left ventricular endocardial region 33a, a left ventricular myocardial region 33b, and a right ventricular endocardial region 33c, and each region is distinguished from the other. The area can be colored with a unique color or divided by a border and printed out.

Referring again to FIG. 4, the cardiac image segmentation apparatus 100 may determine the center point (axis) of the left ventricular region among the divided regions (S22). In this regard, description will be made with additional reference to FIG. 6.

Figure 6 is an exemplary diagram showing a heart image 43 according to an embodiment of the present invention.

Specifically, the cardiac image segmentation device 100 selects the left ventricular regions 43a and 43b among the left ventricular endocardial region 43a, left ventricular myocardial region 43b and right ventricular endocardial region 43c segmented in the heart image 43. The center of gravity may be determined as the center point 43d based on the average of the coordinate values on the Cartesian coordinate plane or the area of the left ventricle regions 43a and 43b. That is, the heart image segmentation device 100 can determine the center point of the left ventricular blood pool through the divided areas in the above heart image 43.

Next, the heart image segmentation apparatus 100 converts the heart image into a polar coordinate system using the determined center point (S23). This will be described with reference to FIG. 7.

Figure 7(a) shows a heart image in Cartesian coordinates and Figure 7(b) shows a heart image in polar coordinates.

Specifically, the heart image segmentation apparatus 100 uses the center point 51 of the determined left ventricular region as the origin of the polar coordinate system, and divides the heart image on the Cartesian coordinates of FIG. 7(a) into the heart image on the polar coordinates of FIG. 7(b). It can be converted to .

Next, the heart image segmentation apparatus 100 may determine a reference point for re-segmentation of the left ventricular region in the heart image converted to the polar coordinate system (S24). Specifically, the cardiac image segmentation device 100 may determine two reference points using the minimum/maximum Y-axis values of the right ventricular endocardial region and the median value of the X-axis of the left ventricular endocardial region of the cardiac image converted to a polar coordinate system. . In this regard, description will be made with additional reference to FIG. 8.

Referring to FIG. 8(a), the cardiac image segmentation device 100 divides the cardiac image converted into a polar coordinate system into a Y-axis coordinate value that is the minimum value on the Y-axis of the right ventricular endocardial region 63 and a left ventricular endocardial region ( The point 65 having the middle value on the X-axis of 61) as the X-axis coordinate value can be determined as the first reference point.

In addition, the heart image segmentation device 100 includes a Y-axis coordinate value that is the maximum value on the Y-axis of the right ventricular endocardial region 63 and the corresponding left ventricular endocardial region 61 among the heart images converted to the polar coordinate system. The point 67 having the middle value on the X-axis as the X-axis coordinate value can be determined as the second reference point.

That is, the cardiac image segmentation device 100 divides the left ventricular endocardial region 61 based on the Y-axis coordinate values corresponding to the minimum and maximum values on the Y-axis of the right ventricular endocardial region 63 in the cardiac image converted to the polar coordinate system. Among the coordinates of the median value on the X-axis, two points 65 and 67 can be determined as reference points, respectively.

The above reference point determination method can be performed based on pixel values in the input heart image, and the method can be defined as a rule and output through more simplified processing.

Hereinafter, the configuration of the cardiac image segmentation apparatus 100 will be described with reference to FIG. 9 .

Figure 9 is a block diagram showing the configuration of a cardiac image segmentation device according to an embodiment of the present invention.

Referring to FIG. 9 , the cardiac image segmentation apparatus 100 may include a first reference point determination unit 110, a guide line calculation unit 120, and a second reference point determination unit 130.

The first reference point determination unit 110 may determine a reference point where the endocardium of the right ventricle (RV) and the left ventricle (LV) of the plurality of heart images are connected according to a predetermined rule.

Specifically, the first reference point determination unit 110 divides the cardiac image acquired using a learned neural network into a plurality of regions according to predetermined standards on orthogonal coordinates, and selects the center point of the left ventricular region among the divided regions. After determining, the heart image can be converted to a polar coordinate system using the determined center point, and a reference point for segmentation can be determined from the heart image converted to the polar coordinate system.

The guide line calculation unit 120 may calculate a guide line for the reference point by performing curve fitting in a three-dimensional space based on the reference point of adjacent heart images.

Specifically, the guide line calculation unit 120 may calculate a function representing a trend in three-dimensional space of reference points of adjacent heart images, and calculate a guide line in three-dimensional space based on the function.

If the reference point of the heart image is not determined according to the above rule, the second reference point determination unit 130 may determine the reference point of the heart image for which the reference point has not been determined based on the heart image adjacent to the heart image for which the reference point has not been determined. .

Additionally, the second reference point determination unit 130 may determine a reference point of a heart image for which the reference point has not been determined based on the guide line calculated by the guide line calculation unit 120. At this time, the second reference point determination unit 130 may determine the reference point of the heart image for which the reference point has not been determined by considering the axial gap between adjacent heart images and the heart image for which the reference point has not been determined.

Hereinafter, according to the present invention described above, even when the reference point is not determined according to a predetermined rule, the reference point can be determined using an adjacent heart image.

Additionally, by using adjacent heart images, the reference point of a heart image for which the reference point has not been determined can be accurately determined.

Additionally, by determining the reference point using adjacent heart images, the reference point of all heart images can be accurately determined, and through this, accurate anatomical segmentation can be performed.

Furthermore, various embodiments described herein may be implemented in a recording medium readable by a computer or similar device, for example, using software, hardware, or a combination thereof.

According to hardware implementation, the embodiments described herein include application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), and field programmable gate arrays (FPGAs). It may be implemented using at least one of processors, controllers, micro-controllers, microprocessors, and other electrical units for performing functions. In some cases, as described herein, The described embodiments may be implemented as a control module itself.

According to software implementation, embodiments such as procedures and functions described in this specification may be implemented as separate software modules. Each of the software modules may perform one or more functions and operations described herein. Software code can be implemented as a software application written in an appropriate programming language. The software code may be stored in a memory module and executed by a control module.

The above description is merely an illustrative explanation of the technical idea of the present invention, and various modifications, changes, and substitutions can be made by those skilled in the art without departing from the essential characteristics of the present invention. will be.

Accordingly, the embodiments disclosed in the present invention and the accompanying drawings are not intended to limit the technical idea of the present invention, but are for illustrative purposes, and the scope of the technical idea of the present invention is not limited by these embodiments and the attached drawings. . The scope of protection of the present invention should be interpreted in accordance with the claims below, and all technical ideas within the equivalent scope should be interpreted as being included in the scope of rights of the present invention.

Claims (13)

In a method for determining reference points for anatomical segmentation of cardiac images performed on a computer device,
Segmenting a heart image on a Cartesian coordinate plane obtained using a learned neural network into a plurality of regions;
determining a center point on the Cartesian coordinate plane of a left ventricular region among the divided regions;
converting the heart image into a polar coordinate system using the determined center point; and
Determining a reference point where the endocardium of the right ventricle (RV) and the left ventricle (LV) are connected for segmentation in the heart image converted to the polar coordinate system. According to claim 1,
The heart image is a continuous heart cross-sectional image according to a predetermined reference height on the axis in the height direction with respect to the heart,
It further includes calculating a guide line for the reference point by curve fitting the reference point of the consecutive cardiac cross-sectional images in a three-dimensional space,
The step of determining the reference point is characterized in that determining a point on the calculated guide line as a reference point of a heart image for which the reference point has not been determined. According to claim 2,
The step of determining the reference point includes determining a point of the guide line determined according to the height of the heart image for which the reference point has not been determined as the reference point. According to claim 1,
The step of determining the reference point involves determining the reference point using the coordinate value on the Y-axis of the right ventricular endocardial region and the X-axis coordinate value of the left ventricular endocardial region of the heart image converted to the polar coordinate system. method. According to claim 1,
The step of determining the center point includes determining the center of gravity determined as the area of the left ventricle region on the Cartesian coordinate plane as the center point. delete In a cardiac image segmentation device for anatomical segmentation of cardiac images,
processor; and
comprising a memory that loads a computer program executed by the processor,
The computer program is,
Segmenting a heart image on a Cartesian coordinate plane obtained using a learned neural network into a plurality of regions;
determining a center point on the Cartesian coordinate plane of a left ventricular region among the divided regions;
converting the heart image into a polar coordinate system using the determined center point; and
Determining a reference point where the endocardium of the right ventricle (RV) and the left ventricle (LV) are connected for segmentation in the heart image converted to the polar coordinate system;
A cardiac image segmentation device comprising: According to claim 7,
The heart image is a continuous heart cross-sectional image according to a predetermined reference height on the axis in the height direction with respect to the heart,
It further includes calculating a guide line for the reference point by curve fitting the reference point of the consecutive cardiac cross-sectional images in a three-dimensional space,
The step of determining the reference point is a cardiac image segmentation device characterized in that a point on the calculated guide line is determined as a reference point of a heart image for which the reference point has not been determined. According to claim 8,
The step of determining the reference point is a heart image segmentation device characterized in that a point of the guide line determined according to the height of the heart image for which the reference point has not been determined is determined as the reference point. According to claim 7,
The step of determining the reference point is a heart image characterized in that the reference point is determined using the coordinate value on the Y-axis of the right ventricular endocardial region and the coordinate value on the X-axis of the left ventricular endocardial region of the heart image converted to the polar coordinate system. Splitting device. According to claim 7,
The step of determining the center point includes determining the center of gravity determined as the area of the left ventricle region on the Cartesian coordinate plane as the center point. A computer-readable recording medium storing a program for executing a method for determining reference points for anatomical segmentation of cardiac images performed by the computer device according to any one of claims 1 to 5. delete