KR20210135682A - Apparatus and Method for Analyzing Image Distinguishing Boundary Portion of Cardiac Chamber Image - Google Patents

Abstract

An image analysis apparatus and method for classifying a boundary of a cardiac chamber image recognize an object of a cardiac chamber in a 3D MRI image by using an artificial neural network, form a 3D model by classifying a boundary region between a left atrium and a left ventricle, and have an excellent effect of segmentation accuracy of classifying the boundary region. The image analysis apparatus includes: an input unit which receives a cardiac MRI data sheet including a plurality of slice images representing cardiac chamber images of a left ventricle and a left atrium from a 3D cardiac magnetic resonance imaging apparatus; a pre-processing unit which pre-processes each slice image by removing noise; a feature extracting unit which extracts a plurality of features from each slice image of all objects by using an algorithm linking the pre-processed slide image; a machine learning unit; and a control unit.

Description

Apparatus and Method for Analyzing Image Distinguishing Boundary Portion of Cardiac Chamber Image

The present invention relates to an image analysis apparatus, and more particularly, to a cardiac chamber image boundary that recognizes an object of a heart chamber in a 3D MRI image using an artificial neural network, and forms a 3D model by dividing a boundary region between a left atrium and a left ventricle It relates to an image analysis apparatus and method for discriminating.

3D Cardiac Magnetic Resonance Imaging is widely used to diagnose heart diseases such as congenital heart defects, left ventricular hypertrophy and left atrial hypertrophy. It is one of the non-invasive techniques for examining cardiac anatomy.

Cardiac imaging plays an important role in the diagnosis of various cardiovascular diseases and abnormalities, such as ventricular tachycardia and atrial fibrillation.

These medical imaging technologies include intravascular ultrasound, 2D and 3D cardiac MRI. 3D cardiac MRI requires image segmentation for doctors to read or interpret, and specific technical and medical training is required to understand and classify raw 3D MRI.

In other words, since the image segmentation operation in 3D MRI depends on the ability of a doctor, it takes time to identify the target anatomy, perform an examination and diagnosis, and there is a possibility that an error may occur in image analysis.

Korea Registered Patent No. 10-1579948

In order to solve this problem, the present invention recognizes an object of a heart chamber in a 3D MRI image using an artificial neural network, and divides the boundary between the left atrium and the left ventricle to form a 3D model. An object of the present invention is to provide an image analysis apparatus and method.

According to a feature of the present invention for achieving the above object, there is provided an image analysis apparatus for dividing a boundary of a cardiac chamber image,

an input unit for receiving a cardiac MRI data sheet including a plurality of slice images representing images of the heart chambers of the left ventricle and the left atrium from a 3D cardiac magnetic resonance imaging apparatus;

a pre-processing unit for pre-processing each slice image by removing noise;

a feature extracting unit for extracting a plurality of features from each slice image of all objects using an algorithm linking the pre-processed slide image;

a machine learning unit generating a left atrium and left ventricle junction detection model that determines whether there is a junction region between the left ventricle and the left atrium of an object existing in each of the slice images by using the plurality of extracted features; and

A plurality of features extracted from each slice image are transmitted as input data of the machine learning unit, and the plurality of features are controlled to output whether there is a junction region between the left atrium and the left atrium by the junction detection model of the left atrium and left ventricle. It is characterized in that it comprises a control unit.

An image analysis apparatus for dividing a boundary of a cardiac chamber image according to a feature of the present invention,

Receiving a cardiac MRI data sheet including a plurality of slice images of an axial image, a plurality of slice images of a coronal image, and a plurality of slice images of a sagittal image from a 3D cardiac magnetic resonance imaging apparatus input unit;

a preprocessor for preprocessing each slice image using a threshold value that sets a maximum brightness level to a limit of 85% to generate a binary image;

By using the preprocessed slide image connection algorithm, all objects in each slice image are elliptic-related parameters Eccentricity, Major Axis, Object Area (Amount of Pixels), Minor Axis, Object a feature extracting unit for extracting a feature of a location; and

and a machine learning unit that generates a junction detection model of the left atrium and the left ventricle that determines whether there is a junction region between the left ventricle and the left atrium of the object present in each of the sliced images by using the five extracted features.

An image analysis method for classifying a boundary of a cardiac chamber image according to a feature of the present invention,

Receiving a cardiac MRI data sheet including a plurality of slice images of an axial image, a plurality of slice images of a coronal image, and a plurality of slice images of a sagittal image from a 3D cardiac magnetic resonance imaging apparatus step;

generating a binary image by pre-processing each slice image using a threshold value that sets a maximum brightness level to a limit of 85%;

By using the preprocessed slide image connection algorithm, all objects in each slice image are elliptic-related parameters Eccentricity, Major Axis, Object Area (Amount of Pixels), Minor Axis, Object extracting a feature of the location; and

A junction detection model of the left atrium and left ventricle that determines whether there is a junction region between the left ventricle and the left atrium of the object present in each slice image is generated using the extracted five features, and the left atrium and the left atrium using the five features as input data and outputting whether there is a junction region between the left ventricle and the left atrium by the junction detection model of the left ventricle.

According to the above-described configuration, the present invention has an effect of recognizing an object of a cardiac chamber in a 3D MRI image and having excellent segmentation accuracy for dividing a boundary region between the left atrium and the left ventricle.

1 is a diagram showing the configuration of an image analysis apparatus for dividing a boundary of a cardiac chamber image according to an embodiment of the present invention.
2 is a diagram illustrating an example of an axial image, a coronal image, and a sagittal image according to an embodiment of the present invention.
3 is a diagram illustrating a junction region between a left ventricle and a left atrium in an axial image, a coronal image, and a sagittal image according to an embodiment of the present invention.
4 is a diagram illustrating a configuration of a neural network for detecting a junction region between LV and LA according to an embodiment of the present invention.
5 is a diagram illustrating loading of each slice image according to an embodiment of the present invention.
6 is a diagram illustrating an image analysis method for dividing a boundary of a cardiac chamber image according to an embodiment of the present invention.
7 is a diagram illustrating a result of division of axial slice images according to an embodiment of the present invention.
8 is a diagram illustrating a 3D model by reconstructing a segmented slice image according to an embodiment of the present invention.
9 is a diagram illustrating chamber mass partitioning in each partitioned slice according to an embodiment of the present invention.

Throughout the specification, when a part "includes" a certain element, it means that other elements may be further included, rather than excluding other elements, unless otherwise stated.

1 is a diagram showing the configuration of an image analysis apparatus for classifying boundaries of cardiac chamber images according to an embodiment of the present invention, and FIG. 2 is an example of an axial image, a coronal image, and a sagittal image according to an embodiment of the present invention 3 is a view showing the junction area between the left ventricle and the left atrium of the axial image, coronal image, and sagittal image according to an embodiment of the present invention, and FIG. 4 is LV and LA according to an embodiment of the present invention It is a diagram showing the configuration of a neural network for detecting a junction region of It is a diagram showing an image analysis method for classifying the boundaries of .

The image analysis apparatus 100 for classifying the boundaries of cardiac chamber images according to an embodiment of the present invention includes an input unit 110 , a preprocessor 120 , a feature extraction unit 130 , a control unit 140 , and a machine learning unit 150 . ), a storage unit 160 , a memory unit 170 , and an image reconstruction unit 180 .

As shown in FIG. 2 , the input unit 110 receives an axial image of cardiac MRI (FIG. 2(a)) and a coronal image of cardiac MRI from a 3D cardiac magnetic resonance imaging device. A 3D cardiac MRI datasheet including a (Sagittal Image) ((b) of FIG. 2) and a sagittal image of a cardiac MRI ((c) of FIG. 2) may be received (S100).

In other words, the input unit 110 reads the DICOM file from the 3D cardiac magnetic resonance imaging apparatus using the image processing library and receives the cardiac chamber image. Here, the cardiac chamber image represents a left ventricle image and a left atrium image.

DICOM (Digital Imaging and Communications in Medicine) is the most commonly used standard for receiving medical scans, in which DICOM files contain multiple "segments" representing distinct scan layers, i.e. multiple representations of an axial image of a cardiac chamber image. a slice image of , a plurality of slice images indicating a coronal image of the cardiac chamber image, and a plurality of slice images indicating a sagittal image of the cardiac chamber image.

The pre-processing unit 120 generates each of a plurality of slice images representing an axial image of the heart chamber image, a plurality of slice images representing a coronal image of the heart chamber image, and a plurality of slice images representing a sagittal image of the heart chamber image, respectively. Pre-processing is performed for (S110).

The preprocessor 120 generates a binary image by removing noise from the bone and muscle tissue, and preprocessing the slice image using a threshold value indicating the maximum brightness level. Here, the threshold sets the maximum brightness level to the 85% limit. This maximum brightness level of 85% is a numerical value obtained through experiments on various data sets.

As shown in FIG. 3 , in the axial plane image of the heart chamber image, the coronal plane image of the heart chamber image, and the sagittal plane image of the heart chamber image, the sizes of the left ventricle (Left Ventricle) and the left atrium (Left Atrium) are larger than other regions in the heart region. It can be seen that is much larger. The left ventricle and left atrium are oval in shape.

The junction of the left ventricle and left atrium is shown in red, and the blue oval indicates that the shape of the chamber is similar.

Taken together, these are parameters related to the ellipse, such as Eccentricity, Major Axis, and Object Area (amount of pixels).

Simulations using these features show that the junction region of the left atrium and left ventricle can be identified with acceptable accuracy.

However, as shown in Table 1 below, when two features (minor axis and object position) are added to the simulation result, it can be seen that the accuracy of image segmentation is further improved.

These results show that using the five features, the algorithm becomes more robust against noise caused by bones and small blood vessels.

The preprocessor 120 removes noise including bones, small blood vessels, muscle tissue, etc. using five features from each slice image received from the input unit 110, and then preprocesses the maximum brightness level of 85% to generate a binary image. create

The feature extraction unit 130 extracts five features from each slice image of all objects by using a concatenation algorithm (S120). Here, the concatenation algorithm is based on the concatenated component algorithm in the binary image.

The concatenation algorithm performs detection on the binary image generated by the preprocessor 120 . The linked component algorithm provides a list of pixels of individual objects in the slice image.

When the pixel list is used, the feature extractor 130 finds all five features of an ellipse (eccentricity, major axis, object region, minor axis, and object position) for each object in all sliced images.

Here, all slice images include a plurality of slice images representing an axial image of the heart chamber image, a plurality of slice images representing a coronal image of the heart chamber image, and a plurality of slice images representing a sagittal image of the heart chamber image. do.

These five features are used as input data to the neural network.

For example, for an individual object region, add all binary pixels of an object.

The control unit 140 transmits five features as input data of the machine learning unit 150 .

The machine learning unit 150 uses a prediction deep neural network that receives input data from an input layer and outputs a predicted value to a buffer of an output layer, and the structure or shape of the prediction deep neural network is not limited, and a representative method is a deep neural network (DNN). ), Convolutional Neural Network (CNN), and Recurrent Neural Network (RNN).

The machine learning unit 150 is a technique for finding the junction regions of various left atrium and left ventricle using five feature vectors using a pattern recognition technique. do.

The pattern recognition technique is a prediction method using an artificial neural network, and when the result value of the output layer is predicted from the input layer, the input value can be predicted from the result values in the learning process. Since the artificial neural network does not have a one-to-one correspondence between the input value and the output value, it is impossible to restore the input layer as an output layer as it is. If it is different from the input data of , it can be considered that the prediction of the artificial neural network is inaccurate.

A deep neural network refers to a neural network composed of several layers among neural network algorithms. A layer consists of several nodes, and actual calculations are performed at the nodes, and this calculation process is designed to mimic the processes occurring in the neurons constituting the human neural network. A typical artificial neural network is divided into an input layer, a hidden layer, and an output layer. becomes the input of the output layer, and the output of the output layer becomes the final output.

The machine learning unit 150 uses a prediction deep neural network that receives input data from an input layer and outputs a predicted value to a buffer of an output layer, and the structure or shape of the prediction deep neural network is not limited, and a representative method is a deep neural network (DNN). ), Convolutional Neural Network (CNN), Recurrent Neural Network (RNN), etc., and by combining each neural network, a predictive deep neural network can be constructed to construct a deep neural network of various structures.

The machine learning unit 150 may store the junction detection model of the left atrium and the left ventricle in the storage unit 160 to create a predictive model and use it as a learning data set for machine learning.

The machine learning process extracts feature vectors from a lot of data stored in the past, creates a training data set based on the extracted feature vectors, and creates a predictive model based on the machine learning algorithm. Learning data is a set of data used to extract desired information in machine learning.

As shown in FIG. 4 , the machine learning unit 150 loads the junction detection model of the left atrium and the left ventricle stored in the storage unit 160 into the memory unit 170 , and uses 5 features as input data, and 3 slices. The left atrium and the left ventricle are found in the image, and the junction region between the left atrium and the left ventricle is machine-learned (S130).

The detection of the junction region between the left atrium and left ventricle is based on an artificial neural network and uses a backpropagation algorithm for training purposes.

The image analysis apparatus 100 receives three slice images that are 3D data through the input unit 110 , and extracts 5 features from each slice image through the feature extractor 130 , and the extracted 5 features It is fed into a trained artificial neural network.

The machine learning unit 150 performs a plurality of slice images representing an axial image of the heart chamber image, a plurality of slice images representing a coronal image of the heart chamber image, and a plurality of slice images representing a sagittal image of the heart chamber image. Learn to identify slice images in which junction regions of the left atrium and left ventricle exist for each.

The machine learning unit 150 may achieve maximum performance for each layer having 5 neurons and 5 hidden layers (Table 2 below)

The feature extractor 130 may include a plurality of slice images representing an axial image of the heart chamber image received through the input unit 110 , a plurality of slice images representing a coronal image of the heart chamber image, and a sagittal plane of the heart chamber image. Five features of an object are extracted using a component algorithm connected to each of a plurality of slice images representing an image.

The controller 140 transmits five features extracted from each slice image as input data of the machine learning unit 150 .

The machine learning unit 150 generates a left atrium and left ventricle junction detection model that determines whether there is a junction region between the left ventricle and the left atrium of an object existing in each slice image using the plurality of extracted features.

The control unit 140 controls the five features to output 1 (the object has a junction region between the left atrium and the left ventricle) or 0 (the object does not have a junction region between the left atrium and left ventricle) by the left atrium and left ventricle junction detection model. .

5 , the controller 140 loads a subsequent slice image downward based on the junction region of the left atrium and the left ventricle in each slice image, and loads a preceding slice image upward based on the junction region in the left atrium. and the left ventricle are extracted (S140).

When there is a junction region between the left atrium and the left ventricle in each slice image, the controller 140 detects the first slice image of the left ventricle by loading a subsequent slice image below the junction region.

The controller 140 pre-processes the slice image, and various objects are detected using a connected component algorithm.

The controller 140 extracts an object having a minimum Euclidean distance from the center of a junction region between the left atrium and the left ventricle into the left ventricle, in which the object having the maximum size in the sensed object is the heart chamber image. Thereafter, the image reconstruction unit 180 loads the next slice image, and similar processes of preprocessing and object detection are performed, except that the minimum Euclidean distance from the previously searched left ventricle is calculated.

This process establishes a minimum threshold for detecting the end of the left ventricle (LV).

When there is no object having an area larger than a preset minimum threshold value in the slice image, the controller 140 determines that it is the end of the left ventricle. This is because the size of the left ventricle decreases as it goes down in the image of the heart chamber.

The controller 140 detects the first left atrial slice image by loading the preceding slice image above the junction region.

The controller 140 pre-processes the slice image, and various objects are detected using a connected component algorithm.

The controller 140 extracts an object having a minimum Euclidean distance from the center of a junction region between the left atrium and the left ventricle as an image of the heart chamber as an object having the maximum size in the sensed object as the left atrium. Thereafter, the image reconstruction unit 180 loads the next slice image, and similar processes of preprocessing and object detection are performed, except that the minimum Euclidean distance from the previously searched left atrium is calculated.

This process establishes a minimum threshold for detecting the end of the left atrium (LA).

The image reconstruction unit 180 stacks all the segmented individual slice images (segmented left ventricle and left atrium) on top of each other using a connectivity algorithm (S150), and generates a segmented 3D shape (left ventricle and left atrium) do (S160).

As shown in FIG. 8 , a 3D model of two chambers (left ventricle image and left atrium image) is generated by stacking all the divided slice images.

The experimental setup consists of a data set with a total of 21400 3D cardiac MRI images, the data sets are in DICOM format, the axial image of cardiac MRI, the sagittal image of cardiac MRI, and sagittal of cardiac MRI. Includes all coronal images.

In addition, the data set consists of high-resolution sliced images, and the details are shown in Table 3 below.

7 is a diagram illustrating a result of segmentation of axial slice images.

Fig. 7 (c) is a view showing the divided junction region between the left ventricle and the left atrium, Fig. 7 (a) is the divided left atrium (the 220th slice), and Fig. 7 (b) is the divided left atrium (the 270th slice) ), and (d) of FIG. 7 is a divided left ventricle (405th slice), and FIG. 7(e) is a divided left ventricle (490th slice).

Non-chamber mass (representing non-chamber tissue such as arteries) is measured in pixels.

The percentage of non-cardiac chamber mass is regarded as a segmentation error (SE) as in Equation 1 below.

Here, Total Mass represents the total mass measured in pixels in the slice image, and NonChamber Mass represents the mass, not the cardiac chamber, measured in pixels in the slice image.

As a performance evaluation matrix, the mean segmentation error (MSE) of Equation 3 below is used.

The Percent Inverse of Segmentation Error represents the accuracy (SA). SA is calculated by Equation 2 below.

Mean segmentation accuracy (MSA) represents the average of all accuracies of three slice images (axial, coronal, and sagittal) (Equation 3).

Here, SA _Axial is the accuracy in the axial image, SA _Coronal is the accuracy in the coronal image, and SA _Sagittal is the accuracy in the sagittal image.

The effect of non-chamber mass on individual slice images is shown in FIG. 9 .

9 shows segmentation accuracy in a three-slice image. The achievement accuracy of the axial slice is 94.4%, the achievement accuracy of the coronal slice is 91.71%, and the achievement accuracy of the sagittal slice is 88.61%. The overall average segmentation accuracy is 91.57%.

In the above, the embodiment of the present invention is not implemented only through an apparatus and/or method, and may be implemented through a program for realizing a function corresponding to the configuration of the embodiment of the present invention, a recording medium in which the program is recorded, etc. And, such an implementation can be easily implemented by an expert in the technical field to which the present invention belongs from the description of the above-described embodiment.

Although the embodiments of the present invention have been described in detail above, the scope of the present invention is not limited thereto, and various modifications and improved forms of the present invention are also provided by those skilled in the art using the basic concept of the present invention as defined in the following claims. is within the scope of the right.

100: image analysis device
110: input unit
120: preprocessor
130: feature extraction unit
140: control unit
150: machine learning unit
160: storage
170: memory unit
180: image reconstruction unit

Claims (9)

an input unit for receiving a cardiac MRI data sheet including a plurality of slice images representing images of the heart chambers of the left ventricle and the left atrium from a 3D cardiac magnetic resonance imaging apparatus;
a pre-processing unit for pre-processing each slice image by removing noise;
a feature extracting unit for extracting a plurality of features from each slice image of all objects using an algorithm linking the pre-processed slide image;
a machine learning unit generating a left atrium and left ventricle junction detection model that determines whether there is a junction region between the left ventricle and the left atrium of an object existing in each of the slice images by using the plurality of extracted features; and
A plurality of features extracted from each slice image are transmitted as input data of the machine learning unit, and the plurality of features are controlled to output whether there is a junction region between the left atrium and the left atrium by the junction detection model of the left atrium and left ventricle. An image analysis apparatus comprising a control unit to. According to claim 1,
The cardiac chamber image is an image analysis apparatus, characterized in that it comprises a plurality of slice images of an axial image, a plurality of slice images of a coronal image, and a plurality of slice images of a sagittal image. According to claim 1,
The image analysis apparatus according to claim 1, wherein the pre-processing unit pre-processes each slice image using a threshold value that limits the maximum brightness level to 85% to generate a binary image. According to claim 1,
The plurality of features is an image analysis apparatus, characterized in that it includes eccentricity, a major axis, an object area (amount of pixels), a minor axis, and an object position, which are parameters related to the ellipse. According to claim 1,
The feature extraction unit uses an algorithm to connect the preprocessed slide image to eccentricity, major axis, object area (amount of pixels), minor axis, and object in each slice image for all objects. Image analysis apparatus, characterized in that extracting five features of the location. According to claim 1,
The control unit detects the junction region between the left atrium and the left ventricle in each of the slice images, loads a subsequent slice image downward based on the junction region, and loads a preceding slice image upward based on the junction region, so that the left atrium and the left ventricle An image analysis apparatus for extracting the left ventricle. Receiving a cardiac MRI data sheet including a plurality of slice images of an axial image, a plurality of slice images of a coronal image, and a plurality of slice images of a sagittal image from a 3D cardiac magnetic resonance imaging apparatus input unit;
a preprocessor for preprocessing each slice image using a threshold value that sets a maximum brightness level to a limit of 85% to generate a binary image;
By using the preprocessed slide image connection algorithm, all objects in each slice image are elliptic-related parameters Eccentricity, Major Axis, Object Area (Amount of Pixels), Minor Axis, Object a feature extracting unit for extracting a feature of a location; and
and a machine learning unit for generating a junction detection model of the left atrium and the left ventricle that determines whether there is a junction region between the left ventricle and the left atrium of the object present in each of the sliced images by using the five extracted features. Receiving a cardiac MRI data sheet including a plurality of slice images of an axial image, a plurality of slice images of a coronal image, and a plurality of slice images of a sagittal image from a 3D cardiac magnetic resonance imaging apparatus step;
generating a binary image by pre-processing each slice image using a threshold value that sets a maximum brightness level to a limit of 85%;
By using the preprocessed slide image connection algorithm, all objects in each slice image are elliptic-related parameters Eccentricity, Major Axis, Object Area (Amount of Pixels), Minor Axis, Object extracting a feature of the location; and
A junction detection model of the left atrium and left ventricle that determines whether there is a junction region between the left ventricle and the left atrium of the object present in each slice image is generated using the extracted five features, and the left atrium and the left atrium using the five features as input data and outputting whether there is a junction region between the left ventricle and the left atrium by the junction detection model of the left ventricle. 9. The method of claim 8,
The left atrium and left ventricle are extracted by detecting the junction region of the left atrium and the left ventricle in each of the slice images, loading the subsequent slice image downward based on the junction region, and loading the preceding slice image upward relative to the junction region. to do; and
and generating a 3D model of a left ventricle image and a left atrium image by stacking the divided slice images on each other.

Images