TWI787980B - Method and system for automatically interpreting cardiac function through deep learning models - Google Patents

Abstract

The present invention discloses a method and a system for automatically interpreting cardiac function through deep learning models. In the method, an area of a heart to be marked can be calculated, and an outer contour of the heart can be obtained. The outer contour is divided into multiple segments. A maximum area is calculated from a diastolic heart, and a minimum area is calculated from a systolic heart from the continuous cardiac images. Accordingly, a union-set image and an intersection-set image of the maximum area and the minimum area can be obtained. Afterwards, the union-set image is used to subtract the intersection-set image, and the subtraction is referred to for automatically interpreting abnormal contractions in multiple areas of the heart while introducing an abnormal interpreting model.

Description

Method and system for automatically interpreting cardiac function through deep learning model

The present invention relates to a method for interpreting cardiac function, in particular to a method and system for automatically interpreting abnormal cardiac local contraction through a deep learning model.

Generally speaking, when training a deep learning model, a professional data set is required to train the model. The collection of professional data sets for medical use is often the most difficult part of the entire R&D process. Because most of the medical professional data sets are marked by professional medical practitioners or even specialist physicians. Therefore, the cost of completing a professional medical data set is often very large. In the past, the clinical detection of abnormal cardiac systole was done manually with the naked eye, which required a high threshold of experience and was time-consuming and labor-intensive. Left ventricular photography is a technique widely performed clinically to judge cardiac ejection rate, cardiac hypertrophy, and local systolic abnormalities. Because of its long history, it is also defined as the gold standard (gold standard) for evaluating heart disease.

For example, coronary artery disease (CAD) is the deadliest and most urgent of cardiovascular diseases. In recent years, with the popularization of cardiac catheterization, the survival rate of patients can be greatly improved, Knowing the infarct related artery (IRA) as early as possible is extremely helpful for the preoperative preparation of the subsequent cardiac catheterization and the evaluation of the strategy of opening the blood vessel.

However, the clinical detection of cardiac systole abnormalities is still only judged by the naked eye. In addition to requiring a high threshold of experience, the judge is often time-consuming and labor-intensive, but there is no automatic interpretation system. In addition, it is not easy to train new recruits, and there is often the possibility of human error in interpretation. In recent years, deep learning has been approaching maturity in image recognition technology. Coupled with the rapid release of CPU and GPU hardware, the real-time automatic image interpretation system has been quite easy to implement. In other fields (such as: face recognition, The practical application of self-driving vehicles, etc.) has been quite extensive. But in the field of smart medical care, it is just about to emerge. The main reason why the development of artificial intelligence in the smart medical field is slower than other fields is that it is more difficult to obtain training data sets than other fields. Because professional model training sets used in the medical field often require professionals with medical backgrounds to mark them, the cost of obtaining professional training data sets is quite high.

In view of the known lack of an automatic interpretation system for heart diseases, the present invention proposes a method and system for automatic interpretation of cardiac function through a deep learning model. The method is especially suitable for regional wall motion abnormality (RWMA) Carry out automatic interpretation, in which the deep learning method is used to learn the abnormal data of the local systole of the heart, and the abnormal interpretation model is established to replace the manual judgment.

According to an embodiment, the system for implementing the method for automatically interpreting cardiac function through a deep learning model proposes a computer system, which acquires continuous cardiac images through an image capture unit, obtains changes in the area of the heart during diastole and systole, and evaluates the image features by A deep learning algorithm establishes an abnormal interpretation model for automatically interpreting the heart function.

In the method of system operation, the area of the marked heart is calculated, the peripheral contour of the heart is obtained, and the peripheral contour of the heart is divided into multiple regions, and then the maximum area of the heart in diastole and the maximum area of the heart in systole are calculated from the continuous cardiac images. The minimum area, and then obtain the maximum area and the minimum area of the joint graphics, intersection graphics, and the results of the union graphics minus the intersection graphics, introduce the abnormal interpretation model to classify and automatically interpret the abnormal contraction of multiple areas of the heart.

Preferably, the color difference between the heart and other body tissues can be generated on the continuous heart image by a contrast agent input into the human body, and then the image is marked, and the maximum area and contraction of the heart in diastole can be obtained and calculated by using the color difference in the image The minimum area when .

Furthermore, a deep learning image semantic segmentation algorithm can be used to distinguish the peripheral contour of the heart, and obtain the maximum diastolic area and the minimum systolic area of the heart.

Further, in the multiple regions of the peripheral contour of the heart, the thinner part of the graph obtained by subtracting the intersection graph from the union graph is determined to be a region with abnormal cardiac local contraction, while the thicker part is judged to be a region with normal cardiac contraction . According to the regional location of systolic abnormality, it was classified as a left anterior descending terminal lesion, a right coronary artery or left circumflex artery local abnormality, and a left anterior descending base lesion.

In order to fully understand the purpose, features and effects of the present invention, the present invention will be described in detail through the following specific embodiments and accompanying drawings, as follows:

In the present disclosure, "a" or "an" is used to describe the elements, elements and components described herein. This is done for convenience of description only and to provide a general sense of the scope of the invention. Accordingly, unless it is obvious that it is otherwise indicated, such description should be read to include one, at least one, and the singular also includes the plural.

As used herein, the terms "comprises", "including", "has", "containing" or any other similar terms are intended to cover a non-exclusive inclusion. For example, an element, structure, article, or device that contains a plurality of elements is not limited to those elements listed herein, but may include elements that are not explicitly listed but are generally inherent in the element, structure, article, or apparatus. other requirements. In addition, unless expressly stated to the contrary, the word "or" means an inclusive "or" and not an exclusive "or".

Clinical detection of cardiac systole abnormalities, including regional wall motion abnormality (RWMA), generally relies on doctors' professional judgment with the naked eye, and lacks an automatic interpretation system. In view of this, the present invention proposes a deep learning model The method for automatic interpretation of cardiac function and the system for realizing this method, through the deep learning of abnormal heart data to establish an abnormal interpretation model, can automatically interpret the abnormal features of the obtained cardiac images, one of the purposes is to allow clinicians to obtain real-time The abnormal contraction of the heart (such as the left ventricle) obtained by photography is used as a reference for treatment.

The regional wall motion abnormality (RWMA) of the heart is judged according to the lack of contraction of the local myocardium during the diastolic and systolic periods during the left ventricle angiography. Under normal circumstances, the left ventricle dilates to return blood during diastole, and shrinks at the end of systole to eject blood from the aorta. However, if the coronary arteries (such as left anterior descending (LAD), left circumflex, LCA) and right coronary artery (right coronary artery, RCA)) are blocked, which will cause local hypoxia of the myocardium, and the contraction will become worse. In general, the systolic state of the heart can be divided into normal (normokinesis), poor systole (hypokinesis), no systole (akinesis) and reverse systole (dyskinesis). Therefore, abnormal cardiac local contraction judged by ventriculography is extremely helpful for inferring ischemic heart disease with coronary artery obstruction.

Firstly, Fig. 1 shows a diagram of an embodiment of the system architecture for realizing the method. In the figure, a computer system 10 is shown, and a method for automatically interpreting cardiac function through a deep learning model is realized through a processor, a memory, and related software and hardware. The computer system 10 Obtain images of patients 110 through the image capture unit 12, especially heart images, including dynamic or static images, store them in the storage unit 101 in the computer system 10, and perform calculations through the computing unit 103, including image acquisition, feature extraction, and storage. , and after image processing, the necessary information for interpreting cardiac function, such as the area change during diastole and systole, is obtained. A deep learning unit 10 is provided in the computer system 10, and the purpose of deep learning is realized by means of software, especially for deep learning of the image features stored in the storage unit 101. Taking heart images as an example, the formation of area changes during diastole and systole of the heart is studied. Based on the abnormal features of the heart, an abnormal interpretation model can be established, and this intelligent method can replace the traditional way of manually interpreting abnormal cardiac function. The relevant interpretation results and images can be output by the output unit 107 .

According to the embodiment of the method for automatically interpreting cardiac function through the deep learning model proposed by the invention, the computer system 10 runs the deep learning method, and the flow chart of the embodiment shown in FIG. 2 can be referred to.

The flow shown can assist medical personnel to judge whether the function of a specific part of the heart is abnormal. The method adopted is to use a deep learning model to automatically judge whether the local contraction of the heart is abnormal. The deep learning model can be a convolutional neural network (convolutional neural network, CNN) executes the abnormal interpretation model established by deep learning.

At the beginning, as in step S201, continuous heart images are acquired by an image capture device. For example, when performing left ventricular cardiac catheterization, as in step S203, the color difference between the continuous heart image and other body tissues can be generated on the image by injecting the contrast agent of the human body, and image marking can be performed through image processing methods. For example, inject iodine-containing contrast agent into the coronary arteries and perform coronary angiography on X-rays to obtain the color difference between the heart and other body tissues. You can refer to the example of the peripheral contour mark of the heart part shown in Figure 3 and the example shown in Figure 4. Schematic diagram of images showing peripheral contour markers (with contrast agent color difference) of the heart site, so that the area of the marked heart or a specific part can be calculated, that is, the maximum diastolic area of the heart can be obtained and calculated using the color difference in consecutive heart images ( Step S205) and the minimum area during systole (step S207).

After the above processing, the image features of the heart in operation can be obtained. At this time, the method continues to use the image semantic segmentation algorithm of deep learning. The image segmentation in the process is aimed at the interesting parts of the continuous images. pixels for detection and classification. In this example, the proposed image semantic segmentation algorithm can correctly distinguish the boundary pixels in each region of interest (such as the heart or a specific part), that is, the maximum area graph in diastole and the minimum area graph in systole Union set (union set), as in step S209, generating a union graph obtained by overlapping two images; and, as in step S211, obtaining an intersection graph of the maximum area graph during diastole and the minimum area graph during systole. Then use the image processing method to subtract the intersection graphics from the union graphics (step S213). According to the deducted results, refer to Figure 7(a)(b)(c)(d) to introduce the abnormal interpretation model classification graphics (step S215), The abnormality of cardiac systole can be judged (step S217).

For the method of discriminating abnormal cardiac systole, please refer to the flow chart of an embodiment of the classification of cardiac systolic states shown in FIG. For cardiac images, the labeled data provide training for the abnormal interpretation model, in which the running deep learning algorithm is trained based on these labeled data.

In this example, you can refer to the schematic diagram of an embodiment of the heart contour segmentation area shown in FIG. 6, and divide the part of the peripheral contour of the heart into multiple areas (step S503), such as the five areas shown in FIG. 6, which are marked as area number 1 To 5, obtain continuous images according to these areas, as in step S505, the systolic status of multiple areas can be marked, and then as in step S507, mark and classify the systolic status of these 5 areas, refer to Figure 7 ( a)(b)(c)(d) shows a schematic diagram of an example of cardiac abnormality. In this example, the location of abnormal cardiac systole is determined based on the area change of the union or intersection of images during diastole and systole, which can be classified as Apical Anterior (refers to Left anterior descending terminal damage), Basal (referring to right coronary artery or left circumflex artery local abnormality) and Septal (referring to left anterior descending base damage), or Normal (normal).

Discrimination of abnormal cardiac local systole in the method is based on the feature of the graphics obtained by subtracting the intersection graphics from the union graphics, wherein the result of the area difference between the union and intersection of the area of the heart diastole and systole is shown, and can refer to Figure 7 (a )(b)(c)(d), the graph of the abnormal heart part shows that the area is relatively small, that is, the thinner part in the graph obtained by deducting the intersection graph from the union graph, and it is determined as a partial heart The area with abnormal contraction, and the thicker part is judged as the area with normal contraction of the heart.

Referring to Figure 7(a), compare Figure 6 and the result obtained by subtracting the intersection graph from the union graph in the above method, when the local contraction of the area numbers 2, 3 and 4, or the area numbers 2 and 3, or the area number 3 is abnormal , classified as Apical Anierior, what is to be expressed is left anterior descending distal lesion (LAD (left anterior descending) distal lesion).

Referring to Figure 7(b), compared with Figure 6, when the local contraction of area numbers 1 and 2 is abnormal, it is classified as Septal, which means the damage of the left anterior descending base (LAD proximal).

Referring to Figure 7(c), compared with Figure 6, when the area numbers 3, 4 and 5, or area numbers 4 and 5, or area number 5 have local systolic abnormalities, it is classified as Basal, meaning the right coronary artery (right coronary artery) artery, RCA), or left circumflex artery (Left circumflex Artery, LCX) local abnormalities.

Referring to FIG. 7( d ), compared with FIG. 6 , the image obtained by subtracting the intersection graph from the union graph, if there is no relatively thin area, is judged to be normal.

Furthermore, the method for automatically interpreting cardiac function through a deep learning model proposed by the invention uses a deep learning algorithm to train an abnormal interpretation model using a large number of acquired cardiac images. The flow chart of the embodiment can be referred to in FIG. 8 .

In the process of this model training, a large number of input cardiac image files, preferably continuous image files (step S801), are first obtained. In this embodiment, a dilated cardiomyopathy (DCM) continuous image file is input as an example. , the deep learning model performs semantic segmentation and recognition for each pixel of each frame of medical images. In this example, the maximum diastolic area and the minimum systolic area are obtained for all consecutive images in the dilated cardiomyopathy file (step S803), so as to obtain the union graph of the maximum diastolic area and the minimum systolic area (step S805) and the maximum diastolic area The intersection graph with the contracted minimum area (step S807 ), and then subtract the intersection graph with the union graph (step S809 ), and finally use the subtracted graph result for model training and prediction. In the process, after using the graphic marking tool to mark the peripheral contour of the heart part with the polygon marking method, the marked image can be trained by the deep learning algorithm, and the graphic classifier in the deep learning algorithm (such as an EfficientNet classifier Model) to classify and identify various local abnormalities of the heart, that is, to distinguish blocked blood vessels, and use the above data and judgment results to train and form an abnormal interpretation model (step S811), which can automatically interpret local abnormalities of the heart according to the input image files ( Step S813).

In summary, according to the method and system for automatically interpreting cardiac function through the deep learning model described in the above embodiments, the system establishes an abnormal interpretation model through deep learning to automatically detect the maximum diastole of the heart (such as the left ventricle) in photography. The area and the smallest area of contraction are used to establish an early warning platform for heart failure and structural abnormalities. The deep learning method used in the method uses a deep learning image classifier algorithm to establish an abnormal interpretation model and perform images of local abnormal cardiac contractions Identification is used to replace clinical manual judgment and achieve the purpose of automatic interpretation, which can effectively reduce clinical interpretation errors, and quickly provide information on cardiac function and local systolic abnormalities, allowing clinicians to have a basis for judgment on subsequent treatment, for example, Help detect myocardial hypoxia and help doctors decide whether to perform coronary artery surgery. In addition, an instant early warning function can be achieved, which can be fed back to clinicians in real time to make the most correct treatment decision.

The present invention has been disclosed above with preferred embodiments, but those skilled in the art should understand that the embodiments are only used to describe the present invention, and should not be construed as limiting the scope of the present invention. It should be noted that all changes and substitutions equivalent to the embodiment should be included in the scope of the present invention. Therefore, the scope of protection of the present invention should be defined by the scope of the patent application.

110: Patient 12: Image capture unit 10:Computer system 101: storage unit 103: Calculation unit 10: Deep Learning Unit 107: output unit 1, 2, 3, 4, 5: area number Steps S201-S217: the process of automatically interpreting cardiac function through the deep learning model Steps S501-S507: the process of classifying the systolic state Steps S801-S813: the process of training the abnormal interpretation model

Fig. 1 shows the embodiment diagram of the system architecture of the method for automatically interpreting the heart function through the deep learning model;

Fig. 2 shows the flow chart of a method embodiment for automatically interpreting cardiac function by a deep learning model;

Fig. 3 shows an example diagram of an image of a peripheral contour marker of a heart region;

Figure 4 shows a schematic diagram of the image of the peripheral contour mark (with developer color difference) of the heart site;

Figure 5 shows an embodiment flow diagram for classifying the systolic state;

Figure 6 shows a schematic diagram of an embodiment of a heart contour segmentation region;

Figure 7(a)(b)(c)(d) shows a schematic diagram of an example of a cardiac abnormality type; and

FIG. 8 shows a flowchart of an embodiment of training an anomaly interpretation model.

S201-S217: Method

Claims (8)

A method for automatically interpreting cardiac function through a deep learning model, comprising: obtaining continuous heart images; calculating the area of a marked heart, obtaining the peripheral contour of the heart, and dividing the peripheral contour of the heart into multiple regions; Calculating a maximum area when the heart is diastolic and a minimum area when the heart is systolic in the heart image; obtaining a joint graph of the maximum area and the minimum area and an intersection graph; and subtracting the The intersection graph, according to the result of the subtraction, classifies and automatically interprets the abnormal contractions of the regions of the heart with an abnormal interpretation model, wherein the plurality of marked peripheral contours of the heart in the continuous cardiac images are trained with a deep learning algorithm The heart abnormality data of the area is used to establish the abnormality interpretation model, and through the abnormality interpretation model, in the multiple areas of the peripheral contour of the heart, the thinner part in the figure obtained by interpreting the joint figure and subtracting the intersection figure is the The region where the local contraction of the heart is abnormal, and the thicker part is interpreted as the region where the heart contraction is normal. The method for automatically interpreting heart function through a deep learning model as described in claim 1, wherein a contrast agent injected into the human body is used to generate a color difference between the heart and other body tissues on the continuous heart image, and then the image is marked. The method for automatically interpreting cardiac function through a deep learning model as described in claim 2, wherein the maximum area in diastole and the minimum area in systole are obtained and calculated by using the color difference in the image. The method for automatically interpreting cardiac function through a deep learning model as described in Claim 1, wherein a deep learning image semantic segmentation algorithm is used to distinguish the peripheral contour of the heart, and obtain the maximum area of the heart in diastole and the maximum area of the heart during systole Minimum area. The method for automatically interpreting cardiac function through a deep learning model as described in Claim 1, wherein, according to the location of the abnormal cardiac systole, it is classified into a left anterior descending terminal damage, a right coronary artery or a left circumflex artery local abnormality, and a left anterior descending Basal damage. A system for automatically interpreting cardiac function through a deep learning model, including: a computer system that obtains continuous cardiac images through an image capture unit, obtains changes in the area of the heart during diastole and systole, and performs a deep learning algorithm on the image features A method for establishing an abnormal interpretation model for automatically interpreting the heart function, wherein the computer system runs a method for automatically interpreting the heart function through a deep learning model, including: calculating the marked area of the heart, obtaining the peripheral contour of the heart, and dividing the peripheral contour of the heart into a plurality of regions; calculating a maximum area of the heart diastole and a minimum area of the heart systole from the continuous heart image; obtaining a union of the maximum area and the minimum area graph, and an intersection graph; and subtracting the intersection graph from the union graph, and according to the deducted result, using the abnormal interpretation model to classify and automatically judge the abnormal contractions of the regions of the heart, wherein a deep learning algorithm is used to train The cardiac abnormality data of the plurality of regions of the peripheral contour of the heart in the continuous cardiac image are marked to establish the abnormality interpretation model, and the joint graph is interpreted in the plurality of regions of the cardiac peripheral contour through the abnormality interpretation model The thinner part of the graph obtained by subtracting the intersection graph is the area of abnormal local systole of the heart, and the thicker part is the area of normal systole of the heart. The system for automatically interpreting cardiac function through a deep learning model as described in Claim 6, wherein a deep learning image semantic segmentation algorithm is used to distinguish the peripheral contour of the heart, and obtain the maximum area of the heart in diastole and the maximum area in systole Minimum area. The system for automatically interpreting cardiac function through a deep learning model as described in claim 6 or 7, wherein, according to the location of the abnormal cardiac systole, it is classified into a left anterior descending terminal lesion, a local abnormality of the right coronary artery or left circumflex artery, and a Left anterior descending base lesion.

Images