TWI792055B - Establishing method of echocardiography judging model with 3d deep learning, echocardiography judging system with 3d deep learning and method thereof - Google Patents
Establishing method of echocardiography judging model with 3d deep learning, echocardiography judging system with 3d deep learning and method thereof Download PDFInfo
- Publication number
- TWI792055B TWI792055B TW109133402A TW109133402A TWI792055B TW I792055 B TWI792055 B TW I792055B TW 109133402 A TW109133402 A TW 109133402A TW 109133402 A TW109133402 A TW 109133402A TW I792055 B TWI792055 B TW I792055B
- Authority
- TW
- Taiwan
- Prior art keywords
- ultrasound
- cardiac
- model
- cardiac ultrasound
- judgment
- Prior art date
Links
Images
Landscapes
- Ultra Sonic Daignosis Equipment (AREA)
Abstract
Description
本發明是關於一種醫療資訊判斷模型之建立方法、系統以及方法,特別是關於一種利用三維深度學習之心臟超音波判斷模型之建立方法、利用三維深度學習之心臟超音波判斷系統及其方法。The present invention relates to a method, system and method for establishing a medical information judgment model, in particular to a method for establishing a cardiac ultrasound judgment model using three-dimensional deep learning, a cardiac ultrasound judgment system and its method using three-dimensional deep learning.
在心血管疾病中,急性冠心症(Acute Coronary Syndrome,ACS)最為致命且危急。近幾年隨著心導管的普及,患者的生存率大幅提升。若能盡早了解梗塞相關血管(Infarct Related Artery,IRA)對於心導管的術前準備和評估打通血管的策略極有幫助。Among cardiovascular diseases, acute coronary syndrome (ACS) is the most deadly and critical. In recent years, with the popularity of cardiac catheterization, the survival rate of patients has increased significantly. Early understanding of infarct-related arteries (Infarct Related Artery, IRA) is extremely helpful for preoperative preparation of cardiac catheterization and evaluation of strategies to open blood vessels.
然而,目前僅以ST上升型心肌梗塞(STEMI)的患者在急性期中可透過心電圖判斷梗塞相關血管,而非ST上升型心肌梗塞(NSTEMI)與不穩定型心絞痛(Unstable angina,UA)的患者則往往需要做了心導管才能評估再灌流計畫。However, at present, only patients with ST-elevating myocardial infarction (STEMI) can judge infarction-related vessels through ECG in the acute phase, while patients with non-ST-elevating myocardial infarction (NSTEMI) and unstable angina (UA) can Cardiac catheterization is often required to assess reperfusion planning.
心臟超音波被廣泛應用在臨床上,但是心臟超音波往往無法判斷區域性室壁運動異常(RWMA),即便有豐富經驗的操作者也不一定能依據患者的心臟超音波而正確判斷是否具有梗塞相關血管。目前雖有亮點追蹤超音波(Speckle Tracking Echocardiography,STE)能在早期偵測肉眼不易辨識的梗塞相關血管,卻因價格較高且較費時而不普遍。Cardiac ultrasound is widely used clinically, but cardiac ultrasound is often unable to judge regional wall motion abnormality (RWMA), even experienced operators may not be able to correctly judge whether there is an infarction based on the patient's cardiac ultrasound associated blood vessels. Although there is a bright spot at present, Speckle Tracking Echocardiography (STE) can detect infarction-related blood vessels that are difficult to identify with the naked eye at an early stage, but it is not popular because of its high price and time-consuming.
由此可知,若能快速地在第一時間利用心臟超音波判斷出梗塞相關血管,實為民眾所殷切企盼,亦係相關業者須努力研發突破之目標及方向。It can be seen from this that if the blood vessels related to infarction can be judged quickly by using cardiac ultrasound in the first time, it is indeed the eager expectation of the public, and it is also the goal and direction that the relevant industry must work hard to develop breakthroughs.
因此,針對非ST上升型心肌梗塞與不穩定型心絞痛,本發明之目的在於提供一種利用三維深度學習之心臟超音波判斷系統,其可將心臟超音波判斷結果比對心導管確診結果,以修正心臟超音波判斷模型,達到提高心臟超音波判斷模型對於判斷梗塞相關血管的準確率。Therefore, for non-ST ascending myocardial infarction and unstable angina, the purpose of the present invention is to provide a cardiac ultrasound judgment system using three-dimensional deep learning, which can compare the cardiac ultrasound judgment results with the cardiac catheter diagnosis results to correct Cardiac ultrasound judgment model to improve the accuracy of cardiac ultrasound judgment model for judging blood vessels related to infarction.
依據本發明的一態樣之一實施方式提供一種利用三維深度學習之心臟超音波判斷模型之建立方法,其包含以下步驟:資料取得步驟、影像前處理步驟、參照特徵選取步驟以及模型訓練步驟。資料取得步驟係取得參照資料,其中參照資料包含複數個參照心臟超音波資料及對應此些參照心臟超音波資料之複數個心導管確診結果。影像前處理步驟係切割各個參照心臟超音波資料,以取得複數個參照心臟超音波影像。參照特徵選取步驟係利用參照特徵選取模組分析各個參照心臟超音波影像後,於各個參照心臟超音波影像中得到參照影像特徵值,以取得對應此些參照心臟超音波影像之複數個參照影像特徵值。模型訓練步驟係依據此些參照影像特徵值訓練而產生心臟超音波判斷模型,並輸入此些參照心臟超音波資料至心臟超音波判斷模型而產生對應此些參照心臟超音波資料之複數心臟超音波判斷結果,然後將此些心臟超音波判斷結果分別比對此些心導管確診結果以修正心臟超音波判斷模型。An embodiment according to an aspect of the present invention provides a method for establishing a cardiac ultrasound judgment model using 3D deep learning, which includes the following steps: data acquisition step, image preprocessing step, reference feature selection step, and model training step. The data obtaining step is obtaining reference data, wherein the reference data includes a plurality of reference echocardiographic data and a plurality of diagnostic results of cardiac catheterization corresponding to the reference echocardiographic data. The image preprocessing step is to cut each reference echocardiogram data to obtain a plurality of reference echocardiogram images. The reference feature selection step is to use the reference feature selection module to analyze each reference cardiac ultrasound image, and obtain reference image feature values in each reference cardiac ultrasound image, so as to obtain a plurality of reference image features corresponding to these reference cardiac ultrasound images value. The model training step is to generate a cardiac ultrasound judgment model based on the reference image feature value training, and input the reference cardiac ultrasound data into the cardiac ultrasound judgment model to generate a plurality of cardiac ultrasound corresponding to the reference cardiac ultrasound data Judgment results, and then compare these heart ultrasound judgment results with the cardiac catheter diagnosis results to modify the heart ultrasound judgment model.
藉此,本發明係透過心臟超音波資料訓練出心臟超音波判斷模型,並依據心導管確診結果修正心臟超音波判斷模型,進而得到最佳化的心臟超音波判斷模型。In this way, the present invention trains the cardiac ultrasound judgment model through the cardiac ultrasound data, and corrects the cardiac ultrasound judgment model according to the diagnosis result of the cardiac catheterization, and then obtains the optimized cardiac ultrasound judgment model.
前述實施方式之其他實施例如下:前述心臟超音波判斷模型包含依序連接的密集卷積神經網路(DenseNet)模型、全連接神經網路(Fully-connected neural network,FCNN)模型及歸一化指數函數(Softmax)模型。Other examples of the aforementioned embodiments are as follows: the aforementioned cardiac ultrasound judgment model includes a sequentially connected dense convolutional neural network (DenseNet) model, a fully-connected neural network (Fully-connected neural network, FCNN) model and a normalized Exponential function (Softmax) model.
前述實施方式之其他實施例如下:前述密集卷積神經網路模型包含複數個密集卷積神經網路模塊(DenseNet Block,DNB)、複數個最大池化模組(Max pooling)及平坦模組(Flatten)。Other examples of the foregoing embodiments are as follows: the aforementioned dense convolutional neural network model includes a plurality of dense convolutional neural network modules (DenseNet Block, DNB), a plurality of maximum pooling modules (Max pooling) and flat modules ( Flatten).
前述實施方式之其他實施例如下:前述各個密集卷積神經網路模塊包含複數個密集卷積神經網路單元,此些密集卷積神經網路單元彼此依序級聯,且各個密集卷積神經網路單元包含三維卷積層(Conv)、線性整流單元訓練層(ReLU)及批標準化層(Batch Normalization,BN)。Other examples of the foregoing embodiments are as follows: each of the aforementioned dense convolutional neural network modules includes a plurality of dense convolutional neural network units, and these dense convolutional neural network units are cascaded to each other in sequence, and each dense convolutional neural network The network unit includes a three-dimensional convolutional layer (Conv), a linear rectification unit training layer (ReLU) and a batch normalization layer (Batch Normalization, BN).
依據本發明的另一態樣之一實施方式提供一種利用三維深度學習之心臟超音波判斷系統,其包含超音波檢測器與處理器。超音波檢測器用以產生受試者之目標心臟超音波影像。處理器電性連接超音波檢測器,處理器儲存一程式,當程式由處理器執行時,程式依據目標心臟超音波影像判斷受試者是否具有梗塞相關血管(Infarct Related Artery,IRA)。所述程式包含參照資料取得模組、參照影像前處理模組、參照特徵選取模組、模型訓練模組及分析模組。參照資料取得模組係用以取得參照資料,參照資料包含複數個參照心臟超音波資料及對應此些參照心臟超音波資料之複數個心導管確診結果。參照影像前處理模組係用以切割各個參照心臟超音波資料,以取得複數個參照心臟超音波影像。參照特徵選取模組係用以分析各個參照心臟超音波影像後,於各個參照心臟超音波影像中得到參照影像特徵值,以取得對應此些參照心臟超音波影像之複數個參照影像特徵值。模型訓練模組係依據此些參照影像特徵值訓練而產生心臟超音波判斷模型,並輸入此些參照心臟超音波資料至心臟超音波判斷模型而產生對應此些參照心臟超音波資料之複數個心臟超音波判斷結果,然後將此些心臟超音波判斷結果分別比對此些心導管確診結果以修正心臟超音波判斷模型。分析模組係依據心臟超音波判斷模型分析目標心臟超音波影像而產生對應梗塞相關血管之目標心臟超音波判斷結果。An embodiment according to another aspect of the present invention provides a cardiac ultrasound diagnosis system using three-dimensional deep learning, which includes an ultrasound detector and a processor. The ultrasonic detector is used to generate the target cardiac ultrasonic image of the subject. The processor is electrically connected to the ultrasound detector, and the processor stores a program. When the program is executed by the processor, the program judges whether the subject has an Infarct Related Artery (IRA) according to the target cardiac ultrasound image. The program includes a reference data acquisition module, a reference image preprocessing module, a reference feature selection module, a model training module and an analysis module. The reference data obtaining module is used to obtain reference data, and the reference data includes a plurality of reference echocardiographic data and a plurality of diagnostic results of cardiac catheterization corresponding to the reference echocardiographic data. The reference image preprocessing module is used for cutting each reference echocardiogram data to obtain a plurality of reference echocardiogram images. The reference feature selection module is used to obtain reference image feature values in each reference echocardiogram image after analyzing each reference echocardiogram image, so as to obtain a plurality of reference image feature values corresponding to the reference echocardiogram images. The model training module is based on the training of these reference image feature values to generate a cardiac ultrasound judgment model, and input these reference cardiac ultrasound data into the cardiac ultrasound judgment model to generate a plurality of hearts corresponding to these reference cardiac ultrasound data Ultrasonic judgment results, and then compare the cardiac ultrasound judgment results with the cardiac catheter diagnosis results to modify the cardiac ultrasound judgment model. The analysis module analyzes the target cardiac ultrasound image according to the cardiac ultrasound judgment model to generate the target cardiac ultrasound judgment result corresponding to the infarction-related blood vessels.
藉此,本發明係透過心臟超音波資料訓練出心臟超音波判斷模型,進而可分析受試者之目標心臟超音波影像,達到準確地判斷受試者是否具有梗塞相關血管。In this way, the present invention trains a cardiac ultrasound judgment model through cardiac ultrasound data, and then analyzes target cardiac ultrasound images of the subject to accurately determine whether the subject has infarct-related blood vessels.
前述實施方式之其他實施例如下:前述心臟超音波判斷模型包含依序連接的密集卷積神經網路模型、全連接神經網路模型及歸一化指數函數模型。Other examples of the aforementioned embodiments are as follows: the aforementioned cardiac ultrasound judgment model includes a sequentially connected dense convolutional neural network model, a fully connected neural network model, and a normalized exponential function model.
前述實施方式之其他實施例如下:前述密集卷積神經網路模型包含複數個密集卷積神經網路模塊、複數個最大池化模組及平坦模組。Other examples of the foregoing embodiments are as follows: the foregoing dense convolutional neural network model includes a plurality of dense convolutional neural network modules, a plurality of maximum pooling modules and a flattening module.
依據本發明的又一態樣之一實施方式提供一種利用三維深度學習之心臟超音波判斷方法,其包含下述步驟。提供前段所述心臟超音波判斷模型。提供受試者之目標心臟超音波影像。依據心臟超音波判斷模型分析目標心臟超音波影像而產生目標心臟超音波判斷結果。利用目標心臟超音波判斷結果判斷受試者是否具有梗塞相關血管。An embodiment according to yet another aspect of the present invention provides a method for judging cardiac ultrasound using 3D deep learning, which includes the following steps. Provide the cardiac ultrasound judgment model mentioned in the preceding paragraph. Provide the subject's target cardiac ultrasound image. The target cardiac ultrasound image is analyzed according to the cardiac ultrasound judgment model to generate a target cardiac ultrasound judgment result. Whether the subject has an infarct-related blood vessel is judged by using the judgment result of the target cardiac ultrasound.
藉此,本發明係透過心臟超音波判斷模型分析受試者之目標心臟超音波影像,進而可準確地判斷受試者是否具有梗塞相關血管。Therefore, the present invention analyzes the subject's target cardiac ultrasound image through the cardiac ultrasound judgment model, and then can accurately determine whether the subject has infarct-related blood vessels.
前述實施方式之其他實施例如下:前述心臟超音波判斷模型包含依序連接的密集卷積神經網路模型、全連接神經網路模型及歸一化指數函數模型。Other examples of the aforementioned embodiments are as follows: the aforementioned cardiac ultrasound judgment model includes a sequentially connected dense convolutional neural network model, a fully connected neural network model, and a normalized exponential function model.
前述實施方式之其他實施例如下:前述密集卷積神經網路模型包含複數個密集卷積神經網路模塊、複數個最大池化模組及平坦模組。Other examples of the foregoing embodiments are as follows: the foregoing dense convolutional neural network model includes a plurality of dense convolutional neural network modules, a plurality of maximum pooling modules and a flattening module.
以下將參照圖式說明本發明之複數個實施例。為明確說明起見,許多實務上的細節將在以下敘述中一併說明。然而,應瞭解到,這些實務上的細節不應用以限制本發明。也就是說,在本發明部分實施例中,這些實務上的細節是非必要的。此外,為簡化圖式起見,一些習知慣用的結構與元件在圖式中將以簡單示意的方式繪示之;並且重複之元件將可能使用相同的編號表示之。Several embodiments of the present invention will be described below with reference to the drawings. For the sake of clarity, many practical details are included in the following narrative. It should be understood, however, that these practical details should not be used to limit the invention. That is, in some embodiments of the present invention, these practical details are unnecessary. In addition, for the sake of simplifying the drawings, some commonly used structures and elements will be shown in a simple and schematic way in the drawings; and repeated elements may be denoted by the same reference numerals.
第1圖係繪示本發明一實施方式之利用三維深度學習之心臟超音波判斷模型之建立方法100的步驟流程圖。如第1圖所示,利用三維深度學習之心臟超音波判斷模型之建立方法100包含步驟110、步驟120、步驟130及步驟140。建立後的心臟超音波判斷模型可用以分析目標心臟超音波影像,以利後續判斷是否具有梗塞相關血管(Infarct Related Artery,IRA)並產生目標心臟超音波判斷結果。FIG. 1 is a flow chart showing the steps of a
步驟110是進行資料取得步驟,資料取得步驟係取得參照資料,參照資料包含複數個參照心臟超音波資料及對應此些參照心臟超音波資料之複數個心導管確診結果。
步驟120是進行影像前處理步驟,影像前處理步驟係切割各個參照心臟超音波資料,以取得複數個參照心臟超音波影像。
步驟130是進行參照特徵選取步驟,參照特徵選取步驟係利用參照特徵選取模組分析各個參照心臟超音波影像後,於各個參照心臟超音波影像中得到參照影像特徵值,以取得對應此些參照心臟超音波影像之複數個參照影像特徵值。
步驟140是進行模型訓練步驟,模型訓練步驟係依據此些參照影像特徵值訓練而產生心臟超音波判斷模型,並輸入此些參照心臟超音波資料至心臟超音波判斷模型而產生對應此些參照心臟超音波資料之複數心臟超音波判斷結果,然後將此些心臟超音波判斷結果分別比對此些心導管確診結果以修正心臟超音波判斷模型。
所述心臟超音波判斷模型可包含依序連接的一密集卷積神經網路(DenseNet)模型、一全連接神經網路(Fully-connected neural network,FCNN)模型及一歸一化指數函數(Softmax)模型。較佳地,密集卷積神經網路模型可包含複數個密集卷積神經網路模塊(DenseNet Block,DNB)、複數個最大池化模組(Max pooling)及一平坦模組(Flatten),且各個密集卷積神經網路模塊可包含複數個密集卷積神經網路單元,此些密集卷積神經網路單元彼此依序級聯,且各個密集卷積神經網路單元包含三維卷積層(Conv)、線性整流單元訓練層(ReLU)和批標準化層(Batch Normalization,BN)。The cardiac ultrasound judgment model may include a dense convolutional neural network (DenseNet) model, a fully-connected neural network (Fully-connected neural network, FCNN) model and a normalized exponential function (Softmax )Model. Preferably, the dense convolutional neural network model may include a plurality of dense convolutional neural network modules (DenseNet Block, DNB), a plurality of maximum pooling modules (Max pooling) and a flat module (Flatten), and Each dense convolutional neural network module may contain a plurality of dense convolutional neural network units, and these dense convolutional neural network units are cascaded to each other in sequence, and each dense convolutional neural network unit contains a three-dimensional convolutional layer (Conv ), linear rectification unit training layer (ReLU) and batch normalization layer (Batch Normalization, BN).
第2圖係繪示本發明另一實施方式之利用三維深度學習之心臟超音波判斷系統200的方塊示意圖。如第2圖所示,利用三維深度學習之心臟超音波判斷系統200包含超音波檢測器210和處理器220。FIG. 2 is a schematic block diagram of a cardiac
超音波檢測器210可為心臟超音波檢測器,用以產生受試者之目標心臟超音波影像。The
處理器220電性連接超音波檢測器210,所述處理器220儲存一程式,當程式由處理器220執行時,程式依據目標心臟超音波影像判斷受試者是否具有梗塞相關血管。所述程式包含參照資料取得模組221、參照影像前處理模組222、參照特徵選取模組223、模型訓練模組224及分析模組225。The
參照資料取得模組221係用以取得參照資料,參照資料包含複數個參照心臟超音波資料及對應此些參照心臟超音波資料之複數個心導管確診結果。The reference
參照影像前處理模組222係用以切割各個參照心臟超音波資料,以取得複數個參照心臟超音波影像。The reference
參照特徵選取模組223係用以分析各個參照心臟超音波影像後,於各個參照心臟超音波影像中得到參照影像特徵值,以取得對應此些參照心臟超音波影像之複數個參照影像特徵值。The reference
模型訓練模組224係依據此些參照影像特徵值訓練而產生心臟超音波判斷模型,並輸入此些參照心臟超音波資料至心臟超音波判斷模型而產生對應此些參照心臟超音波資料之複數個心臟超音波判斷結果,然後將此些心臟超音波判斷結果分別比對此些心導管確診結果以修正心臟超音波判斷模型。The
分析模組225係依據心臟超音波判斷模型分析目標心臟超音波影像而產生對應梗塞相關血管之目標心臟超音波判斷結果。The
第3圖係繪示本發明又一實施方式之利用三維深度學習之心臟超音波判斷方法300的步驟流程圖。利用三維深度學習之心臟超音波判斷方法300包含步驟310、步驟320、步驟330和步驟340。FIG. 3 is a flow chart showing the steps of a cardiac
步驟310係提供心臟超音波判斷模型。所述心臟超音波判斷模型係經由前述步驟110至步驟140所建立而成。Step 310 is to provide a cardiac ultrasound judgment model. The cardiac ultrasound judgment model is established through the
步驟320係提供目標心臟超音波影像,亦即提供受試者之目標心臟超音波影像。Step 320 is to provide the target ultrasound image of the heart, that is, to provide the target ultrasound image of the heart of the subject.
步驟330係產生目標心臟超音波判斷結果,亦即分析模組225依據心臟超音波判斷模型分析目標心臟超音波影像而產生目標心臟超音波判斷結果。Step 330 is to generate a target ultrasonic cardiac judgment result, that is, the
步驟340係判斷是否具有梗塞相關血管,亦即利用目標心臟超音波判斷結果判斷受試者是否具有梗塞相關血管。Step 340 is to determine whether there is an infarction-related vessel, that is, to determine whether the subject has an infarction-related vessel by using the target cardiac ultrasound determination result.
[試驗例][Test example]
一、參照資料1. Reference materials
本發明所使用的參照資料包含患有急性冠心症(Acute coronary syndrome,ACS)之複數個病患的複數個參照心臟超音波資料及經醫師診斷後的複數個心導管確診結果,並可分類為以下三組:The reference data used in the present invention include multiple reference echocardiographic data of multiple patients with acute coronary syndrome (ACS) and multiple cardiac catheterization results after diagnosis by doctors, and can be classified into the following three groups:
(1)ACS:心導管有看到冠狀動脈阻塞,且心臟超音波也看到區域性室壁運動異常(RWMA)。(1) ACS: Coronary artery obstruction was seen in cardiac catheterization, and regional wall motion abnormalities (RWMA) were also seen in cardiac ultrasound.
(2)Insignificant:心導管有看到冠狀動脈阻塞,但心臟超音波未看到區域性室壁運動異常。(2) Insignificant: Coronary artery obstruction was seen in cardiac catheterization, but regional wall motion abnormalities were not seen in cardiac ultrasound.
(3)Control:心導管未看到冠狀動脈阻塞,且心臟超音波也未看到區域性室壁運動異常。(3) Control: Cardiac catheterization did not see coronary artery obstruction, and cardiac ultrasound did not see regional wall motion abnormalities.
值得注意的是,將一參照心臟超音波資料和一心導管確診結果定義為一組樣本。試驗上共有9000個樣本組,並進一步分類為7500個訓練樣本組、1000個驗證樣本組與500個測試樣本組。 It is worth noting that a reference echocardiographic data and a cardiac catheterization result were defined as a group of samples. There are 9000 sample groups in the experiment, which are further classified into 7500 training sample groups, 1000 verification sample groups and 500 test sample groups.
二、影像前處理 2. Image pre-processing
在取得參照資料後,先進行影像前處理步驟,其係切割各個參照心臟超音波資料,以取得複數個參照心臟超音波影像。 After obtaining the reference data, the image pre-processing step is performed firstly, which is to cut each reference echocardiogram data to obtain a plurality of reference echocardiogram images.
三、本發明之心臟超音波判斷模型 Three, the cardiac ultrasonic judgment model of the present invention
第4圖係繪示本發明之心臟超音波判斷模型400的架構示意圖。如第4圖所示,心臟超音波判斷模型400主要由密集卷積神經網路模型500、全連接神經網路模型600及歸一化指數函數模型700依序連接所構成。密集卷積神經網路模型500可包含4個密集卷積神經網路模塊510、4個最大池化模組520及1個平坦模組530。首先,將參照心臟超音波影像401經過4次的密集卷積神經網路模塊510和最大池化模組520運算,並利用平坦模組530攤平成一維的陣列,接著透過全連接神經網路模型600將特徵值透過權值矩陣做合成,最後輸入至歸一化指數函數模型700進行運算而產生三種深度學習機率800,其中深度學習機率800分別為異常機率、正常機率及疑似異常機率。深度學習機率800透過交叉熵(Cross Entropy)分類成三種心臟超音波判斷結果900,心臟超音波判斷結果900分別為異常、正常及疑似異常。所得到的心臟超音波
判斷結果900可與心導管確診結果進行比對以修正心臟超音波判斷模型400。其中全連接神經網路模型600是一種神經元的連接模式,其特色是上一層神經元與下一層所有的神經元相連接,而歸一化指數函數模型700在計算機深度學習的應用主要在處理多分類問題上,且其使多分類的輸出數值轉換為相對論概率,使其更容易理解和比較。
FIG. 4 is a schematic diagram showing the structure of the cardiac
請一併參照第4圖與第5圖,其中第5圖係繪示第4圖中密集卷積神經網路模塊510的架構示意圖。密集卷積神經網路模塊510包含4個密集卷積神經網路單元511、512、513、514,參照心臟超音波影像401輸入至輸入端515,且經由4個密集卷積神經網路單元511、512、513、514彼此依序級聯並計算後,再輸出至輸出端516。
Please refer to FIG. 4 and FIG. 5 together, wherein FIG. 5 is a schematic diagram of the architecture of the dense convolutional
詳細地說,各個密集卷積神經網路單元511、512、513、514可包含三維卷積層540、線性整流單元訓練層550及批標準化層560。各個密集卷積神經網路單元511、512、513、514的輸出都將傳送至級聯單元C進行相加,並做為下一個的輸入,且以此類推,最終輸出至輸出端516。特別的是,使用填充(Padding)方法,將三維卷積層540的輸出影像放大而保持整個密集卷積神經網路模塊510的影像大小(Size),亦即影像經過各個密集卷積神經網路單元511、512、513、514運算後圖形大小不變,只有通道(Channel)數量的改變。
In detail, each dense convolutional
此外,本發明的心臟超音波判斷模型400以線性
整流單元訓練層550作為激活函數(Activation Function),相較於傳統神經網路激活函數,線性整流單元訓練層550具有仿生物學原理、更加有效率的梯度下降以及反向傳播、避免梯度消失與簡化計算過程等優點。
In addition, the cardiac
再者,本發明的心臟超音波判斷模型400所使用的損失函數(Loss Function)主要功能在於提供深度學習網路評估訓練結果與目標結果之誤差程度,且其如式子(1)與(2)所示:
針對損失函數利用梯度下降法的計算,進而達到網路自我調整與學習之目的。其中為網路參數,Y r 為訓練結果,為目標結果,ΩReg為正規化項(regularization term),λ為正規化因子(regularization factor)。 For the calculation of the loss function, the gradient descent method is used to achieve the purpose of network self-adjustment and learning. in is the network parameter, Y r is the training result, is the target result, Ω Reg is the regularization term, and λ is the regularization factor.
接著,本發明的心臟超音波判斷模型400的網路參數w l+1是藉由使用小批次梯度下降矩估算(Mini Batch Gradient Descent Moment Estimation)技術加以遞迴更新,且其如式子(3)所示:
其中η為學習率(Learning Rate),是一個值很小的常數主要用於避免式子(3)中第二項出現分母為零的現象發生。在更新網路參數時本發明的心臟超音波判斷模
型400考慮了數量為B個小批次(Mini Batch),因此在下列式子(4)與(5)中出現一次式梯度下降法(Gradient Descent)▽L r (w l )與二次式梯度下降法[▽L r (w l )]2的平均數,其平均數量為小批次的數量B。此外,β1與β2為衰減率(decay rate)。每次心臟超音波判斷模型400從測試樣本組中,隨機抽取數量為B個樣本組進行深度學習的方式稱為小批次。
Where η is the learning rate (Learning Rate), is a constant with a very small value, which is mainly used to avoid the phenomenon that the denominator of the second term in the formula (3) is zero. Cardiac
式子(4)與(5)說明心臟超音波判斷模型400每次調整時除了使用小批次技術外,還考慮先前每組小批次的梯度下降數值,亦即本次調整網路參數時會根據先前的調整經驗作為本次調整的參考依據,其具有依據先前經驗配合本次小批次產生動態調整網路參數的智能特性。
Equations (4) and (5) show that the cardiac
請一併參照第4圖至第6圖,其中第6圖係繪示第5圖中批標準化層560的架構示意圖。本發明的心臟超音波判斷模型400藉由批標準化層560減少訓練所需的時間、防止梯度下降消失問題與降低參數初始化造成擬合過度(Overfitting)的現象。心臟超音波判斷模型400在學習過程中會從測試樣本組中,隨機抽取數量為B個樣本組(小批次)進行學習。請一併參照下列式子(6)與式子(7),由輸入值X1、X2、X3與權重W1、W2、W3計算每個神經元輸出值A1、A2、A3的平均值μ與標準差σ,再將所有輸出值A1、A2、A3以平均值μ與標準差σ進行正規化
得到數值Ã1、Ã2、Ã3(即式子(6)的數值Ã i ),但是我們希望小批次進入激活函數時平均值μ與標準差σ與另一個小批次是不同的,因此引進參數β與參數γ調整此現象並獲得數值Â1、Â2、Â3(即式子(7)的數值Â i )。
Please refer to FIG. 4 to FIG. 6 together, wherein FIG. 6 is a schematic diagram of the architecture of the
四、模型訓練步驟 4. Model training steps
將7500個訓練樣本組與1000個驗證樣本組,放入心臟超音波判斷模型400中進行訓練。根據訓練結果,若Loss小於2%的目標無法被滿足時,將執行深度學習參數修正程序並重新訓練,直到目標被滿足為止。進一步可修正相關參數包含:初始參數、學習率、正規化因子、衰退率、每層神經元個數、神經元階層數、小批次大小、訓練樣本組與驗證樣本組,並繼續監控與參數調整直至狀況被改善。
7500 training sample groups and 1000 verification sample groups are put into the cardiac
當訓練結果滿足Loss小於2%且無擬合過度發生達到收斂而得到最佳化的心臟超音波判斷模型400後,前述心臟超音波判斷模型400輸出複數個心臟超音波判斷結果,並將各個心臟超音波判斷結果分別與相應的各個心導管確診結果進行比對,以驗證心臟超音波判斷模型400的準確性,若比對結果未通過,則回到模型訓練步驟並執行深度學習參數修正程序。
When the training result satisfies that the Loss is less than 2% and there is no over-fitting to achieve convergence and obtain the optimized cardiac
綜所上述,本發明具有下列優點:其一,本發明利 用三維深度學習訓練出心臟超音波判斷模型,並將心臟超音波判斷結果比對心導管確診結果而修正心臟超音波判斷模型,最終得到最佳化的心臟超音波判斷模型。其二,本發明透過心臟超音波判斷模型可即時分析目標心臟超音波影像而產生對應梗塞相關血管之目標心臟超音波判斷結果,進而改善人眼無法辨識區域性室壁運動異常的問題,達到可提早擬定後續心導管的術前準備。 In summary, the present invention has the following advantages: one, the present invention utilizes Three-dimensional deep learning is used to train the cardiac ultrasound judgment model, and the cardiac ultrasound judgment results are compared with the cardiac catheter diagnosis results to correct the cardiac ultrasound judgment model, and finally an optimized cardiac ultrasound judgment model is obtained. Second, the present invention can analyze the target cardiac ultrasound image in real time through the cardiac ultrasound judgment model to generate the target cardiac ultrasound judgment result corresponding to the infarct-related blood vessel, thereby improving the problem that the human eye cannot identify regional abnormal wall motion, and achieving Prepare early for subsequent cardiac catheterization.
雖然本發明已以實施方式揭露如上,然其並非用以限定本發明,任何熟習此技藝者,在不脫離本發明之精神和範圍內,當可作各種之更動與潤飾,因此本發明之保護範圍當視後附之申請專利範圍所界定者為準。 Although the present invention has been disclosed above in terms of implementation, it is not intended to limit the present invention. Anyone skilled in this art can make various changes and modifications without departing from the spirit and scope of the present invention. Therefore, the protection of the present invention The scope shall be defined by the appended patent application scope.
100:利用三維深度學習之心臟超音波判斷模型之建立方法 100: Establishment method of cardiac ultrasound judgment model using three-dimensional deep learning
110,120,130,140:步驟 110, 120, 130, 140: steps
200:利用三維深度學習之心臟超音波判斷系統 200: Cardiac Ultrasound Judgment System Using 3D Deep Learning
210:超音波檢測器 210: Ultrasonic detector
220:處理器 220: Processor
221:參照資料取得模組 221:Refer to the information to obtain the module
222:參照影像前處理模組 222: Refer to image preprocessing module
223:參照特徵選取模組 223: Reference feature selection module
224:模型訓練模組 224:Model training module
225:分析模組 225: Analysis module
300:利用三維深度學習之心臟超音波判斷方法 300: Cardiac ultrasound judgment method using three-dimensional deep learning
310,320,330,340:步驟 310, 320, 330, 340: steps
400:心臟超音波判斷模型 400: Cardiac Ultrasound Judgment Model
401:參照心臟超音波影像 401: Refer to cardiac ultrasound images
500:密集卷積神經網路模型 500: Dense Convolutional Neural Network Models
510:密集卷積神經網路模塊 510:Dense Convolutional Neural Network Module
511,512,513,514:密集卷積神經網路單元 511,512,513,514: Dense Convolutional Neural Network Units
515:輸入端 515: input terminal
516:輸出端 516: output terminal
520:最大池化模組 520: Maximum pooling module
530:平坦模組 530: flat module
540:三維卷積層 540: Three-dimensional convolution layer
550:線性整流單元訓練層 550: Linear rectification unit training layer
560:批標準化層 560: Batch normalization layer
600:全連接神經網路模型 600: Fully connected neural network model
700:歸一化指數函數模型 700:Normalized exponential function model
800:深度學習機率 800: Deep Learning Probability
900:心臟超音波判斷結果 900: Judgment result of cardiac ultrasound
X1,X2,X3:輸入值 X 1 , X 2 , X 3 : input value
W1,W2,W3:權重 W 1 , W 2 , W 3 : weights
C:級聯單元 C: cascade unit
μ:平均值 μ: average value
σ:標準差 σ: standard deviation
β,γ:參數 β, γ: parameters
Ã1,Ã2,Ã3,Â1,Â2,Â3:數值 Ã 1 , Ã 2 , Ã 3 , Â 1 , Â 2 , Â 3 : numeric values
為讓本發明之上述和其他目的、特徵、優點與實施例能更明顯易懂,所附圖式之說明如下: 第1圖係繪示本發明一實施方式之利用三維深度學習之心臟超音波判斷模型之建立方法的步驟流程圖; 第2圖係繪示本發明另一實施方式之利用三維深度學習之心臟超音波判斷系統的方塊示意圖; 第3圖係繪示本發明又一實施方式之利用三維深度學習之心臟超音波判斷方法的步驟流程圖; 第4圖係繪示本發明之心臟超音波判斷模型的架構示意圖; 第5圖係繪示第4圖中密集卷積神經網路模塊的架構示意圖;以及 第6圖係繪示第5圖中批標準化層的架構示意圖。 In order to make the above and other objects, features, advantages and embodiments of the present invention more clearly understood, the accompanying drawings are described as follows: Figure 1 is a flow chart showing the steps of a method for establishing a cardiac ultrasound judgment model using three-dimensional deep learning according to an embodiment of the present invention; Fig. 2 is a schematic block diagram of a heart ultrasound judgment system using three-dimensional deep learning according to another embodiment of the present invention; FIG. 3 is a flow chart showing the steps of a cardiac ultrasound judgment method using three-dimensional deep learning according to another embodiment of the present invention; Figure 4 is a schematic diagram showing the structure of the cardiac ultrasound judgment model of the present invention; FIG. 5 is a schematic diagram showing the architecture of the dense convolutional neural network module in FIG. 4; and FIG. 6 is a schematic diagram showing the architecture of the batch normalization layer in FIG. 5 .
200:利用三維深度學習之心臟超音波判斷系統 200: Cardiac Ultrasound Judgment System Using 3D Deep Learning
210:超音波檢測器 210: Ultrasonic detector
220:處理器 220: Processor
221:參照資料取得模組 221:Refer to the information to obtain the module
222:參照影像前處理模組 222: Refer to image preprocessing module
223:參照特徵選取模組 223: Reference feature selection module
224:模型訓練模組 224:Model training module
225:分析模組 225: Analysis module
Claims (7)
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
TW109133402A TWI792055B (en) | 2020-09-25 | 2020-09-25 | Establishing method of echocardiography judging model with 3d deep learning, echocardiography judging system with 3d deep learning and method thereof |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
TW109133402A TWI792055B (en) | 2020-09-25 | 2020-09-25 | Establishing method of echocardiography judging model with 3d deep learning, echocardiography judging system with 3d deep learning and method thereof |
Publications (2)
Publication Number | Publication Date |
---|---|
TW202213384A TW202213384A (en) | 2022-04-01 |
TWI792055B true TWI792055B (en) | 2023-02-11 |
Family
ID=82197115
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
TW109133402A TWI792055B (en) | 2020-09-25 | 2020-09-25 | Establishing method of echocardiography judging model with 3d deep learning, echocardiography judging system with 3d deep learning and method thereof |
Country Status (1)
Country | Link |
---|---|
TW (1) | TWI792055B (en) |
Citations (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN105518684A (en) * | 2013-08-27 | 2016-04-20 | 哈特弗罗公司 | Systems and methods for predicting location, onset, and/or change of coronary lesions |
CN105825509A (en) * | 2016-03-17 | 2016-08-03 | 电子科技大学 | Cerebral vessel segmentation method based on 3D convolutional neural network |
CN110232383A (en) * | 2019-06-18 | 2019-09-13 | 湖南省华芯医疗器械有限公司 | A kind of lesion image recognition methods and lesion image identifying system based on deep learning model |
CN110475505A (en) * | 2017-01-27 | 2019-11-19 | 阿特瑞斯公司 | Utilize the automatic segmentation of full convolutional network |
US10706545B2 (en) * | 2018-05-07 | 2020-07-07 | Zebra Medical Vision Ltd. | Systems and methods for analysis of anatomical images |
US20200272857A1 (en) * | 2019-02-22 | 2020-08-27 | Neuropace, Inc. | Systems and methods for labeling large datasets of physiologial records based on unsupervised machine learning |
-
2020
- 2020-09-25 TW TW109133402A patent/TWI792055B/en active
Patent Citations (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN105518684A (en) * | 2013-08-27 | 2016-04-20 | 哈特弗罗公司 | Systems and methods for predicting location, onset, and/or change of coronary lesions |
CN105825509A (en) * | 2016-03-17 | 2016-08-03 | 电子科技大学 | Cerebral vessel segmentation method based on 3D convolutional neural network |
CN110475505A (en) * | 2017-01-27 | 2019-11-19 | 阿特瑞斯公司 | Utilize the automatic segmentation of full convolutional network |
US10706545B2 (en) * | 2018-05-07 | 2020-07-07 | Zebra Medical Vision Ltd. | Systems and methods for analysis of anatomical images |
US20200272857A1 (en) * | 2019-02-22 | 2020-08-27 | Neuropace, Inc. | Systems and methods for labeling large datasets of physiologial records based on unsupervised machine learning |
CN110232383A (en) * | 2019-06-18 | 2019-09-13 | 湖南省华芯医疗器械有限公司 | A kind of lesion image recognition methods and lesion image identifying system based on deep learning model |
Also Published As
Publication number | Publication date |
---|---|
TW202213384A (en) | 2022-04-01 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN109584254B (en) | Heart left ventricle segmentation method based on deep full convolution neural network | |
EP3445250B1 (en) | Echocardiographic image analysis | |
CN111493935B (en) | Artificial intelligence-based automatic prediction and identification method and system for echocardiogram | |
CN111626114B (en) | Electrocardiosignal arrhythmia classification system based on convolutional neural network | |
Alqudah et al. | COVID-19 detection from x-ray images using different artificial intelligence hybrid models | |
US8308646B2 (en) | Trainable diagnostic system and method of use | |
CN111009324B (en) | Auxiliary diagnosis system and method for mild cognitive impairment through multi-feature analysis of brain network | |
Nurmaini et al. | Accurate detection of septal defects with fetal ultrasonography images using deep learning-based multiclass instance segmentation | |
CN107391900A (en) | Atrial fibrillation detection method, disaggregated model training method and terminal device | |
KR102439082B1 (en) | Method for predicting liver disease of ordinary perseon using ecg analysis data based on deep running | |
JPH05176932A (en) | Diagnostic method for cerebral infarction | |
US8064670B2 (en) | Analysis of multidimensional data | |
Sapitri et al. | Segmentation atrioventricular septal defect by using convolutional neural networks based on U-NET architecture | |
Kabir et al. | Multi-classification based Alzheimer's disease detection with comparative analysis from brain MRI scans using deep learning | |
CN117274185B (en) | Detection method, detection model product, electronic device, and computer storage medium | |
TWI792055B (en) | Establishing method of echocardiography judging model with 3d deep learning, echocardiography judging system with 3d deep learning and method thereof | |
CN109003264B (en) | Retinopathy image type identification method and device and storage medium | |
CN109191425B (en) | Medical image analysis method based on multilayer neural network model | |
WO2020199692A1 (en) | Method and apparatus for screening predictive image features for cancer metastasis, and storage medium | |
Thakral et al. | An innovative intelligent solution incorporating artificial neural networks for medical diagnostic application | |
Karan et al. | Heart Sound Abnormality Detection using Wavelet Packet Features and Machine Learning | |
CN114224354A (en) | Arrhythmia classification method, device and readable storage medium | |
CN114021630A (en) | Ordinal regression problem solving method for category-unbalanced data set | |
KR20220045653A (en) | Acute myocardial infarction judgment method, device and program | |
CN110786839A (en) | Method, device, equipment and medium for generating instantaneous waveform-free ratio |