CN111915562A - Deep learning children echocardiogram standard tangent plane identification method and device - Google Patents
Deep learning children echocardiogram standard tangent plane identification method and device Download PDFInfo
- Publication number
- CN111915562A CN111915562A CN202010625258.6A CN202010625258A CN111915562A CN 111915562 A CN111915562 A CN 111915562A CN 202010625258 A CN202010625258 A CN 202010625258A CN 111915562 A CN111915562 A CN 111915562A
- Authority
- CN
- China
- Prior art keywords
- section
- echocardiogram
- parasternal
- blood flow
- region
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 66
- 238000013135 deep learning Methods 0.000 title claims abstract description 25
- 238000010586 diagram Methods 0.000 claims abstract description 35
- 238000012805 post-processing Methods 0.000 claims abstract description 14
- 238000007781 pre-processing Methods 0.000 claims abstract description 12
- 238000009499 grossing Methods 0.000 claims abstract description 9
- 238000000605 extraction Methods 0.000 claims abstract description 8
- 238000001228 spectrum Methods 0.000 claims description 42
- 230000000747 cardiac effect Effects 0.000 claims description 38
- 230000017531 blood circulation Effects 0.000 claims description 36
- 230000008569 process Effects 0.000 claims description 32
- 230000002861 ventricular Effects 0.000 claims description 30
- 210000002376 aorta thoracic Anatomy 0.000 claims description 14
- 238000012545 processing Methods 0.000 claims description 13
- 230000001746 atrial effect Effects 0.000 claims description 12
- 210000001147 pulmonary artery Anatomy 0.000 claims description 12
- 238000002592 echocardiography Methods 0.000 claims description 11
- 238000004422 calculation algorithm Methods 0.000 claims description 8
- 210000004115 mitral valve Anatomy 0.000 claims description 6
- 210000000591 tricuspid valve Anatomy 0.000 claims description 6
- 238000003745 diagnosis Methods 0.000 abstract description 9
- 238000002604 ultrasonography Methods 0.000 abstract description 9
- 238000003860 storage Methods 0.000 description 8
- 238000004590 computer program Methods 0.000 description 7
- 230000000694 effects Effects 0.000 description 6
- 230000006870 function Effects 0.000 description 4
- 238000011156 evaluation Methods 0.000 description 3
- 208000002330 Congenital Heart Defects Diseases 0.000 description 2
- 238000013145 classification model Methods 0.000 description 2
- 208000028831 congenital heart disease Diseases 0.000 description 2
- 238000001514 detection method Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 230000003068 static effect Effects 0.000 description 2
- 208000024172 Cardiovascular disease Diseases 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 239000003086 colorant Substances 0.000 description 1
- 238000013136 deep learning model Methods 0.000 description 1
- 201000010099 disease Diseases 0.000 description 1
- 208000037265 diseases, disorders, signs and symptoms Diseases 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000011897 real-time detection Methods 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 230000008439 repair process Effects 0.000 description 1
- 238000004904 shortening Methods 0.000 description 1
- 210000002417 xiphoid bone Anatomy 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0012—Biomedical image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/25—Determination of region of interest [ROI] or a volume of interest [VOI]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10024—Color image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10132—Ultrasound image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30004—Biomedical image processing
- G06T2207/30048—Heart; Cardiac
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30004—Biomedical image processing
- G06T2207/30101—Blood vessel; Artery; Vein; Vascular
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V2201/00—Indexing scheme relating to image or video recognition or understanding
- G06V2201/03—Recognition of patterns in medical or anatomical images
- G06V2201/031—Recognition of patterns in medical or anatomical images of internal organs
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Engineering & Computer Science (AREA)
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Computation (AREA)
- General Health & Medical Sciences (AREA)
- Computational Linguistics (AREA)
- Evolutionary Biology (AREA)
- Molecular Biology (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Biophysics (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Biomedical Technology (AREA)
- Computing Systems (AREA)
- Multimedia (AREA)
- Medical Informatics (AREA)
- Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
- Radiology & Medical Imaging (AREA)
- Quality & Reliability (AREA)
- Ultra Sonic Daignosis Equipment (AREA)
- Image Analysis (AREA)
Abstract
The invention provides a method and a device for identifying a standard section of a deep-learning children echocardiogram, wherein the method comprises the following steps: obtaining an echocardiogram, and preprocessing the echocardiogram to obtain a plurality of frames of echocardiogram images; extracting an interested region of the multi-frame echocardiogram image; performing feature extraction on the image of the region of interest by using a classical residual error network ResNet to obtain section features of the region of interest; classifying the section characteristics of the region of interest according to preset categories to obtain classification results; carrying out smoothing post-processing on the classification result to obtain a processed classification result; and generating and displaying a characteristic thermodynamic diagram corresponding to the echocardiogram. Not only greatly shortens the operation time of the heart color Doppler ultrasound doctor, accelerates the diagnosis speed of the doctor, but also can effectively reduce misdiagnosis caused by artificial observation.
Description
Technical Field
The invention relates to the field of computers, in particular to a method and a device for recognizing a standard section of a deep learning children echocardiogram.
Background
The diagnosis accuracy rate of the echocardiogram on the congenital heart disease is closely related to the technical level of an operator, and an excellent hypercardiogram doctor not only has high-level operation skill and image recognition and interpretation capability, but also has overall comprehension on of various diseases of the congenital heart disease. Because the technical level requirement is high, the talent culture period is long, the population base of China is large, and pediatricians are in short supply; the ultrasonic diagnosis of the cardiovascular diseases of children has the severe states of difficult hospitalization, expensive hospitalization and poor hospitalization effect; this conflict is even more pronounced in remote areas where economy is not reached. Therefore, the requirement for rapidly and automatically identifying the standard section in the echocardiogram is gradually increased.
Deep learning methods have been widely used in the field of computer vision. The method has the advantages that the medical image features are extracted and classified by means of the deep learning network, and a good effect is achieved. In the field of cardiac ultrasound, Ali Madani et al attempted to classify 12 echocardiograms and 3 static slices of an adult using a modified VGG network (Ali. Madani, Ramy A., et al: Fast and acquisition view classification of electronic diagnosis and repair in: Proc. nature (2018)), but were limited to grayscale images and did not process color ultrasounds. In addition, the standard section in the paper is not comprehensive, and the real-time detection effect cannot be achieved in practical application. In addition, the heart structure of children is different from that of adults, and a new ultrasonic section classification mode needs to be designed to achieve the effect of real-time automatic detection.
The existing ultrasound section classification aims at adult ultrasound images and adopts an improved VGG network. There are three main problems. One is that the section classification is not perfect and color ultrasound images are not considered. Secondly, the standard ultrasonic section for adults can not be completely suitable for children. Thirdly, the original model input and the VGG model are simple, the image feature extraction capability is not strong, and the real-time section detection result is not realized.
Disclosure of Invention
The present invention aims to provide a method and apparatus for deep learning standard sectional echocardiography for children which overcomes or at least partially solves the above mentioned problems.
In order to achieve the purpose, the technical scheme of the invention is realized as follows:
one aspect of the present invention provides a method for identifying a standard section of a deep-learning children echocardiogram, comprising: obtaining an echocardiogram, and preprocessing the echocardiogram to obtain a plurality of frames of echocardiogram images; extracting an interested region of the multi-frame echocardiogram image; performing feature extraction on the image of the region of interest by using a classical residual error network ResNet to obtain section features of the region of interest; classifying the section characteristics of the region of interest according to preset categories to obtain classification results; carrying out smoothing post-processing on the classification result to obtain a processed classification result; and generating and displaying a characteristic thermodynamic diagram corresponding to the echocardiogram.
Wherein the preset categories include: cut surfaces of 27 categories; the 27 categories of cuts include: continuous wave spectrum, aortic blood flow spectrum, pulmonary artery blood flow spectrum, mitral valve blood flow spectrum, tricuspid valve blood flow spectrum, descending aorta blood flow spectrum, other blood flow spectrum, aortic type M, left ventricular minor axis M type, other types M, glad process inferior atrial septal frontal section, glad process inferior atrial septal sagittal section, glad process inferior four-chamber cardiac section, glad process inferior left ventricular minor axis section, glad process inferior right ventricular outflow tract section, parasternal left ventricular major axis section, parasternal aortic minor axis section, parasternal pulmonary artery section, parasternal left ventricular minor axis section, parasternal four-chamber sternal section, low parasternal five-chamber cardiac section, four-chamber cardiac section of cardiac apex, five-chamber cardiac section of cardiac apex, suprasternal aortic arch section, double-amplitude, other standard section and non-standard section.
Wherein, the step of carrying out smooth post-processing on the classification result to obtain the processed classification result comprises the following steps: and combining the prediction result of the current frame with the prediction result of the previous N frames, and correcting the prediction result of the current frame by adopting a majority voting algorithm to obtain a processed classification result.
Wherein N is 4.
Wherein, the method further comprises: and comparing the thermodynamic diagram with the classification result.
In another aspect, the present invention provides a device for recognizing a standard section of a deep-learning echocardiogram for children, comprising: the preprocessing module is used for acquiring the echocardiogram and preprocessing the echocardiogram to obtain a plurality of frames of echocardiogram images; the interesting region extracting module is used for extracting an interesting region of the multi-frame echocardiography image; the characteristic extraction module is used for extracting the characteristics of the image of the region of interest by using a classical residual error network ResNet to obtain the section characteristics of the region of interest; the classification module is used for classifying the section characteristics of the region of interest according to preset categories to obtain classification results; the processing module is used for carrying out smooth post-processing on the classification result to obtain a processed classification result; and the display module is used for generating and displaying the characteristic thermodynamic diagram corresponding to the echocardiogram.
Wherein the preset categories include: cut surfaces of 27 categories; the 27 categories of cuts include: continuous wave spectrum, aortic blood flow spectrum, pulmonary artery blood flow spectrum, mitral valve blood flow spectrum, tricuspid valve blood flow spectrum, descending aorta blood flow spectrum, other blood flow spectrum, aortic type M, left ventricular minor axis M type, other types M, glad process inferior atrial septal frontal section, glad process inferior atrial septal sagittal section, glad process inferior four-chamber cardiac section, glad process inferior left ventricular minor axis section, glad process inferior right ventricular outflow tract section, parasternal left ventricular major axis section, parasternal aortic minor axis section, parasternal pulmonary artery section, parasternal left ventricular minor axis section, parasternal four-chamber sternal section, low parasternal five-chamber cardiac section, four-chamber cardiac section of cardiac apex, five-chamber cardiac section of cardiac apex, suprasternal aortic arch section, double-amplitude, other standard section and non-standard section.
The processing module performs smooth post-processing on the classification result in the following way to obtain a processed classification result: and the processing module is specifically used for combining the prediction result of the current frame with the prediction result of the previous N frames, and correcting the prediction result of the current frame by adopting a majority voting algorithm to obtain a processed classification result.
Wherein N is 4.
Wherein, the device still includes: and the comparison module is used for comparing the thermodynamic diagram with the classification result.
Therefore, the method and the device for identifying the standard tangent plane of the deep learning children echocardiogram adopt the classification module and the interpretation module, the classification module classifies the ultrasonic images into 27 categories, the interpretation module displays the tangent plane characteristic thermodynamic diagram obtained by the model in real time, and a doctor can evaluate a machine display result according to the thermodynamic diagram provided by the system and compare the result with a decision judgment basis of the doctor, so that the reliability of the system identification result is enhanced, and the diagnosis of the doctor is assisted.
In addition, the invention designs a children heart ultrasonic section classification method, and provides section characteristic thermodynamic diagrams for explaining the image characteristics focused by section classification, thereby greatly shortening the operation time of a heart color ultrasonic doctor, accelerating the diagnosis speed of the doctor and more effectively reducing misdiagnosis caused by artificial observation.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings needed to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on the drawings without creative efforts.
Fig. 1 is a flowchart of a method for identifying a standard section of a deep-learning echocardiogram for children according to an embodiment of the present invention;
FIG. 2 is a schematic diagram of a standard section classification model of an echocardiogram according to an embodiment of the present invention;
FIG. 3 is a schematic diagram of standard section classification provided in the embodiment of the present invention;
FIG. 4 is a schematic diagram illustrating a classification result smoothing post-processing according to an embodiment of the present invention;
FIG. 5 is a characteristic thermodynamic diagram corresponding to an ultrasound image provided by an embodiment of the present invention;
fig. 6 is a schematic structural diagram of a deep learning standard echocardiography section identification device for children according to an embodiment of the present invention.
Detailed Description
Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be embodied in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.
In order to solve the problem of detecting a standard section in real time in practical ultrasonic application, the invention designs a deep learning children echocardiogram standard section identification method, which basically covers all useful section types related to children's heart ultrasound. And a deep learning model is designed, and each standard tangent plane is classified in real time.
Fig. 1 shows a flowchart of a deep learning method for identifying a standard section of an echocardiogram of a child according to an embodiment of the present invention, and referring to fig. 1, the deep learning method for identifying a standard section of an echocardiogram of a child according to an embodiment of the present invention includes:
and S1, obtaining the echocardiogram, preprocessing the echocardiogram and obtaining a multi-frame echocardiogram image.
Specifically, firstly, the echocardiogram acquired by the acquisition card is adopted to zoom and convert the image data into a uniform size. To facilitate subsequent use of the image data. The model inputs may include: multiple frames of echocardiograms, e.g., 3-5 frames.
S2, extracting the interested region of the multi-frame echocardiogram image;
s3, performing feature extraction on the image of the region of interest by using a classical residual error network ResNet to obtain section features of the region of interest;
and S4, classifying the section features of the region of interest according to preset categories to obtain classification results.
Specifically, after an interest region of a multi-frame image is extracted, a direct association channel is established between input and output by using a classical residual error network ResNet, so that a strong reference layer is concentrated to learn residual errors between the input and the output, the quantity of parameters of the network is effectively reduced by stacking residual error modules with dimension reduction and dimension increase, the time required by network calculation is shortened, deep residual error learning is performed on the multi-frame image to extract features, and the extracted features are classified into the preset categories.
Wherein the classification model may refer to fig. 2.
As an optional implementation manner of the embodiment of the present invention, the preset category includes: cut surfaces of 27 categories; the 27 categories of cuts include: continuous wave spectrum, aortic blood flow spectrum, pulmonary artery blood flow spectrum, mitral valve blood flow spectrum, tricuspid valve blood flow spectrum, descending aorta blood flow spectrum, other blood flow spectrum, aortic type M, left ventricular minor axis M type, other types M, glad process inferior atrial septal frontal section, glad process inferior atrial septal sagittal section, glad process inferior four-chamber cardiac section, glad process inferior left ventricular minor axis section, glad process inferior right ventricular outflow tract section, parasternal left ventricular major axis section, parasternal aortic minor axis section, parasternal pulmonary artery section, parasternal left ventricular minor axis section, parasternal four-chamber sternal section, low parasternal five-chamber cardiac section, four-chamber cardiac section of cardiac apex, five-chamber cardiac section of cardiac apex, suprasternal aortic arch section, double-amplitude, other standard section and non-standard section.
As an alternative to the embodiment of the present invention, referring to fig. 3, 27 categories of slices can be divided into 10 groups.
And S5, performing smoothing post-processing on the classification result to obtain a processed classification result.
As an optional implementation manner of the embodiment of the present invention, performing smoothing post-processing on the classification result, and obtaining the processed classification result includes: and combining the prediction result of the current frame with the prediction result of the previous N frames, and correcting the prediction result of the current frame by adopting a majority voting algorithm to obtain a processed classification result. Preferably, N is 4. Specifically, the current frame prediction result is combined with the previous four frames prediction result, and the current frame prediction result is corrected by adopting a majority voting algorithm. The method effectively prevents the fluctuation of misdiagnosis sections and correct sections, and improves the stability and accuracy of system diagnosis. As shown in fig. 4 below, fig. 4 was misdiagnosed as a subconjunctival septum surface of the xiphoid process. The superior sternal fossa aortic arch tangent plane takes the majority of votes by using the majority voting principle, so the result of fig. 4 is revised as the superior sternal fossa aortic arch tangent plane. The algorithm fully utilizes the dynamic characteristics of the heart, solves the problem that the network is very sensitive to the quality of a single frame image, has higher accuracy and robustness compared with single frame image identification, and can achieve the real-time section identification effect while keeping higher accuracy.
And S6, generating and displaying a characteristic thermodynamic diagram corresponding to the echocardiogram.
Specifically, a characteristic thermodynamic diagram corresponding to the ultrasonic image is displayed in real time by the system during the generation of the cardiac ultrasonography. For example, referring to fig. 5, the colors in the characteristic thermodynamic diagram range from blue to red, indicating that the importance of the effect on the classification result in the original pixels ranges from light to heavy. The doctor can evaluate the machine display result according to the thermodynamic diagram provided by the system and compare the evaluation result with the decision judgment basis of the doctor.
As an optional implementation manner of the embodiment of the present invention, the method for identifying a standard section of a deep-learning children echocardiogram further includes: and comparing the thermodynamic diagram with the classification result. The judgment can be made according to the comparison result.
Therefore, the children echocardiogram standard section identification method for deep learning provided by the embodiment of the invention provides a children echocardiogram preprocessing method and a standard section classification method, which are divided into 10 groups of 27 sections, the classification result is smoothed and then processed, the result is smoothed by adopting a voting method, a section classification thermodynamic diagram is provided for comparison and evaluation with machine judgment and doctor judgment, the reliability of the system identification result is enhanced, and the diagnosis of a doctor is assisted.
Fig. 6 is a schematic structural diagram of a deep-learning standard child echocardiography section identification device according to an embodiment of the present invention, in which the above method is applied to the deep-learning standard child echocardiography section identification device, and only the structure of the deep-learning standard child echocardiography section identification device is briefly described below, and for other things that are not the least, the deep-learning standard child echocardiography section identification device according to an embodiment of the present invention includes:
the preprocessing module is used for acquiring the echocardiogram and preprocessing the echocardiogram to obtain a plurality of frames of echocardiogram images;
the interesting region extracting module is used for extracting an interesting region of the multi-frame echocardiography image;
the characteristic extraction module is used for extracting the characteristics of the image of the region of interest by using a classical residual error network ResNet to obtain the section characteristics of the region of interest;
the classification module is used for classifying the section characteristics of the region of interest according to preset categories to obtain classification results;
the processing module is used for carrying out smooth post-processing on the classification result to obtain a processed classification result;
and the display module is used for generating and displaying the characteristic thermodynamic diagram corresponding to the echocardiogram.
As an optional implementation manner of the embodiment of the present invention, the preset category includes: cut surfaces of 27 categories; the 27 categories of cuts include: continuous wave spectrum, aortic blood flow spectrum, pulmonary artery blood flow spectrum, mitral valve blood flow spectrum, tricuspid valve blood flow spectrum, descending aorta blood flow spectrum, other blood flow spectrum, aortic type M, left ventricular minor axis M type, other types M, glad process inferior atrial septal frontal section, glad process inferior atrial septal sagittal section, glad process inferior four-chamber cardiac section, glad process inferior left ventricular minor axis section, glad process inferior right ventricular outflow tract section, parasternal left ventricular major axis section, parasternal aortic minor axis section, parasternal pulmonary artery section, parasternal left ventricular minor axis section, parasternal four-chamber sternal section, low parasternal five-chamber cardiac section, four-chamber cardiac section of cardiac apex, five-chamber cardiac section of cardiac apex, suprasternal aortic arch section, double-amplitude, other standard section and non-standard section.
As an optional implementation manner of the embodiment of the present invention, the processing module performs smoothing post-processing on the classification result in the following manner to obtain a processed classification result: and the processing module is specifically used for combining the prediction result of the current frame with the prediction result of the previous N frames, and correcting the prediction result of the current frame by adopting a majority voting algorithm to obtain a processed classification result. Preferably, N is 4.
As an optional implementation manner of the embodiment of the present invention, the apparatus for recognizing a standard section of a deep-learning echocardiogram for children further includes: and the comparison module is used for comparing the thermodynamic diagram with the classification result.
Therefore, the children echocardiogram standard section recognition device for deep learning provided by the embodiment of the invention provides a children echocardiogram preprocessing method and a standard section classification method, which are divided into 10 groups of 27 sections, the classification result is smoothed and then processed, the result is smoothed by adopting a voting method, a section classification thermodynamic diagram is provided for comparison and evaluation with machine judgment and doctor judgment, the reliability of the system recognition result is enhanced, and the diagnosis of a doctor is assisted.
As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
In a typical configuration, a computing device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory.
The memory may include forms of volatile memory in a computer readable medium, Random Access Memory (RAM) and/or non-volatile memory, such as Read Only Memory (ROM) or flash memory (flash RAM). The memory is an example of a computer-readable medium.
Computer-readable media, including both non-transitory and non-transitory, removable and non-removable media, may implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), Read Only Memory (ROM), Electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), Digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information that can be accessed by a computing device. As defined herein, a computer readable medium does not include a transitory computer readable medium such as a modulated data signal and a carrier wave.
The above are merely examples of the present application and are not intended to limit the present application. Various modifications and changes may occur to those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.
Claims (10)
1. A method for recognizing a standard section of a deep-learning children echocardiogram is characterized by comprising the following steps:
obtaining an echocardiogram, and preprocessing the echocardiogram to obtain a multi-frame echocardiogram image;
extracting a region of interest of the multi-frame echocardiography image;
performing feature extraction on the image of the region of interest by using a classical residual error network ResNet to obtain section features of the region of interest;
classifying the section characteristics of the region of interest according to preset categories to obtain classification results;
carrying out smoothing post-processing on the classification result to obtain a processed classification result;
and generating and displaying a characteristic thermodynamic diagram corresponding to the echocardiogram.
2. The method of claim 1, wherein the preset categories comprise: cut surfaces of 27 categories;
the 27 categories of cut surfaces include: continuous wave spectrum, aortic blood flow spectrum, pulmonary artery blood flow spectrum, mitral valve blood flow spectrum, tricuspid valve blood flow spectrum, descending aorta blood flow spectrum, other blood flow spectrum, aortic type M, left ventricular minor axis M type, other types M, glad process inferior atrial septal frontal section, glad process inferior atrial septal sagittal section, glad process inferior four-chamber cardiac section, glad process inferior left ventricular minor axis section, glad process inferior right ventricular outflow tract section, parasternal left ventricular major axis section, parasternal aortic minor axis section, parasternal pulmonary artery section, parasternal left ventricular minor axis section, parasternal four-chamber sternal section, low parasternal five-chamber cardiac section, four-chamber cardiac section of cardiac apex, five-chamber cardiac section of cardiac apex, suprasternal aortic arch section, double-amplitude, other standard section and non-standard section.
3. The method according to claim 1, wherein the performing smoothing post-processing on the classification result to obtain a processed classification result comprises:
and combining the prediction result of the current frame with the prediction results of the previous N frames, and correcting the prediction result of the current frame by adopting a majority voting algorithm to obtain a processed classification result.
4. The method of claim 3, wherein N-4.
5. The method of claim 1, further comprising:
and comparing the thermodynamic diagram with the classification result.
6. A children echocardiography standard tangent plane recognition device for deep learning is characterized by comprising:
the preprocessing module is used for acquiring an echocardiogram and preprocessing the echocardiogram to obtain a plurality of frames of echocardiogram images;
the interesting region extracting module is used for extracting an interesting region of the multi-frame echocardiography image;
the characteristic extraction module is used for extracting the characteristics of the image of the region of interest by using a classical residual error network ResNet to obtain the section characteristics of the region of interest;
the classification module is used for classifying the section characteristics of the region of interest according to preset categories to obtain classification results;
the processing module is used for carrying out smooth post-processing on the classification result to obtain a processed classification result;
and the display module is used for generating and displaying the characteristic thermodynamic diagram corresponding to the echocardiogram.
7. The apparatus of claim 6, wherein the preset categories comprise: cut surfaces of 27 categories;
the 27 categories of cut surfaces include: continuous wave spectrum, aortic blood flow spectrum, pulmonary artery blood flow spectrum, mitral valve blood flow spectrum, tricuspid valve blood flow spectrum, descending aorta blood flow spectrum, other blood flow spectrum, aortic type M, left ventricular minor axis M type, other types M, glad process inferior atrial septal frontal section, glad process inferior atrial septal sagittal section, glad process inferior four-chamber cardiac section, glad process inferior left ventricular minor axis section, glad process inferior right ventricular outflow tract section, parasternal left ventricular major axis section, parasternal aortic minor axis section, parasternal pulmonary artery section, parasternal left ventricular minor axis section, parasternal four-chamber sternal section, low parasternal five-chamber cardiac section, four-chamber cardiac section of cardiac apex, five-chamber cardiac section of cardiac apex, suprasternal aortic arch section, double-amplitude, other standard section and non-standard section.
8. The apparatus of claim 6, wherein the processing module performs smoothing post-processing on the classification result to obtain a processed classification result by:
the processing module is specifically configured to combine the prediction result of the current frame with the prediction result of the previous N frames, and correct the prediction result of the current frame by using a majority voting algorithm to obtain a processed classification result.
9. The apparatus of claim 8, wherein N-4.
10. The apparatus of claim 6, further comprising: and the comparison module is used for comparing the thermodynamic diagram with the classification result.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010625258.6A CN111915562A (en) | 2020-07-02 | 2020-07-02 | Deep learning children echocardiogram standard tangent plane identification method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010625258.6A CN111915562A (en) | 2020-07-02 | 2020-07-02 | Deep learning children echocardiogram standard tangent plane identification method and device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111915562A true CN111915562A (en) | 2020-11-10 |
Family
ID=73227139
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010625258.6A Pending CN111915562A (en) | 2020-07-02 | 2020-07-02 | Deep learning children echocardiogram standard tangent plane identification method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111915562A (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113658145A (en) * | 2021-08-20 | 2021-11-16 | 合肥合滨智能机器人有限公司 | Liver ultrasonic standard tangent plane identification method and device, electronic equipment and storage medium |
| CN113990459A (en) * | 2021-10-26 | 2022-01-28 | 卡本(深圳)医疗器械有限公司 | Method for full-automatically measuring lower sternum end/xiphoid process in CT/MRI volume |
| CN114521914A (en) * | 2020-11-23 | 2022-05-24 | 深圳迈瑞生物医疗电子股份有限公司 | Ultrasonic parameter measuring method and ultrasonic parameter measuring system |
| CN116704305A (en) * | 2023-06-20 | 2023-09-05 | 华中科技大学同济医学院附属协和医院 | Ultrasonic cardiogram multi-mode multi-section classification method based on deep learning algorithm |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20110310964A1 (en) * | 2010-06-19 | 2011-12-22 | Ibm Corporation | Echocardiogram view classification using edge filtered scale-invariant motion features |
| CN108701354A (en) * | 2016-05-09 | 2018-10-23 | 深圳迈瑞生物医疗电子股份有限公司 | Identify the method and system of area-of-interest profile in ultrasonoscopy |
| CN109925002A (en) * | 2019-01-15 | 2019-06-25 | 胡秋明 | Artificial intelligence echocardiogram data collection system and its collecting method |
| CN110543912A (en) * | 2019-09-02 | 2019-12-06 | 李肯立 | Method for automatically acquiring cardiac cycle video in fetal key section ultrasonic video |
| CN110555836A (en) * | 2019-09-05 | 2019-12-10 | 李肯立 | Automatic identification method and system for standard fetal section in ultrasonic image |
| CN111012377A (en) * | 2019-12-06 | 2020-04-17 | 北京安德医智科技有限公司 | Echocardiogram heart parameter calculation and myocardial strain measurement method and device |
-
2020
- 2020-07-02 CN CN202010625258.6A patent/CN111915562A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20110310964A1 (en) * | 2010-06-19 | 2011-12-22 | Ibm Corporation | Echocardiogram view classification using edge filtered scale-invariant motion features |
| CN108701354A (en) * | 2016-05-09 | 2018-10-23 | 深圳迈瑞生物医疗电子股份有限公司 | Identify the method and system of area-of-interest profile in ultrasonoscopy |
| CN109925002A (en) * | 2019-01-15 | 2019-06-25 | 胡秋明 | Artificial intelligence echocardiogram data collection system and its collecting method |
| CN110543912A (en) * | 2019-09-02 | 2019-12-06 | 李肯立 | Method for automatically acquiring cardiac cycle video in fetal key section ultrasonic video |
| CN110555836A (en) * | 2019-09-05 | 2019-12-10 | 李肯立 | Automatic identification method and system for standard fetal section in ultrasonic image |
| CN111012377A (en) * | 2019-12-06 | 2020-04-17 | 北京安德医智科技有限公司 | Echocardiogram heart parameter calculation and myocardial strain measurement method and device |
Non-Patent Citations (1)
| Title |
|---|
| 陶攀，付忠良，朱锴，王莉莉: "基于深度学习的超声心动图切面识别方法", 《计算机应用》 * |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114521914A (en) * | 2020-11-23 | 2022-05-24 | 深圳迈瑞生物医疗电子股份有限公司 | Ultrasonic parameter measuring method and ultrasonic parameter measuring system |
| CN113658145A (en) * | 2021-08-20 | 2021-11-16 | 合肥合滨智能机器人有限公司 | Liver ultrasonic standard tangent plane identification method and device, electronic equipment and storage medium |
| CN113990459A (en) * | 2021-10-26 | 2022-01-28 | 卡本(深圳)医疗器械有限公司 | Method for full-automatically measuring lower sternum end/xiphoid process in CT/MRI volume |
| CN113990459B (en) * | 2021-10-26 | 2024-08-09 | 卡本(深圳)医疗器械有限公司 | Method for fully automatically measuring lower sternum end/xiphoid process in CT/MRI (computed tomography/magnetic resonance imaging) volume |
| CN116704305A (en) * | 2023-06-20 | 2023-09-05 | 华中科技大学同济医学院附属协和医院 | Ultrasonic cardiogram multi-mode multi-section classification method based on deep learning algorithm |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111915562A (en) | Deep learning children echocardiogram standard tangent plane identification method and device | |
| CN107591200B (en) | Bone age mark identification and evaluation method and system based on deep learning and image omics | |
| CN110428432B (en) | Deep neural network algorithm for automatically segmenting colon gland image | |
| US11301996B2 (en) | Training neural networks of an automatic clinical workflow that recognizes and analyzes 2D and doppler modality echocardiogram images | |
| US8744152B2 (en) | Echocardiogram view classification using edge filtered scale-invariant motion features | |
| CN111915557B (en) | Deep learning atrial septal defect detection method and device | |
| Silberman et al. | Instance segmentation of indoor scenes using a coverage loss | |
| CN110543912B (en) | Method for automatically acquiring cardiac cycle video in fetal key section ultrasonic video | |
| KR100442834B1 (en) | Method and system for face detecting using classifier learned decision boundary with face/near-face images | |
| CN110298844B (en) | X-ray radiography image blood vessel segmentation and identification method and device | |
| Xia et al. | Extracting membrane borders in IVUS images using a multi-scale feature aggregated U-Net | |
| CN107993221B (en) | Automatic identification method for vulnerable plaque of cardiovascular Optical Coherence Tomography (OCT) image | |
| CN110765921B (en) | Video object positioning method based on weak supervised learning and video spatiotemporal features | |
| CN111493935A (en) | Artificial intelligence-based automatic prediction and identification method and system for echocardiogram | |
| CN111462049A (en) | Automatic lesion area form labeling method in mammary gland ultrasonic radiography video | |
| CN111814620A (en) | Face image quality evaluation model establishing method, optimization method, medium and device | |
| CN111274955A (en) | Emotion recognition method and system based on audio-visual feature correlation fusion | |
| CN111915626B (en) | Automatic segmentation method, device and storage medium for heart ultrasonic image ventricular region | |
| CN110992352A (en) | Automatic infant head circumference CT image measuring method based on convolutional neural network | |
| Zhang et al. | Artifact detection in endoscopic video with deep convolutional neural networks | |
| CN114882325A (en) | Semi-supervisor detection and training method and device based on two-stage object detector | |
| Khajontantichaikun et al. | Emotion Detection of Thai Elderly Facial Expressions using Hybrid Object Detection | |
| CN116524297B (en) | Weak supervision learning training method based on expert feedback | |
| CN117893843A (en) | Model training method, target detection device, and storage medium | |
| Xia et al. | Retinal vessel segmentation via a coarse-to-fine convolutional neural network |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20201110 |