CN115049660A - Method and device for positioning characteristic points of cardiac anatomical structure - Google Patents
Method and device for positioning characteristic points of cardiac anatomical structure Download PDFInfo
- Publication number
- CN115049660A CN115049660A CN202210971904.3A CN202210971904A CN115049660A CN 115049660 A CN115049660 A CN 115049660A CN 202210971904 A CN202210971904 A CN 202210971904A CN 115049660 A CN115049660 A CN 115049660A
- Authority
- CN
- China
- Prior art keywords
- image
- neural network
- module
- dimensional
- residual
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0012—Biomedical image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
- G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30004—Biomedical image processing
- G06T2207/30048—Heart; Cardiac
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- General Health & Medical Sciences (AREA)
- Health & Medical Sciences (AREA)
- Artificial Intelligence (AREA)
- Data Mining & Analysis (AREA)
- Radiology & Medical Imaging (AREA)
- Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
- Life Sciences & Earth Sciences (AREA)
- Medical Informatics (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Quality & Reliability (AREA)
- Evolutionary Computation (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Image Processing (AREA)
- Image Analysis (AREA)
Abstract
The invention provides a method and a device for positioning characteristic points of a heart anatomical structure, wherein the method comprises the following steps: acquiring an echocardiogram of the cardiac anatomy; performing data enhancement processing on the echocardiogram to form an enhanced image; generating a corresponding real heat map according to the enhanced image; inputting the enhanced image into a residual error neural network model for training, outputting a prediction heat map by the residual error neural network model according to the enhanced image, and calculating a loss function error value of the residual error neural network model according to the prediction heat map and a real heat map so as to perform gradient descent; repeating the steps to train a residual error neural network model; and positioning the characteristic points of the heart anatomical structure by adopting the trained residual error neural network model. The invention uses the heat map regression method to position the heart anatomical structure feature points, reduces the dependence on a huge training data set, lightens the burden of a marking doctor, and uses the residual error neural network to extract the heat map semantic information, thereby improving the accuracy of feature point positioning.
Description
Technical Field
The invention relates to the technical field of computer vision, in particular to a method and a device for positioning characteristic points of a cardiac anatomical structure.
Background
With the development of cardiac ultrasound technology, the requirement of medical image analysis is more and more extensive, and the analysis of a large batch of medical images in a short time is difficult to complete by depending on the manual labeling of the anatomical structure feature points of the medical images by doctors.
At present, although there is a medical image analysis technology based on deep learning, a coordinate regression method is generally adopted, the method needs a large amount of training data, needs a large amount of manpower preparation work, and is not deep enough in network, and in the face of complicated anatomical structures of the heart, it is difficult to grasp all semantic information of each structural feature point, and it is difficult to obtain accurate prediction with small error.
Disclosure of Invention
In order to solve the above technical problems, a first object of the present invention is to provide a method for locating feature points of cardiac anatomy.
A second object of the present invention is to provide a device for locating characteristic points of cardiac anatomy.
The technical scheme adopted by the invention is as follows:
an embodiment of the first aspect of the present invention provides a method for locating feature points of cardiac anatomical structures, including the following steps: acquiring an echocardiogram of a cardiac anatomy, the echocardiogram including: the feature points and semantic information corresponding to the feature points; performing data enhancement processing on the echocardiogram to form an enhanced image; generating a corresponding real heat map according to the enhanced image, wherein the real heat map is a multi-dimensional matrix with the same size as the enhanced image, a highlight area which is based on two-dimensional Gaussian distribution, takes the characteristic point as a central point and has a radius of r is designed in the real heat map aiming at each characteristic point of the enhanced image, and r is a positive integer; inputting the enhanced image into a residual error neural network model for training, outputting a prediction heat map by the residual error neural network model according to the enhanced image, calculating a loss function error value of the residual error neural network model according to the prediction heat map and the real heat map, and performing gradient descent according to the loss function error value; repeating the steps to train the residual error neural network model until the error value of the loss function reaches the set requirement; and positioning the characteristic points of the heart anatomical structure by using the trained residual neural network model.
The method for locating the heart anatomical structure feature points provided by the invention can also have the following additional technical features:
according to one embodiment of the invention, the residual neural network model comprises:
a 7 × 7 convolution kernel, wherein the 7 × 7 convolution kernel is used to perform a convolution operation on the enhanced image and perform a Batch Norm (an algorithm that keeps the input of each layer of the neural network in the same distribution during the training process) and Relu (linear rectification function) operations to obtain 64-dimensional image data; a Max Pooling module connected to the 7 × 7 convolution kernel, the Max Pooling module being configured to perform a Max Pooling operation on the 64-dimensional image data; the first residual learning module, the second residual learning module, the third residual learning module, the fourth residual learning module, the third residual learning module, the fourth residual learning module, the first image, the fourth image is 64 dimensional image data after Max Pooling, the fourth image is 64 dimensional image data respectively, the 128 dimensional image data is 64 dimensional image, the 128 dimensional image respectively, the fourth image, the first to obtain 64 dimensional image, the fourth image, the 256 dimensional image data respectively, the 64 dimensional image, the 128 dimensional image respectively; the first up-sampling module is used for performing up-sampling operation on the fourth image, then performing connection operation on the fourth image and the third image, and performing dimensionality reduction operation by using a 1 × 1 convolution kernel to generate a 256-dimensional 64 × 32 fifth image; the second up-sampling module is used for performing up-sampling operation on the fifth image, then performing connection operation on the fifth image and the second image, and then performing dimensionality reduction operation by using a 1 × 1 convolution kernel to generate a 128-dimensional 64 × 128 sixth image; the third up-sampling module is configured to perform connection operation on the sixth image after performing up-sampling operation on the sixth image and the first image, and then perform dimension reduction operation by using a 1 × 1 convolution kernel to generate a 64-dimensional 256 × 128 seventh image; a Sigmoid module for Sigmoid processing of the seventh image to generate the predicted heat map.
According to one embodiment of the invention, the Loss function of the residual neural network model adopts a Focal local Loss function.
According to an embodiment of the present invention, when the echocardiogram is a four-chamber view of the apex of the heart, a first highlight line is used to connect two right ventricular endocardium when the echocardiogram is generated, all positions on the first highlight line are defined as right ventricular endocardium points, a second highlight line is used to connect two right ventricular septa points, and all positions on the second highlight line are defined as right ventricular septa points.
According to an embodiment of the invention, the semantic information of the feature points includes apex labeling points, and the apex labeling points are designed with a Loss weight 3-10 times that of the apex labeling points when the residual neural network model is trained.
An embodiment of the second aspect of the present invention provides a device for locating a feature point of a cardiac anatomical structure, comprising: an acquisition module for acquiring an echocardiogram of a cardiac anatomy, the echocardiogram comprising: the feature points and semantic information corresponding to the feature points; an enhancement module for performing data enhancement processing on the echocardiogram to form an enhanced image; a generating module, configured to generate a corresponding real heat map according to the enhanced image, where the real heat map is a multidimensional matrix having the same size as the enhanced image, and a highlight region based on two-dimensional gaussian distribution, with the feature point as a central point and a radius of r is designed in the real heat map for each feature point of the enhanced image, where r is a positive integer; the training module is used for inputting the enhanced image into a residual error neural network model for training, the residual error neural network model outputs a prediction heat map according to the enhanced image, calculates a loss function error value of the residual error neural network model according to the prediction heat map and the real heat map, and performs gradient descent according to the loss function error value until the loss function error value reaches a set requirement; and the positioning module is used for positioning the characteristic points of the heart anatomical structure by adopting the trained residual error neural network model.
The above proposed device for locating the characteristic points of the cardiac anatomy structure can also have the following additional technical features:
according to one embodiment of the invention, the residual neural network model comprises: a 7 × 7 convolution kernel, the 7 × 7 convolution kernel to perform a convolution operation on the enhanced image and perform a Batch Norm and Relu operation to obtain 64-dimensional image data; a Max Pooling module, connected to the 7 × 7 convolution kernel, configured to perform Max Pooling operation on the 64-dimensional image data; the first residual learning module, the second residual learning module, the third residual learning module, the fourth residual learning module, the third residual learning module, the fourth residual learning module, the first image, the fourth image is 64 dimensional image data after Max Pooling, the fourth image is 64 dimensional image data respectively, the 128 dimensional image data is 64 dimensional image, the 128 dimensional image respectively, the fourth image, the first to obtain 64 dimensional image, the fourth image, the 256 dimensional image data respectively, the 64 dimensional image, the 128 dimensional image respectively; the first up-sampling module is used for performing up-sampling operation on the fourth image, then performing connection operation on the fourth image and the third image, and performing dimensionality reduction operation by using a 1 × 1 convolution kernel to generate a 256-dimensional 64 × 32 fifth image; the second up-sampling module is used for performing up-sampling operation on the fifth image, then performing connection operation on the fifth image and the second image, and then performing dimensionality reduction operation by using a 1 × 1 convolution kernel to generate a 128-dimensional 64 × 128 sixth image; the third up-sampling module is configured to perform connection operation on the sixth image after performing up-sampling operation on the sixth image and the first image, and then perform dimension reduction operation by using a 1 × 1 convolution kernel to generate a 64-dimensional 256 × 128 seventh image; a Sigmoid module to perform Sigmoid processing on the seventh image to generate the prediction heatmap.
According to one embodiment of the invention, the Loss function of the residual neural network model adopts a Focal local Loss function.
According to an embodiment of the present invention, when the echocardiogram is a four-chamber view of the apex of the heart, the generation module generates the real heat map by connecting two right ventricular endocardium points by using a first highlight line, all positions on the first highlight line are defined as right ventricular endocardium points, connecting two right ventricular epicardium points by using a second highlight line, and all positions on the second highlight line are defined as right ventricular epicardium points.
According to an embodiment of the invention, the semantic information of the feature points includes apex labeling points, and the training module designs a Loss weight 3-10 times of the apex labeling points when training the residual neural network model.
The invention has the beneficial effects that:
the invention positions the heart anatomical structure feature points by using the heat map regression method, greatly reduces the complexity of a training model function, reduces the dependence on a huge training data set, lightens the burden of a marking doctor, extracts heat map semantic information by using a residual error neural network, deepens the network depth by using the excellent learning capacity of the residual error neural network, extracts the semantic information in a multi-scale manner, improves the accuracy of feature point positioning, reduces errors and ensures the accuracy of medical image analysis.
Drawings
FIG. 1 is a flow chart of a method for locating features points of cardiac anatomy according to one embodiment of the present invention;
FIG. 2 is a block schematic diagram of a residual neural network model according to one embodiment of the present invention;
fig. 3 is a block schematic diagram of an apparatus for locating features of cardiac anatomy in accordance with an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be obtained by a person skilled in the art without making any creative effort based on the embodiments in the present invention, belong to the protection scope of the present invention.
Fig. 1 is a flow chart of a method for locating points of anatomical features of a heart, as shown in fig. 1, comprising the steps of:
s1, obtaining an echocardiogram of the cardiac anatomy, the echocardiogram including: and the feature points and semantic information corresponding to the feature points.
Specifically, a physician labels feature points (such as an apex marker, a right ventricular endocardium, a right ventricular septum, and the like) in advance on the echocardiogram, and represents different semantic information in different labeling forms, for example, different labeling shapes or different labeling colors are used to represent different semantic information, so that the labeled echocardiogram is directly obtained when the echocardiogram is used.
S2, the echocardiogram is subjected to data enhancement processing to form an enhanced image.
Specifically, data enhancement includes: scaling, translation, and rotation to unify the echocardiograms into 1024 × 512 as an input image of a residual neural network model described below.
And S3, generating a corresponding real heat map according to the enhanced image, wherein the real heat map is a multi-dimensional matrix with the same size as the enhanced image, and a highlight area which is based on two-dimensional Gaussian distribution, takes the feature point as a central point and has a radius of r is designed in the real heat map for each feature point of the enhanced image, and r is a positive integer.
Specifically, for each feature point in the echocardiogram, a highlight region with a radius r (the representative value is from 0 to 1 from the periphery to the center) based on the two-dimensional gaussian distribution and taking P as a center point is designed in the real heat map by using the two-dimensional gaussian distribution, for the sake of convenience of representation, the representative value of the center point of the region is the largest and is recorded as 1, the representative values gradually decrease to 0 as the periphery approaches, and the representative value is designed to mean the similarity degree of the coordinates of the point and the coordinates of the real feature point. After the heat map regression processing, the more the obtained predicted point coordinate is close to the real point coordinate, the more the representative value in the heat map is close to 1, the calculation of the accuracy of the predicted point by the model is greatly facilitated, compared with the traditional coordinate regression method, the heat map regression not only provides richer monitoring information in space, but also avoids the high nonlinearity of a function of network learning, and the requirement on data quantity is reduced.
That is, the real heat map is a matrix with the same scale as the enhanced image, the matrix further includes integer coordinates and no floating point number, and a highlight region is designed in the matrix for the feature points in the enhanced image and represented by a numerical value from 0 to 1.
And S4, inputting the enhanced image into a residual error neural network model for training, outputting a prediction heat map by the residual error neural network model according to the enhanced image, calculating a loss function error value of the residual error neural network model according to the prediction heat map and the real heat map, and performing gradient descent according to the loss function error value.
In an embodiment of the present invention, as shown in fig. 2, the residual neural network model specifically includes: the system comprises a 7 x 7 convolution kernel, a Max Pooling module, a first residual learning module, a second residual learning module, a third residual learning module, a first upsampling module, a second upsampling module, a third residual learning module, a Sigmoid module and a filter module.
Wherein, the convolution kernel of 7 × 7 is used for performing convolution operation on the enhanced image (1024 × 512) and performing Batch Norm and Relu operation to obtain 64-dimensional image data (512 × 256 × 64); the Max Pooling module is connected with the convolution kernel of 7 multiplied by 7 and used for carrying out Max Pooling operation on 64-dimensional image data so as to compress pictures, eliminate noise and reduce calculated amount; the first to fourth residual modules are respectively used for sequentially performing residual processing on the 64-dimensional image data after the Max Pooling operation to respectively obtain first to fourth images, wherein the first to fourth images are respectively 64-dimensional (256 × 128 × 64), 128-dimensional (128 × 64 × 128), 256-dimensional (64 × 32 × 256) and 512-dimensional (32 × 16 × 512) image data; the first up-sampling module is used for performing Concat connection operation on the fourth image after up-sampling operation is performed on the fourth image and the third image, and then performing dimensionality reduction operation by using a 1 × 1 convolution kernel to generate a 256-dimensional 64 × 32 fifth image; the second up-sampling module is used for performing Concat connection operation on the fifth image after up-sampling operation is performed on the fifth image and the second image, and then performing dimensionality reduction operation by using a 1 × 1 convolution kernel to generate a 128-dimensional 64 × 128 sixth image; the third up-sampling module is used for performing Concat connection operation on the sixth image after up-sampling operation is performed on the sixth image and the first image, and then performing dimensionality reduction operation by using a 1 × 1 convolution kernel to generate a 64-dimensional 256 × 128 seventh image; the Sigmoid module is used to perform Sigmoid processing on the seventh image to generate a prediction heat map (256 × 128).
And S5, repeating the steps (S1-S4) to train the residual neural network model until the loss function error value reaches the set requirement.
Specifically, the enhanced image is input into a residual error neural network model, the residual error neural network outputs a predicted heat map according to the enhanced image, meanwhile, a real heat map is used as an image regression target in a training model, the predicted heat map is compared with the real heat map, a loss function error value is calculated, gradient reduction is carried out, and finally a good regression effect is achieved.
And S6, positioning the feature points of the heart anatomical structure by using the trained residual neural network model.
Therefore, by adopting the heat map regression method, the function of the network model needing to be learned is simplified, the complexity of the training model function is reduced to the greatest extent, the dependence on a huge training data set is reduced, the burden of a marking doctor is reduced, the contrast relation between a foreground (characteristic point) and a background map is easily captured, the residual error neural network is used for extracting the heat map semantic information, the network depth is deepened by using the excellent learning capability of the residual error neural network, the problems of gradient disappearance, gradient explosion and the like which possibly occur are solved, the characteristic information in the map is extracted in a multi-scale mode, the accuracy of the positioning of the characteristic point is improved, the error is reduced, and the accuracy of medical image analysis is ensured.
In one embodiment of the invention, the Loss function of the residual neural network model adopts a Focal local Loss function to balance positive and negative samples and accelerate regression.
In an embodiment of the invention, the semantic information of the feature points comprises apex annotation points, and because the accuracy of each anatomical structure feature point in positioning is different, and the apex annotation points are most likely to cause errors, when a residual neural network model is trained, the apex annotation points are designed with Loss weights 3-10 times, so that when the model is trained, the positioning of the apex annotation points is learnt emphatically.
In an embodiment of the present invention, the anatomical structure feature points of different semantic information correspond to different real heat maps, where when the echocardiogram is a four-chamber view of the apex of the heart, a first highlight line is used to connect two right ventricular endocardium when generating the real heat map, all positions on the first highlight line are defined as right ventricular endocardium points, a second highlight line is used to connect two right ventricular septa points, and all positions on the second highlight line are defined as right ventricular septa points. The first highlight line and the second highlight line are different in color.
Specifically, different heat maps are designed for anatomical structure feature points with different semantic information, in order to reduce the deviation of labeling points of the apex of the heart, endocardium of the right ventricle and septa of the right ventricle, in four-cavity views of the apex of the heart, the heat maps of the endocardium of the right ventricle and the septa of the right ventricle are modified, a first highlight line is adopted to connect the two endocardium of the right ventricle, so that all positions on the line can be considered as endocardium points of the right ventricle, a second highlight line is adopted to connect the two endocardium points of the right ventricle, all positions on the line can be considered as right ventricular septa points, and the medical semantic information of the endocardium is more attached.
That is to say, the method can modify the heat map, design the correlation of a plurality of feature points, guide the network to learn, obtain better effect and keep more semantic information.
In summary, according to the method for locating the characteristic points of the cardiac anatomical structure provided by the embodiment of the present invention, the characteristic points of the cardiac anatomical structure are located by using a heat map regression method, so that the complexity of a training model function is greatly reduced, the dependence on a huge training data set is reduced, the burden of a marking physician is reduced, a residual error neural network is used to extract heat map semantic information, the learning ability of the residual error neural network is used to deepen the depth of the network, the semantic information is extracted in a multi-scale manner, the accuracy of the feature point location is improved, the error is reduced, and the accuracy of medical image analysis is ensured.
Corresponding to the above method for locating the feature points of the cardiac anatomical structure, the present invention further provides a block diagram of a device for locating the feature points of the cardiac anatomical structure, and details that are not disclosed in the device embodiment may refer to the above method embodiment, and are not described again in the present invention.
Fig. 3 is a block schematic diagram of an apparatus for locating features of cardiac anatomy according to one embodiment of the present invention, as shown in fig. 3, comprising: the system comprises an acquisition module 1, an enhancement module 2, a generation module 3, a training module 4 and a positioning module 5.
Wherein the acquisition module 1 is configured to acquire an echocardiogram of the cardiac anatomical structure, the echocardiogram comprising: the feature points and semantic information corresponding to the feature points; the enhancement module 2 is used for performing data enhancement processing on the echocardiogram to form an enhanced image; the generating module 3 is used for generating a corresponding real heat map according to the enhanced image, the real heat map is a multi-dimensional matrix with the same size as the enhanced image, a highlight area which is based on two-dimensional Gaussian distribution, takes the feature point as a central point and has a radius of r is designed in the real heat map for each feature point of the enhanced image, and r is a positive integer; the training module 4 is used for inputting the enhanced image into the residual error neural network model for training, the residual error neural network model outputs a prediction heat map according to the enhanced image, calculates a loss function error value of the residual error neural network model according to the prediction heat map and the real heat map, and performs gradient descent according to the loss function error value until the loss function error value reaches a set requirement; the positioning module 5 is used for positioning the heart anatomical structure feature points by adopting the trained residual error neural network model.
According to one embodiment of the invention, the residual neural network model comprises: 7 × 7 convolution kernels, wherein the 7 × 7 convolution kernels are used for performing convolution operation on the enhanced image and performing Batch Norm and Relu operation to obtain 64-dimensional image data; the Max Pooling module is connected with the 7 multiplied by 7 convolution kernel and used for carrying out Max Pooling operation on 64-dimensional image data; the first residual learning module, the second residual learning module, the third residual learning module, the fourth residual learning module, the third residual learning module, the fourth residual learning module and the fourth residual learning module are respectively used for sequentially carrying out residual processing on 64-dimensional image data after Max Pooling operation to respectively obtain first images, second images and fourth images, wherein the first images, the second images and the fourth images are respectively 64-dimensional image data, 128-dimensional image data, 256-dimensional image data and 512-dimensional image data; the first up-sampling module is used for performing up-sampling operation on a fourth image, then performing connection operation on the fourth image and a third image, and performing dimensionality reduction operation by using a 1 × 1 convolution kernel to generate a 256-dimensional 64 × 32 fifth image; the second up-sampling module is used for performing up-sampling operation on the fifth image, then performing connection operation on the fifth image and the second image, and then performing dimensionality reduction operation by using a 1 × 1 convolution kernel to generate a 128-dimensional 64 × 128 sixth image; the third up-sampling module is used for performing up-sampling operation on the sixth image, then performing connection operation on the sixth image and the first image, and then performing dimensionality reduction operation by using a 1 × 1 convolution kernel to generate a 64-dimensional 256 × 128 seventh image; a Sigmoid module to Sigmoid process the seventh image to generate a predicted heat map.
According to one embodiment of the invention, the Loss function of the residual neural network model adopts a Focal local Loss function.
According to an embodiment of the present invention, the anatomical feature points of different semantic information correspond to different real heat maps, wherein when the echocardiogram is a four-chamber view of the cardiac apex, the generation module 3 generates the real heat maps by connecting the two right ventricular endocardium points by using a first highlight line, all the positions on the first highlight line are defined as right ventricular endocardium points, connecting the two right ventricular septum points by using a second highlight line, and all the positions on the second highlight line are defined as right ventricular septum points.
According to one embodiment of the invention, the semantic information of the feature points comprises apex annotation points, and the training module designs a Loss weight 3-10 times of the apex annotation points when training the residual neural network model.
In summary, according to the positioning device for the cardiac anatomical structure feature points of the embodiment of the present invention, the cardiac anatomical structure feature points are positioned by using the heat map regression method, so that the complexity of the training model function is greatly reduced, the dependence on a huge training data set is reduced, the burden of the annotating physician is reduced, the residual error neural network is used to extract the heat map semantic information, the learning ability of the residual error neural network is used to deepen the network depth, the semantic information is extracted in multiple scales, the accuracy of feature point positioning is improved, the error is reduced, and the accuracy of medical image analysis is ensured.
In the description of the present invention, the terms "first" and "second" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance or implying any number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. The meaning of "plurality" is two or more unless specifically limited otherwise.
In the description herein, references to the description of the term "one embodiment," "some embodiments," "an example," "a specific example," or "some examples," etc., mean that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the invention. In this specification, the schematic representations of the terms used above are not necessarily intended to refer to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples. Furthermore, various embodiments or examples and features of different embodiments or examples described in this specification can be combined and combined by one skilled in the art without contradiction. In the description herein, references to the description of the term "one embodiment," "some embodiments," "an example," "a specific example," or "some examples," etc., mean that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the invention. In this specification, the schematic representations of the terms used above are not necessarily intended to refer to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples. Furthermore, various embodiments or examples and features of different embodiments or examples described in this specification can be combined and combined by one skilled in the art without contradiction.
Any process or method descriptions in flow charts or otherwise described herein may be understood as representing modules, segments, or portions of code which include one or more executable instructions for implementing steps of a custom logic function or process, and alternate implementations are included within the scope of the preferred embodiment of the present invention in which functions may be executed out of order from that shown or discussed, including substantially concurrently or in reverse order, depending on the functionality involved, as would be understood by those reasonably skilled in the art of the present invention.
The logic and/or steps represented in the flowcharts or otherwise described herein, e.g., an ordered listing of executable instructions that can be considered to implement logical functions, can be embodied in any computer-readable medium for use by or in connection with an instruction execution system, apparatus, or device, such as a computer-based system, processor-containing system, or other system that can fetch the instructions from the instruction execution system, apparatus, or device and execute the instructions. For the purposes of this description, a "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of the computer-readable medium would include the following: an electrical connection (electronic device) having one or more wires, a portable computer diskette (magnetic device), a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disc read-only memory (CDROM). Additionally, the computer-readable medium could even be paper or another suitable medium upon which the program is printed, as the program can be electronically captured, via for instance optical scanning of the paper or other medium, then compiled, interpreted or otherwise processed in a suitable manner if necessary, and then stored in a computer memory.
It should be understood that portions of the present invention may be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, the various steps or methods may be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. If implemented in hardware, as in another embodiment, any one or combination of the following techniques, which are known in the art, may be used: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, an application specific integrated circuit having an appropriate combinational logic gate circuit, a Programmable Gate Array (PGA), a Field Programmable Gate Array (FPGA), or the like.
It will be understood by those skilled in the art that all or part of the steps carried by the method for implementing the above embodiments may be implemented by hardware related to instructions of a program, which may be stored in a computer readable storage medium, and when the program is executed, the program includes one or a combination of the steps of the method embodiments.
In addition, functional units in the embodiments of the present invention may be integrated into one processing module, or each unit may exist alone physically, or two or more units are integrated into one module. The integrated module can be realized in a hardware mode, and can also be realized in a software functional module mode. The integrated module, if implemented in the form of a software functional module and sold or used as a stand-alone product, may also be stored in a computer readable storage medium.
The storage medium mentioned above may be a read-only memory, a magnetic or optical disk, etc. Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention, and that variations, modifications, substitutions and alterations can be made to the above embodiments by those of ordinary skill in the art within the scope of the present invention.
Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that changes, modifications, substitutions and alterations can be made in these embodiments without departing from the principles and spirit of the invention, the scope of which is defined in the appended claims and their equivalents.
Claims (10)
1. A method for locating a feature point of an anatomical structure of a heart, comprising the steps of:
acquiring an echocardiogram of a cardiac anatomy, the echocardiogram including: the feature points and semantic information corresponding to the feature points;
performing data enhancement processing on the echocardiogram to form an enhanced image;
generating a corresponding real heat map according to the enhanced image, wherein the real heat map is a multi-dimensional matrix with the same size as the enhanced image, each feature point of the enhanced image is provided with a highlight area which is based on two-dimensional Gaussian distribution, takes the feature point as a central point and has a radius of r in the real heat map, and r is a positive integer;
inputting the enhanced image into a residual error neural network model for training, outputting a prediction heat map by the residual error neural network model according to the enhanced image, calculating a loss function error value of the residual error neural network model according to the prediction heat map and the real heat map, and performing gradient descent according to the loss function error value;
repeating the steps to train the residual error neural network model until the error value of the loss function reaches the set requirement;
and positioning the characteristic points of the heart anatomical structure by adopting the trained residual error neural network model.
2. The method of claim 1, wherein the residual neural network model comprises:
a 7 × 7 convolution kernel, the 7 × 7 convolution kernel to perform a convolution operation on the enhanced image and perform a Batch Norm and Relu operation to obtain 64-dimensional image data;
a Max Pooling module, connected to the 7 × 7 convolution kernel, configured to perform Max Pooling operation on the 64-dimensional image data;
the first residual learning module, the second residual learning module, the third residual learning module, the fourth residual learning module, the third residual learning module, the fourth residual learning module, the first image, the fourth image is 64 dimensional image data after Max Pooling, the fourth image is 64 dimensional image data respectively, the 128 dimensional image data is 64 dimensional image, the 128 dimensional image respectively, the fourth image, the first to obtain 64 dimensional image, the fourth image, the 256 dimensional image data respectively, the 64 dimensional image, the 128 dimensional image respectively;
the first up-sampling module is used for performing up-sampling operation on the fourth image, then performing connection operation on the fourth image and the third image, and performing dimensionality reduction operation by using a 1 × 1 convolution kernel to generate a 256-dimensional 64 × 32 fifth image; the second up-sampling module is used for performing up-sampling operation on the fifth image, then performing connection operation on the fifth image and the second image, and then performing dimensionality reduction operation by using a 1 × 1 convolution kernel to generate a 128-dimensional 64 × 128 sixth image; the third up-sampling module is configured to perform connection operation on the sixth image after performing up-sampling operation on the sixth image and the first image, and then perform dimension reduction operation by using a 1 × 1 convolution kernel to generate a 64-dimensional 256 × 128 seventh image;
a Sigmoid module to perform Sigmoid processing on the seventh image to generate the prediction heatmap.
3. The method of claim 2, wherein the Loss function of the residual neural network model is a Focal local Loss function.
4. The method according to claim 3, wherein the anatomical feature points of different semantic information correspond to different real heat maps, wherein,
when the echocardiogram is a four-chamber view of the apex of the heart, a first highlight line is adopted to connect two right ventricular endocardium when the real heat map is generated, all positions on the first highlight line are defined as right ventricular endocardium points, a second highlight line is adopted to connect two right ventricular septa points, and all positions on the second highlight line are defined as right ventricular septa points.
5. The method as claimed in claim 3, wherein the semantic information of the feature points includes apex labeling points, and the apex labeling points are weighted 3-10 times less when training the residual neural network model.
6. An apparatus for locating a feature point of a cardiac anatomical structure, comprising:
an acquisition module for acquiring an echocardiogram of a cardiac anatomy, the echocardiogram comprising: the feature points and semantic information corresponding to the feature points;
an enhancement module for performing data enhancement processing on the echocardiogram to form an enhanced image;
a generating module, configured to generate a corresponding real heat map according to the enhanced image, where the real heat map is a multidimensional matrix having the same size as the enhanced image, and a highlight region based on two-dimensional gaussian distribution, with the feature point as a central point and a radius of r is designed in the real heat map for each feature point of the enhanced image, where r is a positive integer;
the training module is used for inputting the enhanced image into a residual error neural network model for training, the residual error neural network model outputs a prediction heat map according to the enhanced image, calculates a loss function error value of the residual error neural network model according to the prediction heat map and the real heat map, and performs gradient descent according to the loss function error value until the loss function error value reaches a set requirement;
and the positioning module is used for positioning the characteristic points of the heart anatomical structure by adopting the trained residual error neural network model.
7. The apparatus for locating cardiac anatomical feature points according to claim 6, wherein the residual neural network model comprises:
a 7 × 7 convolution kernel, the 7 × 7 convolution kernel to perform a convolution operation on the enhanced image and perform a Batch Norm and Relu operation to obtain 64-dimensional image data;
a Max Pooling module, connected to the 7 × 7 convolution kernel, configured to perform Max Pooling operation on the 64-dimensional image data;
the first residual learning module, the second residual learning module, the third residual learning module, the fourth residual learning module, the third residual learning module, the fourth residual learning module, the first image, the fourth image is 64 dimensional image data after Max Pooling, the fourth image is 64 dimensional image data respectively, the 128 dimensional image data is 64 dimensional image, the 128 dimensional image respectively, the fourth image, the first to obtain 64 dimensional image, the fourth image, the 256 dimensional image data respectively, the 64 dimensional image, the 128 dimensional image respectively;
the first up-sampling module is used for performing up-sampling operation on the fourth image, then performing connection operation on the fourth image and the third image, and performing dimensionality reduction operation by using a 1 × 1 convolution kernel to generate a 256-dimensional 64 × 32 fifth image; the second up-sampling module is used for performing up-sampling operation on the fifth image, then performing connection operation on the fifth image and the second image, and then performing dimensionality reduction operation by using a 1 × 1 convolution kernel to generate a 128-dimensional 64 × 128 sixth image; the third up-sampling module is configured to perform up-sampling on the sixth image, perform connection operation on the sixth image and the first image, and perform dimension reduction operation using a 1 × 1 convolution kernel to generate a 64-dimensional 256 × 128 seventh image;
a Sigmoid module for performing Sigmoid processing on the seventh image to generate the prediction heatmap.
8. The apparatus for locating cardiac anatomical feature points according to claim 7, wherein the Loss function of the residual neural network model is a Focal local Loss function.
9. The apparatus for locating anatomical feature points of the heart according to claim 8, wherein the anatomical feature points of different semantic information correspond to different real heat maps,
when the echocardiogram is a four-chamber view of the apex of the heart, the generation module generates the real heat map and connects the two right ventricular endocardium points by adopting a first highlight line, all positions on the first highlight line are defined as right ventricular endocardium points, the two right ventricular septum points are connected by adopting a second highlight line, and all positions on the second highlight line are defined as right ventricular septum points.
10. The apparatus as claimed in claim 8, wherein the semantic information of the feature points includes apex label points, and the training module designs a Loss weight 3-10 times of the apex label points when training the residual neural network model.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210971904.3A CN115049660B (en) | 2022-08-15 | 2022-08-15 | Method and device for positioning characteristic points of cardiac anatomical structure |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210971904.3A CN115049660B (en) | 2022-08-15 | 2022-08-15 | Method and device for positioning characteristic points of cardiac anatomical structure |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115049660A true CN115049660A (en) | 2022-09-13 |
| CN115049660B CN115049660B (en) | 2022-11-29 |
Family
ID=83167401
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210971904.3A Active CN115049660B (en) | 2022-08-15 | 2022-08-15 | Method and device for positioning characteristic points of cardiac anatomical structure |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115049660B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115761381A (en) * | 2022-12-14 | 2023-03-07 | 安徽鲲隆康鑫医疗科技有限公司 | Method and device for classifying echocardiograms |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2015045595A (en) * | 2013-08-29 | 2015-03-12 | 日本メジフィジックス株式会社 | Visualization of heart muscle motion |
| CN110675406A (en) * | 2019-09-16 | 2020-01-10 | 南京信息工程大学 | CT image kidney segmentation algorithm based on residual double-attention depth network |
| CN111353978A (en) * | 2020-02-26 | 2020-06-30 | 合肥凯碧尔高新技术有限公司 | Method and device for identifying cardiac anatomical structure |
| CN111652054A (en) * | 2020-04-21 | 2020-09-11 | 北京迈格威科技有限公司 | Joint point detection method, posture recognition method and device |
| CN113284136A (en) * | 2021-06-22 | 2021-08-20 | 南京信息工程大学 | Medical image classification method of residual error network and XGboost of double-loss function training |
| CN114010227A (en) * | 2021-11-02 | 2022-02-08 | 中国科学技术大学 | Right ventricle characteristic information identification method and device |
-
2022
- 2022-08-15 CN CN202210971904.3A patent/CN115049660B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2015045595A (en) * | 2013-08-29 | 2015-03-12 | 日本メジフィジックス株式会社 | Visualization of heart muscle motion |
| CN110675406A (en) * | 2019-09-16 | 2020-01-10 | 南京信息工程大学 | CT image kidney segmentation algorithm based on residual double-attention depth network |
| CN111353978A (en) * | 2020-02-26 | 2020-06-30 | 合肥凯碧尔高新技术有限公司 | Method and device for identifying cardiac anatomical structure |
| CN111652054A (en) * | 2020-04-21 | 2020-09-11 | 北京迈格威科技有限公司 | Joint point detection method, posture recognition method and device |
| CN113284136A (en) * | 2021-06-22 | 2021-08-20 | 南京信息工程大学 | Medical image classification method of residual error network and XGboost of double-loss function training |
| CN114010227A (en) * | 2021-11-02 | 2022-02-08 | 中国科学技术大学 | Right ventricle characteristic information identification method and device |
Non-Patent Citations (2)
| Title |
|---|
| FATEMEH TAHERI DEZAKI等: ""Cardiac Phase Detection in Echocardiograms With Densely Gated Recurrent Neural Networks and Global Extrema Loss"", 《IEEE TRANSACTIONS ON MEDICAL IMAGING》 * |
| 陶攀等: "基于深度学习的超声心动图切面识别方法", 《计算机应用》 * |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115761381A (en) * | 2022-12-14 | 2023-03-07 | 安徽鲲隆康鑫医疗科技有限公司 | Method and device for classifying echocardiograms |
| CN115761381B (en) * | 2022-12-14 | 2023-11-07 | 安徽鲲隆康鑫医疗科技有限公司 | Method and device for classifying echocardiogram |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115049660B (en) | 2022-11-29 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112992308B (en) | Training method of medical image report generation model and image report generation method | |
| CN108022238B (en) | Method, computer storage medium, and system for detecting object in 3D image | |
| CN106056595B (en) | Based on the pernicious assistant diagnosis system of depth convolutional neural networks automatic identification Benign Thyroid Nodules | |
| CN109166133B (en) | Soft tissue organ image segmentation method based on key point detection and deep learning | |
| US20230104173A1 (en) | Method and system for determining blood vessel information in an image | |
| Nurmaini et al. | Accurate detection of septal defects with fetal ultrasonography images using deep learning-based multiclass instance segmentation | |
| US10853409B2 (en) | Systems and methods for image search | |
| Li et al. | Automated measurement network for accurate segmentation and parameter modification in fetal head ultrasound images | |
| CN103854276A (en) | Image registration device and method, image segmentation device and method and medical image device | |
| CN112614133B (en) | Three-dimensional pulmonary nodule detection model training method and device without anchor point frame | |
| CN112308846B (en) | Blood vessel segmentation method and device and electronic equipment | |
| CN114693933A (en) | Medical image segmentation device based on generation of confrontation network and multi-scale feature fusion | |
| de Siqueira et al. | Artificial intelligence applied to support medical decisions for the automatic analysis of echocardiogram images: A systematic review | |
| CN115049660B (en) | Method and device for positioning characteristic points of cardiac anatomical structure | |
| US20200357162A1 (en) | Modeling method, apparatus, device and storage medium of dynamic cardiovascular system | |
| CN117392473A (en) | Interpretable medical image classification system based on multi-modal prototype network | |
| Yong et al. | Automatic ventricular nuclear magnetic resonance image processing with deep learning | |
| CN113344873A (en) | Blood vessel segmentation method, device and computer readable medium | |
| CN112258476A (en) | Echocardiography myocardial abnormal motion mode analysis method, system and storage medium | |
| CN111599444A (en) | Intelligent tongue diagnosis detection method and device, intelligent terminal and storage medium | |
| Yu et al. | Pyramid feature adaptation for semi-supervised cardiac bi-ventricle segmentation | |
| CN112862786B (en) | CTA image data processing method, device and storage medium | |
| CN112914610B (en) | Contrast-enhanced echocardiography wall thickness automatic analysis system and method based on deep learning | |
| CN115471512A (en) | Medical image segmentation method based on self-supervision contrast learning | |
| CN115294023A (en) | Liver tumor automatic segmentation method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |