CN112085711B - Method for automatically tracking muscle feather angle by combining convolutional neural network and Kalman filtering - Google Patents
Method for automatically tracking muscle feather angle by combining convolutional neural network and Kalman filtering Download PDFInfo
- Publication number
- CN112085711B CN112085711B CN202010865743.0A CN202010865743A CN112085711B CN 112085711 B CN112085711 B CN 112085711B CN 202010865743 A CN202010865743 A CN 202010865743A CN 112085711 B CN112085711 B CN 112085711B
- Authority
- CN
- China
- Prior art keywords
- angle
- muscle
- neural network
- image
- convolutional neural
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 49
- 210000003205 muscle Anatomy 0.000 title claims abstract description 37
- 210000003746 feather Anatomy 0.000 title claims abstract description 28
- 238000001914 filtration Methods 0.000 title claims abstract description 28
- 238000013527 convolutional neural network Methods 0.000 title claims abstract description 16
- 210000001087 myotubule Anatomy 0.000 claims abstract description 65
- 238000004422 calculation algorithm Methods 0.000 claims abstract description 26
- 238000001514 detection method Methods 0.000 claims abstract description 17
- 238000007781 pre-processing Methods 0.000 claims abstract description 4
- 229910052704 radon Inorganic materials 0.000 claims description 17
- SYUHGPGVQRZVTB-UHFFFAOYSA-N radon atom Chemical compound [Rn] SYUHGPGVQRZVTB-UHFFFAOYSA-N 0.000 claims description 17
- 238000013528 artificial neural network Methods 0.000 claims description 13
- 239000013598 vector Substances 0.000 claims description 12
- 238000003708 edge detection Methods 0.000 claims description 10
- 230000009466 transformation Effects 0.000 claims description 10
- 238000005259 measurement Methods 0.000 claims description 8
- 230000008569 process Effects 0.000 claims description 7
- 102100028717 Cytosolic 5'-nucleotidase 3A Human genes 0.000 abstract description 11
- 241000219745 Lupinus Species 0.000 abstract description 11
- 238000002372 labelling Methods 0.000 abstract description 5
- 238000012937 correction Methods 0.000 abstract description 3
- 238000002604 ultrasonography Methods 0.000 description 12
- 230000006870 function Effects 0.000 description 8
- 238000013135 deep learning Methods 0.000 description 7
- 238000012549 training Methods 0.000 description 6
- 238000012545 processing Methods 0.000 description 4
- 238000003672 processing method Methods 0.000 description 3
- 238000004458 analytical method Methods 0.000 description 2
- 239000011159 matrix material Substances 0.000 description 2
- 238000012952 Resampling Methods 0.000 description 1
- 230000006978 adaptation Effects 0.000 description 1
- 238000007792 addition Methods 0.000 description 1
- 238000013459 approach Methods 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 238000002591 computed tomography Methods 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000013136 deep learning model Methods 0.000 description 1
- 238000003745 diagnosis Methods 0.000 description 1
- 201000010099 disease Diseases 0.000 description 1
- 208000037265 diseases, disorders, signs and symptoms Diseases 0.000 description 1
- 230000009977 dual effect Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 239000000835 fiber Substances 0.000 description 1
- 238000007667 floating Methods 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 238000012886 linear function Methods 0.000 description 1
- 230000004807 localization Effects 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 239000000203 mixture Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 230000000877 morphologic effect Effects 0.000 description 1
- 201000000585 muscular atrophy Diseases 0.000 description 1
- 230000004092 musculoskeletal function Effects 0.000 description 1
- 230000001537 neural effect Effects 0.000 description 1
- 239000002245 particle Substances 0.000 description 1
- 231100000915 pathological change Toxicity 0.000 description 1
- 230000036285 pathological change Effects 0.000 description 1
- 238000004393 prognosis Methods 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 230000001629 suppression Effects 0.000 description 1
- 230000026676 system process Effects 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
- 210000001519 tissue Anatomy 0.000 description 1
- 238000011426 transformation method Methods 0.000 description 1
- 230000007704 transition Effects 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0012—Biomedical image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/13—Edge detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10132—Ultrasound image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20024—Filtering details
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
Landscapes
- Engineering & Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Physics & Mathematics (AREA)
- Medical Informatics (AREA)
- Quality & Reliability (AREA)
- Radiology & Medical Imaging (AREA)
- Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Image Processing (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a method for automatically tracking a muscle feather angle by combining a convolutional neural network and Kalman filtering, which specifically comprises the following steps: (1) preprocessing an ultrasonic image; (2) myomembrane detection; (3) obtaining a myofiber direction observation value; (4) myofiber direction correction; and (5) calculating the feather angle. The invention utilizes the deep convolutional neural network to measure the direction of the current muscle fiber, and combines the direction with the Kalman filter to realize the tracking of the feather angle. The method improves the robustness of the lupin angle calculation algorithm, expands the application field of the automatic lupin angle labeling algorithm, and provides a method for automatically tracking the lupin angle for an ultrasonic image sequence with poor quality.
Description
Technical Field
The invention relates to the technical field of muscle feather angle detection, in particular to a method for automatically tracking muscle feather angle by combining a convolutional neural network and Kalman filtering.
Background
The muscle bundle feathering angle is an important parameter related to the musculoskeletal function, and changes when the muscle stretches or contracts. Detection of the feathered angle of the muscle can detect the pathological changes of the muscle at an early stage. Ultrasound images for specific quantitative measurement of morphological parameters of muscle tissue are defined as Sonomyograms (SMG) from which muscle structural parameters such as muscle cross-sectional area, muscle thickness, feather angle and muscle fiber length can be obtained.
Deep learning is used as a hot spot in the machine learning field in recent years, and by virtue of the ultrahigh prediction accuracy in identification application, great attention is paid to the image processing field, so that the performance of the existing image processing system can be improved, a new application field can be created, and the method is also very suitable for analyzing and detecting medical images. The method has strong automatic feature extraction and complex model construction capability, can avoid complicated manual feature labeling, effectively utilizes unsupervised data, has excellent generalization capability, and can be applied to different medical fields. Particularly, for medical images with relatively large noise, the conversion of the medical images into visual problems of deep learning can greatly improve the performance of a medical analysis and detection system, so that the medical analysis and detection system is widely focused by researchers in the medical field.
Kalman filtering (Kalman filtering) is an algorithm that uses a linear system state equation to optimally estimate the state of the system by inputting and outputting observed data through the system. A typical example of kalman filtering predicts the coordinate position and velocity of an object from a finite set of observed sequences of object positions, including noise. Its statue can be found in many engineering applications (radar, computer vision). The feathering of the muscles is continuous, regularly circulated, and thus, if kalman filter tracking is suitably applied to a tracking system of the feathering.
Clinicians typically rely on manual labeling to determine these muscle structure parameters, which is time consuming, laborious, and subjective and may affect the accuracy of the parameters. The prior art is mainly based on an image processing method, and has the following steps:
(1) A method of tracking the direction of muscle fibers in a muscle ultrasound image based on hough transform was proposed in 2007. This study proposes an improved hough transform, a voting-based hough transform (RVHT), aimed at automatically estimating the direction of a straight-line pattern, such as muscle fibers and muscle-bone interfaces in ultrasound images. Firstly, the original ultrasonic image is required to be subjected to edge detection, the data volume is greatly reduced by the edge detection, irrelevant information can be considered to be removed, and important structural attributes of the ultrasonic image of the muscle, namely the texture of the muscle, are reserved. Next, the edge map of the ultrasound image is converted into hough space and a global peak is found that corresponds to the straight line position in image space. Then, all feature points close to the detection line are removed from the edge map, and the same process of searching for another straight line is performed using the updated edge map. The iteration is repeated until a predetermined termination condition is met. Thus, the method will extract straight lines in the edge map one by one until the maximum in Hough space is below a prescribed threshold.
(2) Muscle fibers in the muscle ultrasound image are tracked by Radon Transform (RT). Firstly, the original ultrasonic image is subjected to edge detection, the data volume is reduced, and important structural attributes of the muscle ultrasonic image are reserved. The edge detection converts the original gray image into a binary black-and-white edge image, and a local radon transform algorithm is used for the detected edge image. The radon transform is an integral transform that transforms a two-dimensional plane function f into a linear function Rf defined in two dimensions, with the value of Rf being the value of the function f integrating the line Rf. For detection of muscle bundles, some a priori knowledge is used for localization. For example, the myofibers must lie between the superficial and deep myomembrane lines, which generally have a high echogenic intensity and are relatively easy to detect. Myomembrane lines can be first extracted over the entire image. Then, clusters of bundles can be found in the areas between the myomembrane wires. The area may be further divided into smaller portions until the muscle bundles are properly detected.
(3) And (3) adding a Bayesian Kalman filtering algorithm on the basis of the step (2) to track the direction of the muscle fiber. First, the direction of muscle fibers from the current frame to the next frame is predicted using a Bayesian Kalman Filter (BKF) tracking strategy. BKF adopts Gaussian Mixture (GM) to represent system state, and performs optimal state estimation according to observed value and given state dynamics. In addition, it employs a new direct density reduction algorithm that avoids the use of resampling techniques in conventional Particle Filtering (PF). Finally, fiber orientation is extracted from the enhanced image using a set of local radon transforms to provide observations for BKF correction.
In the prior art, (1) angle of myomembrane line and myofiber bundle is calculated on an ultrasonic image by voting Hough transformation, and although Hough transformation works well in positioning myomembranes, the myofibers cannot be well positioned; (2) the angles of the myomembrane and the myofiber bundles are calculated by using the local radon transformation of the ultrasonic image, and when the myofibers are unclear and the noise is large, the radon transformation performance is obviously reduced; (3) the Kalman filter is added to improve the performance of the method based on the original method, but the method also depends on the measured value of the traditional image processing method, namely the detection of local radon transformation, so the limitation is still larger. The muscle fiber has the characteristics of time-out and time-out, and the robustness of the image processing-based method is not strong.
Disclosure of Invention
In order to solve the technical problems, the invention discloses a method for automatically tracking the feather angle of a muscle by combining a convolutional neural network and Kalman filtering.
In order to achieve the above purpose, the present invention adopts the following technical scheme:
a method for automatically tracking a muscle feather angle by combining a convolutional neural network and Kalman filtering specifically comprises the following steps:
step (1), preprocessing an ultrasonic image;
step (2), myomembrane detection;
step (3), obtaining a myofiber direction observation value;
step (4), correcting the direction of the muscle fiber;
and (5) calculating the feather angle.
Further, in the step (2), the myomembrane detection specifically includes the following steps:
(21) Performing Gaussian filtering operation on the image by adopting a Canni edge detection algorithm to enable the image to be smooth; selecting an operator to calculate the gradient value and gradient direction of the image to obtain a brightness difference approximation value;
(22) Carrying out local radon transformation on the edge map;
(23) Marking the myofilm by a straight line with the maximum probability of the point position gradient map, and calculating the vector of the myofilm
Further, in the step (21), the following formula is adopted in the canny edge detection algorithm:
wherein, A is the original image, G x G (G) y Respectively, images detected by transverse and longitudinal edges, and θ is the gradient direction.
Further, in step (23), the edge map is subjected to local radon transform, and the following formula is adopted:
where, in the x-y image plane, F (x, y) is the image intensity at the location (x, y), δ is the dirac function, ρ is the distance from the origin to the line, θ is the angle between the x-axis and the line perpendicular to the line from the origin.
Further, in the step (3), a reference line is marked randomly, then whether the reference line is parallel to the direction of the muscle fiber is judged through the neural network, if not, adjustment is carried out until the neural network judges that the reference line is parallel, and the angle is recorded and used as an observation value of Kalman filtering.
Further, in the step (4), a kalman filter tracking system is used to correct the direction of the muscle fiber of the current frame according to the direction of the muscle fiber of the previous frame and the measurement of the angle of the muscle fiber in the step (3), and the direction of the muscle fiber of the next frame is predicted.
Further, the step (4) specifically includes:
(41) Predicting a system of a next state using a process model of the system;
(42) Combining the predicted value and the measured value obtained in the last step to obtain the optimized estimated value of the current state, namely the optimized myofiber vector
(43) And obtaining the optimal estimated value in the current state, and updating the covariance.
Further, in the step (5), the feather angle value is calculated by the vector of the myomembrane line and the myofiber bundle line:
wherein θ is the gradient direction, myomembrane vectorMyofiber vector->
The invention has the advantages that,
(1) The algorithm is strong in robustness. The method of deep learning is used in the measurement stage, and the method is more reliable compared with the result obtained by the traditional image processing method, and can be suitable for ultrasound images with poor quality. The Kalman filtering algorithm is used for tracking the lupin angle on the basis of the method, so that the lupin angle is more robust.
(2) The algorithm can be suitable for various front-end ultrasonic image acquisition equipment without adjustment, and when new ultrasonic equipment exists, the algorithm effective for the new equipment can be obtained by only adding a new image into a training set for fine adjustment.
In short, the direction of the current muscle fiber is measured by using the deep convolutional neural network, and the feather angle is tracked by combining the direction of the current muscle fiber with a Kalman filter. The method improves the robustness of the lupin angle calculation algorithm, expands the application field of the automatic lupin angle labeling algorithm, and provides a method for automatically tracking the lupin angle for an ultrasonic image sequence with poor quality.
Drawings
FIG. 1 is a schematic flow chart of the present invention;
the myofiber direction measurement process of the ultrasound image of fig. 2;
the system of fig. 3 outputs the result.
Detailed Description
The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
The method for tracking the feather angle of the muscle based on the ultrasonic image of the deep learning and the Kalman filtering utilizes the depth residual error convolution neural network in the deep learning to measure the feather angle of the feather muscle tissue from the ultrasonic image of the feather muscle, and corrects the value by using the Kalman filtering algorithm to obtain alpha, and the parameter has important guiding significance for function assessment, diagnosis and monitoring of diseases, later rehabilitation planning, tissue function prognosis and the like. The invention expands the new application field of ultrasonic imaging technology and realizes the automatic tracking of the feather angle of the muscle.
A method for automatically tracking muscle feathering angle by combining convolutional neural network and kalman filtering, comprising the following steps:
step one, ultrasound image preprocessing
The raw data stored by the ultrasound instrument often contains other information than the image, such as time, patient name, instrument model, etc. Thus, the image sequence needs to be further cropped, denoised, and the interference processed, leaving only the ultrasound image.
Step two, myomembrane detection
An edge map of the ultrasound image is obtained using convolution calculations, and the linear segments in the edge map are detected using a local radon transform, thereby locating the myomembrane.
The myomembrane detection specifically comprises the following steps:
(1) And deriving the original ultrasonic image to obtain a gradient map of the original image, so that the edge of the myomembrane in the original image is clear. Firstly, performing Gaussian filtering operation on an image by adopting a Canni edge detection algorithm to enable the image to be smooth; secondly, selecting an operator to calculate the gradient value and the gradient direction of the image, taking a Sobel operator as an example, wherein the operator comprises two groups of 3X 3 matrixes which are respectively in the transverse direction and the longitudinal direction, and carrying out plane convolution on the matrixes and the image to obtain the brightness difference approximate values in the transverse direction and the longitudinal direction respectively.
In the Canni edge detection algorithm, the following formula is adopted:
wherein, A is the original image, G x G (G) y Respectively, images detected by transverse and longitudinal edges, and θ is the gradient direction.
In addition, the Canni algorithm further determines the boundaries of the image using non-maximal suppression and a dual threshold approach.
(2) Local radon transform of edge map
The radon transform is an integral transform that integrates a function f (x, y) defined in a two-dimensional plane along any one line in the plane, corresponding to a CT scan of the function f (x, y). Because the position and the angle of the deep and shallow myomembrane line are within a certain range, the detection is carried out by taking the position and the angle as priori knowledge. And carrying out local radon transformation on the edge map, and adopting the following formula:
where, in the x-y image plane, F (x, y) is the image intensity at the location (x, y), δ is the dirac function, ρ is the distance from the origin to the line, θ is the angle between the x-axis and the line perpendicular to the line from the origin.
(3) Marking the myofilm by a straight line with the maximum probability of the point position gradient map, and calculating the vector of the myofilm
Step three, obtaining the observation value of the myofiber direction
Randomly marking a reference line, judging whether the reference line is parallel to the direction of the muscle fiber through the neural network, and adjusting if not, until the neural network judges that the reference line is parallel, and recording the angle to be used as an observation value of Kalman filtering.
The specific operation steps of deep learning model to detect the direction of muscle fiber can be divided into two parts of training and detection:
(1) The training section provides a method of generating a myofiber ultrasound image database, comprising: taking points randomly in the middle area of two myomembranes in an ultrasonic image as coordinates of the upper left corner, and determining a square image with 224 multiplied by 224 pixels as a myofiber image according to each upper left corner coordinate; in the process, the square is randomly rotated by an angle relative to the original image to expand the training set.
To improve accuracy, the type of reference line that floats up and down parallel to the muscle fibers is marked, also as part of the training set.
(2) In the process of processing an image, generally, the upper left corner of the image is taken as an origin, a reference line parallel to muscle fibers in an ultrasonic image is marked first, the slope k of the reference line is taken as a standard, a random point is taken at the left boundary of the image as a reference line starting point A, a reference line ending point B on the right boundary of the image is determined according to the reference line starting point, and the method for determining the ending point is as follows: according to k and A, if the reference line parallel muscle fiber needs to be generated, the slope of the straight line where the point B and the point A are positioned is equal to k; if the reference line is required to be generated to be higher than the myofiber direction image, the slope of the straight line where the point B and the point A are positioned is smaller than k; if the reference line is required to be generated to be lower than the muscle fiber, the slope of the straight line where the point B and the point A are positioned is larger than k;
(3) After the database is generated, a depth residual convolutional neural (ResNet) network is trained, the muscle fiber ultrasonic images of the training sample and the test sample are normalized and converted in type, and for each pixel pix in each image, the converted pixel pix' is:
the value of the muscle beam ultrasonic image is normalized from an integer of [0,255] to a floating point number of [ -1,1] for the deep neural network to calculate.
(4) And inputting images with different reference lines into a trained network, and if the output results are not parallel, adjusting the slope of the reference lines according to the output results until the output results are parallel.
(5) When the output result of the neural network is parallel, the angle is recorded as the observation value of the following Kalman filtering algorithm.
Step four, correcting the direction of the muscle fiber
And (3) correcting the muscle fiber direction of the current frame according to the muscle fiber direction of the previous frame and the measurement of the muscle fiber angle in the step (3) by using a Kalman filtering tracking system, and predicting the direction of the muscle fiber of the next frame.
The method specifically comprises the following steps:
(1) Predicting a system of a next state using a process model of the system;
assuming that the current system state is k, according to the model of the system, the occurrence in the state can be predicted based on the last state of the system:
X(k|k-1)＝AX(k-1|k-1) (6)
in the formula (6), X (k|k-1) is the result of the prediction of the previous state, and X (k-1|k-1) is the optimal result of the previous state, namely the angle of the direction of the muscle fiber of the previous frame.
Wherein θ k For the predictive value of the prior model, i.e. predicting the angle value of the current muscle fiber direction, U k For the speed of the angular transformation of the myofiber direction, a is a state transition matrix, and is set as shown in (8) in this system.
P(k|k-1)＝AP(k-1|k-1)A′+Q (9)
The covariance of X (k|k-1) is updated using equation (9), P represents covariance (covariance), P (k|k-1) is the covariance corresponding to X (k|k-1), P (k-1|k-1) is the covariance corresponding to X (k-1|k-1), A' represents the transpose matrix of A, and Q is the covariance of the system process.
(2) Combining the predicted value and the measured value obtained in the last step, obtaining an optimized estimated value X (k|k) of the current state (k):
X(k|k)＝X(k|k-1)+Kg(k)(Z(k)-HX(k|k-1)) (10)
wherein Z (k) is a measured value at the moment k, namely the measurement of the direction angle of the muscle fiber by the neural network of the current frame; h is a parameter of the measurement system.
Where Kg (k) is Kalman Gain, where R is the covariance of the measured noise.
After obtaining the optimized estimation value X (k|k), the optimized myofiber vector is obtained
(3) Obtaining the optimal estimated value under the current state and updating the covariance
An optimal estimate X (k|k) in the k-state has been obtained in order to keep the Kalman filter running straight
By the end of the system procedure, the covariance of X (k|k) in the k state needs to be updated:
P(k|k)＝(I-Kg(k)H)P(k|k-1) (13)
and fifthly, calculating the feather angle.
The feather angle value is calculated by the vector of the myomembrane line and the myofiber bundle line:
wherein θ is the gradient direction, myomembrane vectorMyofiber vector->
The invention provides a method and a system for tracking feather angle in a muscle ultrasonic image sequence by using a method combining deep learning and Kalman filtering,
and measuring the direction of the current muscle fiber by using a deep convolutional neural network, and combining with a Kalman filter to realize the tracking of the feather angle. The method improves the robustness of the lupin angle calculation algorithm, expands the application field of the automatic lupin angle labeling algorithm, and provides a method for automatically tracking the lupin angle for an ultrasonic image sequence with poor quality.
The method of the invention has the following simple outline:
(1) The direction of the muscle fibers is obtained using a deep learning method.
And firstly randomly marking a reference line for each ultrasonic myofiber image, judging whether the direction of the reference line is parallel to the direction of the myofiber bundles through a neural network, if not, adjusting until the neural network judges that the reference lines are parallel, and outputting the angle of the reference line.
(2) The direction of the muscle fibers from the current frame to the next frame is predicted using a Kalman Filter (KF) tracking strategy. And (3) providing observation for KF correction by using the method (1) to the measured value of the current myofiber direction, and finally obtaining the corrected myofiber reference line direction.
(3) And obtaining the direction of the deep and shallow myomembrane line by using a local radon transformation method.
(4) And (3) obtaining the value of the current feather angle according to the results of the steps (2) and (3).
To verify the feasibility and effectiveness of the present invention, gastrocnemius ultrasonic images of normal persons and patients with muscular atrophy acquired by an ultrasonic diagnostic apparatus were analyzed, and the feathered angle α of the muscle tissue was estimated from the ultrasonic images using the method proposed by the present invention, as shown in fig. 3.
In addition, in the present invention:
(1) The detection of the myomembrane line is changed into other algorithms, such as a radon algorithm, hough transformation and the like.
(2) According to the invention, not only can a deep residual neural network be used, but also the system can obtain better effects, such as AlexNet, VGG-Net and the like, under the condition of using other deep convolutional neural networks through experiments.
(3) The kalman tracking part may also use variations of the kalman algorithm to achieve tracking.
It should be understood that the above description is not intended to limit the invention to the particular embodiments disclosed, but to limit the invention to the particular embodiments disclosed, and that the invention is not limited to the particular embodiments disclosed, but is intended to cover modifications, adaptations, additions and alternatives falling within the spirit and scope of the invention.
Claims (5)
1. The method for automatically tracking the muscle feather angle by combining the convolutional neural network and the Kalman filtering is characterized by comprising the following steps of:
step (1), preprocessing an ultrasonic image;
step (2), myomembrane detection;
step (3), obtaining a myofiber direction observation value; randomly marking a reference line, judging whether the reference line is parallel to the direction of the muscle fiber through a neural network, and adjusting if not, until the neural network judges that the reference line is parallel, and recording the angle to be used as an observation value of Kalman filtering;
step (4), correcting the direction of the muscle fiber; correcting the direction of the muscle fiber of the current frame according to the direction of the muscle fiber of the previous frame and the measurement of the angle of the muscle fiber in the step (3) by using a Kalman filtering tracking system, and predicting the direction of the muscle fiber of the next frame; the method specifically comprises the following steps:
(41) Predicting a system of a next state using a process model of the system;
(42) Combining the predicted value and the measured value obtained in the last step to obtain the optimized estimated value of the current state, namely the optimized myofiber vector
(43) Obtaining an optimal estimated value in the current state, and updating covariance;
and (5) calculating the feather angle.
2. The method for automatically tracking the feather angle of a muscle by combining a convolutional neural network with a kalman filter as claimed in claim 1, wherein in the step (2), the myomembrane detection specifically comprises the following steps:
(21) Performing Gaussian filtering operation on the image by adopting a Canni edge detection algorithm to enable the image to be smooth; selecting an operator to calculate the gradient value and gradient direction of the image to obtain a brightness difference approximation value;
(22) Carrying out local radon transformation on the edge map;
(23) Marking the myofilm by a straight line with the maximum probability of the point position gradient map, and calculating the vector of the myofilm
3. The method for automatically tracking muscle feathering angle by combining convolutional neural network and kalman filtering according to claim 2, wherein in step (21), the following formula is adopted in the canny edge detection algorithm:
wherein, A is the original image, G x G (G) y Respectively, images detected by transverse and longitudinal edges, and θ is the gradient direction.
4. The method for automatically tracking muscle feathering angle by combining convolutional neural network and kalman filtering according to claim 2, wherein in step (23), the local radon transform is performed on the edge map by using the following formula:
where in the x-y image plane, F (x, y) is the image intensity at the location (x, y), δ is the dirac function, ρ is the distance from the origin to the line, and α is the angle between the x-axis and the line perpendicular to the line from the origin.
5. The method for automatically tracking the feather angle of the muscle by combining the convolutional neural network with the kalman filter as claimed in claim 1, wherein in the step (5), the feather angle value is calculated by vectors of the myomembrane line and the myofiber bundle line:
wherein beta is gradient direction, myomembrane vectorMyofiber vector->
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010865743.0A CN112085711B (en) | 2020-08-25 | 2020-08-25 | Method for automatically tracking muscle feather angle by combining convolutional neural network and Kalman filtering |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010865743.0A CN112085711B (en) | 2020-08-25 | 2020-08-25 | Method for automatically tracking muscle feather angle by combining convolutional neural network and Kalman filtering |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112085711A CN112085711A (en) | 2020-12-15 |
| CN112085711B true CN112085711B (en) | 2023-12-22 |
Family
ID=73729006
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010865743.0A Active CN112085711B (en) | 2020-08-25 | 2020-08-25 | Method for automatically tracking muscle feather angle by combining convolutional neural network and Kalman filtering |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112085711B (en) |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110111386A (en) * | 2019-05-10 | 2019-08-09 | 深圳大学 | The method for automatic measurement and system of structural angle in a kind of musculature |
| CN110930394A (en) * | 2019-11-29 | 2020-03-27 | 深圳先进技术研究院 | Method and terminal equipment for measuring slope and pinnate angle of muscle fiber bundle line |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7395251B2 (en) * | 2005-07-01 | 2008-07-01 | International Business Machines Corporation | Neural networks for prediction and control |
-
2020
- 2020-08-25 CN CN202010865743.0A patent/CN112085711B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110111386A (en) * | 2019-05-10 | 2019-08-09 | 深圳大学 | The method for automatic measurement and system of structural angle in a kind of musculature |
| CN110930394A (en) * | 2019-11-29 | 2020-03-27 | 深圳先进技术研究院 | Method and terminal equipment for measuring slope and pinnate angle of muscle fiber bundle line |
Non-Patent Citations (1)
| Title |
|---|
| "基于神经网络与卡尔曼滤波的手部实时追踪方法";曾公任等;《液晶与显示》;第第35卷卷(第第5期期);第464-468页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112085711A (en) | 2020-12-15 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11633169B2 (en) | Apparatus for AI-based automatic ultrasound diagnosis of liver steatosis and remote medical diagnosis method using the same | |
| CN111047572B (en) | Automatic spine positioning method in medical image based on Mask RCNN | |
| CN107480677B (en) | Method and device for identifying interest region in three-dimensional CT image | |
| CN106934821B (en) | Conical beam CT and CT image registration method based on ICP algorithm and B spline | |
| US7916919B2 (en) | System and method for segmenting chambers of a heart in a three dimensional image | |
| US8724865B2 (en) | Method, computer software, and system for tracking, stabilizing, and reporting motion between vertebrae | |
| US20030086596A1 (en) | Method, computer software, and system for tracking, stabilizing, and reporting motion between vertebrae | |
| CN110772286B (en) | System for discernment liver focal lesion based on ultrasonic contrast | |
| CN109124662B (en) | Rib center line detection device and method | |
| US11455720B2 (en) | Apparatus for ultrasound diagnosis of liver steatosis using feature points of ultrasound image and remote medical-diagnosis method using the same | |
| CN110458859B (en) | Multi-sequence MRI-based multiple myeloma focus segmentation system | |
| JP2018064627A (en) | Radiographic apparatus, radiographic system, radiographic method, and program | |
| US20150154765A1 (en) | Tomosynthesis reconstruction with rib suppression | |
| CN110782428B (en) | Method and system for constructing clinical brain CT image ROI template | |
| CN109801276B (en) | Method and device for calculating heart-chest ratio | |
| Zheng et al. | Fast and automatic heart isolation in 3D CT volumes: Optimal shape initialization | |
| US11526991B2 (en) | Medical image processing apparatus, and medical imaging apparatus | |
| CN104732520A (en) | Cardio-thoracic ratio measuring algorithm and system for chest digital image | |
| CN116580068B (en) | Multi-mode medical registration method based on point cloud registration | |
| CN109064473B (en) | 2.5D ultrasonic panoramic image segmentation method | |
| CN110111386B (en) | Method and system for automatically measuring structural angle in muscle tissue | |
| CN110580697B (en) | Video image processing method and system for measuring thickness of fetal nape transparency from ultrasonic video image | |
| CN112215878B (en) | X-ray image registration method based on SURF feature points | |
| CN112085711B (en) | Method for automatically tracking muscle feather angle by combining convolutional neural network and Kalman filtering | |
| CN113012127A (en) | Cardiothoracic ratio measuring method based on chest medical image |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |