CN111310669B - Fetal head circumference real-time measurement method and device - Google Patents
Fetal head circumference real-time measurement method and device Download PDFInfo
- Publication number
- CN111310669B CN111310669B CN202010100707.5A CN202010100707A CN111310669B CN 111310669 B CN111310669 B CN 111310669B CN 202010100707 A CN202010100707 A CN 202010100707A CN 111310669 B CN111310669 B CN 111310669B
- Authority
- CN
- China
- Prior art keywords
- fetal
- image
- current frame
- head circumference
- fetal image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 230000001605 fetal effect Effects 0.000 title claims abstract description 258
- 238000000691 measurement method Methods 0.000 title claims abstract description 24
- 238000000034 method Methods 0.000 claims abstract description 43
- 238000012545 processing Methods 0.000 claims abstract description 16
- 238000000605 extraction Methods 0.000 claims description 31
- 238000013527 convolutional neural network Methods 0.000 claims description 30
- 238000004422 calculation algorithm Methods 0.000 claims description 22
- 230000015654 memory Effects 0.000 claims description 17
- 238000004590 computer program Methods 0.000 claims description 9
- 238000005259 measurement Methods 0.000 claims description 9
- 238000007781 pre-processing Methods 0.000 claims description 9
- 230000011218 segmentation Effects 0.000 claims description 8
- 230000008569 process Effects 0.000 abstract description 20
- 230000006870 function Effects 0.000 description 14
- 238000001914 filtration Methods 0.000 description 7
- 230000004913 activation Effects 0.000 description 6
- 210000003754 fetus Anatomy 0.000 description 5
- 230000000694 effects Effects 0.000 description 4
- 238000009499 grossing Methods 0.000 description 4
- 238000010606 normalization Methods 0.000 description 4
- 238000011176 pooling Methods 0.000 description 4
- 238000010295 mobile communication Methods 0.000 description 3
- 238000005260 corrosion Methods 0.000 description 2
- 238000013135 deep learning Methods 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 230000018109 developmental process Effects 0.000 description 2
- 238000003745 diagnosis Methods 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 230000002708 enhancing effect Effects 0.000 description 2
- 239000000284 extract Substances 0.000 description 2
- 230000008175 fetal development Effects 0.000 description 2
- 239000013589 supplement Substances 0.000 description 2
- 238000002604 ultrasonography Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 230000009547 development abnormality Effects 0.000 description 1
- 238000012631 diagnostic technique Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000002068 genetic effect Effects 0.000 description 1
- 238000010191 image analysis Methods 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 239000003550 marker Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 210000000056 organ Anatomy 0.000 description 1
- 238000011524 similarity measure Methods 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000012285 ultrasound imaging Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/22—Matching criteria, e.g. proximity measures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/25—Fusion techniques
- G06F18/253—Fusion techniques of extracted features
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
- G06V10/443—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components by matching or filtering
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Computation (AREA)
- General Engineering & Computer Science (AREA)
- Artificial Intelligence (AREA)
- Life Sciences & Earth Sciences (AREA)
- Bioinformatics & Computational Biology (AREA)
- Evolutionary Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Multimedia (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Human Computer Interaction (AREA)
- Ultra Sonic Daignosis Equipment (AREA)
Abstract
The invention discloses a fetal head circumference real-time measurement method and device, and belongs to the field of image processing. Based on the characteristic that most of ultrasonic images have relevance in the scanning process, the invention filters out images with strong relevance (such as images with high similarity or matched images) in the process of real-time identification by matching the fetal image of the current frame with the fetal image of the previous frame, thereby realizing the purpose of reducing the calculated amount; and identifying the unmatched images to obtain contour information (contour of the fetal head), performing image fitting on the contour information to obtain fetal head circumference ellipse, and calculating fetal head circumference length contour information to achieve the purpose of identifying the fetal head circumference in real time.
Description
Technical Field
The invention relates to the field of image processing, in particular to a fetal head circumference real-time measurement method and device.
Background
In medical diagnostic techniques, ultrasound imaging is an important medical diagnostic tool to aid in obstetrical examinations. In the obstetrical ultrasound image aided diagnosis system, various growth parameters of a fetus in the development process (such as checking whether the fetus is terated or not, organ function and development abnormality, genetic factors are stable and the like) can be monitored by measuring the fetal head circumference (Head circumference, HC) in the ultrasound image, so that the weight and the age of the fetus are estimated. In the ultrasonic image, the fetal head circumference is one of important indexes for assessing fetal development, and a doctor generally manually performs ellipse fitting on the fetal head circumference in clinic, so that a high error exists, and the fetal development diagnosis is easy to deviate. With the development of technology, a method for measuring the fetal head circumference by deep learning has been developed. However, the current method for measuring the fetal head circumference based on deep learning has large calculation amount, can only use static image analysis, and cannot perform online real-time measurement.
Disclosure of Invention
Aiming at the problem that the existing fetal head circumference measuring method does not support real-time measurement, the invention provides a method and a device for measuring the fetal head circumference in real time.
The invention provides a fetal head circumference real-time measurement method, which comprises the following steps:
acquiring a fetal image of a current frame;
matching the current frame of fetal image with the previous frame of fetal image;
if the current frame of fetal image is matched with the previous frame of fetal image, acquiring the fetal head circumference length corresponding to the previous frame of fetal image;
if the current frame of fetal image is not matched with the previous frame of fetal image, identifying the current frame of fetal image and obtaining contour information;
and performing image fitting on the contour information to obtain a fetal head circumference ellipse, and calculating the fetal head circumference length.
Preferably, matching the current frame of fetal image with the previous frame of fetal image includes:
calculating a similarity value between the fetal image of the current frame and the fetal image of the previous frame by adopting a similarity matching algorithm;
if the similarity value is greater than or equal to a preset threshold value, the current frame of fetal image is matched with the previous frame of fetal image;
and if the similarity value is smaller than the preset threshold value, the current frame of fetal image is not matched with the previous frame of fetal image.
Preferably, if the current frame of fetal image is not matched with the previous frame of fetal image, identifying the current frame of fetal image, and obtaining contour information includes:
if the current frame of fetal image is not matched with the previous frame of fetal image, identifying the current frame of fetal image by adopting a convolutional neural network model, and extracting candidate feature information;
and processing the candidate feature information to obtain the contour information.
Preferably, the identifying the fetal image of the current frame by using a convolutional neural network model, and extracting candidate feature information includes:
preprocessing the fetal image of the current frame, identifying the preprocessed fetal image of the current frame by adopting a convolutional neural network model, and extracting candidate characteristic information.
Preferably, the convolutional neural network model includes: the device comprises a first feature extraction module, a second feature extraction module and a classification module;
when the current frame of fetal image is not matched with the previous frame of fetal image, identifying the current frame of fetal image by adopting a convolutional neural network model, and extracting candidate characteristic information comprises the following steps:
when the current frame of fetal image is not matched with the previous frame of fetal image, extracting first characteristic information through the first characteristic extraction module;
performing feature extraction on the basis of the first feature information through the second feature extraction module to obtain second feature information, and gradually fusing the first feature information and the second feature information to generate third feature information;
and generating the candidate feature information according to a pixel prediction semantic segmentation result of the third feature information through the classification module.
Preferably, the processing the candidate feature information to obtain the profile information includes:
and processing the candidate feature information by adopting a binarization method to obtain the contour information.
Preferably, image fitting is performed on the profile information to obtain a fetal head circumference ellipse, and the fetal head circumference length profile information is calculated, including:
and performing image fitting on the profile information by adopting a least square fitting algorithm to obtain a fetal head circumference ellipse, and calculating the fetal head circumference length.
The invention also provides a fetal head circumference real-time measurement device, which comprises:
the receiving unit is used for acquiring a fetal image of the current frame;
the matching unit is used for matching the current frame of fetal image with the previous frame of fetal image;
the identification unit is used for identifying the current frame of fetal image and acquiring contour information when the current frame of fetal image is not matched with the previous frame of fetal image;
and the calculating unit is used for carrying out image fitting on the contour information to obtain a fetal head circumference ellipse and calculating the fetal head circumference length.
The invention also provides a computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the steps of the fetal head circumference real-time measurement method are realized when the processor executes the computer program.
The present invention also provides a computer-readable storage medium having stored thereon a computer program characterized in that: the computer program when executed by a processor implements the steps of the fetal head circumference real-time measurement method described above.
The beneficial effects of the technical scheme are that:
in the technical scheme, the fetal head circumference real-time measurement method is based on the characteristic that most of ultrasonic images in the scanning process have relevance, and the aim of reducing the calculated amount is fulfilled by matching the fetal image of the current frame with the fetal image of the previous frame and filtering out images with strong relevance (such as images with high similarity or matched images) in the real-time identification process; and identifying the unmatched images to obtain contour information (contour of the fetal head), performing image fitting on the contour information to obtain fetal head circumference ellipse, and calculating fetal head circumference length contour information to achieve the purpose of identifying the fetal head circumference in real time.
Drawings
FIG. 1 is a flow chart of a method of one embodiment of a fetal head circumference real-time measurement method of the present invention;
FIG. 2 is a flowchart of an embodiment of identifying the fetal image of the current frame and obtaining profile information according to the present invention;
FIG. 3 is a flowchart of an embodiment of the present invention for identifying the fetal image of the current frame and extracting candidate feature information using a convolutional neural network model;
FIG. 4 is a flow chart of another embodiment of a method for fetal head circumference real-time measurement according to the present invention;
FIG. 5 is a block diagram of one embodiment of a fetal head circumference real-time measurement apparatus according to the present invention;
fig. 6 is a schematic diagram of a hardware architecture of an embodiment of a computer device according to the present invention.
Detailed Description
The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
It should be noted that, without conflict, the embodiments of the present invention and features of the embodiments may be combined with each other.
The invention is further described below with reference to the drawings and specific examples, which are not intended to be limiting.
Example 1
As shown in fig. 1, the present invention provides a method for measuring fetal head circumference in real time, comprising:
s1, acquiring a fetal image of a current frame;
s2, matching the current frame of fetal image with the previous frame of fetal image, and if so, executing a step S3; if not, executing step S4;
in the step, considering the characteristic that most of ultrasonic images have relevance in the fetal scanning process, the current frame of fetal image is matched with the previous frame of fetal image, so that the image with strong relevance (such as the image with high similarity or the matched image) is filtered in the real-time identification process, and the aim of reducing the calculated amount while carrying out real-time identification is fulfilled.
Further, in step S2, matching the current frame of fetal image with the previous frame of fetal image includes:
calculating a similarity value between the fetal image of the current frame and the fetal image of the previous frame by adopting a similarity matching algorithm;
if the similarity value is greater than or equal to a preset threshold (e.g., 2), the current frame of fetal image is matched with the previous frame of fetal image;
and if the similarity value is smaller than the preset threshold value, the current frame of fetal image is not matched with the previous frame of fetal image.
It should be noted that: the similarity matching algorithm adopts a contrast algorithm (Mean Absolute Differences, abbreviated as MAD algorithm) to take a plurality of subgraphs with M multiplied by N (such as 21 multiplied by 15) in the fetal image of the current frame, and calculates the similarity between the fetal image of the current frame and the fetal image of the previous frame. Traversing the whole fetal image of the current frame, and finding out the most similar sub-image with the fetal image of the previous frame from all the available sub-images as a final matching result.
The similarity measure formula of the MAD algorithm is as follows:
taking (i, j) as the upper left corner in the fetal image S of the current frame, taking a subgraph with the size of M multiplied by N, and calculating the similarity of the fetal image of the current frame and the fetal image T M multiplied by N of the previous frame; traversing the whole fetal image of the current frame, and finding out the most similar sub-image with the fetal image of the previous frame from all the available sub-images as a final matching result, namely, the minimum D (i, j).
The smaller the average absolute difference D (i, j) in the MAD algorithm, the more similar the average absolute difference D (i, j), so that the most similar sub-graph position can be determined by only finding the minimum D (i, j). The smaller the average absolute difference D (i, j) is, the more similar the fetal image of the current frame is to the fetal image of the previous frame, and the minimum similarity is taken as the similarity value of the fetal image of the current frame. The MAD algorithm has the advantages of small operand and high precision.
S3, obtaining the fetal head circumference length corresponding to the previous frame of fetal image;
s4, identifying the fetal image of the current frame to obtain contour information;
further, referring to fig. 2, in step S4, if the current frame of fetal image does not match the previous frame of fetal image, identifying the current frame of fetal image, and obtaining contour information includes:
s41, if the current frame of fetal image is not matched with the previous frame of fetal image, identifying the current frame of fetal image by adopting a convolutional neural network model, and extracting candidate feature information;
it should be noted that: in step S41, a convolutional neural network model is used to identify the fetal image of the current frame, and candidate feature information is extracted, including:
preprocessing the fetal image of the current frame, identifying the preprocessed fetal image of the current frame by adopting a convolutional neural network model, and extracting candidate characteristic information.
Further, preprocessing the current frame fetal image includes:
and preprocessing the fetal image of the current frame by adopting at least one of bilinear interpolation, smoothing and median filtering.
Preprocessing the fetal image of the current frame by adopting a smoothing processing and median filtering processing mode, so that the purposes of eliminating irrelevant information in the image, enhancing the detectability of relevant information and simplifying data to the maximum extent can be achieved; the image of the fetus of the current frame can be scaled (the size of the image is adjusted) by adopting a bilinear interpolation method, and the compressed image is subjected to edge extraction by a Sobel operator (sobel operator) to obtain an image boundary.
In this embodiment, the convolutional neural network model may include: the device comprises a first feature extraction module, a second feature extraction module and a classification module;
step S41, when the current frame fetal image is not matched with the previous frame fetal image, uses a convolutional neural network model to identify the current frame fetal image, extracts candidate feature information, and referring to fig. 3, includes:
s411, when the current frame of fetal image is not matched with the previous frame of fetal image, extracting first characteristic information through the first characteristic extraction module;
s412, carrying out feature extraction on the basis of the first feature information through the second feature extraction module to obtain second feature information, and merging the first feature information and the second feature information step by step to generate third feature information;
s413, generating the candidate feature information through the classification module according to the pixel prediction semantic segmentation result of the third feature information.
It should be noted that: the convolutional neural network model is constructed by a full convolutional neural network of U-Net, and feature extraction is carried out on ultrasonic images in obstetrical ultrasonic data according to the convolutional neural network model. The first feature extraction module of the convolutional neural network model is used for downsampling and extracting deep features of the region of interest, so that the fetal head can be identified.
The first feature extraction module includes eight convolution layers (convolutions conv) for downsampling. Wherein, the former two convolutions conv are composed of 2 convolution layers of 3×3 to increase the speed and accuracy of feature extraction; the second convolution layer stride is 2 to replace the pooling effect, the third to fifth convolution layers have 4 convolution layers of 3×3, the fourth convolution layer stride is 2 to replace the pooling effect, the multi-layer convolution makes the decision function more sensitive to different areas and has stronger recognition capability, each convolution layer passes Batch Normalization and uses the leakey relu as the activation function.
The second feature extraction module includes a five upsampling layer for upsampling. Wherein the five upsampling layers are all deconvolution layers, using Relu as the activation function.
The classification module comprises three full-connection layers, namely: the convolution layers of 8×8×4096, 1×1×4096, and 1×1×2, with the final convolution depth of 2, are due to the fact that fetal head section segmentation is a one-pixel classification problem (i.e., whether it is a fetal head pixel or not). Each convolution layer passes through Batch Normalization and uses the Leakey ReLu as an activation function.
The convolutional neural network model of the embodiment adopts a VGG 16-like network as a main network for extracting deep features of the region of interest by a downsampling layer, so that automatic semantic segmentation of fetal head sections is realized. The VGG 16-like network gradually and upwardly samples 5 times from the 5 th layer through deconvolution (deconv 1, deconv2, deconv3, deconv4 and deconv 5), and interpolates with the BN layer each time so as to supplement the lost detail information in the downsampling process, and finally, the upsampling is restored to a mask image.
The following table is a VGG 16-like full convolutional neural network:
s42, processing the candidate feature information to obtain the contour information.
Further, step S42 may use a binarization method to process the candidate feature information, so as to obtain the profile information.
In step S42, the extracted candidate feature information is processed by adopting a binarization method, adjacent independent features are combined through a closed operation (such as a closed operation of expansion-corrosion), the relevance of images is increased, the outline is extracted, the outline coordinates can be inversely operated to the size of the original image according to the preprocessed scaling value, and the outline information is obtained by adopting a boundary tracking algorithm.
S5, performing image fitting on the contour information to obtain a fetal head circumference ellipse, and calculating the fetal head circumference length.
Further, step S5 performs image fitting on the profile information to obtain a fetal head circumference ellipse, and calculates fetal head circumference length profile information, including:
and (5) performing image fitting on the profile information by adopting a least square fitting algorithm (Direct Least Squares Fitting of Ellipses), obtaining a fetal head circumference ellipse, and calculating the fetal head circumference length.
Generating a contour sample point set based on contour information, fitting the contour sample point set through a specified fitting, and judging whether a fitting result is similar to elements in a preset candidate parameter list, wherein the specified fitting is elliptical, and can be directly performed by adopting a least square fitting algorithm, and the elements comprise, but are not limited to, long half shafts, short half shafts and circle center positions of fitted ellipses. The similarity judging principle is whether the circle center distance, the minor axis, the major axis and the radian of the ellipse are similar to those of the previous fitted ellipse, and the ellipse is regarded as the same ellipse, wherein the similarity judging principle is that the difference between the corresponding parameters is smaller than the corresponding threshold value. In the embodiment of the invention, the following are preferable: the difference of the center distances is smaller than 4, the difference of the short half shaft and the long half shaft is smaller than 3, and the radian of the ellipse is smaller than 6, namely, the two ellipses are judged to be similar, of course, the practicality can also adjust the threshold value parameters according to actual conditions, and the smaller the selected threshold value is, the more similar the two ellipses are, and the higher the precision is. The perimeter of the ellipse, which is the fetal head circumference, is calculated.
By way of example and not limitation, reference to fig. 4 may further include:
s6, fusing the fetal head circumference length (step S3 or step S5) with the fetal image of the current frame to generate an ultrasonic image for marking the fetal head circumference length so as to be convenient for a doctor or a patient to watch.
Specifically, a superposition of the fetal head circumference length with the fetal image of the current frame may be employed to generate the marker image.
In the embodiment, the fetal head circumference real-time measurement method is based on the characteristic that most of ultrasonic images in the scanning process have relevance, and the aim of reducing the calculated amount is fulfilled by matching the fetal image of the current frame with the fetal image of the previous frame and filtering out images with strong relevance (such as images with high similarity or matched images) in the real-time identification process; and identifying the unmatched images to obtain contour information (contour of the fetal head), performing image fitting on the contour information to obtain fetal head circumference ellipse, and calculating fetal head circumference length contour information to achieve the purpose of identifying the fetal head circumference in real time.
Example two
As shown in fig. 5, the present invention further provides a fetal head circumference real-time measurement apparatus 1, which may include: a receiving unit 11, a matching unit 12, an identifying unit 13, and a calculating unit 14;
a receiving unit 11 for acquiring a fetal image of a current frame;
a matching unit 12, configured to match the current frame of fetal image with a previous frame of fetal image;
in the step, considering the characteristic that most of ultrasonic images have relevance in the fetal scanning process, the current frame of fetal image is matched with the previous frame of fetal image, so that the image with strong relevance (such as the image with high similarity or the matched image) is filtered in the real-time identification process, and the aim of reducing the calculated amount while carrying out real-time identification is fulfilled.
Further, the matching unit 12 may calculate a similarity value between the current frame fetal image and the previous frame fetal image using a similarity matching algorithm;
if the similarity value is greater than or equal to a preset threshold (e.g., 2), the current frame of fetal image is matched with the previous frame of fetal image;
and if the similarity value is smaller than the preset threshold value, the current frame of fetal image is not matched with the previous frame of fetal image.
It should be noted that: the similarity matching algorithm adopts a contrast algorithm (Mean Absolute Differences, abbreviated as MAD algorithm) to take a plurality of subgraphs with M multiplied by N (such as 21 multiplied by 15) in the fetal image of the current frame, and calculates the similarity between the fetal image of the current frame and the fetal image of the previous frame. Traversing the whole fetal image of the current frame, and finding out the most similar sub-image with the fetal image of the previous frame from all the available sub-images as a final matching result. The MAD algorithm has the advantages of small operand and high precision.
A recognition unit 13, configured to recognize the current frame of fetal image and acquire contour information when the current frame of fetal image and the previous frame of fetal image are not matched;
specifically, if the current frame of fetal image is not matched with the previous frame of fetal image, the identification unit 13 adopts a convolutional neural network model to identify the current frame of fetal image, and extracts candidate feature information;
it should be noted that: the current frame fetal image may also be preprocessed before being identified by the identification unit 13 using the convolutional neural network model.
Further, preprocessing the current frame fetal image includes:
and preprocessing the fetal image of the current frame by adopting at least one of bilinear interpolation, smoothing and median filtering.
Preprocessing the fetal image of the current frame by adopting a smoothing processing and median filtering processing mode, so that the purposes of eliminating irrelevant information in the image, enhancing the detectability of relevant information and simplifying data to the maximum extent can be achieved; the image of the fetus of the current frame can be scaled (the size of the image is adjusted) by adopting a bilinear interpolation method, and the compressed image is subjected to edge extraction by a Sobel operator (sobel operator) to obtain an image boundary.
In this embodiment, the convolutional neural network model may include: the device comprises a first feature extraction module, a second feature extraction module and a classification module;
when the current frame of fetal image is not matched with the previous frame of fetal image, extracting first characteristic information through the first characteristic extraction module;
performing feature extraction on the basis of the first feature information through the second feature extraction module to obtain second feature information, and gradually fusing the first feature information and the second feature information to generate third feature information;
and generating the candidate feature information according to a pixel prediction semantic segmentation result of the third feature information through the classification module.
It should be noted that: the convolutional neural network model is constructed by a full convolutional neural network of U-Net, and features of an ultrasonic image in obstetrical ultrasonic data are extracted from a current frame according to the convolutional neural network model. The first feature extraction module of the convolutional neural network model is used for downsampling and extracting deep features of the region of interest, so that the fetal head can be identified.
The first feature extraction module includes eight convolution layers (convolutions conv) for downsampling. Wherein, the former two convolutions conv are composed of 2 convolution layers of 3×3 to increase the speed and accuracy of feature extraction; the second convolution layer stride is 2 to replace the pooling effect, the third to fifth convolution layers have 4 convolution layers of 3×3, the fourth convolution layer stride is 2 to replace the pooling effect, the multi-layer convolution makes the decision function more sensitive to different areas and has stronger recognition capability, each convolution layer passes Batch Normalization and uses the leakey relu as the activation function.
The second feature extraction module includes a five upsampling layer for upsampling. Wherein the five upsampling layers are all deconvolution layers, using Relu as the activation function.
The classification module comprises three full-connection layers, namely: the convolution layers of 8×8×4096, 1×1×4096, and 1×1×2, with the final convolution depth of 2, are due to the fact that fetal head section segmentation is a one-pixel classification problem (i.e., whether it is a fetal head pixel or not). Each convolution layer passes through Batch Normalization and uses the Leakey ReLu as an activation function.
The convolutional neural network model of the embodiment adopts a VGG 16-like network as a main network for extracting deep features of the region of interest by a downsampling layer, so that automatic semantic segmentation of fetal head sections is realized. The VGG 16-like network gradually and upwardly samples 5 times from the 5 th layer through deconvolution (deconv 1, deconv2, deconv3, deconv4 and deconv 5), and interpolates with the BN layer each time so as to supplement the lost detail information in the downsampling process, and finally, the upsampling is restored to a mask image.
The identification unit 13 processes the candidate feature information to acquire the profile information.
Further, the identifying unit 13 may process the candidate feature information by using a binarization method, to obtain the profile information. The extracted candidate feature information is processed by adopting a binarization method, adjacent independent features are combined through closed operation (such as expansion-corrosion closed operation), image relevance is increased, contours are extracted, contour coordinates can be inversely operated to the size of an original image according to a preprocessed scaling value, and the contour information is obtained by adopting a boundary tracking algorithm.
And the calculating unit 14 is used for performing image fitting on the profile information to obtain a fetal head circumference ellipse and calculating the fetal head circumference length.
And (5) performing image fitting on the profile information by adopting a least square fitting algorithm (Direct Least Squares Fitting of Ellipses), obtaining a fetal head circumference ellipse, and calculating the fetal head circumference length.
Generating a contour sample point set based on contour information, fitting the contour sample point set through a specified fitting, and judging whether a fitting result is similar to elements in a preset candidate parameter list, wherein the specified fitting is elliptical, and can be directly performed by adopting a least square fitting algorithm, and the elements comprise, but are not limited to, long half shafts, short half shafts and circle center positions of fitted ellipses. The similarity judging principle is whether the circle center distance, the minor axis, the major axis and the radian of the ellipse are similar to those of the previous fitted ellipse, and the ellipse is regarded as the same ellipse, wherein the similarity judging principle is that the difference between the corresponding parameters is smaller than the corresponding threshold value. In the embodiment of the invention, the following are preferable: the difference of the center distances is smaller than 4, the difference of the short half shaft and the long half shaft is smaller than 3, and the radian of the ellipse is smaller than 6, namely, the two ellipses are judged to be similar, of course, the practicality can also adjust the threshold value parameters according to actual conditions, and the smaller the selected threshold value is, the more similar the two ellipses are, and the higher the precision is. The perimeter of the ellipse, which is the fetal head circumference, is calculated.
In the embodiment, the fetal head circumference real-time measurement method is based on the characteristic that most of ultrasonic images in the scanning process have relevance, and the aim of reducing the calculated amount is fulfilled by matching the fetal image of the current frame with the fetal image of the previous frame and filtering out images with strong relevance (such as images with high similarity or matched images) in the real-time identification process; and identifying the unmatched images to obtain contour information (contour of the fetal head), performing image fitting on the contour information to obtain fetal head circumference ellipse, and calculating fetal head circumference length contour information to achieve the purpose of identifying the fetal head circumference in real time.
Example III
As shown in fig. 6, a computer device 2, the computer device 2 comprising:
a memory 21 for storing executable program code; and
a processor 22 for invoking said executable program code in said memory 21, the execution steps comprising the fetal head circumference real-time measurement method described above.
One processor 22 is illustrated in fig. 6.
The memory 21 is a non-volatile computer readable storage medium, and may be used to store a non-volatile software program, a non-volatile computer executable program, and modules, such as program instructions/modules (e.g., the receiving unit 11, the matching unit 12, the identifying unit 13, and the calculating unit 14 shown in fig. 5) corresponding to the fetal head circumference real-time measurement method in the embodiment of the present application. The processor 22 performs various functional applications of the computer device 2 and data processing, i.e. implements the above-described embodiments for the fetal head circumference real-time measurement method, by running non-volatile software programs, instructions and modules stored in the memory 21.
The memory 21 may include a storage program area and a storage data area, wherein the storage program area may store an application program required for operating the system and the at least one function; the storage data area may store playback information of the user at the computer device 2. In addition, memory 21 may include high-speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other non-volatile solid-state storage device. In some embodiments, the memory 21 optionally includes a memory 21 remotely located with respect to the processor 22, and these remote memories 21 may be connected to the fetal head circumference real-time measurement apparatus 1 via a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
The one or more modules are stored in the memory 21, and when executed by the one or more processors 22, perform the fetal head circumference real-time measurement method in any of the above-described method embodiments, for example, perform the method steps S1 to S5 in fig. 1 described above, implementing the functions of the receiving unit 11, the matching unit 12, the identifying unit 13, and the calculating unit 14 shown in fig. 5.
The computer device 2 of the embodiments of the present application exists in a variety of forms including, but not limited to:
(1) A mobile communication device: such devices are characterized by mobile communication capabilities and are primarily aimed at providing voice, data communications. Such terminals include: smart phones (e.g., iPhone), multimedia phones, functional phones, and low-end phones, etc.
(2) Ultra mobile personal computer device: such devices are in the category of personal computers, having computing and processing functions, and generally also having mobile internet access characteristics. Such terminals include: PDA, MID, and UMPC devices, etc., such as iPad.
(3) Portable entertainment device: such devices may display and play multimedia content. The device comprises: audio, video players (e.g., iPod), palm game consoles, electronic books, and smart toys and portable car navigation devices.
(4) And (3) a server: the configuration of the server includes a processor, a hard disk, a memory, a system bus, and the like, and the server is similar to a general computer architecture, but is required to provide highly reliable services, and thus has high requirements in terms of processing capacity, stability, reliability, security, scalability, manageability, and the like.
(5) Other electronic devices with data interaction function.
Example IV
Embodiments of the present application provide a non-transitory computer readable storage medium storing computer executable instructions that are executed by one or more processors, such as the one processor 22 in fig. 6, to enable the one or more processors 22 to perform the fetal head circumference real-time measurement method in any of the above-described method embodiments, such as performing the method steps S1 to S5 in fig. 1 described above, to implement the functions of the fetal head circumference real-time measurement method shown in fig. 5.
The apparatus embodiments described above are merely illustrative, wherein elements illustrated as separate elements may or may not be physically separate, and elements shown as elements may or may not be physical elements, may be located in one place, or may be distributed over at least two network elements. Some or all of the modules may be selected according to actual needs to achieve the purposes of the embodiments of the present application. Those of ordinary skill in the art will understand and implement the present invention without undue burden.
From the above description of embodiments, it will be apparent to those skilled in the art that the embodiments may be implemented by means of software plus a general purpose hardware platform, or may be implemented by hardware. Those skilled in the art will appreciate that all or part of the processes implementing the methods of the above embodiments may be implemented by a computer program for instructing relevant hardware, where the program may be stored in a computer readable storage medium, and where the program may include processes implementing the embodiments of the methods described above. The storage medium may be a magnetic disk, an optical disk, a Read-only memory (ROM), a random access memory (RandomAccessMemory, RAM), or the like.
Finally, it should be noted that: the above embodiments are only for illustrating the technical solution of the present application, and not for limiting the same; although the present application has been described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some or all of the technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit of the corresponding technical solutions from the scope of the technical solutions of the embodiments of the present application.
Claims (9)
1. A method for real-time measurement of fetal head circumference, comprising:
acquiring a fetal image of a current frame;
matching the current frame of fetal image with the previous frame of fetal image;
if the current frame of fetal image is matched with the previous frame of fetal image, acquiring the fetal head circumference length corresponding to the previous frame of fetal image;
if the current frame of fetal image is not matched with the previous frame of fetal image, identifying the current frame of fetal image and obtaining contour information;
performing image fitting on the contour information to obtain a fetal head circumference ellipse, and calculating the fetal head circumference length;
matching the current frame of fetal image with the previous frame of fetal image comprises the following steps:
calculating a similarity value between the fetal image of the current frame and the fetal image of the previous frame by adopting a similarity matching algorithm;
if the similarity value is greater than or equal to a preset threshold value, the current frame of fetal image is matched with the previous frame of fetal image;
and if the similarity value is smaller than the preset threshold value, the current frame of fetal image is not matched with the previous frame of fetal image.
2. The fetal head circumference real-time measurement method of claim 1, wherein if the current frame fetal image does not match the previous frame fetal image, identifying the current frame fetal image, and obtaining contour information comprises:
if the current frame of fetal image is not matched with the previous frame of fetal image, identifying the current frame of fetal image by adopting a convolutional neural network model, and extracting candidate feature information;
and processing the candidate feature information to obtain the contour information.
3. The fetal head circumference real-time measurement method of claim 2, wherein identifying the current frame fetal image using a convolutional neural network model, extracting candidate feature information, comprises:
preprocessing the fetal image of the current frame, identifying the preprocessed fetal image of the current frame by adopting a convolutional neural network model, and extracting candidate characteristic information.
4. The fetal head circumference real-time measurement method of claim 2, wherein the convolutional neural network model comprises: the device comprises a first feature extraction module, a second feature extraction module and a classification module;
when the current frame of fetal image is not matched with the previous frame of fetal image, identifying the current frame of fetal image by adopting a convolutional neural network model, and extracting candidate characteristic information comprises the following steps:
when the current frame of fetal image is not matched with the previous frame of fetal image, extracting first characteristic information through the first characteristic extraction module;
performing feature extraction on the basis of the first feature information through the second feature extraction module to obtain second feature information, and gradually fusing the first feature information and the second feature information to generate third feature information;
and generating the candidate feature information according to a pixel prediction semantic segmentation result of the third feature information through the classification module.
5. The fetal head circumference real-time measurement method of claim 2, wherein processing the candidate feature information to obtain the profile information comprises:
and processing the candidate feature information by adopting a binarization method to obtain the contour information.
6. The method of claim 1, wherein performing image fitting on the profile information to obtain a fetal head circumference ellipse, and calculating fetal head circumference length profile information comprises:
and performing image fitting on the profile information by adopting a least square fitting algorithm to obtain a fetal head circumference ellipse, and calculating the fetal head circumference length.
7. A fetal head circumference real-time measurement apparatus, characterized in that a fetal head circumference real-time measurement method according to any one of claims 1-6 is used, comprising:
the receiving unit is used for acquiring a fetal image of the current frame;
the matching unit is used for matching the current frame of fetal image with the previous frame of fetal image;
the identification unit is used for identifying the current frame of fetal image and acquiring contour information when the current frame of fetal image is not matched with the previous frame of fetal image;
and the calculating unit is used for carrying out image fitting on the contour information to obtain a fetal head circumference ellipse and calculating the fetal head circumference length.
8. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the steps of the fetal head circumference real-time measurement method of any of claims 1 to 6 when the computer program is executed.
9. A computer-readable storage medium having stored thereon a computer program, characterized by: the computer program, when executed by a processor, performs the steps of the fetal head circumference real-time measurement method of any one of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010100707.5A CN111310669B (en) | 2020-02-19 | 2020-02-19 | Fetal head circumference real-time measurement method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010100707.5A CN111310669B (en) | 2020-02-19 | 2020-02-19 | Fetal head circumference real-time measurement method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111310669A CN111310669A (en) | 2020-06-19 |
| CN111310669B true CN111310669B (en) | 2024-02-02 |
Family
ID=71161695
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010100707.5A Active CN111310669B (en) | 2020-02-19 | 2020-02-19 | Fetal head circumference real-time measurement method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111310669B (en) |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5795296A (en) * | 1996-03-29 | 1998-08-18 | University Of Washington | Pipeline process for automatically measuring object boundary from ultrasound image samples |
| CN108171739A (en) * | 2017-12-25 | 2018-06-15 | 清华大学苏州汽车研究院（吴江） | Binocular vision solid matching method based on movable information |
| CN110448335A (en) * | 2019-07-11 | 2019-11-15 | 暨南大学 | A kind of fetus head circumference full-automatic measuring method and device based on ultrasound image |
-
2020
- 2020-02-19 CN CN202010100707.5A patent/CN111310669B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5795296A (en) * | 1996-03-29 | 1998-08-18 | University Of Washington | Pipeline process for automatically measuring object boundary from ultrasound image samples |
| CN108171739A (en) * | 2017-12-25 | 2018-06-15 | 清华大学苏州汽车研究院（吴江） | Binocular vision solid matching method based on movable information |
| CN110448335A (en) * | 2019-07-11 | 2019-11-15 | 暨南大学 | A kind of fetus head circumference full-automatic measuring method and device based on ultrasound image |
Non-Patent Citations (1)
| Title |
|---|
| 邢妍妍 ; 杨丰 ; 唐宇姣 ; 张利云 ; .融合型UNet++网络的超声胎儿头部边缘检测.中国图象图形学报.2020,(02),全文. * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111310669A (en) | 2020-06-19 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107784282B (en) | Object attribute identification method, device and system | |
| CN110874594B (en) | Human body appearance damage detection method and related equipment based on semantic segmentation network | |
| CN111161275B (en) | Method and device for segmenting target object in medical image and electronic equipment | |
| WO2020103676A1 (en) | Image identification method and apparatus, system, and storage medium | |
| CN110689025B (en) | Image recognition method, device and system and endoscope image recognition method and device | |
| CN110974294A (en) | Ultrasonic scanning method and device | |
| KR20210048523A (en) | Image processing method, apparatus, electronic device and computer-readable storage medium | |
| CN109124662B (en) | Rib center line detection device and method | |
| CN111310705A (en) | Image recognition method and device, computer equipment and storage medium | |
| JP2016533782A (en) | Method and system for characterizing images acquired by medical imaging equipment | |
| US11967181B2 (en) | Method and device for retinal image recognition, electronic equipment, and storage medium | |
| CN113826143A (en) | Feature point detection | |
| EP4181059A1 (en) | Medical image processing method, apparatus, device, storage medium, and product | |
| CN116228787A (en) | Image sketching method, device, computer equipment and storage medium | |
| CN116897012A (en) | Method and device for identifying physique of traditional Chinese medicine, electronic equipment, storage medium and program | |
| CN116311370A (en) | Multi-angle feature-based cow face recognition method and related equipment thereof | |
| CN111461202B (en) | Thyroid nodule ultrasonic image real-time identification method and device | |
| CN113822323A (en) | Brain scanning image identification processing method, device, equipment and storage medium | |
| CN111310669B (en) | Fetal head circumference real-time measurement method and device | |
| Kinasih et al. | Two-stage multiple object detection using CNN and correlative filter for accuracy improvement | |
| CN110210314B (en) | Face detection method, device, computer equipment and storage medium | |
| CN111325709A (en) | Wireless capsule endoscope image detection system and detection method | |
| CN113658145B (en) | Liver ultrasonic standard tangent plane identification method and device, electronic equipment and storage medium | |
| CN113379770B (en) | Construction method of nasopharyngeal carcinoma MR image segmentation network, image segmentation method and device | |
| WO2022227193A1 (en) | Liver region segmentation method and apparatus, and electronic device and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20201015 Address after: Room 3388-c, building 2, No. 1077, Zuchongzhi Road, China (Shanghai) pilot Free Trade Zone, Minhang District, Shanghai, 200241 Applicant after: Shanghai Shenzhi Information Technology Co.,Ltd. Address before: Room 3388-c, building 2, 1077 Zuchongzhi Road, China (Shanghai) pilot Free Trade Zone, Pudong New Area, Shanghai Applicant before: Shanghai Zhuxing Biotechnology Co.,Ltd. |
|
| TA01 | Transfer of patent application right | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |