CN111242926A - Focus detection method and device and electronic equipment - Google Patents

Focus detection method and device and electronic equipment Download PDF

Info

Publication number
CN111242926A
CN111242926A CN202010034320.4A CN202010034320A CN111242926A CN 111242926 A CN111242926 A CN 111242926A CN 202010034320 A CN202010034320 A CN 202010034320A CN 111242926 A CN111242926 A CN 111242926A
Authority
CN
China
Prior art keywords
image
focus
network
scanning image
denoised
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202010034320.4A
Other languages
Chinese (zh)
Inventor
常德杰
孔飞
刘邦长
李荣华
姜鹏
刘朝振
刘红霞
张航飞
季科
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Miaoyijia Health Technology Group Co Ltd
Original Assignee
Beijing Miaoyijia Health Technology Group Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Miaoyijia Health Technology Group Co Ltd filed Critical Beijing Miaoyijia Health Technology Group Co Ltd
Priority to CN202010034320.4A priority Critical patent/CN111242926A/en
Publication of CN111242926A publication Critical patent/CN111242926A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/0002Inspection of images, e.g. flaw detection
    • G06T7/0012Biomedical image inspection
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/04Architecture, e.g. interconnection topology
    • G06N3/045Combinations of networks
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/10Image acquisition modality
    • G06T2207/10072Tomographic images
    • G06T2207/10081Computed x-ray tomography [CT]
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/20Special algorithmic details
    • G06T2207/20081Training; Learning
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/30Subject of image; Context of image processing
    • G06T2207/30004Biomedical image processing
    • G06T2207/30061Lung

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Health & Medical Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • General Physics & Mathematics (AREA)
  • Computing Systems (AREA)
  • Mathematical Physics (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • Biophysics (AREA)
  • Molecular Biology (AREA)
  • Biomedical Technology (AREA)
  • General Engineering & Computer Science (AREA)
  • Artificial Intelligence (AREA)
  • Computational Linguistics (AREA)
  • Software Systems (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Medical Informatics (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Radiology & Medical Imaging (AREA)
  • Quality & Reliability (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Image Analysis (AREA)

Abstract

The invention provides a focus detection method, a focus detection device and electronic equipment, wherein the method is used for acquiring a medical scanning image to be detected; preprocessing the medical scanning image to obtain a de-noised scanning image corresponding to the medical scanning image; inputting the de-noised scanning image into a pre-trained focus detection model to obtain a detection result output by the focus detection model; wherein, the detection result comprises one or more of the coordinates of the focus central point, the focus size and the focus offset; and labeling the detection result to the medical scanning image. The invention can effectively improve the accuracy of detecting the focus.

Description

Focus detection method and device and electronic equipment
Technical Field
The present invention relates to the field of image processing technologies, and in particular, to a method and an apparatus for detecting a lesion, and an electronic device.
Background
The focus is also the part of the body with pathological changes, and medical staff can better cure the patient aiming at the focus by detecting the position of the focus. Currently, the detection of a lesion is performed on the basis of a CT (Computed Tomography) image, for example, a 3D (3Dimensions, three-dimensional) CT (Computed Tomography) image is reduced to a 2D (2Dimensions, two-dimensional) CT image, and the 2D CT image is input into a neural network model to obtain a bounding box of the lesion, so as to represent the position of the lesion in the CT through the bounding box.
Disclosure of Invention
In view of the above, the present invention provides a method, an apparatus and an electronic device for detecting a lesion, which can effectively improve the accuracy of detecting the lesion.
In a first aspect, an embodiment of the present invention provides a method for detecting a lesion, including: acquiring a medical scanning image to be detected; preprocessing the medical scanning image to obtain a de-noised scanning image corresponding to the medical scanning image; inputting the de-noised scanning image into a pre-trained focus detection model to obtain a detection result output by the focus detection model; wherein the detection result comprises one or more of a focus central point coordinate, a focus size and a focus offset; and labeling the detection result to the medical scanning image.
In one embodiment, the medical scan image is a 3D image; the preprocessing comprises normalization processing and/or resampling processing; the step of preprocessing the medical scanning image to obtain a de-noised scanning image corresponding to the medical scanning image comprises the following steps: if the preprocessing comprises the normalization processing, mapping the color values of the pixels in the medical scanning image to a plurality of target color values in a preset color range respectively; if the preprocessing comprises the resampling processing, removing noise in the medical scanning image by using the resampling processing, and adjusting the current size of the medical scanning image to a first target size; and taking the preprocessed medical scanning image as a de-noising scanning image.
In one embodiment, the lesion detection model includes a feature map generation network and a first convolution network; the first convolution network comprises a central point convolution sub-network, a size convolution sub-network and an offset convolution sub-network which are respectively connected with the feature map generation network; the step of inputting the de-noised scanning image into a pre-trained focus detection model to obtain a detection result output by the focus detection model comprises the following steps: inputting the de-noised scanned image into the characteristic map generation network to obtain a characteristic map of the de-noised scanned image; inputting the feature map into the central point convolution sub-network, the size convolution sub-network and the offset convolution sub-network respectively to obtain detection results; the central point convolution sub-network is used for detecting the central point coordinates of the focus, the size convolution sub-network is used for detecting the size of the focus, and the offset convolution sub-network is used for detecting the offset of the focus.
In one embodiment, the feature map generation network comprises a second convolution network and a feature skeletal model; the step of inputting the denoised scanned image into the feature map generation network to obtain the feature map of the denoised scanned image includes: inputting the denoised scanned image into the second convolution network, and resampling the denoised scanned image through the second convolution network to obtain a denoised scanned image with a second target size; and inputting the de-noised scanning image with the second target size into the characteristic skeleton model to obtain a characteristic image of the de-noised scanning image.
In one embodiment, the step of training the lesion detection model includes: acquiring an image training set; training the focus detection model based on image samples in the image training set; calculating a loss function of the lesion detection model based on a training result of each training; stopping training the lesion detection model when a loss function of the lesion detection model converges.
In one embodiment, the step of acquiring the training set of images includes: acquiring an original image sample carrying a focus label; wherein the lesion label comprises a lesion bounding box; the original image sample is a 3D image; setting the color value of each pixel in the peripheral area of the focus bounding box in the original image sample as a preset color value; the area where the focus bounding box is located in the original image sample is preprocessed to obtain a denoised image sample corresponding to the original image sample; performing feature enhancement processing on the denoised image sample to obtain an enhanced image sample corresponding to the denoised image sample; and taking the denoised image sample and the enhanced image sample as an image training set.
In an embodiment, the step of performing feature enhancement processing on the denoised image sample to obtain an enhanced image sample corresponding to the denoised image sample includes: inputting the denoised image sample into a pre-trained confrontation characteristic generation network to obtain an enhanced image sample output by the confrontation characteristic generation network; and/or; carrying out image processing on the de-noised image sample to obtain an enhanced image sample output by the countermeasure characteristic generation network; wherein the image processing comprises one or more of rotation, scaling, translation, and warping.
In one embodiment, the step of calculating a loss function of the lesion detection model based on the training result of each training includes: calculating a loss function L of the lesion detection model according to the following formulafl
Figure BDA0002364353470000031
Wherein a represents a first hyperparameter, γ represents a second hyperparameter, y represents the lesion label, and y' represents the training result.
In a second aspect, an embodiment of the present invention further provides a lesion detection apparatus, including: the image acquisition module is used for acquiring a medical scanning image to be detected; the preprocessing module is used for preprocessing the medical scanning image to obtain a denoising scanning image corresponding to the medical scanning image; the detection module is used for inputting the de-noising scanning image into a pre-trained focus detection model to obtain a detection result output by the focus detection model; wherein the detection result comprises one or more of a focus central point coordinate, a focus size and a focus offset; and the marking module is used for marking the detection result to the medical scanning image.
In a third aspect, an embodiment of the present invention further provides an electronic device, including a processor and a memory; the memory has stored thereon a computer program which, when executed by the processor, performs the method of any one of the aspects as provided in the first aspect.
In a fourth aspect, an embodiment of the present invention further provides a computer storage medium for storing computer software instructions for use in any one of the methods provided in the first aspect.
According to the focus detection method, the focus detection device and the electronic equipment, firstly, an image scanning image to be detected is obtained, image medical scanning is preprocessed to obtain a de-noised scanning image, the de-noised scanning image is input into a pre-trained focus detection model to obtain a detection result, the detection result comprises one or more of focus center coordinates, focus size and focus offset, and the detection result is finally marked to the medical scanning image. According to the method, the denoising scanning image can be obtained by preprocessing the medical scanning image, so that the problems of inconsistent size and inconsistent precision of the medical scanning image are well solved, and the accuracy of detecting the focus is effectively improved.
Additional features and advantages of the invention will be set forth in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. The objectives and other advantages of the invention will be realized and attained by the structure particularly pointed out in the written description and claims hereof as well as the appended drawings.
In order to make the aforementioned and other objects, features and advantages of the present invention comprehensible, preferred embodiments accompanied with figures are described in detail below.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and other drawings can be obtained by those skilled in the art without creative efforts.
Fig. 1 is a schematic flow chart of a lesion detection method according to an embodiment of the present invention;
FIG. 2 is a diagram illustrating a second convolutional network according to an embodiment of the present invention;
fig. 3 is a schematic diagram of a first convolutional network according to an embodiment of the present invention;
fig. 4 is a schematic flow chart illustrating another lesion detection method according to an embodiment of the present invention;
fig. 5 is a schematic structural diagram of a lesion detection apparatus according to an embodiment of the present invention;
fig. 6 is a schematic structural diagram of an electronic device according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below with reference to the embodiments, and it is obvious that the described embodiments are some, but not all embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The problem that the existing focus detection method is low in accuracy exists at present, and based on the problem, the invention provides the focus detection method, the focus detection device and the electronic equipment, so that the focus detection accuracy can be effectively improved.
To facilitate understanding of the present embodiment, first, a detailed description is given of a lesion detection method disclosed in the present embodiment, referring to a schematic flow chart of a lesion detection method shown in fig. 1, where the method mainly includes the following steps:
step S102, acquiring a medical scanning image to be detected.
In one embodiment, the medical scanning image, that is, the CT image, may be a 2D image or a 3D image, and the 3D medical scanning image showing the contour of the skeleton and the organ of the human body is obtained by emitting X-rays to the human body through a CT scanning machine and scanning the human body, and is used as the medical scanning image to be detected.
And step S104, preprocessing the medical scanning image to obtain a de-noised scanning image corresponding to the medical scanning image.
If the medical scanned image is a 3D image, because the medical scanned image may have different-sized intervals which affect the detection result, in order to improve the accuracy of the detection result, the embodiment of the present invention needs to preprocess the medical scanned image, and in one implementation, the noise in the medical scanned image can be removed through normalization processing and resampling processing, so as to obtain the denoised scanned image.
And S106, inputting the de-noised scanning image into a pre-trained focus detection model to obtain a detection result output by the focus detection model.
For example, the trained lesion detection model is obtained by training a 3D CT image and a label labeled in the CT image to the CNN (Convolutional Neural Network). The detection result comprises one or more of focus central point coordinates, focus size and focus offset. The coordinates of the central point of the lesion, that is, the coordinates corresponding to the central point of the region where the lesion is located, and the size of the lesion may include the length, width, etc. of the lesion.
And step S108, labeling the detection result to the medical scanning image.
In order to facilitate medical staff or patients to know the detection result well, the detection result can be labeled to the medical scanning image, so that the medical staff or patients can clearly know the position and the size of the medical staff or patients in combination with the medical scanning image.
The focus detection method provided by the embodiment of the invention comprises the steps of firstly obtaining an image scanning image to be detected, preprocessing the image medical scanning to obtain a de-noised scanning image, inputting the de-noised scanning image into a pre-trained focus detection model to obtain a detection result, wherein the detection result comprises one or more of focus center coordinates, focus size and focus offset, and finally marking the detection result to the medical scanning image. According to the method, the denoising scanning image can be obtained by preprocessing the medical scanning image, so that the problems of inconsistent size and inconsistent precision of the medical scanning image are well solved, and the accuracy of detecting the focus is effectively improved.
In consideration of the fact that in the prior art, before a lesion is detected, a 3D CT image needs to be reduced to a 2D image, and a lesion detection result is obtained based on the 2D image, but the stereoscopic effect of the 2D image is significantly slightly inferior to that of the 3D image, so that the lesion detection result obtained based on the 2D image has a poor expression effect, the medical scanning image in the embodiment of the present invention uses the 3D image. In addition, due to the influence of the model of the CT scanning machine or the scanning parameters, each layer of the 3D medical scanning image may have a different size of space, thereby affecting the accuracy of the output detection result. On this basis, the embodiment of the present invention provides a specific implementation manner for preprocessing a medical scanning image to obtain a denoised scanning image corresponding to the medical scanning image, wherein: (1) if the preprocessing includes normalization, color values of individual pixels in the medical scan image are mapped to a plurality of target color values within a preset color range, respectively. Since the color value of the CT image is different from the color value of the common image, in one embodiment, the CT image with the color value in the (-1024,1024) range can be intercepted, and the other color ranges can be discarded, and the color value (-1024,1024) can be scaled to the range of (0, 225), so that the medical staff and the patient can clearly distinguish the region where the lesion is located by naked eyes, which is beneficial to debugging and checking the relevant data of the lesion. When the color value (-1024,1024) is scaled to be within the (0, 225), there is a one-to-one mapping of color values within the (-1024,1024) range to color values within the (0, 225) range. (2) If the preprocessing comprises resampling processing, noise in the medical scanning image is removed by the resampling processing, and the current size of the medical scanning image is adjusted to the first target size. In practical applications, the size and the scanning distance of the medical scanning image may be different due to the model of the CT scanning device and the problem of the scanning parameters (for example, different layer thickness parameters or different position parameters, etc.), and if the medical scanning image is not subjected to the temporal resampling process, the accuracy of the detection result will be affected. Wherein, the first target size may be (W (Width, Width), H (Height ), L (Length, Length)), for example, if the first target size is set to (0.8,0.8,0.3) (mm), then the scanning distance of the medical scanning image is unified by performing down-sampling processing and then performing up-sampling processing on the medical scanning image according to the preset sampling magnification, and the medical scanning image can be adjusted from the current size to the first target size. (3) And taking the preprocessed medical scanning image as a de-noised scanning image.
The existing method for detecting the focus by using the neural network model usually outputs a surrounding frame of the focus to represent the position of the focus in a medical scanning image by the surrounding frame, however, the single prediction mode has the problem of low reliability, in order to improve the reliability of the detection result, the embodiment of the invention outputs a plurality of detection results such as focus central coordinates, focus sizes, focus offset and the like by using the focus detection model, and the reliability of the detection result can be effectively improved by combining the focus central coordinates, the focus sizes and the focus offset. In order to enable the lesion detection model to output the above detection result, an embodiment of the present invention provides a lesion detection model including a feature map generation network and a first convolution network, where the first convolution network further includes a central point convolution sub-network, a size convolution sub-network, and an offset convolution sub-network respectively connected to the feature map generation network. The input of the characteristic map generation network is a de-noised scanning image, and the output of the characteristic map generation network is a characteristic map; inputting the central point convolution sub-network into a feature map, and outputting a focus central point coordinate P which is (x, y, z); the input of the size convolution sub-network is a feature map, and the output is the lesion size (w, h, l); the input to the offset convolution sub-network is the feature map and the output is the lesion offset (x1, y1, z 1).
On the basis of the focus detection model, the embodiment of the present invention provides a step that the denoising scanning image is input to the pre-trained focus detection model according to the following steps 1 to 2, and the detection result output by the focus detection model is obtained:
step 1, inputting the de-noised scanned image into a characteristic map generation network to obtain a characteristic map of the de-noised scanned image. To facilitate understanding of step 1, the embodiment of the present invention further illustrates a feature map generation network. The feature map generation network further comprises a second convolution network and a feature skeleton model. The second convolution network is used for resampling the de-noised scanned image, and the characteristic skeleton model is used for generating a characteristic image corresponding to the de-noised scanned image. On this basis, the embodiment of the present invention further provides a step of inputting the denoised scanned image into the feature map generation network to obtain the feature map of the denoised scanned image, which is shown in the following steps 1.1 to 1.2:
and 1.1, inputting the denoised scanned image into a second convolution network, and resampling the denoised scanned image through the second convolution network to obtain the denoised scanned image with a second target size. Referring to fig. 2, a schematic diagram of a second convolutional network is shown, the second convolutional network includes a downsampling portion and an upsampling portion, the resampling processing of the denoised scanned image can be realized by utilizing different sampling multiplying factors (X, Y, Z) to carry out downsampling processing and upsampling processing on each dimension of the input denoised scanned image, so as to obtain the denoised scanned image with a second target size, wherein X is a sampling magnification corresponding to a width (W) dimension, Y is a sampling magnification corresponding to a height (H) dimension, and Z is a sampling magnification corresponding to a length (L) dimension, for example, the input to the second convolution network is a 16 by 384 by 512 size de-noised scan image, then 4 x 2 in fig. 2 indicates that 1 pixel is sampled every 4 pixels in the W dimension, 1 pixel is sampled every 4 pixels in the H dimension, and 1 pixel is sampled every 2 pixels in the L dimension.
And 1.2, inputting the de-noised scanned image with the second target size into the characteristic skeleton model to obtain a characteristic diagram of the de-noised scanned image. The characteristic skeleton model can adopt a 3D rest-net50 skeleton model, and the 3D rest-net50 skeleton model is used for extracting characteristics in the de-noised scanned image with the second target size so as to generate a characteristic map of the de-noised scanned image.
And 2, respectively inputting the feature maps into a central point convolution sub-network, a size convolution sub-network and an offset convolution sub-network to obtain a detection result. The central point convolution sub-network is used for detecting the central point coordinates of the focus, the size convolution sub-network is used for detecting the size of the focus, and the offset convolution sub-network is used for detecting the offset of the focus. In one embodiment, referring to fig. 3, a schematic diagram of a first convolutional network is shown, wherein each of the centroid convolutional subnetwork, the size convolutional subnetwork, and the offset convolutional subnetwork may employ two convolutional layers connected together, where one convolutional layer has a parameter of 3 × 3 and the other convolutional layer has a parameter of 1 × 1.
In order to improve the accuracy of the detection result output by the focus detection model, the embodiment of the invention needs to train the focus detection model in advance, and stop the training of the focus detection model when the loss function of the focus detection model is converged. The embodiment of the invention provides a method for training a focus detection model, which comprises the following steps of a to d:
step a, obtaining an image training set. The image training set adopted by the embodiment of the invention comprises a plurality of 3D original image samples, and each original image sample carries a focus label. Wherein the lesion label includes a lesion bounding box that is used to characterize where the lesion is located in the original image sample. The embodiment of the present invention provides a specific implementation manner for obtaining an image training set, which is shown in the following steps a1 to a 5:
step a1, obtaining an original image sample carrying a lesion label. The original image sample is a 3D image, the original image sample can be obtained by scanning a human body through a CT scanning machine, and a focus surrounding frame is marked in the original image sample in a public marking mode or a neural network detection mode.
Step a2, setting the color value of each pixel in the peripheral area of the focus bounding box in the original image sample as a preset color value. In order to reduce the influence of noise in the original image sample on the training thereof, the embodiment of the present invention performs a background removal operation on the original image sample, so as to only retain the focus image, wherein a peripheral region of the focus bounding box may be regarded as a background region, and the background region is not greatly related to a region where the focus is located, so that in a specific implementation, a preset color value may be set to 0, so as to set a color value of each pixel in the peripheral region of the focus bounding box to 0, thereby removing the background region of the original image sample.
Step a3, preprocessing the region where the focus surrounding frame in the original image sample is located, and obtaining a denoised image sample corresponding to the original image sample. In practical application, the order of removing the background, the normalization processing, and the resampling processing may be set based on an actual situation, which is not limited in this embodiment of the present invention.
Step a4, performing feature enhancement processing on the denoised image sample to obtain an enhanced image sample corresponding to the denoised image sample. The feature enhancement processing may include feature enhancement processing of an image dimension and feature enhancement processing of a feature dimension, among others. Because the CT image data volume of the patient is less, and a large number of CT images are needed for more accurate detection results output by the detection model, the embodiment of the invention performs characteristic enhancement processing on the denoised image sample. The embodiment of the present invention exemplarily provides a method for performing feature enhancement processing on a denoised image sample, which can be specifically seen as follows:
the first method is as follows: and inputting the denoised image sample into a pre-trained confrontation characteristic generation network to obtain an enhanced image sample output by the confrontation characteristic generation network. The method comprises the steps that a confrontation feature generation network (GAN) (generic adaptive networks) model is also formed, wherein the GAN model comprises a feature generator and a feature discriminator which are connected, the feature generator is used for generating confrontation features corresponding to a denoised image sample, the input of the confrontation features is the denoised image sample, and the output of the confrontation features is the confrontation features; the feature discriminator is used for discriminating the input features as the confrontation features generated by the feature generator or the real features extracted aiming at the denoised image samples, and in practical application, the loss function can be calculated through the output of the feature discriminator, and then the feature generator and the feature discriminator are trained based on the loss function, so that the feature generator generates the confrontation features closer to the real features.
The second method comprises the following steps: and carrying out image processing on the denoised image sample to obtain an enhanced image sample output by the countermeasure feature generation network. Wherein the image processing comprises one or more of rotation, scaling, translation and warping. For example, the denoised image sample is rotated to obtain enhanced image samples at different angles, the denoised image sample is scaled or translated to obtain enhanced image samples at different regions, and the denoised image sample is distorted to obtain a deformed enhanced image sample.
Step a5, taking the denoised image sample and the enhanced image sample as an image training set.
And b, training a focus detection model based on the image samples in the image training set. In specific implementation, the enhanced image sample and the label carried by the enhanced image sample can be jointly input into the focus detection model, the focus detection model outputs a corresponding training result aiming at the input enhanced image sample, and the training result and the label are calculated to calculate a loss function of the focus detection model, so that the focus detection model is trained through the loss function.
And c, calculating a loss function of the focus detection model based on the training result of each training. In a kind of implementationIn this way, the loss function L of the lesion detection model may be calculated as followsfl
Figure BDA0002364353470000111
Wherein a represents a first hyperparameter, gamma represents a second hyperparameter, y represents a lesion label, and y' represents a training result. The above formula provided by the embodiment of the invention is obtained by improving FocalLoss.
Since the lesion detection model provided in the embodiment of the present invention includes the centroid convolution sub-network, the size convolution sub-network, and the offset convolution sub-network, it needs to be trained on the centroid convolution sub-network, the size convolution sub-network, and the offset convolution sub-network, and further needs to calculate the loss function of the convolution sub-networks, specifically, a first loss value of the centroid convolution sub-network is calculated for the coordinates of the lesion centroid output by the centroid convolution sub-network using the above formula, a second loss value of the size convolution sub-network is calculated for the size of the lesion output by the size convolution sub-network using the above formula, and a third loss value of the offset convolution sub-network is calculated for the lesion offset output by the offset sub-.
And d, stopping training the focus detection model when the loss function of the focus detection model is converged.
To facilitate understanding of the above-mentioned lesion detection method, another lesion detection method is provided in the embodiments of the present invention, and the above-mentioned lesion detection method can be preferably applied to lung lesion detection, referring to a flowchart of another lesion detection method shown in fig. 4, the method mainly includes the following steps S402 to S412:
in step S402, first lung CT data (i.e., the original image sample) scanned by the CT apparatus is obtained.
And S404, processing the first lung CT data to obtain an image training set. Wherein the processing comprises (1) resampling processing; (2) background removal processing; (3) normalization processing; (4) data enhancement processing (i.e., the aforementioned feature enhancement processing). For details, reference may be made to the foregoing embodiments, which are not described herein again.
Step S406, training a focus detection model by using the image training set. The training process of the lesion detection model can be referred to the foregoing embodiments, and the embodiments of the present invention are not described herein again.
In step S408, second lung CT data (i.e., the medical scan image) scanned by the CT apparatus is obtained.
And S410, processing the second lung CT data, and inputting the processed second lung CT data into the trained lesion detection model to obtain a lung lesion detection result. The processing performed on the second lung CT data may include the resampling processing and the normalization processing mentioned in step S404.
Step S412, labeling the detection result to the second lung CT data, and sending the second lung CT data labeled with the detection result to the designated terminal, so as to display the second lung CT data labeled with the detection result through the designated terminal. The designated terminal can comprise various devices such as a mobile phone, a tablet or a computer, so that medical staff and patients can be better assisted to view detection results.
In summary, the embodiments of the present invention have at least one of the following features:
(1) the embodiment of the invention performs resampling processing on the CT data, so that the distribution of the CT data is more uniform.
(2) According to the embodiment of the invention, 3D images are adopted for medical scanning images and enhanced image samples in an image training set, so that the focus detection process is scientifically based, and the embodiment of the invention does not need to convert 3D CT images into 2D CT images, so that the detection result is more consistent with the characteristics of CT data.
(3) According to the embodiment of the invention, the GAN model is adopted to perform data enhancement on the CT data required by training, so that the number of samples is effectively increased.
(4) The invention directly predicts the focus central point coordinate, the focus size and the focus offset of the focus, so that the detection result has better robustness.
(5) The embodiment of the invention trains the focus detection model by adopting the improved FocalLoss loss function, so that the trained focus detection model can output more accurate detection results.
With respect to the lesion detection method provided in the foregoing embodiment, an embodiment of the present invention provides a lesion detection apparatus, referring to a schematic structural diagram of a lesion detection apparatus shown in fig. 5, the apparatus mainly includes the following components:
an image obtaining module 502, configured to obtain a medical scanning image to be detected;
a preprocessing module 504, configured to preprocess the medical scanning image to obtain a denoised scanning image corresponding to the medical scanning image;
the detection module 506 is configured to input the denoised scan image into a pre-trained lesion detection model to obtain a detection result output by the lesion detection model; wherein, the detection result comprises one or more of the coordinates of the focus central point, the focus size and the focus offset;
and the labeling module 508 is configured to label the detection result to the medical scanning image.
According to the focus detection device provided by the embodiment of the invention, the denoising scanning image can be obtained by preprocessing the medical scanning image, so that the problems of inconsistent size and inconsistent precision of the medical scanning image are better solved, and the accuracy of focus detection is further effectively improved.
In one embodiment, the medical scan image is a 3D image; the pretreatment comprises normalization treatment and/or resampling treatment; the preprocessing module 504 is further configured to: if the preprocessing comprises normalization processing, mapping the color values of all pixels in the medical scanning image to a plurality of target color values in a preset color range respectively; if the preprocessing comprises resampling processing, removing noise in the medical scanning image by utilizing the resampling processing, and adjusting the current size of the medical scanning image to be a first target size; and taking the preprocessed medical scanning image as a de-noised scanning image.
In one embodiment, the lesion detection model includes a feature map generation network and a first convolution network; the first convolution network comprises a central point convolution sub-network, a size convolution sub-network and an offset convolution sub-network which are respectively connected with the feature map generation network; the detection module 506 is further configured to: inputting the denoised scanned image into a characteristic map generation network to obtain a characteristic map of the denoised scanned image; respectively inputting the feature maps into a central point convolution sub-network, a size convolution sub-network and an offset convolution sub-network to obtain detection results; the central point convolution sub-network is used for detecting the central point coordinates of the focus, the size convolution sub-network is used for detecting the size of the focus, and the offset convolution sub-network is used for detecting the offset of the focus.
In one embodiment, the feature map generation network comprises a second convolution network and a feature skeletal model; the detection module 506 is further configured to: inputting the denoised scanned image into a second convolution network, and resampling the denoised scanned image through the second convolution network to obtain a denoised scanned image with a second target size; and inputting the de-noised scanned image with the second target size into the characteristic skeleton model to obtain a characteristic diagram of the de-noised scanned image.
In one embodiment, the lesion detection model further includes a training module, configured to: acquiring an image training set; training a focus detection model based on image samples in the image training set; calculating a loss function of the focus detection model based on the training result of each training; and when the loss function of the focus detection model is converged, stopping training the focus detection model.
In one embodiment, the training module is further configured to: acquiring an original image sample carrying a focus label; wherein the lesion label includes a lesion bounding box; the original image sample is a 3D image; setting the color value of each pixel in the peripheral area of the focus bounding box in the original image sample as a preset color value; preprocessing a region where a focus surrounding frame in an original image sample is located to obtain a denoised image sample corresponding to the original image sample; carrying out characteristic enhancement processing on the denoised image sample to obtain an enhanced image sample corresponding to the denoised image sample; and taking the denoised image sample and the enhanced image sample as an image training set.
In one embodiment, the training module is further configured to: inputting the denoised image sample into a pre-trained confrontation characteristic generation network to obtain an enhanced image sample output by the confrontation characteristic generation network; and/or; carrying out image processing on the denoised image sample to obtain an enhanced image sample output by a countermeasure characteristic generation network; wherein the image processing comprises one or more of rotation, scaling, translation and warping.
In one embodiment, the training module is further configured to: calculating a loss function L of a lesion detection model according to the following formulafl
Figure BDA0002364353470000151
Wherein a represents a first hyperparameter, gamma represents a second hyperparameter, y represents a lesion label, and y' represents a training result.
The device provided by the embodiment of the present invention has the same implementation principle and technical effect as the method embodiments, and for the sake of brief description, reference may be made to the corresponding contents in the method embodiments without reference to the device embodiments.
The embodiment of the invention provides electronic equipment, which particularly comprises a processor and a storage device; the storage means has stored thereon a computer program which, when executed by the processor, performs the method of any of the above described embodiments.
Fig. 6 is a schematic structural diagram of an electronic device 100 according to an embodiment of the present invention, where the electronic device 100 includes: a processor 60, a memory 61, a bus 62 and a communication interface 63, wherein the processor 60, the communication interface 63 and the memory 61 are connected through the bus 62; the processor 60 is arranged to execute executable modules, such as computer programs, stored in the memory 61.
The Memory 61 may include a high-speed Random Access Memory (RAM) and may also include a non-volatile Memory (non-volatile Memory), such as at least one disk Memory. The communication connection between the network element of the system and at least one other network element is realized through at least one communication interface 63 (which may be wired or wireless), and the internet, a wide area network, a local network, a metropolitan area network, and the like can be used.
The bus 62 may be an ISA bus, PCI bus, EISA bus, or the like. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one double-headed arrow is shown in FIG. 6, but that does not indicate only one bus or one type of bus.
The memory 61 is used for storing a program, the processor 60 executes the program after receiving an execution instruction, and the method executed by the apparatus defined by the flow process disclosed in any of the foregoing embodiments of the present invention may be applied to the processor 60, or implemented by the processor 60.
The processor 60 may be an integrated circuit chip having signal processing capabilities. In implementation, the steps of the above method may be performed by integrated logic circuits of hardware or instructions in the form of software in the processor 60. The Processor 60 may be a general-purpose Processor, and includes a Central Processing Unit (CPU), a Network Processor (NP), and the like; the device can also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA) or other Programmable logic device, a discrete Gate or transistor logic device, or a discrete hardware component. The various methods, steps and logic blocks disclosed in the embodiments of the present invention may be implemented or performed. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like. The steps of the method disclosed in connection with the embodiments of the present invention may be directly implemented by a hardware decoding processor, or implemented by a combination of hardware and software modules in the decoding processor. The software module may be located in ram, flash memory, rom, prom, or eprom, registers, etc. storage media as is well known in the art. The storage medium is located in a memory 61, and the processor 60 reads the information in the memory 61 and, in combination with its hardware, performs the steps of the above method.
The computer program product of the readable storage medium provided in the embodiment of the present invention includes a computer readable storage medium storing a program code, where instructions included in the program code may be used to execute the method described in the foregoing method embodiment, and specific implementation may refer to the foregoing method embodiment, which is not described herein again.
The functions, if implemented in the form of software functional units and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.
Finally, it should be noted that: the above-mentioned embodiments are only specific embodiments of the present invention, which are used for illustrating the technical solutions of the present invention and not for limiting the same, and the protection scope of the present invention is not limited thereto, although the present invention is described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: any person skilled in the art can modify or easily conceive the technical solutions described in the foregoing embodiments or equivalent substitutes for some technical features within the technical scope of the present disclosure; such modifications, changes or substitutions do not depart from the spirit and scope of the embodiments of the present invention, and they should be construed as being included therein. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.

Claims (11)

1. A method of lesion detection, comprising:
acquiring a medical scanning image to be detected;
preprocessing the medical scanning image to obtain a de-noised scanning image corresponding to the medical scanning image;
inputting the de-noised scanning image into a pre-trained focus detection model to obtain a detection result output by the focus detection model; wherein the detection result comprises one or more of a focus central point coordinate, a focus size and a focus offset;
and labeling the detection result to the medical scanning image.
2. The method of claim 1, wherein the medical scan image is a 3D image; the preprocessing comprises normalization processing and/or resampling processing;
the step of preprocessing the medical scanning image to obtain a de-noised scanning image corresponding to the medical scanning image comprises the following steps:
if the preprocessing comprises the normalization processing, mapping the color values of the pixels in the medical scanning image to a plurality of target color values in a preset color range respectively;
if the preprocessing comprises the resampling processing, removing noise in the medical scanning image by using the resampling processing, and adjusting the current size of the medical scanning image to a first target size;
and taking the preprocessed medical scanning image as a de-noised scanning image.
3. The method of claim 1, wherein the lesion detection model comprises a feature map generation network and a first convolution network; the first convolution network comprises a central point convolution sub-network, a size convolution sub-network and an offset convolution sub-network which are respectively connected with the feature map generation network;
the step of inputting the de-noised scanning image into a pre-trained focus detection model to obtain a detection result output by the focus detection model comprises the following steps:
inputting the de-noised scanned image into the characteristic map generation network to obtain a characteristic map of the de-noised scanned image;
inputting the feature map into the central point convolution sub-network, the size convolution sub-network and the offset convolution sub-network respectively to obtain detection results; the central point convolution sub-network is used for detecting the central point coordinates of the focus, the size convolution sub-network is used for detecting the size of the focus, and the offset convolution sub-network is used for detecting the offset of the focus.
4. The method of claim 3, wherein the feature map generation network comprises a second convolution network and a feature skeletal model;
the step of inputting the denoised scanned image into the feature map generation network to obtain the feature map of the denoised scanned image includes:
inputting the denoised scanned image into the second convolution network, and resampling the denoised scanned image through the second convolution network to obtain a denoised scanned image with a second target size;
and inputting the de-noised scanning image with the second target size into the characteristic skeleton model to obtain a characteristic image of the de-noised scanning image.
5. The method of claim 3, wherein the step of training the lesion detection model comprises:
acquiring an image training set;
training the focus detection model based on image samples in the image training set;
calculating a loss function of the lesion detection model based on a training result of each training;
stopping training the lesion detection model when a loss function of the lesion detection model converges.
6. The method of claim 5, wherein the step of acquiring a training set of images comprises:
acquiring an original image sample carrying a focus label; wherein the lesion label comprises a lesion bounding box; the original image sample is a 3D image;
setting the color value of each pixel in the peripheral area of the focus bounding box in the original image sample as a preset color value;
the area where the focus bounding box is located in the original image sample is preprocessed to obtain a denoised image sample corresponding to the original image sample;
performing feature enhancement processing on the denoised image sample to obtain an enhanced image sample corresponding to the denoised image sample;
and taking the denoised image sample and the enhanced image sample as an image training set.
7. The method according to claim 6, wherein the step of performing feature enhancement processing on the denoised image sample to obtain an enhanced image sample corresponding to the denoised image sample comprises:
inputting the denoised image sample into a pre-trained confrontation characteristic generation network to obtain an enhanced image sample output by the confrontation characteristic generation network;
and/or;
carrying out image processing on the de-noised image sample to obtain an enhanced image sample output by the countermeasure characteristic generation network; wherein the image processing comprises one or more of rotation, scaling, translation, and warping.
8. The method of claim 5, wherein the step of calculating a loss function of the lesion detection model based on the training results of each training comprises:
calculating a loss function of the lesion detection model according to the following formulaNumber Lfl
Figure FDA0002364353460000031
Wherein a represents a first hyperparameter, gamma represents a second hyperparameter, y represents a lesion label, and y' represents a training result.
9. A lesion detection apparatus, comprising:
the image acquisition module is used for acquiring a medical scanning image to be detected;
the preprocessing module is used for preprocessing the medical scanning image to obtain a denoising scanning image corresponding to the medical scanning image;
the detection module is used for inputting the de-noising scanning image into a pre-trained focus detection model to obtain a detection result output by the focus detection model; wherein the detection result comprises one or more of a focus central point coordinate, a focus size and a focus offset;
and the marking module is used for marking the detection result to the medical scanning image.
10. An electronic device comprising a processor and a memory;
the memory has stored thereon a computer program which, when executed by the processor, performs the method of any of claims 1 to 8.
11. A computer storage medium storing computer software instructions for use in the method of any one of claims 1 to 8.
CN202010034320.4A 2020-01-13 2020-01-13 Focus detection method and device and electronic equipment Pending CN111242926A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202010034320.4A CN111242926A (en) 2020-01-13 2020-01-13 Focus detection method and device and electronic equipment

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010034320.4A CN111242926A (en) 2020-01-13 2020-01-13 Focus detection method and device and electronic equipment

Publications (1)

Publication Number Publication Date
CN111242926A true CN111242926A (en) 2020-06-05

Family

ID=70877698

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010034320.4A Pending CN111242926A (en) 2020-01-13 2020-01-13 Focus detection method and device and electronic equipment

Country Status (1)

Country Link
CN (1) CN111242926A (en)

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111739015A (en) * 2020-06-30 2020-10-02 北京青燕祥云科技有限公司 Human chest CT scanning sequence data-based rib fracture AI auxiliary diagnosis method and system
CN112651400A (en) * 2020-12-31 2021-04-13 重庆西山科技股份有限公司 Stereoscopic endoscope auxiliary detection method, system, device and storage medium
CN113538298A (en) * 2021-09-07 2021-10-22 北京大学深圳医院 Focus high-definition image reconstruction method, device, equipment and storage medium
CN113627492A (en) * 2021-07-20 2021-11-09 东软医疗系统股份有限公司 Method for determining size of scanning object, scanning method, scanning device and electronic equipment
CN116862906A (en) * 2023-08-24 2023-10-10 武汉大学人民医院(湖北省人民医院) Eye detection device and method

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106940816A (en) * 2017-03-22 2017-07-11 杭州健培科技有限公司 Connect the CT image Lung neoplasm detecting systems of convolutional neural networks entirely based on 3D
CN108648172A (en) * 2018-03-30 2018-10-12 四川元匠科技有限公司 A kind of CT figure Lung neoplasm detecting systems based on 3D-Unet
CN109299644A (en) * 2018-07-18 2019-02-01 广东工业大学 A kind of vehicle target detection method based on the full convolutional network in region
CN109447963A (en) * 2018-10-22 2019-03-08 杭州依图医疗技术有限公司 A kind of method and device of brain phantom identification
CN109544534A (en) * 2018-11-26 2019-03-29 上海联影智能医疗科技有限公司 A kind of lesion image detection device, method and computer readable storage medium
CN109584218A (en) * 2018-11-15 2019-04-05 首都医科大学附属北京友谊医院 A kind of construction method of gastric cancer image recognition model and its application
CN109801270A (en) * 2018-12-29 2019-05-24 北京市商汤科技开发有限公司 Anchor point determines method and device, electronic equipment and storage medium
CN110210571A (en) * 2019-06-10 2019-09-06 腾讯科技(深圳)有限公司 Image-recognizing method, device, computer equipment and computer readable storage medium

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106940816A (en) * 2017-03-22 2017-07-11 杭州健培科技有限公司 Connect the CT image Lung neoplasm detecting systems of convolutional neural networks entirely based on 3D
CN108648172A (en) * 2018-03-30 2018-10-12 四川元匠科技有限公司 A kind of CT figure Lung neoplasm detecting systems based on 3D-Unet
CN109299644A (en) * 2018-07-18 2019-02-01 广东工业大学 A kind of vehicle target detection method based on the full convolutional network in region
CN109447963A (en) * 2018-10-22 2019-03-08 杭州依图医疗技术有限公司 A kind of method and device of brain phantom identification
CN109584218A (en) * 2018-11-15 2019-04-05 首都医科大学附属北京友谊医院 A kind of construction method of gastric cancer image recognition model and its application
CN109544534A (en) * 2018-11-26 2019-03-29 上海联影智能医疗科技有限公司 A kind of lesion image detection device, method and computer readable storage medium
CN109801270A (en) * 2018-12-29 2019-05-24 北京市商汤科技开发有限公司 Anchor point determines method and device, electronic equipment and storage medium
CN110210571A (en) * 2019-06-10 2019-09-06 腾讯科技(深圳)有限公司 Image-recognizing method, device, computer equipment and computer readable storage medium

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
张声超: "基于特征融合的肺结节检测关键技术研究", 《中国优秀硕士学位论文全文数据库_信息科技辑》 *

Cited By (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111739015A (en) * 2020-06-30 2020-10-02 北京青燕祥云科技有限公司 Human chest CT scanning sequence data-based rib fracture AI auxiliary diagnosis method and system
CN112651400A (en) * 2020-12-31 2021-04-13 重庆西山科技股份有限公司 Stereoscopic endoscope auxiliary detection method, system, device and storage medium
CN112651400B (en) * 2020-12-31 2022-11-15 重庆西山科技股份有限公司 Stereoscopic endoscope auxiliary detection method, system, device and storage medium
CN113627492A (en) * 2021-07-20 2021-11-09 东软医疗系统股份有限公司 Method for determining size of scanning object, scanning method, scanning device and electronic equipment
CN113538298A (en) * 2021-09-07 2021-10-22 北京大学深圳医院 Focus high-definition image reconstruction method, device, equipment and storage medium
CN113538298B (en) * 2021-09-07 2024-01-05 北京大学深圳医院 Focal high-definition image reconstruction method, device, equipment and storage medium
CN116862906A (en) * 2023-08-24 2023-10-10 武汉大学人民医院(湖北省人民医院) Eye detection device and method
CN116862906B (en) * 2023-08-24 2023-12-12 武汉大学人民医院(湖北省人民医院) Eye detection device and method

Similar Documents

Publication Publication Date Title
CN111242926A (en) Focus detection method and device and electronic equipment
KR102581685B1 (en) Classification and 3D modeling of 3D oral and maxillofacial structures using deep learning
KR102560911B1 (en) Image processing apparatus, image processing method, and storage medium
WO2019068741A2 (en) Automated classification and taxonomy of 3d teeth data using deep learning methods
CN110176010B (en) Image detection method, device, equipment and storage medium
CN111597945B (en) Target detection method, device, equipment and medium
WO2022100065A1 (en) Image registration method and apparatus, electronic device and storage medium
CN111488872B (en) Image detection method, image detection device, computer equipment and storage medium
CN110443254B (en) Method, device, equipment and storage medium for detecting metal area in image
CN111598862B (en) Breast molybdenum target image segmentation method, device, terminal and storage medium
CN111862044A (en) Ultrasonic image processing method and device, computer equipment and storage medium
CN113554742A (en) Three-dimensional image reconstruction method, device, equipment and storage medium
JP2023515367A (en) Out-of-distribution detection of input instances to model
CN111951215A (en) Image detection method and device and computer readable storage medium
CN111723836A (en) Image similarity calculation method and device, electronic equipment and storage medium
CN116363038A (en) Ultrasonic image fusion method, device, computer equipment and storage medium
CN113793316B (en) Ultrasonic scanning area extraction method, device, equipment and storage medium
JP7418171B2 (en) Image processing device, radiography system, image processing method and program
CN111369425A (en) Image processing method, image processing device, electronic equipment and computer readable medium
CN111358492A (en) Four-dimensional contrast image generation method, device, equipment and storage medium
CN113393475B (en) Mammary gland molybdenum target image segmentation device, electronic equipment, medical equipment and medium
CN116109891B (en) Image data amplification method, device, computing equipment and storage medium
JP2023100836A (en) Image processing device, image processing method and program
CN114627326A (en) Image identification method and device, electronic equipment and readable storage medium
CN116030073A (en) Aorta region of interest extraction method, electronic device, and storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
RJ01 Rejection of invention patent application after publication

Application publication date: 20200605

RJ01 Rejection of invention patent application after publication