CN113920100A - Knowledge distillation-based weak supervision bone scanning image hot spot segmentation method and system - Google Patents
Knowledge distillation-based weak supervision bone scanning image hot spot segmentation method and system Download PDFInfo
- Publication number
- CN113920100A CN113920100A CN202111216294.8A CN202111216294A CN113920100A CN 113920100 A CN113920100 A CN 113920100A CN 202111216294 A CN202111216294 A CN 202111216294A CN 113920100 A CN113920100 A CN 113920100A
- Authority
- CN
- China
- Prior art keywords
- image
- hot spot
- classification network
- network model
- segmentation
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Withdrawn
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0012—Biomedical image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
- G06F18/2415—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on parametric or probabilistic models, e.g. based on likelihood ratio or false acceptance rate versus a false rejection rate
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/11—Region-based segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/136—Segmentation; Edge detection involving thresholding
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10116—X-ray image
- G06T2207/10128—Scintigraphy
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20021—Dividing image into blocks, subimages or windows
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30004—Biomedical image processing
- G06T2207/30008—Bone
Abstract
The invention discloses a knowledge distillation-based weak supervision bone scanning image hot spot segmentation method and system, belonging to the technical field of bone scanning image processing, and being used for acquiring an image matrix of a bone scanning image, wherein the bone scanning image comprises a precursor image and a posterior image; carrying out preprocessing operation on the bone scanning image and then segmenting the bone scanning image to obtain a segmented part image of the bone; respectively establishing and training a classification network model for different parts, and testing the network effect; constructing a lightweight classification network model for each part, transferring knowledge of the original classification network model to the lightweight classification network model by a knowledge distillation method, and judging whether hot spots exist in the parts by the lightweight classification network model; if the position of the hot spot is obtained, the position is sent to a hot spot segmentation module to extract the position of the hot spot; the method can judge whether the hot spot exists under the condition of less supervision information, can segment the specific hot spot position, and has high efficiency and good adaptability under the weak supervision situation.
Description
Technical Field
The invention belongs to the technical field of bone scanning image processing, and relates to a knowledge distillation-based weak supervision bone scanning image hot spot segmentation method and system.
Background
As one of the most common and serious complications of cancer, bone metastasis has long been recognized as a powerful cause of suffering in cancer patients. According to statistics, the incidence rate of bone metastasis tumor is 35-40 times of that of primary malignant tumor of bone, and the bone metastasis tumor is one of main reasons of cancer pain, and pathological fracture, spinal cord compression, hypercalcemia, bone marrow failure and other complications caused by the bone metastasis tumor are often further accelerated to further accelerate the development of disease conditions, so that the life quality of cancer patients is seriously influenced. 80% -90% of patients with bone metastases from malignant primary tumors die within a few years before they are effectively treated. Therefore, an effective diagnosis of bone metastasis would be one of the determining factors in determining the patient's condition development and subsequent treatment.
In the detection of bone metastases, the examination of medical imaging is a critical step. Among the current series of imaging techniques, PET, CT, MRI and nuclear bone imaging (BS) techniques are relatively predominant in the evaluation of malignant bone metastases. Notably, while PET and PET/CT are the most effective bone metastasis screening techniques in recent years, nuclide bone imaging (BS) remains the most commonly used imaging method in nuclear medicine due to a range of factors such as efficiency cost.
Because the traditional nuclear medicine physician needs to spend a great deal of time on the manual film reading for the relatively mechanical work such as contrast adjustment and the like. In recent years, a large number of scholars try to realize automatic bone scanning detection by an artificial intelligence method, and certain results are obtained. However, the following problems still exist in the practical application of the models:
1) the model which is only classified can only determine whether a hot spot exists in a part, cannot find the exact position of the hot spot, cannot meet the actual requirement, and has limited application value;
2) for a model capable of performing hot spot detection or segmentation, a large amount of data labeling at the early stage cannot be avoided, however, for labeling bone metastasis, a set of unified authoritative standards is not provided in the industry until now, so that the data labeling is seriously dependent on subjective judgment, the reliability and the availability of supervision information are relatively low, and the practical value of the model cannot be in direct proportion to the heavy labeling work at the previous stage;
3) in the absence of supervision information related to the hot spot, the classical hot spot segmentation method is often difficult to handle the more complex environment around the bone metastasis hot spot, including both background and bone;
4) the bone scan detection method usually needs to perform a subdivision judgment for each part, which results in a longer time consumption and more computing resources for outputting the result.
Disclosure of Invention
The invention aims to: the invention provides a knowledge distillation-based method and a knowledge distillation-based system for segmenting hot spots of a weak supervision bone scanning image, which solve the problem that whether hot spots exist or not and the specific hot spot positions cannot be segmented effectively under the condition of less supervision information.
The technical scheme adopted by the invention is as follows:
the weak supervision bone scanning image hot spot segmentation method based on knowledge distillation comprises the following steps:
step 1: obtaining an image matrix of a bone scan image, the bone scan image comprising a precursor image and a posterior image;
step 2: carrying out preprocessing operation on the bone scanning image and then segmenting the bone scanning image to obtain a segmented part image of the bone;
and step 3: respectively establishing and training a classification network model for different parts, and testing the network effect;
and 4, step 4: constructing a lightweight classification network model for each part, transferring knowledge of the original classification network model to the lightweight classification network model by a knowledge distillation method, and judging whether hot spots exist in the parts by the lightweight classification network model;
and 5: and if the position where the hot spot exists is obtained in the step 4, sending the position into a hot spot segmentation module to extract the position of the hot spot.
Further, the step 2 comprises:
step 2.1: respectively and sequentially carrying out contrast adjustment, automatic threshold segmentation, closing operation, median filtering, first opening operation, mask operation, Gaussian blur, equalization, picture cutting, simple threshold segmentation and second opening operation on the front body image and the back body image, and improving the image quality;
step 2.2: and respectively carrying out key point positioning and image segmentation on the preprocessed precursor image and the preprocessed posterior image based on an anatomical segmentation algorithm, and obtaining a plurality of segmentation part images after segmentation.
Further, the step 3 comprises:
step 3.1: establishing a classification network model for each part, and training the classification network model by using the training set slice images, the labels of labelme and the depth-based learning model, wherein the networks can be merged for the symmetrical parts;
step 3.2: and detecting whether the hot spot of the detected image exists or not by using the trained classification network model.
Further, the step 4 comprises:
step 4.1: constructing a lightweight classification network module with the same input and output format as the original network;
step 4.2: training a lightweight classification network model by a knowledge distillation method together with a soft label obtained from an original classification network model and a label given by a doctor;
step 4.3: and detecting whether the hot spot of the detected image exists or not by using the trained lightweight classification network model.
Further, the step 5 comprises:
step 5.1: if the position where the hot spot exists is obtained, regularization, mean filtering and Gaussian filtering fuzzification are sequentially carried out on the corresponding slice image;
step 5.2: separating the skeleton and the background of the image by using one-time threshold segmentation, and filling the background by using a segmented threshold;
step 5.3: and (5) segmenting the hot spot region by using threshold segmentation again for the image obtained in the step 5.2.
The weak supervision bone scanning image hot spot segmentation system based on knowledge distillation comprises an image acquisition module, an image processing module, a classification module, a lightweight module and a hot spot segmentation module;
the image acquisition module is used for acquiring an image matrix of a bone scanning image, wherein the bone scanning image comprises a precursor image and a posterior image;
the image processing module carries out preprocessing operation on the bone scanning image and then carries out segmentation to obtain a segmentation part image of the bone;
the classification module is used for respectively establishing and training classification network models at different parts and testing network effects;
the light-weight module is used for constructing a lighter classification network model for each part, transferring knowledge of the original classification network model to the light classification network model through a knowledge distillation method, and judging whether hot spots exist in the parts through the light classification network model;
and if the position of the hot spot is obtained, sending the position into the hot spot segmentation module to extract the position of the hot spot.
Further, the image processing module for preprocessing the bone scan image comprises: respectively and sequentially carrying out contrast adjustment, automatic threshold segmentation, closing operation, median filtering, first opening operation, mask operation, Gaussian blur, equalization, picture cutting, simple threshold segmentation and second opening operation on the front body image and the back body image, and improving the image quality;
the image processing module segmenting the bone scan image comprises: and respectively carrying out key point positioning and image segmentation on the preprocessed precursor image and the preprocessed posterior image based on an anatomical segmentation algorithm, and obtaining a plurality of segmentation part images after segmentation.
Further, the classification module is used for respectively establishing and training classification network models at different parts, and the testing network effect comprises: establishing a classification network model for each part, and training the classification network model by using the training set slice images, the labels of labelme and the depth-based learning model, wherein the networks can be merged for the symmetrical parts; and detecting whether the hot spot of the detected image exists or not by using the trained classification network model.
Further, the light-weight module is used for constructing a lighter classification network model for each part, transferring the knowledge of the original classification network model to the light-weight classification network model by a knowledge distillation method, and judging whether hot spots exist in the part or not by the light-weight classification network model comprises the following steps: constructing a lightweight classification network module with the same input and output format as the original network; training a lightweight classification network model by a knowledge distillation method together with a soft label obtained from an original classification network model and a label given by a doctor; and detecting whether the hot spot of the detected image exists or not by using the trained lightweight classification network model.
Further, if a part where the hot spot exists is obtained, the part is sent to the hot spot segmentation module to extract the position of the hot spot: if the position where the hot spot exists is obtained, regularization, mean filtering and Gaussian filtering fuzzification are sequentially carried out on the corresponding slice image; separating the skeleton and the background of the image by using one-time threshold segmentation, and filling the background by using a segmented threshold; the hot spot regions are segmented out again using threshold segmentation for the image.
In summary, due to the adoption of the technical scheme, the invention has the beneficial effects that:
1. the invention relates to a weak supervision bone scanning image hotspot segmentation method and system based on knowledge distillation, which comprises the steps of preprocessing and segmenting original data by utilizing an image processing and deep learning technology, merging data of corresponding parts of a left body and a right body based on the symmetry of the bodies, respectively constructing a deep neural network model, training, establishing more efficient and concise light-weight classification network modules for different parts after the training is finished, and transferring the knowledge of a complex network to the light-weight classification network modules through the training of a soft label and a label given by a doctor by using a knowledge distillation method so as to ensure that the model achieves a reliable prediction effect.
2. In the hot spot segmentation stage, based on the characteristic that a bone scanning image has a background-bone-hot spot three-layer structure, a set of simple and efficient method and system is specially designed for the task, the training is performed without monitoring information such as the position and the size of a hot spot in the early stage, a good segmentation effect can be obtained only by two-time threshold segmentation and one-time pixel filling, additional external information is not needed, simplicity and rapidness are realized, and the high efficiency and the good adaptability under the weak monitoring situation are further highlighted.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings that are required to be used in the embodiments will be briefly described below, it should be understood that the following drawings only illustrate some embodiments of the present invention and therefore should not be considered as limiting the scope, and that for those skilled in the art, other relevant drawings can be obtained according to the drawings without inventive effort, wherein:
FIG. 1 is a flow chart of a method for weakly supervised bone scan image hot spot segmentation based on knowledge distillation;
FIG. 2 is a schematic diagram of training a lightweight network using knowledge distillation;
FIG. 3 is a schematic diagram of the output of a chest slice image through the substeps of step 5;
fig. 4 is a schematic diagram of the output of the lumbar spine slice image through the substeps in step 5;
fig. 5 is a schematic diagram of the output of the pelvic slice image through the substeps in step 5.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the detailed description and specific examples, while indicating the preferred embodiment of the invention, are intended for purposes of illustration only and are not intended to limit the scope of the invention. The components of embodiments of the present invention generally described and illustrated in the figures herein may be arranged and designed in a wide variety of different configurations.
Thus, the following detailed description of the embodiments of the present invention, presented in the figures, is not intended to limit the scope of the invention, as claimed, but is merely representative of selected embodiments of the invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments of the present invention without making any creative effort, shall fall within the protection scope of the present invention.
It is noted that relational terms such as "first" and "second," and the like, may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.
A knowledge distillation-based weak supervision bone scanning image hot spot segmentation method and system solve the problem that whether hot spots exist or not and specific hot spot positions cannot be effectively judged under the condition of less supervision information.
The weak supervision bone scanning image hot spot segmentation method based on knowledge distillation comprises the following steps:
step 1: obtaining an image matrix of a bone scan image, the bone scan image comprising a precursor image and a posterior image;
step 2: carrying out preprocessing operation on the bone scanning image and then segmenting the bone scanning image to obtain a segmented part image of the bone;
and step 3: respectively establishing and training a classification network model for different parts, and testing the network effect;
and 4, step 4: constructing a lightweight classification network model for each part, transferring knowledge of the original classification network model to the lightweight classification network model by a knowledge distillation method, and judging whether hot spots exist in the parts by the lightweight classification network model;
and 5: and if the position where the hot spot exists is obtained in the step 4, sending the position into a hot spot segmentation module to extract the position of the hot spot.
The weak supervision bone scanning image hot spot segmentation system based on knowledge distillation comprises an image acquisition module, an image processing module, a classification module, a lightweight module and a hot spot segmentation module;
the image acquisition module is used for acquiring an image matrix of a bone scanning image, wherein the bone scanning image comprises a precursor image and a posterior image;
the image processing module carries out preprocessing operation on the bone scanning image and then carries out segmentation to obtain a segmentation part image of the bone;
the classification module is used for respectively establishing and training classification network models at different parts and testing network effects;
the light-weight module is used for constructing a lighter classification network model for each part, transferring knowledge of the original classification network model to the light classification network model through a knowledge distillation method, and judging whether hot spots exist in the parts through the light classification network model;
and if the position of the hot spot is obtained, sending the position into the hot spot segmentation module to extract the position of the hot spot.
The features and properties of the present invention are described in further detail below with reference to examples.
Example 1
The preferred embodiment of the present invention provides a method for segmenting a hot spot of a weakly supervised bone scan image based on knowledge distillation, as shown in fig. 1 and 2, comprising the following steps:
step 1: obtaining an image matrix of a bone scan image, the bone scan image comprising a precursor image and a posterior image;
step 1.1: detecting the skeleton of the whole body by using a radioactive detection imaging instrument to obtain the scanning images of the bone of the front body and the back body in a DICOM format;
step 1.2: reading the bone scanning image in the DICOM format by using a dcmread method in a pydicom packet of Python, and acquiring the bone scanning image in an image matrix format;
step 2: carrying out preprocessing operation on the bone scanning image and then segmenting the bone scanning image to obtain a segmented part image of the bone;
step 2.1: respectively and sequentially carrying out contrast adjustment, automatic threshold segmentation, closing operation, median filtering, first opening operation, mask operation, Gaussian blur, equalization, picture cutting, simple threshold segmentation and second opening operation on the front body image and the back body image, and improving the image quality;
step 2.1.1: contrast adjustment is respectively carried out on the front body image and the back body image, pixel values can be averaged by adjusting the contrast, and the problem that pixels of partial images are low is avoided;
step 2.1.2: performing automatic threshold segmentation to eliminate noise with low pixel value except bone and soft tissue, wherein the pixel value of the noise is about 0-4;
step 2.1.3: performing closed operation to fill the pixel points in the whole human skeleton;
step 2.1.4: carrying out median filtering to eliminate isolated noise points so as to smooth the image and reduce the fuzziness of the image;
step 2.1.5: performing first opening operation to eliminate burr pixel points at the edge of the human skeleton;
step 2.1.6: performing mask operation to obtain a foreground image of the bone scanning image;
step 2.1.7: carrying out Gaussian blur on the foreground image to ensure that pixel values are widely distributed and prepare for equalization;
step 2.1.8: carrying out equalization treatment to obtain a skeleton and soft tissues;
step 2.1.9: cutting the picture to prepare for positioning key points in image segmentation;
step 2.1.10: performing simple threshold segmentation, and inhibiting soft tissues to obtain a skeleton;
step 2.1.11: performing second opening operation, eliminating isolated pixel points, eliminating some burr noises generated after threshold segmentation, and finishing preprocessing to obtain a preprocessed front body image and a preprocessed back body image;
step 2.2: respectively carrying out key point positioning and image segmentation on the preprocessed precursor image and the preprocessed posterior image based on an anatomical segmentation algorithm to obtain a plurality of segmentation bitmap images after segmentation;
and step 3: respectively establishing and training a classification network model for different parts, and testing the network effect;
step 3.1: establishing a classification network model for each part, and training the classification network model by using the training set slice images, the labels of labelme and the depth-based learning model, wherein the networks can be merged for the symmetrical parts;
step 3.2: detecting whether a hot spot of the detected image exists or not by using the trained classification network model;
and 4, step 4: constructing a lightweight classification network model for each part, transferring knowledge of the original classification network model to the lightweight classification network model by a knowledge distillation method, and judging whether hot spots exist in the parts by the lightweight classification network model;
step 4.1: constructing a lightweight classification network module (lightweight model) with the same input and output format as the original network;
step 4.2: training a lightweight classification network model by a knowledge distillation method together with a soft label obtained from an original classification network model (original model) and a label given by a doctor; labels given by the physician are hard labels;
step 4.2.1: inputting any training sample into the classification network model obtained in the step 3, dividing the classification network model by the temperature parameter T in the previous layer of softmax, and then performing softmax operation to obtain a soft label;
step 4.2.2: and controlling the soft label and the hard label through a parameter lambda, and mixing the soft label and the hard label according to a certain proportion to be used as a label for training a lightweight classification network model.
Step 4.2.3: and removing the temperature parameter T in the lightweight classification network model after the training is finished.
Step 4.3: detecting whether hot spots exist in the detected image by using the trained lightweight classification network model;
and 5: and if the position where the hot spot exists is obtained in the step 4, sending the position into a hot spot segmentation module to extract the position of the hot spot.
Step 5.1: if the position where the hot spot exists is obtained, regularization, mean filtering and Gaussian filtering fuzzification are sequentially carried out on the corresponding slice image;
step 5.1.1: regularizing the divided original image;
step 5.1.2: carrying out mean value filtering and smoothing the image;
step 5.1.3: carrying out Gaussian filtering fuzzification to further make the pixel values of the skeleton part uniform;
step 5.2: separating the skeleton and the background of the image by using one-time threshold segmentation, and filling the background by using a segmented threshold;
step 5.3: and (5) segmenting the hot spot region by using threshold segmentation again for the image obtained in the step 5.2.
As shown in fig. 3, 4 and 5, the output of the slice images of the chest, lumbar and pelvis regions through the sub-steps of step 5 is shown in sequence.
The present embodiment, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a computer-readable storage medium. Based on such understanding, all or part of the flow of the method according to the embodiments of the present invention may also be implemented by a computer program, which may be stored in a computer-readable storage medium, and when the computer program is executed by a processor, the steps of the method embodiments may be implemented. Wherein the computer program comprises computer program code, which may be in the form of source code, object code, an executable file or some intermediate form, etc. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, usb disk, removable hard disk, magnetic disk, optical disk, computer memory, Read-only memory (ROM), Random Access Memory (RAM), electrical carrier wave signals, telecommunications signals, software distribution medium, etc. It should be noted that the computer readable medium may contain content that is subject to appropriate increase or decrease as required by legislation and patent practice in jurisdictions, for example, in some jurisdictions, computer readable media does not include electrical carrier signals and telecommunications signals as is required by legislation and patent practice.
In the embodiment, the deep learning technology and the image processing technology are utilized to carry out preprocessing and segmentation processing on the obtained bone scanning image, the classification network model is established for different parts, the lightweight network corresponding to the classification network model is established on the basis of the classification network model, whether the parts have hot spots is judged through the lightweight network, and the hot spot positions of the parts with the hot spots are extracted through the hot spot segmentation module, so that the problem that whether the hot spots exist or not and the specific hot spot positions are segmented under the condition of less supervision information is solved.
Example 2
The embodiment provides a weak supervision bone scanning image hot spot segmentation terminal device based on knowledge distillation, which comprises: a processor, a memory, and a computer program stored in and executable on the processor, such as a program for automatic labeling of hotspots in a bone scan image based on probability of outlier distribution, illustratively, the computer program may be partitioned into one or more modules, the one or more modules being stored in the memory and executed by the processor to implement the present invention. The one or more modules can be a series of instruction segments of a computer program capable of achieving specific functions, and the instruction segments are used for describing the execution process of the computer program in the weak supervision bone scan image hotspot segmentation terminal equipment based on knowledge distillation. For example, the computer program may be divided into an image acquisition module, an image processing module, a classification module, a lightweight module, and a hot spot division module, and each module has the following specific functions:
the image acquisition module is used for acquiring an image matrix of a bone scanning image, wherein the bone scanning image comprises a precursor image and a posterior image;
the image processing module carries out preprocessing operation on the bone scanning image and then carries out segmentation to obtain a segmentation part image of the bone;
the classification module is used for respectively establishing and training classification network models at different parts and testing network effects;
the light-weight module is used for constructing a lighter classification network model for each part, transferring knowledge of the original classification network model to the light classification network model through a knowledge distillation method, and judging whether hot spots exist in the parts through the light classification network model;
and if the position of the hot spot is obtained, sending the position into the hot spot segmentation module to extract the position of the hot spot.
Further, the image processing module for preprocessing the bone scan image comprises: respectively and sequentially carrying out contrast adjustment, automatic threshold segmentation, closing operation, median filtering, first opening operation, mask operation, Gaussian blur, equalization, picture cutting, simple threshold segmentation and second opening operation on the front body image and the back body image, and improving the image quality;
the image processing module segmenting the bone scan image comprises: and respectively carrying out key point positioning and image segmentation on the preprocessed precursor image and the preprocessed posterior image based on an anatomical segmentation algorithm, and obtaining a plurality of segmentation part images after segmentation.
Further, the classification module is used for respectively establishing and training classification network models at different parts, and the testing network effect comprises: establishing a classification network model for each part, and training the classification network model by using the training set slice images, the labels of labelme and the depth-based learning model, wherein the networks can be merged for the symmetrical parts; and detecting whether the hot spot of the detected image exists or not by using the trained classification network model.
Further, the light-weight module is used for constructing a lighter classification network model for each part, transferring the knowledge of the original classification network model to the light-weight classification network model by a knowledge distillation method, and judging whether hot spots exist in the part or not by the light-weight classification network model comprises the following steps: constructing a lightweight classification network module with the same input and output format as the original network; training a lightweight classification network model by a knowledge distillation method together with a soft label obtained from an original classification network model and a label given by a doctor; and detecting whether the hot spot of the detected image exists or not by using the trained lightweight classification network model.
Further, if a part where the hot spot exists is obtained, the part is sent to the hot spot segmentation module to extract the position of the hot spot: if the position where the hot spot exists is obtained, regularization, mean filtering and Gaussian filtering fuzzification are sequentially carried out on the corresponding slice image; separating the skeleton and the background of the image by using one-time threshold segmentation, and filling the background by using a segmented threshold; the hot spot regions are segmented out again using threshold segmentation for the image.
The weak supervision bone scan image hot spot segmentation terminal device based on knowledge distillation in this embodiment may be a desktop computer, a notebook, a palm computer, a cloud server, and other computing devices, and the weak supervision bone scan image hot spot segmentation terminal device based on knowledge distillation may include, but is not limited to, a processor and a memory. Those skilled in the art will appreciate that the present embodiment is only one example, and does not constitute a limitation to the knowledge-based distillation weak-supervised bone scan image hot spot segmentation terminal device, and may include more or less components than those shown in the drawings, or combine some components, or different components, for example, the knowledge-based distillation weak-supervised bone scan image hot spot segmentation terminal device may further include an input-output device, a network access device, a bus, etc.
The Processor may be a Central Processing Unit (CPU), other general purpose Processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), an off-the-shelf Programmable Gate Array (FPGA) or other Programmable logic device, discrete Gate or transistor logic device, discrete hardware component, etc. The general-purpose processor can be a microprocessor or the processor can be any conventional processor and the like, the processor is a control center of the weak supervision bone scanning image hotspot segmentation terminal device based on knowledge distillation, and various interfaces and lines are used for connecting various parts of the whole bone scanning image processing terminal device.
The memory can be used for storing the computer program and/or the module, and the processor realizes various functions of the knowledge-distillation-based weakly supervised bone scan image hotspot segmentation terminal device by operating or executing the computer program and/or the module stored in the memory and calling data stored in the memory. The memory may mainly include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program required by at least one function (such as a sound playing function, an image playing function, etc.), and the like; the storage data area may store data (such as audio data, a phonebook, etc.) created according to the use of the terminal device, and the like. In addition, the memory may include high speed random access memory, and may also include non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) card, a flash memory card (FlashCard), at least one disk storage device, a flash memory device, or other volatile solid state storage device.
It should be noted that, since the drawings in the specification should not be colored or modified, it is difficult to display the clearly distinguished portions in the present invention, and if necessary, a color picture can be provided.
The above description is only for the purpose of illustrating the preferred embodiments of the present invention and should not be taken as limiting the scope of the present invention, and any modifications, equivalents and improvements made by those skilled in the art within the spirit and principle of the present invention should be included in the scope of the present invention.
Claims (10)
1. The weak supervision bone scanning image hot spot segmentation method based on knowledge distillation is characterized by comprising the following steps of: the method comprises the following steps:
step 1: obtaining an image matrix of a bone scan image, the bone scan image comprising a precursor image and a posterior image;
step 2: carrying out preprocessing operation on the bone scanning image and then segmenting the bone scanning image to obtain a segmented part image of the bone;
and step 3: respectively establishing and training a classification network model for different parts, and testing the network effect;
and 4, step 4: constructing a lightweight classification network model for each part, transferring knowledge of the original classification network model to the lightweight classification network model by a knowledge distillation method, and judging whether hot spots exist in the parts by the lightweight classification network model;
and 5: and if the position where the hot spot exists is obtained in the step 4, sending the position into a hot spot segmentation module to extract the position of the hot spot.
2. The knowledge-distillation-based weakly supervised bone scan image hot spot segmentation method of claim 1, wherein: the step 2 comprises the following steps:
step 2.1: respectively and sequentially carrying out contrast adjustment, automatic threshold segmentation, closing operation, median filtering, first opening operation, mask operation, Gaussian blur, equalization, picture cutting, simple threshold segmentation and second opening operation on the front body image and the back body image, and improving the image quality;
step 2.2: and respectively carrying out key point positioning and image segmentation on the preprocessed precursor image and the preprocessed posterior image based on an anatomical segmentation algorithm, and obtaining a plurality of segmentation part images after segmentation.
3. The knowledge-distillation-based weakly supervised bone scan image hot spot segmentation method of claim 1, wherein: the step 3 comprises the following steps:
step 3.1: establishing a classification network model for each part, and training the classification network model by using the training set slice images, the labels of labelme and the depth-based learning model, wherein the networks can be merged for the symmetrical parts;
step 3.2: and detecting whether the hot spot of the detected image exists or not by using the trained classification network model.
4. The knowledge-distillation-based weakly supervised bone scan image hot spot segmentation method of claim 1, wherein: the step 4 comprises the following steps:
step 4.1: constructing a lightweight classification network module with the same input and output format as the original network;
step 4.2: training a lightweight classification network model by a knowledge distillation method together with a soft label obtained from an original classification network model and a label given by a doctor;
step 4.3: and detecting whether the hot spot of the detected image exists or not by using the trained lightweight classification network model.
5. The knowledge-distillation-based weakly supervised bone scan image hot spot segmentation method of claim 1, wherein: the step 5 comprises the following steps:
step 5.1: if the position where the hot spot exists is obtained, regularization, mean filtering and Gaussian filtering fuzzification are sequentially carried out on the corresponding slice image;
step 5.2: separating the skeleton and the background of the image by using one-time threshold segmentation, and filling the background by using a segmented threshold;
step 5.3: and (5) segmenting the hot spot region by using threshold segmentation again for the image obtained in the step 5.2.
6. A weak supervision bone scanning image hot spot segmentation system based on knowledge distillation is characterized in that: the system comprises an image acquisition module, an image processing module, a classification module, a lightweight module and a hot spot segmentation module;
the image acquisition module is used for acquiring an image matrix of a bone scanning image, wherein the bone scanning image comprises a precursor image and a posterior image;
the image processing module carries out preprocessing operation on the bone scanning image and then carries out segmentation to obtain a segmentation part image of the bone;
the classification module is used for respectively establishing and training classification network models at different parts and testing network effects;
the light-weight module is used for constructing a lighter classification network model for each part, transferring knowledge of the original classification network model to the light classification network model through a knowledge distillation method, and judging whether hot spots exist in the parts through the light classification network model;
and if the position of the hot spot is obtained, sending the position into the hot spot segmentation module to extract the position of the hot spot.
7. The knowledge-distillation-based weakly supervised bone scan image hotspot segmentation system of claim 6, wherein: the image processing module for preprocessing the bone scanning image comprises the following steps: respectively and sequentially carrying out contrast adjustment, automatic threshold segmentation, closing operation, median filtering, first opening operation, mask operation, Gaussian blur, equalization, picture cutting, simple threshold segmentation and second opening operation on the front body image and the back body image, and improving the image quality;
the image processing module segmenting the bone scan image comprises: and respectively carrying out key point positioning and image segmentation on the preprocessed precursor image and the preprocessed posterior image based on an anatomical segmentation algorithm, and obtaining a plurality of segmentation part images after segmentation.
8. The knowledge-distillation-based weakly supervised bone scan image hotspot segmentation system of claim 6, wherein: the classification module is used for establishing and training classification network models at different parts respectively, and the test network effect comprises the following steps: establishing a classification network model for each part, and training the classification network model by using the training set slice images, the labels of labelme and the depth-based learning model, wherein the networks can be merged for the symmetrical parts; and detecting whether the hot spot of the detected image exists or not by using the trained classification network model.
9. The knowledge-distillation-based weakly supervised bone scan image hotspot segmentation system of claim 6, wherein: the light-weight module is used for constructing a lighter classification network model for each part, the knowledge of the original classification network model is transferred to the light classification network model through a knowledge distillation method, and whether hot spots exist in the parts or not is judged through the light classification network model, wherein the light classification network model comprises the following steps: constructing a lightweight classification network module with the same input and output format as the original network; training a lightweight classification network model by a knowledge distillation method together with a soft label obtained from an original classification network model and a label given by a doctor; and detecting whether the hot spot of the detected image exists or not by using the trained lightweight classification network model.
10. The knowledge-distillation-based weakly supervised bone scan image hotspot segmentation system of claim 6, wherein: if the position of the hot spot is obtained, the position is sent to the hot spot segmentation module to extract the position of the hot spot: if the position where the hot spot exists is obtained, regularization, mean filtering and Gaussian filtering fuzzification are sequentially carried out on the corresponding slice image; separating the skeleton and the background of the image by using one-time threshold segmentation, and filling the background by using a segmented threshold; the hot spot regions are segmented out again using threshold segmentation for the image.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111216294.8A CN113920100A (en) | 2021-10-19 | 2021-10-19 | Knowledge distillation-based weak supervision bone scanning image hot spot segmentation method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111216294.8A CN113920100A (en) | 2021-10-19 | 2021-10-19 | Knowledge distillation-based weak supervision bone scanning image hot spot segmentation method and system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113920100A true CN113920100A (en) | 2022-01-11 |
Family
ID=79241475
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111216294.8A Withdrawn CN113920100A (en) | 2021-10-19 | 2021-10-19 | Knowledge distillation-based weak supervision bone scanning image hot spot segmentation method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113920100A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114782915A (en) * | 2022-04-11 | 2022-07-22 | 哈尔滨工业大学 | Intelligent automobile end-to-end lane line detection system and equipment based on auxiliary supervision and knowledge distillation |
-
2021
- 2021-10-19 CN CN202111216294.8A patent/CN113920100A/en not_active Withdrawn
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114782915A (en) * | 2022-04-11 | 2022-07-22 | 哈尔滨工业大学 | Intelligent automobile end-to-end lane line detection system and equipment based on auxiliary supervision and knowledge distillation |
| CN114782915B (en) * | 2022-04-11 | 2023-04-07 | 哈尔滨工业大学 | Intelligent automobile end-to-end lane line detection system and equipment based on auxiliary supervision and knowledge distillation |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109741346B (en) | Region-of-interest extraction method, device, equipment and storage medium | |
| EP1035508B1 (en) | Automated method and system for the segmentation of medical images | |
| Hu et al. | Segmentation of brain from computed tomography head images | |
| Zhang et al. | Design of automatic lung nodule detection system based on multi-scene deep learning framework | |
| Varma et al. | An alternative approach to detect breast cancer using digital image processing techniques | |
| Al-Masni et al. | A deep learning model integrating FrCN and residual convolutional networks for skin lesion segmentation and classification | |
| Kaur et al. | Computer-aided diagnosis of renal lesions in CT images: a comprehensive survey and future prospects | |
| Veluchamy et al. | Brightness preserving optimized weighted bi‐histogram equalization algorithm and its application to MR brain image segmentation | |
| Kumarganesh et al. | An efficient approach for brain image (tissue) compression based on the position of the brain tumor | |
| CN113920100A (en) | Knowledge distillation-based weak supervision bone scanning image hot spot segmentation method and system | |
| CN116228767B (en) | X-ray lung tumor image processing method based on computer vision | |
| Soni et al. | CT scan based brain tumor recognition and extraction using Prewitt and morphological dilation | |
| CN110443792B (en) | Bone scanning image processing method and system based on parallel deep neural network | |
| CN109872307B (en) | Method for detecting tumor in biological tissue image, corresponding device and medium | |
| CN108765399B (en) | Lesion site recognition device, computer device, and readable storage medium | |
| Azli et al. | Ultrasound image segmentation using a combination of edge enhancement and kirsch’s template method for detecting follicles in ovaries | |
| WO2022033598A1 (en) | Breast x-ray radiography acquisition method and apparatus, and computer device and storage medium | |
| Isa et al. | Automatic segmentation and detection of mass in digital mammograms | |
| WO2021139447A1 (en) | Abnormal cervical cell detection apparatus and method | |
| Vijayalakshmi et al. | Liver tumor detection using CNN | |
| Munasinghe et al. | Yuwathi: early detection of breast cancer and classification of mammography images using machine learning | |
| Lynn et al. | Segmentation of skin lesion towards melanoma skin cancer classification | |
| Dzulkifli et al. | Colour thresholding-based automatic Ki67 counting procedure for immunohistochemical staining in meningioma | |
| Mouzai et al. | Xray-Net: Self-supervised pixel stretching approach to improve low-contrast medical imaging | |
| Prakash | Medical image processing methodology for liver tumour diagnosis |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WW01 | Invention patent application withdrawn after publication | ||
| WW01 | Invention patent application withdrawn after publication |
Application publication date: 20220111 |